QA Data Science Engineer

Description

We are seeking a Data Science focused QA engineer to develop next-generation Security Analytics products. You will work closely with Data scientists,engineersand product managers to design andoptimizeAI driven security solutions.

AsQAengineer, the ideal candidate has a strong background in Backend engineering, system integrations,ML,AIand data pipelines.

Responsibilities (QA Engineer – Data Science / ML)

• Establish QA best practices for Traditional ML and Generative AI workflows, including:

• Functional and regression testing of ML pipelines usingpytestand Airflow/Dagstertest utilitiesand API testing tools (e.g., Postman,pytest-httpx).

• Validate data contracts, schemas, and API compatibility across services usingPandera, and custom validation rules.

• Model behavior validation (input/output ranges, invariants, edge cases) using NumPy, SciPy, and statistical assertions

• Runtime and performance testing for inference latency, throughput, and resource usage using Locust, k6, or custom load tests.

• Integrate ML-specific tests into CI/CD pipelines using GitHub Actions, GitLab CI, or Jenkins, alongside containerized workflows (Docker, Kubernetes).
  

• Implement LLM-specific testing, including:

• Prompt and response validation, determinism checks, and regression testing usingLangSmith.

• Evaluation of hallucinations, toxicity, and policy adherence using LLM-as-a-judge and/orrule-based checks.

• Cost, token usage, and timeout monitoring for GenAI workflows
  

• Verify logging, monitoring, and alerting for ML services using Prometheus, Grafana, and cloud-native observability tools.

Requirements:

• BS or MS in Computer Science or a related field.

• 2-5 years of experience in Dataor MachineLearning projects.

• Familiarity and experienceof GenAI applicationsand tools -PyTorch, LangChain, vLLM etc.

• Demonstrates a commitment to continuous learning in this rapidly evolving field.

• Tools listed inthe responsibilitiessection.