About DataOrb 

DataOrb is revolutionizing how organizations understand and utilize their customer data. We enable businesses of all sizes from ambitious startups to Fortune 500 companies to unlock insights from their customer interactions across conversational, transactional, and structured datasets. Founded by veterans from Google, Amazon, Microsoft, and Samsung, we're driven by a shared mission to democratize customer intelligence and make AI accessible to everyone.

‍The Opportunity

We are looking for an experienced QA Manager to lead the quality strategy for our AI-powered SaaS platform. This role requires building and scaling a QA team capable of testing traditional software (UI, APIs, data) as well as AI-driven, agentic architectures. The QA Manager will define methodologies for validating deterministic systems (microservices, data pipelines) and non-deterministic systems (LLMs, AI workflows). The ideal candidate has prior leadership experience in product-based companies, a strong technical background, and a passion for AI quality governance.

Core Responsibilities
‍
Mandatory Skills

• Proven track record in QA leadership for SaaS or product-based companies.
• Strong automation and test strategy background (UI, APIs, microservices, data).
• Expertise in event-driven architectures (SQS, Kafka, Pub/Sub).
• Deep understanding of Generative AI, LLMs, RAG, and agentic frameworks.
• Ability to define AI evaluation strategies for correctness, hallucination detection, and bias testing.
• Strong stakeholder management and team mentoring skills.
• Strong QA fundamentals: test case design, execution, and defect management.
• Hands-on experience with event-driven microservices (e.g., SQS, Kafka, Pub/Sub).
• Expertise in UI/API test automation frameworks (Cypress, Playwright, REST Assured, Postman).
• Proficiency in SQL and advanced data-level testing (accuracy, correctness, transformations).
• Exposure to Generative AI (LLMs, embeddings, RAG) testing.
• Understanding of agentic AI workflows and ability to test multi-agent orchestration.
• Ability to design and execute deep data-level testing (completeness, correctness, accuracy, referential integrity, and transformation validation).
  ‍

Toolkit

• Cypress, Playwright, Selenium, REST Assured, Postman.
• CI/CD & DevOps: Jenkins, GitHub, Docker/K8S familiarity.
• Queues: SQS, Kafka, RabbitMQ, or equivalent.
• Databases: PostgreSQL, MongoDB, Any Vector DB.
• Preferred: Python/Java for custom test harness development.
• Observability & Monitoring: Grafana, Prometheus, ELK stack.
• AI Testing Tools: Prompt evaluation frameworks, golden datasets, synthetic evaluation.
• Collaboration Tools: Jira, Confluence, Git.
• Strong SQL + hands-on experience with data profiling tools, DB query optimization, and preferably Python for data validation scripts.

Education

• Bachelor’s/Master’s degree in Computer Science, Engineering, or related field. Secondary Skills.
• Experience with Vector Databases (Pinecone or similar).
• Cloud platforms (AWS, GCP, or Azure).
• Certifications: ISTQB Advanced, AWS Certified Developer, or AI/ML certification.
• Experience in AI Ethics, Responsible AI, and Data Privacy testing.
• Knowledge of AI monitoring platforms.
• Cloud certifications (AWS/GCP/Azure).
• QA leadership certifications (ISTQB Test Manager, SAFe, Agile certifications).
  ‍

Non-Technical Requirements

• Define and implement the QA vision and strategy for AI-first SaaS products.
• Build and mentor a high-performing QA team with skills across automation, AI testing, and data validation.
• Establish test strategies for non-API microservices, AI-driven outputs, and agentic workflows.
• Partner with Engineering, Product, and Data Science teams to set AI-specific acceptance criteria.
• Drive automation-first and shift-left testing practices.
• Oversee frameworks for continuous AI evaluation in staging and production.
• Define KPIs for QA effectiveness: automation coverage, AI correctness score, defect leakage.
• Collaborate with DevOps for test integration in CI/CD and observability pipelines.
• Ensure timely delivery of high-quality releases with minimal defect leakage.
• Collaborate with developers, product managers, and data operations team to define acceptance criteria.
• Test API and non-API microservices by building message-driven test harnesses.
• Perform deep data validation to ensure correctness and consistency across systems.
• Test Generative AI outputs by designing prompt-based evaluations and ensuring accuracy, factual correctness, and bias detection.
• Validate agentic AI frameworks, ensuring correct orchestration of agents and workflows.
• Contribute to automation coverage across UI, APIs, queues, and data pipelines.
• Participate in Agile ceremonies (sprint planning, stand-ups, retrospectives).
• Provide accurate and detailed defect reports with reproducible steps.
• Strive for continuous improvement in automation reliability and coverage.
• Support testing in staging and production environments, including live troubleshooting.
• Define and enforce data quality SLAs (accuracy %, timeliness, completeness).
• Collaborate with Data Engineering and AI teams to create golden datasets & synthetic datasets for validation.
• Introduce data quality dashboards/metrics as part of release sign-off.
• Own data validation strategy across batch pipelines, streaming events, and API/microservice boundaries.
• Build automated data comparison frameworks (source vs destination, pre vs post transformation).

Why Join DataOrb

• Mission: Be part of democratizing customer intelligence and making AI accessible
• Impact: Shape how organizations understand and serve their customers
• Team: Work with experienced leaders from top tech companies
• Growth: Rapid scaling environment with significant learning opportunities
• Culture: Autonomous, trust-based environment focused on outcomes
• Benefits:
  
  • Flexible work arrangements
  • Comprehensive health coverage
  • Generous PTO policy
  • Professional development support
  • Competitive compensation package

Our Values

• Customer Obsession: We practice what we preach
• Democratizing Technology: Making complex solutions accessible
• Innovation with Purpose: Solving real customer problems
• Trust and Autonomy: Freedom to create and deliver excellence

‍