AI/GenAI Quality Engineer (Playwright, API Testing, LLM Testing)

About the Company
Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd
Teamware Solutions (a division of Quantum Leap Consulting Pvt. Ltd.) is a 20+ year industry leader providing IT staffing and consulting services, supporting 4000+ professionals across India, the USA, the Middle East, and APAC for leading GCC clients.
About the Role
We are hiring an AI/GenAI Quality Engineer (Playwright, API Testing, LLM Testing)
Location: Bangalore
Work Model: Hybrid
Experience: 5 to 7 years
Notice Period: Immediate - 15 days
Shift Timing: 2 PM- 11 PM
QE Consultant- 4 Positions:
Designation: Associate Consultant, Consultant
Key Responsibilities :
Test AI/GenAI, LLM-based, and web applications for quality, reliability, and performance.
Design and execute functional, regression, API, and end-to-end test cases.
Develop and maintain Playwright automation frameworks.
Perform API testing using Postman, REST Assured, or Playwright.
Validate AI outputs for accuracy, relevance, groundedness, consistency, hallucinations, and safety.
Create prompt test suites, benchmark datasets, and AI evaluation metrics.
Integrate automated tests into CI/CD pipelines.
Collaborate with Product Owners, Developers, Data Scientists, and AI Engineers.
Support Responsible AI, quality engineering best practices, and continuous improvement.
Experience
4–6 years of experience in Quality Engineering, Software Testing, or Test Automation.
Hands-on experience testing AI/Generative AI applications.
Experience with modern web application testing and API testing.
Technical Skills
Strong expertise in Playwright automation.
Hands-on experience with API testing and automation.
Experience validating AI/LLM-based applications and AI agents.
Good understanding of AI evaluation methodologies and testing techniques.
Familiarity with:
M365 Copilot
OpenAI/Azure OpenAI
Claude
Gemini
Agentic AI frameworks
Knowledge of test management tools such as Azure DevOps (ADO), Jira, or similar platforms.
Experience working with CI/CD pipelines and DevOps practices.
Preferred Skills
Exposure to Python or TypeScript for automation and AI testing.
Experience with AI evaluation frameworks such as:
RAG evaluation
Prompt evaluation
LLM benchmarking
Human feedback-based evaluation
Knowledge of Responsible AI principles, AI governance, and model risk management.
Experience testing Retrieval-Augmented Generation (RAG) systems and AI agents.
Education / Professional qualifications
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
Other Information
Number of interview rounds: 2
Please let me know if you are interested in this position and send me your resume to netra.s@twsol.com

Work arrangement
No