Quality Engineer

We're Hiring: AI QA Engineer (Generative AI & Web Applications)
Location: Ahmedabad (Onsite)
Experience: 3+ Years
Working Days: 5 Days a Week
Are you passionate about ensuring the quality, reliability, and performance of AI-powered applications?
We're looking for an experienced AI QA Engineer to own quality across a growing portfolio of AI-driven products, including AI-powered dashboards, CRMs, RFP platforms, lead generation systems, agentic workflows, and workflow automations. This is an exciting opportunity to work at the intersection of Artificial Intelligence, Web Applications, and Quality Engineering, ensuring production-ready AI solutions for global clients.
Key Responsibilities
AI & LLM Testing
Design and execute test strategies for LLM-powered features, AI agents, RAG systems, and workflow automations.
Build and maintain evaluation suites, golden datasets, and prompt regression tests.
Validate AI responses for factual accuracy, hallucinations, formatting, instruction-following, and overall quality.
Test agentic workflows, tool calling, multi-step reasoning, error handling, latency, and cost optimization.
Perform adversarial testing, including prompt injection, jailbreak attempts, and edge-case validation.
Application Quality Assurance
Own end-to-end QA for AI-powered web applications, dashboards, CRMs, and client portals.
Perform functional, regression, cross-browser, responsive, and usability testing.
Test REST APIs, webhooks, and third-party integrations.
Validate database integrity, multi-tenant security, authentication, authorization, and role-based access.
Conduct performance, stability, and file-processing validation.
Automation & Quality Engineering
Develop automated test suites using Python and JavaScript.
Integrate automated testing into CI/CD pipelines.
Monitor production systems for AI quality, latency, token usage, and performance.
Collaborate closely with AI Engineers, Full Stack Developers, Product Teams, and Delivery Managers.
Document defects, maintain QA dashboards, and contribute to AI QA best practices.
Required Skills
3+ years of QA experience, including testing AI-enabled applications or LLM-powered features.
Strong Python programming skills.
Working knowledge of JavaScript/TypeScript.
Experience with Generative AI, LLM testing, Prompt Engineering, and Prompt Regression Testing.
Hands-on experience with Promptfoo, DeepEval, RAGAs, LangSmith, Braintrust, or similar evaluation frameworks.
Experience with Playwright, Cypress, or Selenium.
API testing using Postman or similar tools.
Good understanding of SQL, databases, CI/CD, Git, and cloud platforms (AWS, Azure, or GCP).
Knowledge of vector databases, embeddings, and Retrieval-Augmented Generation (RAG).
Preferred Skills
Experience with n8n, Make, Zapier, or AI agent frameworks.
Familiarity with Langfuse, Helicone, LangSmith, or AI observability platforms.
Experience with OpenAI, Claude, or other production LLM APIs.
Knowledge of Supabase, PostgreSQL, MCP, or client-facing SaaS platforms.
Exposure to prompt injection testing, red teaming, SOC 2, or compliance testing.
What We're Looking For
Strong analytical and problem-solving skills
Passion for AI quality and automation
Excellent debugging and communication skills
Ability to work collaboratively in a fast-paced environment
Proactive mindset with attention to detail
If you're excited about building reliable, high-quality AI products and shaping the future of AI Quality Assurance, we'd love to hear from you!
Interested candidates can share their updated resume on avadhi@thepremierconsultants.com.
#Hiring #AIQA #QualityAssurance #GenerativeAI #LLM #PromptEngineering #Python #AutomationTesting #SoftwareTesting #Playwright #Cypress #APItesting #ArtificialIntelligence #MachineLearning #TechJobs #AhmedabadJobs #HiringNow

Work arrangement
No