Airline PSS domain – Conversational AI Testing

Software Quality Engineer – Conversational AI / LLM Testing
Experience: 4-9 Years
Location : Noida / Kolkata
Domain : Airlines / Travel Technology
Job Summary
We are seeking a highly skilled Software Quality Engineer with hands-on experience in testing Conversational AI, LLM-based applications, chatbots, virtual assistants, and Agentic AI systems. The ideal candidate will have a strong automation and quality engineering background, deep understanding of how modern conversational systems operate, and solid experience within the Airline or Travel domain, particularly around Booking, Post-Booking, and Check-In workflows.
4+ years in software quality engineering, with demonstrable ownership of a product's quality
Strong scripting ability in Python
Hands-on experience testing LLM-based, NLP or conversational systems — chatbots, voice assistants, virtual agents, RAG or agentic applications.
Practical understanding of how modern conversational AI works: prompting, context windows, function/tool calling, retrieval, and the characteristic ways these systems fail.
Experience designing evaluation approaches for non-deterministic output — golden datasets, rubric or LLM-as-judge scoring, human review workflows, statistical rather than binary pass criteria.
Strong Airline or travel domain knowledge (Booking, Post Booking & Check-In flows)

Work arrangement
Hybrid