AI QA Engineer

Position Description
Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at cgi.com.
Job Title: AI QA Engineer
Position: Senior Software Engineer
Experience: 5+ Years
Category: Software Development / Quality Engineering
Main Location: HYD/BAN/CHE
Position ID: J1026-0677
Employment Type: Full Time
Position Description:
We are looking for an experienced AI QA Engineer with strong expertise in Python-based test automation, API testing, data validation, and AI/ML model evaluation. The ideal candidate will have hands-on experience testing Generative AI (GenAI) and Large Language Model (LLM) applications, including Retrieval-Augmented Generation (RAG) systems. The role involves evaluating prompt and response quality, validating groundedness, identifying hallucinations, performing Responsible AI and security testing, and ensuring performance and reliability. Experience with cloud AI platforms such as Azure AI Foundry, Azure OpenAI, and Azure Machine Learning is essential.
Your future duties and responsibilities
Design, develop, and maintain automated test frameworks using Python.
Perform API testing and data validation to ensure accuracy, consistency, and reliability.
Evaluate AI/ML models for functional accuracy, performance, and reliability.
Test Generative AI and LLM-based applications, including RAG pipelines.
Evaluate prompt effectiveness and response quality across different use cases.
Perform hallucination detection and groundedness testing to verify that AI-generated responses are supported by source information.
Validate Responsible AI requirements, including relevant safety, security, and risk controls.
Conduct security testing to identify vulnerabilities and potential risks in AI-powered applications.
Perform performance testing to assess response times, scalability, and system reliability.
Integrate automated tests into CI/CD pipelines to enable continuous quality validation.
Work with cloud AI platforms, including Azure AI Foundry, Azure OpenAI, and Azure Machine Learning, to validate AI solutions.
Collaborate with development, data science, and engineering teams to identify defects and improve overall AI solution quality.
Required Qualifications To Be Successful In This Role
Must-Have Skills:
Strong expertise in Python-based test automation.
Hands-on experience in API testing and data validation.
Experience in AI/ML model evaluation and testing.
Experience testing Generative AI and LLM-based applications.
Strong understanding of RAG testing and validation.
Experience in prompt and response quality evaluation.
Knowledge of hallucination detection and groundedness testing.
Experience in Responsible AI validation and security testing.
Experience in performance testing.
Experience integrating automated testing into CI/CD pipelines.
Hands-on experience with cloud AI platforms such as Azure AI Foundry, Azure OpenAI, and Azure Machine Learning.
Good-to-Have Skills:
Experience developing reusable Python-based test automation frameworks for AI applications.
Experience evaluating the quality, reliability, and consistency of LLM-generated responses.
Experience validating RAG retrieval quality and response relevance.
Familiarity with AI safety, responsible AI practices, and security validation techniques.
Experience automating AI quality checks within continuous integration and deployment workflows.
CGI is an equal opportunity employer. In addition, CGI is committed to providing accommodation for people with disabilities in accordance with applicable legislation. Please let us know if you require reasonable accommodation due to a disability during any aspect of the recruitment process, and we will work with you to address your needs.
Life at CGI:
It is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because…
You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction.
Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.
You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.
Come join our team, one of the largest IT and business consulting services firms in the world.
Together, as owners, let’s turn meaningful insights into action.
Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because…
You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction.
Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.
You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.
That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals.
Come join our team—one of the largest IT and business consulting services firms in the world.

Work arrangement
No