Senior Quality Engineer - AI Integrations

We are looking for a Senior Quality Engineer - AI Integration to help achieve this mission. The Senior Quality Engineer will contribute to our Engineering organization by building scalable automation frameworks and quality engineering solutions for artificial intelligence, machine learning, and cybersecurity applications. This role will focus on ensuring reliability, performance, and quality across distributed systems, cloud-native services, and large language model-driven applications.
Responsibilities:
Create and extend software frameworks and tools to execute, monitor, and report on automated tests.
Build and maintain automation frameworks and continuous integration and continuous delivery pipelines.
Review testing strategies, approaches, and test plans while providing guidance to development teams.
Architect, design, and build automation solutions for application programming interface validation, performance testing, load testing, and fuzz testing.
Develop and maintain testing frameworks for generative artificial intelligence and machine learning applications.
Validate model behavior, prompt-response consistency, hallucination detection, and regression testing for artificial intelligence inference application programming interfaces.
Implement evaluation pipelines to measure large language model accuracy, latency, bias, toxicity, and reliability.
Collaborate with highly skilled software engineering professionals across teams.
Continuously improve engineering quality, automation practices, and testing methodologies.
Learn and adopt emerging technologies in artificial intelligence, cybersecurity, and cloud engineering.
Requirements:
Our Tech Stack: Selenium, Playwright, Docker, Kubernetes, Amazon Web Services (including Bedrock, Lambda, Simple Queue Service, and Simple Storage Service), and LangChain; Large language model evaluation frameworks such as Ragas, OpenAPI, PostgreSQL, and Linux.
You are excited to work at the intersection of artificial intelligence, machine learning, big data, and cybersecurity.
You have more than 6 years of experience delivering high-quality software solutions.
You are passionate about improving software quality through automation and testing.
You have experience deploying services on cloud computing platforms such as Amazon Web Services, Microsoft Azure, or Google Cloud Platform.
You have strong expertise with tools used for functional testing, load testing, and fuzz testing such as Locust and CATS.
You have experience building automation frameworks and continuous integration and continuous delivery pipelines.
You have experience working with distributed systems and microservices architectures.
You possess strong programming and problem-solving skills in modern software engineering environments.
Nice to have:
Experience testing artificial intelligence or machine learning applications.
Familiarity with large language model evaluation techniques and prompt engineering.
Experience working in cybersecurity or cloud-native environments.

Experience
6-10 yrs