Automation Test Engineer (AI & Agent)

Role: Automation Test Engineer (AI & Agent)
Experience: 6-12 Years
Location: Hyderabad, Bangalore, Chennai, Delhi
Job Description:
Key Responsibilities
Define risk-based test strategies, quality gates, coverage models, and release criteria for AI agents and automation solutions.
Design automated testing frameworks for prompts, tools, workflows, memory, grounding, governance controls, integrations, and human approval processes.
Develop golden datasets, scenario libraries, adversarial test cases, and outcome-evaluation frameworks with business and technical stakeholders.
Execute functional, regression, integration, end-to-end, performance, resilience, recovery, and operational-readiness testing.
Evaluate hallucination risks, grounding accuracy, relevance, consistency, safety controls, tool execution quality, and decision traceability.
Validate access controls, prompt-injection protection, data leakage prevention, auditability, reversibility, and fail-safe mechanisms.
Test integrations across enterprise systems, operational support platforms, service management systems, inventory repositories, telemetry sources, and workflow orchestration platforms.
Automate evidence collection within CI/CD pipelines and maintain independent test environments, mocks, stubs, and virtualized services.
Manage defect identification, prioritization, root-cause validation, and remediation verification.
Provide release-readiness assessments and independent quality recommendations for testing, deployment, and production releases.
Mandatory Technical Skills
AI & Agent Testing
LLM evaluation and benchmarking
Prompt regression testing
Function and tool validation
Grounding validation
Memory testing
Multi-agent workflow testing
Automation Frameworks
Python
PyTest
API and UI testing
CI/CD pipelines
Test data management
Mocks, stubs, and service virtualization

Work arrangement
No