Quality Assurance Automation Engineer (Voice AI)

Location: Remote (Pakistan) · Type: Full-time contract · Duration: 3 months (estimated), tied to project completion · Experience Required: 2-3 years
About the role
Entropik Labs is hiring a QA automation engineer to own quality for two voice-first AI products: one where users hold spoken conversations with an AI partner, and a voice agent that answers inbound calls and books appointments. You'll own testing for both, and share monitoring and alerting for the voice agent.
Neither system fails cleanly. A call doesn't crash. It goes quiet for four seconds, mishears a date, or books the wrong Thursday. The job is defining what "good" means in measurable terms, automating what can be automated, and staying curious about the rest.
This is not a gate at the end of the process. You'll be in design discussions early, because most of the bugs worth preventing are decided before the code is written.
What you'll do
Own the test strategy: what gets automated, what gets a scripted manual pass, and what gets a production monitor instead.
Write automated API, integration and end-to-end tests against the full stack running in containers.
Test real-time voice: streamed audio over long-lived connections, interruptions, reconnects, dropped calls, and latency budgets measured rather than felt.
Test booking correctness on the voice agent, where the real damage lives: double-bookings, timezones, retries, and confirmations sent for appointments the calendar never accepted.
Evaluate non-deterministic output (transcription accuracy, agent behaviour, model responses) against reference sets, so quality regressions surface before users find them.
Keep monitoring honest: dashboards, alerts, synthetic test calls, and first-line triage when something breaks.
What we need from you
Two or more years testing software in a real release cycle, with a meaningful part of it automated in code. Python is where this work lives; strong automation in another language plus willingness to work in Python is fine.
Comfort testing APIs and services directly, including async behaviour and jobs that finish after the response returns.
SQL, Linux and Docker: enough to stand up an environment yourself and verify what actually landed in the database.
CI experience: adding jobs, killing flakiness, and keeping the suite fast enough to run on every change.
Production monitoring and observability: metrics, dashboards, log search and alerting, owned rather than observed, including synthetic checks and triage when something breaks.
Clear writing, and the confidence to hold a release when the evidence says so.
Nice to have
Telephony or call-platform experience
Speech and audio evaluation
LLM evaluation or golden datasets
Load testing
Mobile automation
Working under HIPAA and/or GDPR
Time on an on-call rotation

Work arrangement
No