Live opening · Posted 27 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Basic Qualifications
Strong proficiency in Python.
Experience operating AI-powered or model-dependent systems in production.
Strong understanding of CI/CD integration and release gating processes.
Experience defining measurable quality standards across distributed systems.
Proven ability to influence architectural and engineering standards across multiple teams.
Requirements:
Experience designing benchmarking systems for LLMs or ML models.
Familiarity with hallucination detection, safety scoring, or response consistency analysis.
Experience building A/B testing infrastructure for model comparison.
Exposure to multi-provider model environments (AWS, OpenAI, Bedrock).
Background in experimentation systems or engineering-led release governance.
Experience
6-10 yrs
Recently tracked roles with full details and direct application links.