Senior AI platform Engineer

Senior AI Platform Engineer
Job Description
Design, build, and operate a production-grade AI Gateway / shared LLM infrastructure layer serving multiple downstream consumers.
Implement multi-provider LLM routing across Anthropic, Gemini, OpenAI, and other providers, with intelligent fallback logic.
Build usage and cost monitoring, quota management, rate limiting, and centralized logging for LLM traffic.
Establish evaluation pipelines, prompt/version management, guardrails, and PII protection mechanisms.
Build and maintain MCP-based tool-use and agent infrastructure to support downstream agentic applications.
Own production operations for the platform — reliability, scaling, incident response, and performance tuning.
Partner with applied ML and product teams to expose LLM capabilities as reliable, governed infrastructure.
Roles & Responsibilities
Strong, hands-on Python engineering skills, with production-grade code quality.
Deep experience with GCP, GKE, and Kubernetes for deploying and scaling services.
Proven experience building multi-provider LLM routing systems (Anthropic, Gemini, OpenAI, etc.).
Experience implementing usage/cost monitoring, quotas, rate limits, logging, and fallback mechanisms in production.
Experience building evaluation frameworks, prompt/version management, guardrails, and PII protection.
Hands-on experience with MCP, tool-use patterns, and agent infrastructure.
Strong production operations background — this must be demonstrable, not POC-only.
Avoid: prompt engineers, POC-only consultants, architecture-only profiles with no hands-on ops experience.

Work arrangement
Hybrid