AI Architect

We are seeking an experienced AI Architect to lead the agentic AI engineering function, responsible for the design, architecture, and quality of all AI agents on the platform.
The role will define engineering standards for prompt design, agent evaluation, and LLM integration across the team.
What You'll Do
Architect the agent engineering framework, including orchestration patterns, prompt governance, and tool-calling standards.
Lead design reviews of all agent implementations across the team.
Own the RAG pipeline architecture, including embedding strategy, retrieval optimization, and accuracy benchmarks.
Define the agent evaluation framework, including golden datasets, confidence calibration, and regression testing.
Set prompt engineering standards and chair the prompt review board.
Mentor JL1 engineers on agent development and evaluation.
Monitor and optimize agent performance in production, including latency, cost, and accuracy.
Required Skills & Experience
10+ years of experience in AI/ML engineering, with 5+ years working with production LLM systems.
Deep expertise in prompt engineering, agent design, and LLM integration.
Experience architecting RAG pipelines at scale.
Experience with agent evaluation, including hallucination testing, confidence scoring, and regression frameworks.
Expert-level Python skills; TypeScript/Node.js familiarity is a plus.
Experience leading AI/ML engineering teams.
Strong understanding of LLM cost management, token budgeting, and latency optimization.
Nice to Have
Experience with Model Context Protocol (MCP) or tool-calling standards.
Background in enterprise AI platform builds.
Knowledge of ITSM or IT operations domains.

Work arrangement
No