AI Architect

About the Role
We are looking for an experienced AI Architect to lead the agentic AI engineering function, taking ownership of the design, architecture, and quality of AI agents across the platform.
The role involves defining engineering standards for prompt design, agent evaluation, and Large Language Model (LLM) integration. The ideal candidate will combine deep technical expertise in AI/ML engineering with strong architectural leadership, ensuring AI agents are reliable, scalable, accurate, and cost-effective in production.
Key Responsibilities
Design and establish the agent engineering framework, including orchestration patterns, prompt governance, and tool-calling standards.
Lead design reviews for AI agent implementations across the engineering team.
Own the Retrieval-Augmented Generation (RAG) pipeline architecture, including embedding strategies, retrieval optimisation, and accuracy benchmarks.
Define and implement agent evaluation frameworks covering golden datasets, confidence calibration, and regression testing.
Establish prompt engineering standards and lead the prompt review board.
Mentor junior-level (JL1) engineers in agent development, testing, and evaluation.
Monitor and optimise AI agent performance in production, focusing on latency, cost, and accuracy.
Drive engineering best practices across agent design, LLM integration, and production deployment.
Required Skills and Experience
10+ years of experience in AI/ML engineering, including at least 5 years working with production-grade LLM systems.
Deep expertise in prompt engineering, AI agent design, and LLM integration.
Proven experience architecting and scaling RAG pipelines.
Strong understanding of agent evaluation techniques, including hallucination testing, confidence scoring, and regression testing frameworks.
Expert-level Python programming skills.
Experience leading AI/ML engineering teams.
Strong understanding of LLM cost management, token budgeting, and latency optimisation.
AI Architect
Familiarity with TypeScript and Node.js.
Experience with Model Context Protocol (MCP) or other tool-calling standards.
Background in building enterprise AI platforms.
Knowledge of IT Service Management (ITSM) or IT operations domains.

Work arrangement
No