AI Product Manager

We're looking for a deeply technical AI Product Manager, someone who can own the product and understand how the underlying AI system behaves.
Responsibilities:
Own AI/agent products end-to-end: user problem, product strategy, requirements, and launch iteration.
Define how AI agents should reason, retrieve, orchestrate tools, and complete research workflows.
Build and own AI evaluation frameworks: eval datasets, LLM-as-judge, human evals, regression testing, hallucination/factuality, and failure analysis.
Define metrics around accuracy, task completion, latency, reliability, and user trust.
Work deeply with engineering/AI teams on LLMs, RAG, agentic workflows, prompting, orchestration, and system architecture.
Translate AI failure modes and user feedback into concrete product and engineering improvements.
Requirements:
Strong product management fundamentals with experience shipping products 01 and iterating them at scale.
Hands-on understanding of AI/LLM systems; this cannot be a surface-level AI PM.
Strong experience with LLM/agent evaluations, experimentation, and AI quality measurement.
Technical background, ideally with prior experience in software engineering, ML/AI, NLP, applied science, or another engineering-heavy role.
Comfortable discussing APIs, system architecture, RAG, agents, prompts, model behavior, and trade-offs directly with engineers.
Experience with LangGraph, LangChain, or similar agent frameworks is a strong advantage.
The bar: We are looking for a product manager + AI systems thinker + evals owner in one person, not a traditional PM who has only worked around AI.

Experience
5-9 yrs