Software Architect - RAG Based Agentic AI Platform

Introduction
At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.
Your Role And Responsibilities
Navi is IBM/HashiCorp's agentic AI platform, spanning multiple product pillars (Assistant, Insight, GTM, with a fourth in development) and handling tens of thousands of queries per month for field, customer-facing, and go-to-market teams. We're looking for a Band 10 AI Architect to serve as the end-to-end technical co-owner of Navi's architecture — someone equally comfortable designing an intuitive user experience, architecting agentic reasoning flows, engineering the underlying data pipelines, and making platform-level infrastructure decisions.
This is a full-stack AI architecture role: you won't hand off between specialists — you'll be expected to move fluently across every layer of the stack, from UI/UX through to deployment infrastructure, and to write production code yourself where needed.
Product ownership across Navi is currently divided among a small group of architects, each with deep context in different areas of the platform. This role requires someone who can operate independently while also actively collaborating with those architects — brainstorming design options, reconciling different perspectives, and converging on the best overall solution rather than optimizing for their own area in isolation.
What You'll Be Working On
UI/UX Layer
Architect intuitive, trustworthy interfaces for surfacing AI-generated answers, confidence signals, and source citations to non-technical business users
Design interaction patterns that make agentic reasoning transparent (e.g., how confidence scoring, source grounding, and escalation paths are presented to end users)
Agentic AI Layer
Design and evolve multi-agent orchestration and reasoning flows (current stack includes Flowise and Langflow), including tool use, retrieval strategies, and prompt/system-message architecture
Own the model-grounding strategy — ensuring answers are anchored in vetted, governed content sources rather than open-ended generation
Design and refine confidence-scoring mechanisms, including evaluating and improving on current LLM-self-assessed scoring approaches
Lead exploration and adoption of MCP (Model Context Protocol) to improve system portability and support broader enterprise integration
Integrate agentic workflows with enterprise orchestration platforms (e.g., watsonx Orchestrate)
Data Layer
Architect data ingestion pipelines from diverse, mixed-reliability sources (formal validated designs, structured GTM/CRM data, informal internal knowledge)
Design and improve retrieval infrastructure (vector search/RAG, e.g. Pinecone), including retrieval quality, chunking strategy, and reducing retrieval-driven answer variance
Partner with data science on predictive modeling initiatives (e.g., churn prediction), including data readiness, feature pipelines, and drift monitoring
Own data governance and content-scoring frameworks that rank source reliability and trigger low-confidence alerts to users
Platform Layer
Make infrastructure and re-platforming decisions as Navi scales across IBM business units, including cloud environment strategy (e.g., Azure to Cirrus transitions)
Architect for reliability, determinism, and reproducibility in LLM-based systems (e.g., addressing answer inconsistency through temperature control, retrieval stability, and regression testing)
Design for compliance, security, and governance requirements specific to enterprise AI deployment at IBM scale
Evaluate and integrate new AI infrastructure/tooling as the ecosystem evolves
What You'll Improve
Beyond maintaining what exists, you're expected to actively advance Navi's current capabilities, including:
Reducing answer inconsistency and improving reproducibility across agentic responses
Strengthening confidence-scoring rigor beyond current LLM self-assessment approaches
Improving text-to-SQL / structured-query reliability for analytics-style queries
Advancing the re-platforming and MCP-adoption effort to support enterprise-wide governance and portability goals
Building more robust content governance and feedback loops with content owners and subject-matter experts
Preferred Education
Master's Degree
Required Technical And Professional Expertise
20+ years of software/AI/data science engineering experience, including significant hands-on architecture experience
Demonstrated ability to design and ship production AI applications end-to-end — not just prototype-level work
Deep, practical expertise in LLM application architecture: RAG/retrieval systems, agentic/multi-agent frameworks, prompt and system-message design, and orchestration tooling
Strong data engineering background: pipeline design, vector databases, data quality/governance frameworks
Experience with cloud infrastructure and platform architecture at enterprise scale
Proficiency across the full stack — able to write production code in UI, backend, and data/ML layers as needed, not purely a "whiteboard architect"
Strong communication skills — able to represent architecture decisions to both engineering teams and executive stakeholders
Proven ability to collaborate effectively with other architects on a shared platform — brainstorming openly, reconciling differing viewpoints, and building consensus around the best overall solution rather than working in isolation
Preferred Technical And Professional Experience
Experience with Flowise, Langflow, or comparable low-code/agentic orchestration tools
Familiarity with Model Context Protocol (MCP) or similar interoperability standards
Experience with enterprise orchestration platforms (e.g., watsonx Orchestrate)
Background in applying AI to GTM, customer success, or sales analytics use cases
Prior experience navigating large-enterprise governance, compliance, and re-platforming initiatives

Work arrangement
Yes