Senior AI Data Engineer

About Amplify Health
Who We Are
Amplify Health Asia Pte. Limited (Amplify Health) is a pan-Asian health technology and analytics company which brings together health technology assets, proprietary data analytics and healthcare expertise to support innovation across the healthcare value chain.
Headquartered in Singapore, Amplify Health operates in 10 markets across the region: Australia, Hong Kong, India, Indonesia, Malaysia, New Zealand, the Philippines, Singapore, Thailand and Vietnam.
Amplify Health develops integrated solutions that empower healthcare payors, providers and policymakers across Asia to identify value and drive new partnerships, understand variation and address systemic inefficiencies—with the aim of improving outcomes for individuals and the sustainability of health systems.
Powered by a clinically enriched data foundation, Amplify Health’s health intelligence platform combines modular AI solutions, context-aware automation and deep regional insights to offer localised, AI-powered health solutions spanning health payment integrity to chronic disease management. The company’s technology has earned recognition at several industry competitions, including the Celent Model Insurer Awards, ITC Asia Awards and SBR Technology Excellence Awards.
As an ISO27001 certified organisation, Amplify Health maintains clearly defined policies, structured oversight mechanisms and ongoing risk assessments that ensure alignment with global best practices on enterprise data security.
Our Vision and Ambition
To build the leading healthcare AI and platform services company in Asia that transforms the delivery of health and wellness for patients and communities by combining and leveraging the distinctive and complementary assets and strengths of AIA and Discovery.
Amplify Health will simplify access to health data and AI Innovation to accelerate distinct and disruptive healthcare value insights and resulting improvements in health outcomes through value-based care, personalised care plans and aligning individuals’ lifestyle/ behavioural choices.
By 2028, Amplify Health will have in place one of Asia’s strongest health-tech and AI capabilities; a comprehensive, integrated health technology stack supported by precision insights derived from proprietary data pools.
The Position
Role Overview
As a Senior AI Data Engineer, you will design, build and run the data foundations that make AI work in production. That means batch and streaming lakehouse pipelines, as well as the unstructured-data, retrieval and evaluation pipelines behind GenAI and agentic applications. You will work mainly on Azure and Databricks, use AI-assisted engineering tools every day, and own complex components end to end with a high degree of autonomy. You will be a go-to engineer on your team, mentoring others and raising the bar on quality, security and cost-efficiency.
Role at a Glance
Experience
6 to 9 years in data engineering, including at least 1 to 2 years delivering AI/ML or GenAI data workloads to production
Team leadership
No direct reports. Mentors 1 to 3 engineers and may act as technical owner of a workstream
Project scope
One project (occasionally two), owning major components or a workstream end to end
Stakeholder / client exposure
Works directly with business, client and partner technical stakeholders on requirements, designs and demos
Hands-on expectation
About 85 to 90% hands-on engineering
Responsibilities
Data and Lakehouse Engineering
Design, build and maintain scalable batch and streaming pipelines on Azure and Databricks (Lakeflow Spark Declarative Pipelines, Lakeflow Jobs, Structured Streaming, Azure Data Factory, Event Hubs / Kafka).
Model data across medallion (bronze, silver, gold) layers using Delta Lake or Apache Iceberg, designing schemas that serve both analytics and AI consumers.
Implement change data capture, incremental processing and schema evolution; build reusable ingestion frameworks for structured, semi-structured and unstructured sources.
Tune Spark and SQL workloads and compute configuration (including serverless) for performance and cost; track and reduce compute, storage and token spend.
AI data and GenAI Engineering
Build pipelines that turn unstructured content (documents, PDFs, images, forms, transcripts) into AI-ready data: parsing and OCR, chunking, metadata enrichment, de-duplication and embedding generation.
Implement and operate vector indexes and retrieval (Databricks Vector Search, Azure AI Search), including hybrid search, re-ranking, incremental index refresh and permission-aware retrieval.
Prepare high-quality, versioned datasets for ML and LLM use cases: features, evaluation (golden) sets, fine-tuning data and, where appropriate, synthetic data.
Integrate data products with GenAI and agentic applications through APIs, Model Context Protocol (MCP) servers and tool / function interfaces; support model serving and AI gateway configuration.
Instrument AI pipelines with tracing and evaluation (for example, MLflow 3 tracing, LLM-as-judge scorers, retrieval quality metrics) and act on the results.
Quality, Governance and Security
Implement data quality checks, data contracts and observability (expectations, freshness, volume, schema and drift monitoring) across pipelines.
Apply Unity Catalog and Microsoft Purview governance: access controls, lineage, tagging and classification of sensitive data.
Protect sensitive data, including health and personal data, through masking, tokenisation, de-identification and least-privilege access; follow responsible-AI and AI-security practices such as the OWASP Top 10 for LLM Applications.
Engineering Practices
Write production-grade, tested Python, PySpark and SQL; contribute to CI/CD and infrastructure-as-code (Databricks Asset Bundles, Terraform / Bicep, GitHub Actions or Azure DevOps).
Use AI coding assistants and agents (for example, GitHub Copilot, Databricks Genie Code) to speed up delivery, and review generated code for correctness, security and cost.
Produce clear technical designs, runbooks and documentation; take part in production support for owned components.
Collaboration and Mentoring
Partner with data scientists, AI engineers, analysts, product owners and business or client stakeholders to turn requirements into technical solutions.
Mentor junior and mid-level engineers through pairing, code reviews and knowledge-sharing sessions.
Evaluate new tools and patterns through spikes and proofs of concept, and share findings with the team.
Candidate Profile
Required Qualifications
6 to 9 years of data engineering experience, including at least 3 years building cloud data platforms on Azure and Databricks (or equivalent).
Strong Python, PySpark and SQL, with solid software engineering fundamentals: testing, version control, code review and packaging.
Deep hands-on experience with Spark, Delta Lake and lakehouse / medallion architectures, plus batch and streaming patterns (Kafka / Event Hubs, Structured Streaming, CDC).
Hands-on delivery of at least one GenAI or ML data workload to production, such as a RAG ingestion and retrieval pipeline, a feature pipeline or an evaluation-dataset pipeline.
Working knowledge of embeddings, vector search, chunking strategies and retrieval-quality trade-offs.
Experience processing unstructured data (documents, PDFs, images or free text) at scale.
Experience with orchestration (Lakeflow Jobs, Azure Data Factory, Airflow), CI/CD and infrastructure-as-code.
Good understanding of data governance, security and privacy controls: Unity Catalog, RBAC / ABAC, encryption, masking, managed identities and private networking.
Practical, responsible use of AI-assisted development tools in day-to-day engineering.
Strong problem-solving skills and clear written and verbal communication.
Technology Stack: (Primary: Azure + Databricks)
Cloud & storage: ADLS Gen2, Azure Data Factory, Event Hubs, Azure SQL, Cosmos DB, Key Vault, Microsoft Fabric / OneLake
Lakehouse & processing: Azure Databricks: Delta Lake, Lakeflow (Connect, Spark Declarative Pipelines, Jobs), Structured Streaming, Photon, serverless compute
Governance & catalog: Unity Catalog (lineage, tags, ABAC, row/column security), Microsoft Purview
AI, retrieval & agents: Databricks Vector Search, Model Serving, Agent Bricks, Lakebase; Azure AI Search; Microsoft Foundry (Azure OpenAI and other models, Agent Service); Azure AI Document Intelligence
MLOps / LLMOps: MLflow 3 (tracking, tracing, evaluation, prompt registry), Feature Store, Unity AI Gateway, inference tables
Engineering & DevOps: Python, PySpark, SQL, Git, GitHub Actions or Azure DevOps, Terraform / Bicep, Databricks Asset Bundles, Docker
AI-assisted engineering: GitHub Copilot, Databricks Genie Code /

Work arrangement
Hybrid