Senior Data Engineer (GCP)

As a Senior Data Engineer, you are a technical leader within the Data Engineering organisation. You don't just build — you architect, mentor, and set the standard. You lead high-priority initiatives end to end, from requirements through to production, and your decisions meaningfully shape the team's technical direction.
You operate at the frontier of modern data engineering. You understand that AI is not a future consideration — it is a present-day design constraint. You build data infrastructure that is AI ready by default: pipelines that serve feature stores, architectures that can support RAG and LLM applications, and platforms capable of integrating AI-assisted tooling at every stage of the engineering lifecycle.
In a global team spanning Europe, the US, and India, you are a connector — bridging technical depth with business context, and aligning local delivery with global standards.
What You Will Do
Technical Architecture & Delivery
Lead the end-to-end design and delivery of complex data engineering solutions on GCP — from architecture through production deployment
Architect scalable, cost-effective data platforms using BigQuery, Dataflow, Cloud Composer, Pub/Sub, Dataplex, and Cloud Storage
Design robust data models using Dimensional (Kimball), 3NF, and Data Vault methodologies — selecting the right approach for each use case
Implement SCD strategies and historical data management patterns for long-lived datasets
Lead the migration of legacy data structures to GCP, defining parallel testing and data parity validation strategies
Provision and govern cloud infrastructure using Terraform; champion IaC as a non-negotiable standard
Design and implement CI/CD pipelines for all data solutions — with automated testing, linting, and deployment gates
AI-Era Responsibilities
AI-ready architecture: Design every data platform component to be downstream-AI compatible — appropriate partitioning, feature store integration, and schema design for ML consumption
GenAI data infrastructure: Architect data pipelines for LLM-based applications, including embedding generation pipelines, vector store population, and RAG data retrieval layers
Feature store engineering: Build and maintain centralised feature stores on Vertex AI, ensuring reproducibility and low-latency serving for ML models
AI-assisted development leadership: Champion GitHub Copilot, Gemini Code Assist, and Cursor as engineering productivity tools — set standards for how the team uses them responsibly
AI-powered data quality: Design ML-based anomaly detection into pipeline monitoring — moving beyond threshold alerts to intelligent pattern recognition
LLMOps data layer: Build the data infrastructure that underpins model evaluation, fine-tuning dataset curation, and prompt tracking pipelines
Leadership & Collaboration
Lead code reviews; hold the bar for quality, testability, and maintainability
Define and document reusable engineering patterns — pipeline templates, transformation standards, naming conventions
Actively mentor junior engineers through pairing, structured feedback, and technical design sessions
Work closely with global Data Engineering counterparts to align on platform standards
Engage directly with senior business stakeholders to translate complex requirements into technical solutions
Contribute to hiring: review take-home tasks, conduct technical interviews, calibrate assessments
Define and execute testing strategies for regulated workloads, including parallel-run validation against legacy systems
Operational Excellence
Own pipeline reliability: define SLAs, implement alerting, lead incident resolution
Drive DataOps practices: automated testing, data contracts, observability-first design
Monitor and optimise GCP costs; propose and implement efficiency improvements
Ensure compliance with data security, encryption, and governance standards in all solutions built
What We're Looking For
Essential — Technical
5+ years of data engineering experience in production, cloud-native environments
5+ years of advanced SQL: BigQuery specifics, query profiling, partitioning/clustering optimisation, complex analytical queries
3+ years of GCP production experience: architecture design and delivery at scale
Deep, hands-on expertise across: BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Dataplex, Cloud Storage, Terraform, Cloud Build
Mastery of data modelling methodologies: Dimensional/Kimball, 3NF, Data Vault — with real world application of each
Production-level Python: OOP design patterns, async processing, unit/integration testing, GCP SDK usage
Demonstrated experience designing CI/CD pipelines for data products
Track record of leading legacy-to-cloud migrations
Essential — Leadership & Professional
Demonstrated technical leadership: you have designed solutions, led reviews, and raised the quality bar of a team
Proven ability to work in high-ambiguity environments and drive clarity through technical design
Strong communication: able to write architecture decision records, run design reviews, and present to non-technical stakeholders
Evidence of mentoring junior engineers and improving team capability
Minimum 2:2 degree (or international equivalent) in Computer Science or related technical field; demonstrated professional experience considered in lieu for internal applicants
Desired
GCP Professional Data Engineer certification
Experience designing AI/ML data pipelines — feature stores, training data pipelines, Vertex AI integration
Hands-on experience with vector databases or embedding pipeline design
Active use of AI-assisted development tools (Copilot, Gemini, Cursor) in production delivery
Experience with dbt Core / Dataform in a production, team setting Data engineering experience in a regulated financial environment (banking, insurance, credit)
Experience designing event-driven architectures with Pub/Sub and Dataflow

Work arrangement
No