Live opening · Posted 1 day ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Requirements:
14+ years of overall experience with 8+ years in data engineering, data platforms, or distributed data systems.
Strong hands-on experience with Google Cloud Platform (GCP), especially BigQuery (data warehousing, performance tuning, cost optimisation, partitioning/clustering); Dataflow (Apache Beam) for large-scale batch and streaming pipelines; Pub/Sub for real-time ingestion and event-driven architectures; Cloud Composer (Airflow) for orchestration and workflow management; and GCS (Cloud Storage) as part of lakehouse or staging architectures.
Deep understanding of modern data architectures, including Lakehouse patterns on GCP and distributed processing systems.
Strong experience designing and operating end-to-end data pipelines (ingestion to transformation to serving) at scale.
Expertise in real-time and streaming architectures, including event design, schema evolution, and fault-tolerant processing.
Hands-on programming skills in Python and Spark (Scala), with experience in building distributed data applications.
Experience implementing CI/CD for data pipelines, including versioning, testing, and deployment automation.
Strong understanding of data modelling and optimisation for analytical workloads in BigQuery.
Practical exposure to data product thinking, including data contracts and schema governance, Discoverability and reuse across domains, ownership and lifecycle management.
Familiarity with MLOps and AI-enabled data platforms on GCP, including support for: Feature engineering pipelines, Model training and inference workflows, Integration with Vertex AI (preferred).
Strong focus on platform reliability and observability, including monitoring, alerting, lineage, and data quality frameworks.
Experience managing cost-performance trade-offs in GCP (e. g., BigQuery cost controls and Dataflow optimisation).
Proven ability to work in globally distributed, federated environments, enabling standardisation across multiple teams and geographies.
Awareness of evolving trends in cloud-native data platforms, data mesh, and event-driven architectures, with the ability to apply them pragmatically.
Experience
14-18 yrs
Recently tracked roles with full details and direct application links.