Live opening · Posted 11 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Responsibilities:
Own the technical design and architecture of scalable data-processing and transformation solutions on GCP.
Design and build production-grade batch, incremental and streaming data pipelines.
Develop and optimise complex transformations using BigQuery, SQL and Dataform / dbt.
Build reusable, metadata-driven and configuration-driven frameworks that reduce bespoke pipeline development.
Design orchestration patterns using Cloud Composer / Apache Airflow, including dynamic workflows, dependencies, retries and recovery.
Develop distributed and streaming processing solutions using Dataflow / Apache Beam and Dataproc / Spark.
Define standards for data modelling, schema design, schema evolution and data contracts.
Design and evolve modern analytical storage patterns, including Apache Iceberg.
Establish engineering patterns for data quality, reconciliation, metadata, lineage and observability.
Optimise large-scale workloads for performance, scalability, latency and GCP cost.
Define reusable engineering standards for testing, versioning, CI/CD and production readiness.
Lead technical design and code reviews and guide engineers on complex implementation decisions.
Identify systemic technical debt and drive improvements to shared platform capabilities.
Work closely with Data Engineering, Platform Engineering, DevOps, Architecture, Governance and Product teams.
Take technical ownership from solution design through production deployment and operational stability.
Experience
3-7 yrs
Recently tracked roles with full details and direct application links.