Live opening · Posted 2 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Looking for a hands-on Lead Data Engineer to build scalable data pipelines and data products using Data Mesh principles. This is a core engineering role involving end-to-end system design, development, and optimisation.
Responsibilities:
Build and maintain batch and real-time (streaming) data pipelines.
Develop domain-driven data products and ensure data quality and reliability.
Implement data contracts, governance, and observability.
Optimise query performance and cost.
Work closely with business and domain teams.
Requirements:
7+ years in Data Engineering.
Strong SQL and Python.
Experience with data modelling (star schema, normalisation).
Hands-on with Apache Airflow (or similar).
Experience with large-scale data processing.
Streaming / Real-Time (Important):
Hands-on experience with real-time streaming pipelines.
Tools like Apache Kafka, Spark Streaming, or similar.
Understanding of event-driven architecture and Kappa architecture.
Tech Stack (Flexible):
Any Data Warehouse (Snowflake / BigQuery / Redshift, etc. )
Spark / PySpark.
dbt / Dataform or similar.
Object storage (S3 or similar).
Hands-on experience with AWS (S3 Redshift, EMR, Glue or similar services).
Good to Have:
Data Mesh / Data Product experience.
Strong ownership and independent working style.
Experience working with analysts/stakeholders.
Experience
7-9 yrs
Recently tracked roles with full details and direct application links.