Live opening · Posted 13 hours ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Job Title: Data Engineering Lead –
Role Summary: We are looking for a Data Engineering Lead to lead a team of data engineers in building and delivering scalable ingestion and transformation pipelines on Databricks + AWS, ensuring engineering execution aligns with the platform's architecture and delivery timelines.
Key Responsibilities
Lead a team of Data Engineers delivering Bronze/Silver/Gold pipelines across batch, near-real-time, streaming, and CDC patterns
Drive hands-on implementation and code/design reviews for ingestion pipelines (AWS Glue, DMS, DataSync) and streaming jobs (Kinesis, Kafka, or Databricks Structured Streaming)
Own sprint planning, task allocation, and delivery tracking for the engineering pod
Ensure adherence to Unity Catalog governance standards, data quality checks, and orchestration best practices (Airflow/MWAA)
Troubleshoot complex pipeline/performance issues and mentor junior engineers
Collaborate closely with the Data Architect to translate architecture into engineering execution
Support client/stakeholder updates on engineering progress and technical blockers
Must-Have Skills
7–10 years in data engineering, including 2+ years leading a team/pod
Strong hands-on Databricks/Spark, Delta Lake, SQL, and Python/PySpark expertise
Strong AWS data services experience: Glue, S3, DMS; working knowledge of Kinesis/MSK or Kafka
Solid understanding of medallion architecture, CDC ingestion, and orchestration tools
Proven ability to manage delivery timelines and mentor a team
Good-to-Have: Exposure to vector databases, RAG pipeline data prep, or agentic/AI tooling (Bedrock, Mosaic AI).
Work arrangement
No
Recently tracked roles with full details and direct application links.