Live opening · Posted 2 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
We are looking for a Data Engineer II with strong hands-on experience in GCP, BigQuery, Python, SQL, and modern data engineering technologies to build and maintain scalable, reliable, and production-grade data platforms. The ideal candidate should have strong expertise in data pipeline development, ETL/ELT, data modelling, pipeline optimization, and production support, with the ability to work effectively with large-scale datasets on GCP.
Responsibilities:
Design, develop, and maintain scalable data pipelines on GCP.
Work with large-scale datasets using BigQuery and other GCP data services.
Develop robust ETL/ELT pipelines for batch and incremental data processing.
Implement idempotent and reliable data processing workflows.
Build and maintain workflows using Airflow / Cloud Composer.
Develop data transformation pipelines using Dataform/dbt.
Work with Dataflow / Apache Beam for distributed data processing.
Work with Dataproc, Spark, and PySpark for large-scale data processing.
Optimize data pipelines for performance, scalability, and cost efficiency.
Implement data quality checks, testing, and monitoring to ensure reliable data products.
Troubleshoot production issues and provide effective production support.
Collaborate with engineering and business teams to deliver high-quality data solutions.
Requirements:
Strong hands-on experience with GCP and BigQuery.
Advanced SQL and Python, Airflow / Cloud Composer, Dataform / dbt, Dataflow / Apache Beam, Dataproc / Spark / PySpark.
Strong understanding of ETL/ELT and data pipeline development.
Experience with batch, incremental, and idempotent processing.
Good understanding of data modelling and data quality.
Experience in pipeline optimization and troubleshooting.
Hands-on experience with Git, CI/CD, and testing.
Strong production support and problem-solving skills.
4-7.5 years of relevant data engineering experience.
Strong experience building production-grade data pipelines.
Excellent SQL and Python skills.
Strong hands-on experience with the GCP data ecosystem, particularly BigQuery.
Ability to work with large datasets and optimize data processing workflows.
Strong analytical and troubleshooting skills.
Good understanding of software engineering best practices, including version control, CI/CD, and testing.
Strong communication and collaboration skills.
Experience
4-8 yrs
Recently tracked roles with full details and direct application links.