Data Engineer II

We are looking for a hands-on Data Engineer II with strong GCP experience to build scalable data pipelines, transformations, and reusable data-platform capabilities. The role involves taking solutions end-to-end from design and development to testing, deployment, and production support while building reliable and reusable data engineering solutions.
Responsibilities:
Design, develop, and maintain production-grade data pipelines on GCP.
Build scalable batch and incremental processing solutions.
Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
Build and maintain workflows using Cloud Composer / Apache Airflow.
Develop reusable transformation components, frameworks, and orchestration patterns.
Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
Design data models for analytical and downstream data products.
Implement data-quality checks, validation, reconciliation, and data lineage.
Optimize pipelines and BigQuery workloads for performance and cloud cost.
Implement automated testing and integrate data workloads with CI/CD.
Monitor production pipelines and troubleshoot issues through root-cause analysis.
Collaborate with data engineering, platform engineering, DevOps, architecture, and business teams.
Requirements:
4+ years of hands-on data engineering experience.
Strong hands-on experience with GCP and BigQuery.
Advanced SQL; complex joins, CTEs, window functions, and query optimization.
Strong Python development skills.
Experience with production ETL/ELT pipelines.
Hands-on experience with Cloud Composer / Apache Airflow.
Experience with Dataform and/or dbt.
Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
Strong understanding of data modelling and large-scale datasets.
Experience with incremental and idempotent processing.
Understanding of data contracts, schema evolution, data quality, and reconciliation.
Experience with Git, automated testing, and CI/CD.
Strong production troubleshooting and problem-solving skills.
Technical Skills:
Cloud: GCP, BigQuery, Google Cloud Storage.
Programming: Python, SQL.
Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark.
Orchestration: Cloud Composer, Apache Airflow.
Transformation: Dataform, dbt.
Engineering: Git, CI/CD, Automated Testing, Monitoring.
Data Engineering: ETL/ELT, Batch Processing, Data Modelling, Data Quality, Data Contracts, Metadata, and Lineage.
Good to Have:
Apache Iceberg or modern lakehouse technologies.
Streaming or event-driven processing.
Data Product / Data Mesh concepts.
Metadata-driven or configuration-driven processing.
Change Data Capture (CDC).
Terraform / Infrastructure as Code.
Experience building reusable components for multiple engineering teams.
A strong ownership mindset, good analytical and debugging skills, the ability to independently design solutions, and the confidence to take data engineering solutions through production.

Experience
4-8 yrs