Data Engineer

Senior Data Engineer – GCP
Experience: 4+ Years
Employment Type: Full-Time
Work Location: Bengaluru, Karnataka, India
Job Level: Senior Data Engineer
Primary Technology: GCP, BigQuery, Dataflow, GCS, Python, SQL
Hiring Details: Bengaluru near Lalbagh | Onsite / Work from Office
Ideal Candidate Profile
4–8 years of experience | GCP | BigQuery | Dataflow | Python | SQL | GCS | Pub/Sub | Airflow |
Data Pipelines | CI/CD
About the Role
We are looking for a Senior Data Engineer with 4+ years of experience to join our data engineering team and build scalable, reliable, and high-performance data platforms using
Google Cloud Platform (GCP). The ideal candidate will have strong hands on experience with BigQuery, Google Cloud Storage, Dataflow, Python, SQL, Apache Spark, and modern cloud data
engineering practises. This is a hands on engineering role where you will be responsible for designing, developing, deploying, and supporting production grade data pipelines and cloud data
solutions.
Data Engineering
Design, develop, and maintain scalable batch and streaming data pipelines on GCP. Build robust ETL/ELT pipelines using Python, SQL, Apache Beam, and/or Spark.
Develop data ingestion frameworks from databases, APIs, files, event streams, and enterprise applications. Design and implement data transformation and processing pipelines.
Implement incremental processing, CDC, partitioning, and efficient data loading strategies. Ensure data pipelines meet requirements for reliability, scalability, performance, and data quality
GCP Data Engineering
Design and develop solutions using BigQuery, GCS, Dataflow, Pub/Sub, Cloud Composer, and Dataproc. Develop optimized BigQuery datasets, tables, views, and SQL transformations.
Implement BigQuery partitioning, clustering, materialized views, and query optimization. Develop batch and streaming pipelines using Dataflow / Apache Beam.
Build data ingestion and processing solutions using Pub/Sub. Use GCS for scalable and secure data storage. Orchestrate data pipelines using Cloud Composer / Apache Airflow.
Implement appropriate IAM, security, and access-control mechanisms.
Cloud Architecture & Integration
Integrate GCP services with enterprise databases, applications, APIs, and third-party systems. Work with relational and NoSQL data sources. Implement cloud-native data architectures following scalability, reliability, security, and cost-optimization principles. Participate in cloud migration and modernization initiatives.
Support migration of traditional data platforms to GCP.
Performance & Cost Optimization
Optimize BigQuery queries and data models for performance and cost. Analyze pipeline performance and identify bottlenecks.
Optimize Dataflow and Spark workloads. Implement appropriate partitioning, clustering, storage, and compute strategies. Identify opportunities for GCP cloud cost optimization\
Production & DevOps
Build production-ready data pipelines with monitoring, logging, alerting, and error handling. Implement CI/CD practices for data engineering workloads.
Use Git and DevOps tools for source control and deployment. Implement Infrastructure as Code using tools such as Terraform. Participate in production support, incident resolution, and root-cause analysis.
. Required Technical Skills
4+ years of professional experience in Data Engineering.
Strong hands-on experience with Google Cloud Platform.
Strong experience with BigQuery.
Strong Python and SQL programming skills.
Experience with Google Cloud Storage (GCS).
Experience with Dataflow / Apache Beam or equivalent distributed processing technologies.
Experience designing and developing enterprise-grade data pipelines.
Strong understanding of ETL/ELT, data modeling, batch processing, streaming processing,
incremental processing, CDC, data partitioning, data quality, and performance optimization.
Experience with Git and CI/CD.
Strong troubleshooting and analytical skills.

Work arrangement
No