AWS Senior Data Engineer

AWS Senior Data Engineer (Kafka & PySpark)
Experience: 10-15 Years
Location: Chennai, Bengaluru, Hyderabad
We are seeking an experienced AWS Data Engineer with strong expertise in Kafka, PySpark, Spark SQL, Python, and AWS to build and optimize large-scale batch and real-time data platforms.
Key Skills
PySpark, Spark SQL, Python, SQL
Apache Kafka (Kafka Connect, Schema Registry, Streaming)
AWS: S3, Glue, EMR, Lambda, IAM, Redshift, Step Functions, CloudWatch
ETL/ELT & Real-Time Data Pipelines
Spark Structured Streaming
Data Modeling, Data Lakes & Lakehouse Architecture
CI/CD, Git, Jenkins/GitHub Actions
Data Quality, Governance & Security
Responsibilities
Design and develop scalable batch and streaming data pipelines on AWS.
Build high-performance PySpark and Spark SQL applications.
Implement Kafka-based event-driven and real-time streaming solutions.
Integrate and optimize AWS data services for reliability and cost efficiency.
Implement data quality, governance, monitoring, and security best practices.
Build CI/CD pipelines and support production deployments.
Troubleshoot performance bottlenecks and production issues.
Good to Have
Amazon MSK, Kinesis
Databricks, Delta Lake, Unity Catalog
Airflow / MWAA
Snowflake, DBT
Terraform
Primary Skills: AWS Data Engineering, Kafka, PySpark, Spark SQL, Python, Streaming Data Engineering.

Work arrangement
No