AWS Pyspark Data Engineer

Overall Experience - 5+
Location - Indore
Job Summary
We are seeking an experienced AWS PySpark Developer with strong expertise in building scalable data engineering solutions on AWS Cloud. The ideal candidate should have hands-on experience in PySpark, Spark, AWS data services, ETL development, and large-scale data processing. The role involves designing, developing, and optimizing cloud-native data pipelines, data lakes, and analytics solutions.
Key Responsibilities
Design and develop scalable ETL/ELT pipelines using PySpark.
Build and maintain data ingestion frameworks on AWS.
Develop batch and real-time data processing solutions.
Optimize Spark jobs for performance and cost efficiency.
Work with structured, semi-structured, and unstructured datasets.
Develop data lake and data warehouse solutions.
Implement data quality checks, monitoring, and alerting.
Collaborate with business stakeholders, architects, and analytics teams.
Support production deployments and troubleshoot critical issues.
Follow DevOps and CI/CD best practices.
Mandatory Technical Skills
AWS Services
Amazon S3
AWS Glue
AWS EMR
AWS Lambda
Amazon Redshift
AWS IAM
CloudWatch
AWS Step Functions
Big Data & Data Engineering
PySpark
Apache Spark
Spark SQL
DataFrames & RDDs
ETL/ELT Development
Data Modeling
Data Warehousing Concepts
Programming
Python
SQL
Shell Scripting
Database Technologies
Redshift
PostgreSQL
MySQL
DynamoDB (Preferred)
DevOps & CI/CD
Git
Jenkins / AWS CodePipeline
Docker
Terraform (Preferred)

Work arrangement
No