Data Engineer

We are looking for a Senior Data Engineer (experience range 1 - 5 years) to design, build, and optimize scalable data pipelines and models that power analytics, reporting, and business insights. You will work across engineering, product, and data teams to deliver reliable, high-quality datasets and modern cloud-based data platforms.
Responsibilities:
Build scalable batch and streaming pipelines using Python, PySpark, and SQL.
Develop and maintain ETL/ELT workflows and data ingestion frameworks.
Design dimensional data models (fact/dimension, star schema, 3NF).
Write and optimise complex SQL transformations for large datasets.
Ensure data quality, validation, lineage, and governance best practices.
Tune Spark jobs and SQL queries for performance and cost.
Work with AWS services (S3 Glue, EMR, Lambda, Step Functions).
Implement CI/CD, testing, and production deployment of data workflows.
Monitor, log, and troubleshoot data pipelines in production.
Participate in architecture discussions and drive technical decisions.
Collaborate with analytics, product, and data science teams.
Mentor junior engineers and promote best practices.
Requirements:
1+ year of experience in data engineering or related roles.
Strong Python development experience.
Advanced SQL (analytical queries, performance tuning).
Expertise in PySpark (DataFrame API, RDD concepts, optimisation).
Solid understanding of data modelling for OLTP/OLAP.
Proven ETL/ELT pipeline design experience.
Cloud experience (AWS preferred; similar GCP/Azure OK).
Knowledge of data warehousing concepts and distributed systems.
Git and CI/CD experience.
Strong debugging, optimisation, and problem-solving skills.
Ability to work with large-scale datasets.
Preferred Skills:
Airflow/Dagster/Prefect.
Kafka/Kinesis or streaming systems.
Docker/Kubernetes.
IaC: CDK, Terraform, CloudFormation.
Delta Lake, Iceberg, or Hudi.
ML feature pipelines.
Monitoring tools: CloudWatch, Datadog, Prometheus, ELK.
System design and architectural experience.

Experience
5-9 yrs