Data Engineer

We are looking for an experienced Data Engineer with 6-8 years of hands-on experience in designing, building, and managing scalable data pipelines, data platforms, and analytics-ready data assets. The ideal candidate should have strong expertise in data engineering, cloud platforms, ETL/ELT pipelines, data lakehouse architecture, data quality, and production-grade data solutions.
Responsibilities:
Design, develop, and maintain scalable data pipelines for structured, semi-structured, and unstructured data.
Build and manage data ingestion frameworks from ERP, CRM, SCADA, IoT, Historian, APIs, databases, and external data sources.
Develop ETL/ELT workflows for data transformation, cleansing, validation, and enrichment.
Implement data lake, data warehouse, and lakehouse architectures to support analytics, AI/ML, and reporting use cases.
Ensure data quality, reliability, lineage, metadata management, and governance across data platforms.
Collaborate with Data Scientists, ML Engineers, Product Managers, Business Teams, and BI Developers to deliver data solutions.
Optimise data pipelines for performance, scalability, cost, and reliability.
Build reusable data assets, data marts, and curated datasets for business intelligence and AI use cases.
Support deployment, monitoring, troubleshooting, and improvement of production data pipelines.
Requirements:
Strong hands-on experience in SQL, Python, PySpark, and data pipeline development.
Experience with cloud data platforms such as Azure Data Factory, Azure Synapse, Databricks, ADLS, AWS Glue, Redshift, S3 BigQuery, Cloud Functions, Data Flow, or Snowflake.
Good understanding of data lake, data warehouse, lakehouse, data modelling, and medallion architecture.
Experience in batch and real-time/streaming data processing.
Knowledge of Apache Spark, Kafka, Airflow, dbt, Delta Lake, or similar technologies.
Experience with data quality, validation, metadata, lineage, and governance frameworks.
Exposure to CI/CD, Git, DevOps practices, and production deployment of data pipelines.
Understanding of APIs, file formats, databases, and enterprise integration patterns.

Experience
6-8 yrs