Data Engineer

Data Engineer (4–5 Years Experience)
Location: Chennai
Experience: 4–5 Years
Employment Type: Full-time
Job Summary
We are seeking a Data Engineer with 4–5 years of experience to build and maintain scalable data pipelines and data integration solutions. The ideal candidate should have strong expertise in PySpark, Databricks, Azure Data Factory (ADF), Python, and SQL, with hands-on experience in cloud-based data platforms.
Key Responsibilities
* Design and develop scalable ETL/ELT pipelines using PySpark and Databricks
* Build and manage data workflows using Azure Data Factory (ADF)
* Integrate data from multiple sources such as APIs, databases, flat files, and cloud storage
* Optimize and maintain large-scale data processing systems
* Ensure data quality, validation, and governance across pipelines
* Collaborate with cross-functional teams including data scientists and analysts
* Monitor, troubleshoot, and improve pipeline performance
* Implement reusable frameworks and best practices for data engineering
* Support deployment using CI/CD pipelines
Required Skills
* Strong experience with Azure Data Factory (ADF) for pipeline orchestration
* Hands-on experience with Databricks and PySpark
* Proficiency in Python
* Advanced SQL skills including complex queries and performance tuning
* Experience working on Azure Cloud (ADLS, Azure SQL, etc.)
* Good understanding of data warehousing and data modeling concepts
Preferred Qualifications
* Experience with Azure ecosystem including ADF, ADLS, Synapse, and Key Vault
* Knowledge of workflow orchestration and scheduling tools
* Familiarity with Delta Lake architecture
* Experience handling large datasets and performance tuning
Nice to Have
* Experience with streaming tools such as Kafka or Event Hubs
* Knowledge of DevOps practices and CI/CD using Azure DevOps
* Exposure to data governance and security practices
Soft Skills
* Strong analytical and problem-solving skills
* Good communication and teamwork abilities
* Ability to work in a fast-paced environment
Interested candidates can also apply at info@kratosolutions.com

Work arrangement
No