Python Data Engineer

We are seeking a highly skilled Data Engineer to join our consulting team for a remote engagement with a client headquartered in Boston, MA. The ideal candidate will have extensive experience working with large-scale data pipelines, real-time streaming technologies, and cloud-based big data platforms. This role requires strong expertise in Apache Spark, Apache Kafka, and data ingestion frameworks, along with hands-on experience in Azure, SQL, and Snowflake.
Title: Python Data Engineer
Location: Krishnarajapuram, Bengaluru, KA - India
Type : Hybrid - 3 days on-site
Duration: 6 months Contract-to-Hire
Experience Required: 5–8 years
About the Role
Key Responsibilities
Design, build, and optimize big data ingestion pipelines to handle large volumes of structured and unstructured data.
Develop and maintain real-time streaming solutions using Kafka and Spark Streaming.
Collaborate with data architects and business stakeholders to define data integration strategies.
Implement scalable solutions on Azure cloud services and integrate with Snowflake for analytics and reporting.
Ensure data quality, reliability, and performance across distributed systems.
Troubleshoot and optimize existing pipelines to improve efficiency and reduce latency.
Document technical designs, workflows, and best practices for long-term maintainability.
Required Skills & Qualifications
5–8 years of professional experience as a Data Engineer or similar role.
Strong Python coding and automation experience, built internal Python libraries.
Strong expertise in Apache Spark (batch and streaming) and Apache Kafka.
Proven experience with large-scale data ingestion and processing.
Hands-on experience with Azure Data Services (e.g., Data Lake, Data Factory, Synapse).
Proficiency in SQL and experience with Snowflake for data warehousing.
Solid understanding of distributed systems and scalable architectures.
Strong problem-solving skills and ability to work independently in a remote environment.
Preferred Qualifications
Experience with performance tuning of big data pipelines.
Familiarity with CI/CD pipelines and DevOps practices for data engineering.
Exposure to data governance and security best practices.
We are an Equal Opportunity Employer

Work arrangement
Hybrid