Data Engineer

Company: Continental
Website: Visit Website
Business Type: Enterprise
Company Type: Product & Service
Business Model: B2B
Funding Stage: Series C
Industry: Retail
Salary Range: ₹ 15-30 Lacs PA
Job Description
This is a Permanent role with one of the valued client of VidPro.
Key Skills
Govern data acquisition and ingestion architecture for enterprise-wide data & analytics integration.
Establish reusable, scalable, and governed interfaces to all source systems across business domains.
Define standards and ensure engineering quality across ingestion pipelines and upstream data flows.
Act as escalation and design authority for ingestion, unstructured data, and advanced analytics integration points.
Oversees ingestion and integration design across 100+ data sources.
Supports 25+ data engineering and data science professionals with ingestion architecture and troubleshooting.
Interfaces with source system experts, data scientists, platform architects, and functional IT stakeholders.
Main Tasks
Define ingestion frameworks for batch, streaming, and APIs.
Establish standards for JSON, CSV, XML, and unstructured data.
Govern manual uploads and exception workflows.
Collaborate with system owners to define interface specifications.
Manage schema evolution and API reliability.
Align ingestion architecture with platform and business requirements.
Collaborate on ingestion for data science and ML pipelines
Ensure robust API and event-driven integration for AI/LLM use cases.
Support complex acquisition scenarios with scalability requirements.
Review high-risk ingestion pipelines for performance and security.
Support data engineers on error handling, retries, and load orchestration.
Ensure ingestion aligns with governance and architecture.
Maintain ingestion libraries and templates.
Promote reuse and automation across engineering teams.
Lead best practices in TDD, code reviews, and pipeline testing.
Maintain architectural blueprints, templates, and best practices.
Publish design guidelines and coding standards.
Create re-usable architecture patterns for lakehouse environments.
Monitor usage and implement auto-scaling policies.
Analyze and optimize cluster configurations for cost-efficiency.
Provide cost transparency and usage reporting to stakeholders.
Required Skills
Degree in Computer Science or related field; certifications in Databricks or Microsoft Azure preferred.
6–10 years in data engineering with strong focus on enterprise-scale data & analytics integration, including ML and LLM scenarios.
Experience building robust ingestion frameworks and reusable components.
Track record of architectural ownership and peer enablement with diverse teams.
Experience working in international teams across multiple time zones and cultures, preferably with teams in India, Germany, and the Philippines.

Work arrangement
No