Senior Data Engineer - CDP

Experience : 10+ years
Work Location : Chennai [ Work from Office]
Interview : In person
ONLY Relevant candidates need to apply [ 10+ years] in Senior Data Engineer - CDP
Job Summary
We are seeking a highly skilled Senior Data Engineer with 10+ years of experience in designing, building, and optimizing enterprise-scale data platforms, databases, and data pipelines. The ideal candidate should possess deep expertise in PostgreSQL, MySQL, ETL/ELT development, data modeling, Databricks, Snowflakes, DBT, performance optimization, cloud data platforms, and distributed data processing.
The role involves architecting scalable data solutions that support transactional applications, analytics, AI/ML initiatives, and enterprise reporting while ensuring high availability, security, and data quality.
The candidate will collaborate with software engineers, solution architects, data scientists, DevOps teams, and business stakeholders to build reliable, secure, and high-performance data ecosystems.
Key Responsibilities
Data Engineering & Platform Development
Design, develop, and maintain enterprise-scale data platforms and data pipelines.
Build scalable ETL/ELT workflows for ingesting, transforming, and integrating data from multiple sources.
Design and optimize relational databases using PostgreSQL and MySQL.
Develop reusable data integration frameworks and automation processes.
Build data ingestion pipelines for structured, semi-structured, and unstructured datasets.
Implement data validation, cleansing, enrichment, and transformation processes.
Design data lakes, operational data stores (ODS), and analytical data marts.
Database Engineering
Design normalized and denormalized database schemas.
Create and maintain tables, indexes, partitions, materialized views, stored procedures, functions, and triggers.
Perform query optimization and SQL tuning for high-volume transactional workloads.
Optimize indexing strategies and execution plans.
Design partitioning strategies for large datasets.
Implement replication, clustering, and high-availability architectures.
Plan and execute database migrations and version upgrades.
Monitor database performance and proactively resolve bottlenecks.
Perform capacity planning and database health monitoring.
Data Modeling
Design conceptual, logical, and physical data models.
Develop enterprise data models aligned with business domains.
Define master data, reference data, and metadata standards.
Ensure data consistency and integrity across systems.
Implement Slowly Changing Dimensions (SCD), star schemas, and snowflake schemas where applicable.
Data Pipeline & Integration
Develop high-performance data pipelines using ETL/ELT frameworks.
Integrate data from APIs, applications, databases, streaming platforms, and cloud services.
Build batch and real-time data processing solutions.
Design event-driven data ingestion pipelines.
Automate workflow scheduling and orchestration.
Performance Optimization
Analyze and optimize complex SQL queries.
Identify performance bottlenecks using execution plans.
Optimize storage utilization and indexing strategies.
Improve data loading performance.
Optimize large-scale reporting queries.
Security & Governance
Implement database security best practices.
Configure role-based access control (RBAC).
Implement Row-Level Security (RLS) where applicable.
Manage encryption at rest and in transit.
Ensure compliance with security and governance policies.
Design backup, recovery, and disaster recovery strategies.
Cloud & DevOps
Design cloud-native database architectures.
Automate database deployments using CI/CD pipelines.
Implement Infrastructure as Code (IaC) where applicable.
Monitor cloud database resources and optimize costs.
Support containerized database deployments.
Collaboration
Collaborate with Software Engineers, Architects, Product Teams, and Data Scientists.
Participate in solution architecture and technical design discussions.
Review database designs and coding standards.
Mentor junior engineers and conduct knowledge-sharing sessions.
Drive database best practices across engineering teams.
Required Technical Skills
Databases
PostgreSQL
MySQL
SQL Performance Tuning
Query Optimization
Partitioning
Replication
High Availability
Database Security
Backup & Recovery
Database Migration
Data Engineering
ETL / ELT Development
Data Pipelines - Databricks
Data Warehousing - Snowflake
Data Lake Architecture
Data Integration
Data Quality
Data Transformation
Batch Processing
Real-time Data Processing
Programming
SQL
Python
Shell Scripting
PL/pgSQL
Stored Procedures
Functions
Cloud Platforms
Experience with one or more:
AWS
Azure
Google Cloud Platform (GCP)
Cloud database services:
Amazon RDS
Aurora PostgreSQL
Cloud SQL
Azure Database for PostgreSQL
Data Technologies
Experience with:
Apache Kafka
Apache Airflow
Spark (preferred)
dbt (preferred)
DevOps & Automation
Git
CI/CD
Docker
Kubernetes (preferred)
Monitoring Tools
Grafana
Prometheus
pgAdmin
CloudWatch
Performance Monitoring Tools
Preferred Qualifications
Experience designing enterprise-scale data platforms.
Experience supporting AI/ML data pipelines.
Knowledge of Data Governance and Metadata Management.
Experience with multi-tenant SaaS platforms.
Experience implementing PostgreSQL Row-Level Security (RLS).
Experience with event-driven architectures and microservices.
Familiarity with data privacy regulations (GDPR, HIPAA, SOC2).
Soft Skills
Strong analytical and problem-solving skills.
Excellent communication and stakeholder management.
Ability to mentor and lead technical teams.
Strong ownership and accountability.
Ability to work in high demanding environments.
Education
Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.
Nice to Have
Experience with AI/ML data pipelines.
Knowledge of vector databases and semantic search.
Experience with PostgreSQL JSONB and full-text search.
Experience with Change Data Capture (CDC) tools such as Debezium.
Knowledge of Delta Lake

Work arrangement
No