Sr. Data Engineer

The Data Engineer (Level III) designs, builds, and operates the data pipelines and lakehouse data products that power P& SC intelligence across device supply chain, reverse logistics, procurement, network supply chain, and real-time control tower capabilities. Operating within a squad focused on a specific SC domain or cross-cutting platform function, the Data Engineer is a hands-on technical contributor who builds to production quality, owns pipeline reliability and KTLO, and contributes to the engineering standard of the team.
Responsibilities:
Design and build production-grade data pipelines spanning source ingestion, bronze landing, silver transformation, and gold-layer data product delivery within the squad's domain scope.
Own pipeline KTLO (Keep the Lights On) for assigned data products, including monitoring, alerting, incident response, and ongoing reliability improvements.
Implement data ingestion patterns for assigned source systems, including batch file ingestion, API-based ingestion, and event-driven streaming (Kafka, Azure Event Hub) depending on squad scope.
Apply medallion architecture (bronze, silver, and gold) and Fabric IQ certification standards consistently across all data product builds.
Collaborate with system analysts to implement field-level transformations, business rule logic, and data quality checks as specified in product requirements documentation.
Participate in and contribute to pipeline design reviews, ensuring solutions align with the organization's Databricks and Fabric engineering standards.
Support the migration and deprecation of legacy platforms, including SCOpsBI SQL Server and SAP boundary systems, following the organization's extract, validate, rebuild, cutover, and decommission pattern.
Write and maintain comprehensive pipeline documentation, including data lineage, transformation logic, SLA definitions, and dependency maps.
Contribute to the organization's DevOps and engineering reliability practices, including CI/CD pipeline setup, testing frameworks, and incident runbooks.
Support and collaborate with junior data engineers within the squad, sharing knowledge and providing guidance on day-to-day engineering tasks.
Requirements:
5-8 years of data engineering experience with a track record of delivering production-grade data pipelines in enterprise environments.
Strong experience with Python, PySpark, Spark SQL, and T-SQL for pipeline development and transformation logic.
Strong proficiency in Azure Data Factory for building and maintaining ELT/ETL pipelines, including parameterization, triggers, linked services, and error handling
Hands-on experience with Azure Databricks, including Delta Lake, Unity Catalog, and workflow orchestration.
Hands-on experience with dbt (Data Build Tool), writing models, tests, sources, and documentation within an established project structure.
Solid SQL skills for data validation, transformation logic, and ad hoc source system analysis.
Expert-level data modelling skills across 3NF, Dimensional (Kimball), and Data Vault 2.0 patterns.
Strong understanding of data quality frameworks, including implementing checks, alerting on anomalies, and maintaining SLA-compliant pipeline health.
Experience with DevOps practices for data pipelines, including version control (Git), CI/CD, and automated testing.
Agile or Scrum practitioner certification or equivalent.
Ability to operate with moderate independence on complex technical problems, escalating appropriately and maintaining clear technical documentation.
Nice to Have:
Experience with real-time streaming technologies, including Kafka, Azure Event Hub, Delta Live Tables, or Spark Structured Streaming.
Working experience with Snowflake table design, query performance tuning, views, stored procedures, and data loading patterns (Snowpipe, COPY INTO).
Experience with enterprise batch scheduling and orchestration tools such as Control-M or Autosys is a plus.
Experience with real-time streaming technologies, including Kafka, Azure Event Hub, Delta Live Tables, or Spark Structured Streaming.
Experience with Microsoft Fabric or Azure Synapse Analytics; familiarity with Fabric IQ and OneLake is a plus.
Background in supply chain, reverse logistics, procurement, or network infrastructure data domains.
Experience working within Agile delivery frameworks, including PI planning, sprint-level technical governance, and cross-squad dependency management.
Familiarity with AI-assisted development tools such as GitHub Copilot or Claude, and experience integrating AI/ML-ready data preparation into ETL/ELT pipeline design.
Working knowledge of SOX and USGCI compliance requirements as they apply to enterprise data platform design and deployment.
Microsoft Azure certifications: DP-203 AZ-305 DP-900 or the Microsoft Fabric Analytics Engineer Associate.
Python development skills beyond PySpark, including utility scripting and framework contributions.

Experience
5-9 yrs