Principal Data Engineer

We're looking for a Principal Data Engineer to design, build, and scale the data pipelines that bring data into our platform and make it ready for analytics. You'll work across the full pipeline lifecycle — from raw ingestion of heterogeneous sources through staging and curated layers — while playing a leading role in setting technical direction for our broader data infrastructure. You'll partner closely with fellow data engineers, data modelers, analysts, and product teams on what gets built next, and help raise the technical bar across the team.
What you'll get to do:
Move data through the full pipeline lifecycle — raw/src, staging, and curated layers — applying appropriate transformation, validation, and testing at each stage to prepare data for analytics consumption
Build and maintain Azure Data Factory (ADF) pipelines to orchestrate ingestion and transformation across source, curated, and serving layers
Design ingestion patterns for a range of raw source formats — REST/SOAP APIs, flat files (CSV, fixed-width), JSON, and XML — landing them reliably into the src layer before transformation
Build resilient API extraction logic handling pagination, rate limiting, incremental/delta pulls, and schema drift from third-party sources
Parse and flatten semi-structured JSON payloads (nested objects, arrays) into queryable relational structures within Snowflake
Handle file-based ingestion at scale — SFTP/blob drops, file validation, schema enforcement, and reprocessing/backfill logic for late or malformed files
Contribute to ADF pipeline CI/CD, deploying through Azure DevOps/GitHub Actions across dev/UAT/prod environments
Set technical standards for pipeline design, code quality, and testing, and mentor other data engineers on the team
Who you are:
Hands-on experience with Azure Data Factory (ADF) — pipelines, data flows, linked services, integration runtimes, and triggers
Experience extracting and normalizing raw data from heterogeneous sources: REST APIs, flat files, JSON/XML, and SFTP/blob-based drops
Comfortable writing extraction logic that handles pagination, authentication (API keys, OAuth), and incremental sync patterns
Experience parsing and modeling semi-structured/nested JSON data within a cloud data warehouse (e.g., Snowflake VARIANT/FLATTEN)
8+ years of experience in data engineering or a related field, including experience operating at a senior or technical lead level
Strong proficiency in SQL and at least one programming language commonly used in data engineering (Python, Scala, or Java)
Strong communication skills and experience working cross-functionally with analytics, product, and engineering teams
Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience preferred
Experience with dbt or similar transformation frameworks
Familiarity with infrastructure-as-code (Terraform, ARM/Bicep) and broader CI/CD practices
Exposure to real-time analytics or streaming technologies

Employment type
Full-time

Work arrangement
Yes