Senior Data Engineer

About the Company
We're looking for a Senior Data Engineer to design, build, and operate modern data platforms at scale. You'll work across Azure, Microsoft Fabric, and AWS — building reliable pipelines, performant data models, and self-service analytics that drive real business decisions. A core part of this role is containerized workload execution: designing and deploying pipeline jobs using Azure Container Apps Jobs for scheduled and event-driven processing. This is a hands-on role for an engineer who thrives in ambiguity, values clean architecture, and moves comfortably between strategy and implementation.
About the Role
Data Platform & Architecture
Design and build scalable data pipelines using Python and cloud-native orchestration tools (Azure Data Factory, Azure Container Apps Jobs, Fabric Data Pipelines)
Architect solutions across Microsoft Fabric Warehouses, Azure SQL, and AWS (S3, Redshift), selecting the right tool for the workload
Implement Medallion/layered architecture (Bronze → Silver → Gold) for structured, governed data delivery
Manage and optimize large-scale data warehouse environments for performance, cost, and maintainability
Responsibilities
Develop Python-based ETL/ELT pipelines ingesting data from APIs, flat files, databases, and SaaS platforms
Build and deploy containerized jobs using Azure Container Apps Jobs, including scheduling, scaling rules, secrets management (Key Vault), and Container Registry integration
Maintain data movement between on-premises SQL Server and cloud targets
Design idempotent, fault-tolerant pipeline patterns with robust logging, alerting, and retry logic
Partner with analytics teams to deliver clean, well-documented data models for Power BI and similar BI tools
Qualifications
5–8 years of hands-on data engineering in production environments
Proven track record delivering data platforms on Azure and/or AWS
Experience migrating legacy on-premises infrastructure to cloud-native solutions
Hands-on experience with Azure Container Apps Jobs or a comparable containerized job execution platform
Required Skills
Python — production-grade pipelines, transformations, and automation
SQL — advanced T-SQL and ANSI SQL; SQL Server, Azure SQL, Redshift, and Fabric query engines
Microsoft Fabric — Warehouses, Lakehouses, Data Pipelines, OneLake
Azure — ADF, Azure SQL, ADLS, Key Vault, Container Apps Jobs, Container Registry
Containerization — Docker image development, container registry management, Container Apps Jobs (scheduling, scaling, secret injection)
AWS — S3 for data lake storage, Redshift for cloud warehousing
Data Modeling — dimensional modeling, star/snowflake schema, ER modeling for OLTP and OLAP
DevOps — Git, GitHub, PR workflows, CI/CD pipelines
Visualization — Power BI, Tableau
Preferred Skills
Microsoft Fabric in production (Fabric Warehouses, OneLake integration)
dbt or similar transformation frameworks
Power BI semantic model development, dataset optimization, DirectQuery/Import tradeoffs
Agile/Scrum experience
Cloud certifications: DP-203 (Azure Data Engineer), AWS Certified Data Analytics, or equivalent
Bachelor's degree in Computer Science, Information Systems, Data Science, or a related field — or equivalent professional experience

Work arrangement
Yes