Senior / Lead Data Engineer - Databricks, AI Engineering & Retail & Omnichannel Analytics

Looking for: Senior / Lead Data Engineer - Databricks, AI Engineering & Retail & Omnichannel Analytics
Job Type: Contract
Location: Remote (Canada)
Description:
About the Role
We are seeking a Senior / Lead Data Engineer to help build the next generation of our Retail & Omnichannel Analytics platform.
This role combines hands-on engineering, technical leadership, and architecture. You will help define scalable data ingestion patterns, establish Databricks best practices, develop reusable frameworks, enable self-service analytics, and drive AI-enabled engineering practices across the organization.
You will work across the full analytics lifecycle, from data ingestion and transformation through semantic models, reporting, governance, and AI-powered data products. The ideal candidate has successfully implemented enterprise-scale data and analytics platforms using Databricks and can bring proven patterns and best practices from previous initiatives.
Key Responsibilities
• Design and implement scalable data ingestion, transformation, and data product frameworks.
• Define and drive adoption of Databricks best practices, including Medallion Architecture (Bronze/Silver/Gold), performance optimization, governance, and operational excellence.
• Build batch, near real-time, and streaming data pipelines using Databricks and Azure technologies.
• Design and deliver end-to-end data products supporting Retail & Omnichannel Analytics use cases.
• Develop trusted analytical datasets, semantic models, and governed consumption layers for reporting and self-service analytics.
• Enable data democratization through capabilities such as Databricks Genie and reusable business-ready data products.
• Lead proof-of-concepts and evaluate emerging platform capabilities across the Databricks ecosystem.
• Define AI SDLC and AI-assisted development patterns, including the use of AI agents and engineering accelerators.
• Implement data quality, lineage, monitoring, and observability practices.
• Partner with engineering, analytics, product, and business teams to deliver scalable solutions and mentor engineers on best practices.
Required Qualifications
• 7+ years of Data Engineering experience.
• Strong hands-on experience with Databricks in enterprise production environments.
• Advanced expertise in:
Python
SQL
Scala
Spark / PySpark
Delta Lake
• Experience building Lakehouse architectures and implementing Medallion Architecture (Bronze/Silver/Gold).
• Strong experience with ETL/ELT design, data integration, and scalable data pipelines.
• Experience creating analytical and dimensional data models.
• Experience delivering end-to-end analytics solutions from ingestion through semantic modeling and reporting.
• Experience implementing CI/CD and Data Engineering SDLC best practices.
• Experience with Azure DevOps, Git, and release management processes.
• Strong understanding of data governance, security, data quality, and performance optimization.
• Strong communication skills and ability to lead technical discussions.
Preferred Qualifications
• Experience with Unity Catalog, LakeFlow, Delta Live Tables, Databricks SQL, and Databricks Genie.
• Experience establishing engineering standards, reusable frameworks, or platform best practices used by multiple teams.
• Experience implementing AI-assisted development workflows and AI engineering practices.
• Experience designing AI agents or automation solutions that improve engineering productivity.
• Experience leading platform modernization initiatives and technical proof-of-concepts.
• Experience in Retail & Omnichannel Analytics, Merchandising, Inventory, Supply Chain, Store Operations, Customer Analytics, or Digital Commerce.
Tech Stack:
Languages
• Python
• SQL
• Scala
Data Engineering
• Spark
• PySpark
• Delta Lake
Data Platform
• Databricks
• Unity Catalog
• LakeFlow
• Delta Live Tables (DLT)
• Databricks SQL
• Databricks Genie
• Databricks Workflows
Data Integration & Orchestration
• Azure Data Factory (ADF)
• Databricks Workflows
• REST APIs
• Batch & Streaming Pipelines
DevOps
• Azure DevOps
• Git
• CI/CD Pipelines
Architecture & Governance
• Lakehouse Architecture
• Medallion Architecture
• Data Products
• Semantic Models
• Data Quality
• Data Lineage
• Observability & Monitoring
AI Engineering
• AI-Assisted Development Tools
• AI Agents
• GenAI-enabled Engineering Workflows

Work arrangement
No