Senior Data Engineer - Databricks (3rd Party Contractor)

to
Position: Senior Data Engineer (Databricks & Cloud Data Platforms)
Reporting to: Practice Head: Data and Automation
ROLE OVERVIEW
As a Senior Data Engineer, you will play a critical consulting role in designing, building, and modernising enterprise data platforms, with a strong focus on Databricks lakehouse implementations across cloud environments.
You will work closely with solution architects, analysts, data scientists, and client stakeholders to deliver secure, scalable, and high-performing analytics platforms that drive real business impact.
This is a hands-on technical role suited to someone passionate about solving complex data challenges and delivering best-in-class cloud-native solutions.
responsibilities
RESPONSIBILITIES
Design and implement scalable Databricks-based data platforms
Build robust ETL / ELT pipelines for batch and streaming workloads
Lead data migration and modernisation initiatives from legacy platforms to cloud-based lakehouse architectures
Develop high-performance data processing solutions using PySpark, Python, and SQL
Implement Delta Lake, Unity Catalog, and modern governance frameworks including Purview
Design enterprise-grade lakehouse and medallion architectures
Optimise Spark workloads for performance, scalability, and cost efficiency
Build orchestration solutions using Databricks Workflows, ADF, Airflow, or similar tools in Azure/AWS
Collaborate with technical and business stakeholders to define solution requirements
Mentor junior engineers and contribute to technical best practices within the Data & AI practice
Produce technical documentation, architecture artefacts, and reusable delivery assets
IDEALLY YOU ARE:
A strong technical leader who can own complex delivery outcomes
Comfortable engaging directly with clients and stakeholders
A proactive problem-solver with excellent analytical skills
Able to clearly communicate technical concepts to both technical and non-technical audiences
Passionate about mentoring and uplifting engineering teams
Delivery-focused, quality-driven, and adaptable in consulting environments
MINIMUM EXPERIENCE
8+ years in Data Engineering projects
3+ years of hands-on Databricks delivery experience
Deep expertise in Python and PySpark
Strong understanding of the Medallion Architecture and Lakehouse Architecture
Hands-on experience building and maintaining live Databricks AI/BI Governance Dashboards using SQL
Strong experience with Azure data services including Synapse, ADF, OneLake, Purview
Practical exposure to AWS data platforms including Glue, S3, Redshift
Proven track record delivering data migration and modernisation projects
Experience working in consulting or client-facing environments
Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent practical experience
TECHNICAL REQUIREMENTS
Databricks Expertise (Essential)
Cloud Data Platform Experience (Azure Fabric, AWS) (Essential)

Employment type
3rd Party Contractor

Work arrangement
No