Senior Data Engineer (Databricks)

This is a Fully Remote Job
1. About Our Client:
The organization operates in the digital product design and development industry, partnering with a diverse range of clients from established corporations to startups. It addresses the challenge of creating impactful digital experiences by building data and AI foundations that support intelligent products and workflows. The team has contributed to projects reaching millions of users, demonstrating significant reach and influence across various sectors.
2. About the Opportunity:
The Senior Data Engineer (Databricks) role is focused on building and optimizing data and AI platforms that enable digital products and intelligent experiences. This position is responsible for designing and implementing data pipelines, lakehouse architectures, and AI foundations on the Databricks platform, ensuring these infrastructures translate into tangible business outcomes for clients. The role requires collaboration across technical and product teams, contributing to both client solutions and internal practice development.
3. Responsibilities:
Design and build production data pipelines using Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming with end-to-end ownership.
Architect and implement Lakehouse solutions on Databricks tailored to analytics, AI, and application requirements.
Build and maintain Databricks transformation layers including DLT pipelines, PySpark notebooks, and dbt with data quality and SLA enforcement.
Design and maintain data and AI foundations such as Unity Catalog, Feature Store, MLflow, and Model Serving.
Collaborate with product and backend engineers to design data models, APIs, and application data contracts.
Consult with clients to assess data challenges and deliver sustainable solutions.
Lead or support data architecture discussions depending on project needs.
Operate within multi-cloud environments, primarily AWS and Azure, aligning data platform recommendations with client goals.
Champion data governance practices through Unity Catalog, including access control, lineage, and compliance.
Design data-to-application architectures connecting governed data to AI workflows and user experiences.
Contribute to the growth of the Databricks practice through accelerators, enablement, certification, and partner materials.
4. Requirements:
7-10 years of data engineering experience with at least 5 years in production Databricks environments.
Working knowledge of AWS and Azure cloud services relevant to Databricks deployments; GCP familiarity is a plus.
Deep expertise with Databricks features including Lakeflow Declarative Pipelines, Autoloader, Structured Streaming, Lakeflow Jobs, Unity Catalog.
Experience designing Lakehouse architectures with medallion patterns, Delta Lake design, partitioning, and query optimization at scale.
Proficiency in data pipeline testing, observability, and CI/CD including unit testing and version-controlled deployments.
Strong SQL and Python skills with emphasis on clean, performant, maintainable code.
Knowledge of data modeling, schema design, and query optimization.
Excellent communication skills to articulate complex data concepts to diverse audiences.
Strong problem-solving abilities.
5. Pay Range and Compensation Package:
The pay range and compensation package for this role will be determined based on the candidate’s experience, skills, and other relevant factors.
Equal Opportunity Statement: Our client is an equal opportunity employer. They celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, or national origin.
Note:
TalentHop is a recruitment partner of this role. Please note that all employment decisions, including candidate assessment, interviews, hiring, compensation, and employment terms, are made exclusively by the hiring employer.

Work arrangement
Yes