Senior Data Engineer - Databricks

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Engineer - Databricks based in Brazil.
We are looking for an experienced Senior Data Engineer to design, develop, and maintain scalable data pipelines using Databricks and modern cloud technologies. In this role, you will integrate and transform data from diverse sources, ensuring it is reliable, secure, well-governed, and ready to support business decisions. You will work extensively with Python, SQL, PySpark, and Delta Lake while applying the Medallion architecture to build robust data solutions. Collaborating with business stakeholders, analysts, data scientists, and platform engineers, you will translate complex requirements into high-quality data products. You will also promote modern DataOps practices and help improve engineering standards across the team. This opportunity combines hands-on technical work, cross-functional collaboration, and the chance to make a meaningful impact through trusted, data-driven solutions.
Accountabilities
Design, develop, maintain, and optimize scalable data pipelines using Databricks, PySpark, Delta Lake, and SQL.
Integrate data from diverse sources, including APIs, relational databases, files, and other enterprise systems.
Build and maintain robust data ingestion and transformation processes aligned with the Medallion architecture and modern data engineering practices.
Collaborate with business and analytics teams to understand data requirements and deliver solutions that support strategic objectives.
Ensure the quality, reliability, security, and governance of ingested and processed data throughout the data lifecycle.
Apply modern DataOps practices, including code versioning, automated data testing, CI/CD pipelines, and infrastructure as code.
Troubleshoot and resolve performance, scalability, and reliability issues across data pipelines and distributed processing environments.
Document data engineering processes, technical decisions, standards, and best practices to support maintainability and knowledge sharing.
Provide technical guidance and mentorship to colleagues while collaborating with data scientists, analysts, and platform engineers.
Contribute to continuous improvement in data architecture, pipeline efficiency, and engineering quality.
Requirements
Proven experience designing, developing, and managing large-scale data pipelines using Databricks, PySpark, Delta Lake, and SQL.
Strong programming skills in Python and SQL for data processing, transformation, and pipeline development.
In-depth understanding of ETL/ELT frameworks, data warehousing concepts, and distributed data processing.
Hands-on experience with modern DataOps practices, including Git version control, CI/CD pipelines, automated testing, and infrastructure as code.
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform and their associated data services.
Strong problem-solving and analytical skills, with the ability to troubleshoot and optimize data solutions for performance, scalability, and reliability.
Proficiency with Git and collaborative software development workflows.
Advanced English proficiency for effective communication, technical collaboration, and documentation.
Nice to have: Experience with data contracts, schema evolution, and maintaining compatibility between services and data consumers.
Familiarity with data quality frameworks such as Great Expectations, Soda, dbt tests, or custom-built validation solutions.
Experience integrating data solutions with Power BI.
Benefits
Health and dental insurance.
Meal and food allowances.
Childcare assistance.
Extended paternity leave.
Access to gyms and health and wellness professionals through Wellhub (Gympass) and TotalPass partnerships.
Profit Sharing and Results Participation (PLR).
Life insurance.
Access to a continuous learning platform and professional development resources.
Discount club with partner offers.
Free online resources dedicated to physical, mental, and overall well-being.
Pregnancy and responsible parenting courses.
Partnerships with online learning platforms.
Access to a language-learning platform.
Additional benefits and well-being initiatives.
How Jobgether Works
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Work arrangement
Yes