Senior Data Engineer

Let's Write Africa's Story Together!
Old Mutual is a firm believer in the African opportunity and our diverse talent reflects this.
Job Description
The Senior Data Engineer is responsible for designing, building, optimising and supporting scalable enterprise data solutions on the Databricks / AWS platform. The role requires strong hands-on engineering capability across PySpark, SQL, Delta Lake, Parquet, Medallion Architecture, ETL/ELT development, data ingestion and reconciliation frameworks. The successful candidate will translate complex business and data requirements into reliable, performant and maintainable data products while ensuring appropriate data quality, security, governance and operational controls.
The role also provides technical leadership to other engineers and works closely with Data Architecture, Data Governance, Data Science, Business Intelligence and business stakeholders. Insurance or financial-services experience is strongly preferred, with the ability to understand business processes and translate them into robust data solutions.
Key Responsibilities
Data Engineering & Development
Design and develop scalable ETL/ELT pipelines using Databricks, PySpark and SQL.
Build batch, incremental and event-driven ingestion pipelines from multiple source systems.
Design and implement solutions using Medallion Architecture (Bronze, Silver and Gold).
Develop and optimise Delta Lake and Parquet-based data solutions.
Implement reusable engineering patterns, frameworks and common components.
Perform performance tuning and cost optimisation across Spark jobs, clusters and pipelines.
Build and manage CI / CD processes on Azure DevOps
Data Quality & Reconciliation
Design and implement automated data reconciliation and control frameworks.
Implement completeness, accuracy, balancing and data-quality controls across source-to-target pipelines.
Investigate data discrepancies and perform root-cause analysis.
Ensure appropriate auditability, lineage and traceability throughout the data lifecycle.
Solution Design & Architecture
Translate business requirements into scalable technical designs and data products.
Contribute to solution architecture, data standards and engineering patterns.
Evaluate existing solutions and define appropriate as-is / to-be designs.
Ensure solutions align with enterprise architecture, information security, governance and control standards.
Operational Excellence
Monitor production pipelines and proactively resolve failures, data incidents and performance issues.
Perform root-cause analysis and implement permanent remediation.
Support production releases through established change-management and deployment processes.
Develop and maintain monitoring, alerting, logging, runbooks and operational documentation.
Manage and support Databricks job schedules to ensure critical business processes run successfully and issues are resolved timeously.
Technical Leadership
Provide technical leadership and mentorship to Data Engineers.
Conduct code reviews and enforce engineering standards, testing and development best practices.
Promote reusable solutions rather than point-to-point development.
Support technical design reviews and challenge solutions where appropriate.
Drive continuous improvement across engineering practices, automation and platform efficiency.
Stakeholder & Business Engagement
Work with business and technical stakeholders to understand data and reporting requirements.
Translate insurance business requirements into appropriate data models and engineering solutions.
Communicate technical issues, dependencies, risks and delivery status clearly to stakeholders.
Partner with architecture, governance, security, operations, finance and business teams.
Minimum Requirements
Bachelor's degree in Computer Science, Computer Engineering, Information Technology, Data Science or a related discipline. Relevant Databricks and cloud certifications are advantageous.
Experience
Ideally 5+ years of Data Engineering / ETL development experience, with demonstrated experience delivering enterprise-scale data solutions and operating production data pipelines.
Core Technical Skills
Advanced PySpark and SQL; strong Databricks experience; Delta Lake and Parquet; Medallion Architecture; ETL/ELT and ingestion pipelines; data modelling; reconciliation and data-quality frameworks; Git/Azure DevOps or equivalent CI/CD; cloud data platforms, preferably AWS; performance optimisation; and production troubleshooting.Advantageous Skills
Databricks Workflows/Jobs, Unity Catalog, Auto Loader, CDC or similar incremental ingestion patterns, orchestration frameworks, CI/CD automation, AWS services, and exposure to AI/ML data workloads.
Industry Experience
Insurance industry experience is highly advantageous, particularly exposure to policy, claims, premium, finance / GL, actuarial or regulatory data.
Business & Behavioural Competencies
Strong problem-solving and analytical capability.
Engineering discipline, ownership and accountability.
Ability to work independently in a complex enterprise environment.
Strong stakeholder management and communication skills.
Ability to coach and uplift the capability of less experienced engineers.
Commercial awareness and a focus on sustainable, cost-effective engineering solutions.
Skills
Analysis of Alternatives (AoA), Analytical Processes, Analytical Sciences, Assessment Testing, Business Intelligence (BI) Analysis, Categorizing Data, Computer Literacy, Data Analysis, Database Reporting, Data Compilation, Data Controls, Data Interpretations, Financial Modeling, Information Retrieval, Managerial Accounting, Numerical Aptitude, Planni