Live opening · Posted 8 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
TNB Tenaga Malaysia to migrate its existing Cloudera data ecosystem to the modern Databricks Data Intelligence Platform (Lakehouse Architecture). This migration is not a mere technical uplift, but a critical enabler aligned directly with TNB's Reimagining TNB 2025 (RT25) strategy and its overarching Energy Transition (ET) Aspiration Plan, specifically supporting the Grid of the Future and Sustainability Pathway 2050 pillars.TNB?s existing Cloudera environment (v7.1.7) to the Databricks Lakehouse Platform using a ?Lift-and-Shift, then Modernize? approach.
This approach ensures minimal operational disruption during migration while establishing a scalable foundation for advanced analytics, AI, and domain-led modernization.
Lead Data Engineer
Role Summary
Leads the design, development, migration, and deployment of data pipelines, ETL processes, and data engineering best practices for the Databricks platform.
Key Responsibilities
Lead migration of HiveQL, Impala, Spark SQL, Bash, and Informatica workloads.
Design and build batch and streaming pipelines using Databricks.
Develop PySpark-based transformation frameworks.
Implement Delta Live Tables and Databricks Workflows.
Establish CI/CD standards and coding best practices.
Conduct performance tuning and optimization.
Support production deployments and hypercare activities.
Mentor and guide development teams.
Required Skills
Databricks Platform
Apache Spark / PySpark
Delta Live Tables (DLT)
Databricks Workflows
Structured Streaming
SQL, Python, Scala
Informatica Migration
Git, CI/CD Pipelines
Data Quality and Testing Frameworks
Agile Delivery Methodologies
Power BI
Work arrangement
No
Recently tracked roles with full details and direct application links.