Iceberg DBA / Lakehouse Operations Engineer - L2

Job Summary
We are seeking a highly skilled Iceberg DBA / Lakehouse Operations Engineer to own the reliability, performance, and operational integrity of the Iceberg data layer powering enterprise analytics and business-critical applications.
This role operates in a large-scale, multi-engine Lakehouse environment, supporting workloads across Spark, Hive, and Impala, and plays a key role in enterprise data modernization initiatives (Hive and Teradata → Iceberg).
The ideal candidate brings deep expertise in Iceberg table operations, metadata management, and query performance optimization, ensuring consistent, high-performance data access across platforms in a cloud-based environment.
This role is critical to ensuring data accuracy and performance—any degradation directly impacts downstream reporting, analytics, and business-critical decision-making.
Key Responsibilities:
Iceberg Data Layer Ownership & Operations
• Own day-to-day operations of Apache Iceberg tables supporting multiple enterprise applications
• Ensure data reliability, consistency, and availability across all Lakehouse workloads
• Maintain operational integrity for datasets at multi-terabyte to petabyte scale
Advanced Table Management & Optimization
• Execute advanced Iceberg table maintenance and optimization strategies:
o Compaction (minor/major) and small file mitigation
o Snapshot expiration and metadata compaction to control metadata growth
o Orphan file cleanup (vacuum) to maintain storage efficiency
• Optimize data layout and performance through:
o File size tuning and distribution strategies
o Partition evolution and pruning optimization
o Clustering and ordering techniques (e.g., Z-ordering or similar patterns)
Data Modeling Standards & Lakehouse Design Alignment
• Support and enforce data modeling best practices aligned with:
o Normalized data structures (3NF) for source-aligned datasets
o Medallion architecture (Bronze / Silver / Gold layers) for curated data flows
• Ensure Iceberg table design aligns with:
o Data ingestion patterns (raw vs curated layers)
o Downstream consumption and performance requirements
• Assist in structuring datasets to balance:
o Data integrity and normalization
o Query performance and analytical efficiency
• Work with data engineering teams to ensure consistent implementation of layered data architecture across multiple applications
Multi-Engine Query Performance & Consistency
• Ensure consistent and performant query behavior across:
o Spark (CDE)
o Hive / Impala (CDW)
• Troubleshoot and resolve:
o Query performance bottlenecks
o Metadata inconsistencies across engines
o Inefficient execution plans and scan patterns
Metadata & Data Lifecycle Management
• Manage Iceberg metadata to ensure:
o Efficient scaling and performance
o Consistent table state across engines
• Execute lifecycle operations:
o Data retention and archival policies
o Snapshot lifecycle management and cleanup
o Time-travel optimization and maintenance
Production Support, Incident Resolution & On-Call
• Provide L2/L3 support for data-related production issues across Iceberg-based Lakehouse workloads
• Participate in on-call rotation to support critical data platforms and ensure timely response to incidents
• Respond to and resolve P1/P2 production incidents within defined SLAs, minimizing impact to downstream applications and reporting
• Troubleshoot:
o Data inconsistencies and reporting discrepancies
o Query failures and performance degradation
• Perform root cause analysis (RCA) and implement preventive measures to avoid recurring issues
• Collaborate with platform and application teams during incident triage and resolution
Security & Data Governance Support
• Support fine-grained access control using:
o Ranger policies and RBAC
• Own and ensure data validation, reconciliation, and accuracy between source and Iceberg datasets
• Ensure secure and compliant access to data across applications
Required Skills
• Strong hands-on experience with Apache Iceberg and/or Hive-based data lakes
• Understanding of data modeling concepts (normal forms) and modern Lakehouse patterns (Medallion architecture)
• Expertise in:
o Table-level optimization and performance tuning
o Large-scale data management (TB/PB scale)
• Experience with:
o Spark SQL, Hive, Impala, NiFI, Trino
• Strong understanding of:
o Partitioning strategies
o File formats (Parquet/ORC)
o Distributed query processing
Preferred Skills
• Experience with:
o Hive-to-Iceberg or Teradata-to-Iceberg migration
o Cloudera CDP (CDE/CDW)
• Familiarity with:
o Cloud platforms (AWS, Azure)
• Scripting/automation (Python, Shell)
What You’ll Work On
• Enterprise-scale Iceberg Lakehouse platform supporting multiple applications
• Large-scale data modernization initiatives
• Performance optimization and stability of mission-critical analytical workloads

Work arrangement
No