Live opening · Posted 1 day ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Requirements
· Strong understanding of AWS architecture best practices, scalability, security, and cost optimization strategies.
· Strong hands-on experience with AWS services including Lambda, Glue ETL, Athena, S3, DynamoDB, Step Functions, EventBridge, SNS, and SQS.
· Deep experience in Apache Spark (PySpark/Scala) development, unit testing, and performance optimization.
· Strong Python programming skills using libraries such as pandas, requests, json, and awswrangler.
· Experience on Apache Kafka and Confluent Kafka.
· Experience designing and optimizing data lakes using Apache Iceberg, including compaction and Iceberg optimization techniques.
· Hands-on experience integrating and optimizing Starburst/Trino, including connecting Starburst from Lambda and Glue ETL jobs.
· Experience with NoSQL databases such as DynamoDB, MongoDB.
· Experience working with data formats including Avro, Parquet, JSON, XML, and CSV.
· Comfortable challenging your peers and leadership team.
· Can prove yourself quickly and decisively.
· Excellent communication skills and Good Customer Centricity
Design and optimize data lakes using Apache Iceberg on AWS, including table compaction and Iceberg performance tuning.
Work arrangement
No
Recently tracked roles with full details and direct application links.