Live opening · Posted 1 day ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Responsibilities:
Design, develop, and deliver end-to-end services for Different.ai 's innovative platform.
Solve complex engineering challenges, including building high-throughput data pipelines for machine learning jobs handling petabyte-scale data.
Lead and mentor a team of dynamic engineers, fostering a collaborative and innovative environment.
Take ownership of critical architectural decisions.
Requirements:
Experience: 7-12 years in software and data engineering.
Solid understanding of data architecture principles and big data technologies/tools & Distributed data systems.
Experience with workflow orchestration tools like Apache Airflow, Dagster, Composer, and Airbyte, etc.
Experience with Real-time streaming (Kafka, Pulsar, Flink) & Query engines (Trino, Presto, Druid)
Experience with Data lakehouse architectures (Iceberg, Delta Lake, Hudi) & Batch processing (PySpark/Spark, Hadoop)
Advanced understanding of cloud platforms (AWS, GCP, Azure), modern data warehouses (Databricks, Snowflake, and Redshift), and databases (SQL family databases, MongoDB, and ElasticSearch).
Deep expertise in Java. Python and GoLang, etc.
Proficiency in Kubernetes, CI/CD tools, and deploying software solutions on-premises.
Experience
8-12 yrs
Recently tracked roles with full details and direct application links.