Live opening · Posted 2 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Responsibilities:
Will collaborate closely with our data science and ML teams across different cybersecurity domains to define ML infrastructure requirements and build critical data services.
Can leverage MLOps best practices to design and develop scalable model training, evaluation, experimentation, and deployment workflows.
Has extensive experience in ML training (local and distributed), feature extraction, and dataset creation.
Is comfortable deploying software with CI / CD tools, including Jenkins, Harness, Terraform, etc.
Is an expert at developing and deploying assets in the cloud, preferably AWS and Kubernetes, using IAC (infrastructure as code).
Can build a workflow orchestration platform to be used by other developers.
Requirements:
Has hands-on experience of 2+ years implementing data pipeline infrastructure for data ingestion and transformation near real-time availability of data for applications and ML pipelines.
Has experience designing optimized solutions for ingestion, curation of large datasets.
Has working knowledge of Data Lake technologies, data storage formats (Parquet, ORC, Avro), and query engines (Athena, Presto, Dremio), and associated concepts for building optimized solutions at scale.
Maintains a proficient level in one of the following programming languages or similar- Python, Java, Go.
Has experience with data pipeline tools (Flink, Spark, or Ray) and orchestration tools such as Airflow, Dagster, or Step Functions.
Is an expert in implementing data streaming and event-based data solutions (Kafka, Kinesis, SQS/SNS or the like).
Experience
7-12 yrs
Recently tracked roles with full details and direct application links.