DevOps Infrastructure Engineer

The ideal candidate will play a critical role in designing, implementing, and maintaining cloud infrastructure and CI/CD pipelines to support scalable, secure, and efficient data and analytics solutions. This role requires a strong understanding of cloud-native technologies, DevOps best practices, and hands-on experience with Azure and Databricks.
Responsibilities:
Architect, deploy, and manage scalable and secure cloud infrastructure on Microsoft Azure.
Implement best practices for Azure Resource Management, including resource groups, virtual networks, and storage accounts.
Optimise cloud costs and ensure high availability and disaster recovery for critical systems
Set up, configure, and maintain Databricks workspaces for data engineering, machine learning, and analytics workloads.
Automate cluster management, job scheduling, and monitoring within Databricks.
Collaborate with data teams to optimise Databricks performance and ensure seamless integration with Azure services.
Design and implement CI/CD pipelines for deploying infrastructure, applications, and data workflows using tools like Azure DevOps, GitHub Actions, or similar.
Automate testing, deployment, and monitoring processes to ensure rapid and reliable delivery of updates.
Implement monitoring and alerting solutions using tools like Dynatrace, Azure Monitor, Log Analytics, and Databricks metrics.
Troubleshoot and resolve infrastructure and application issues, ensuring minimal downtime.
Enforce security best practices, including identity and access management (IAM), encryption, and network security.
Ensure compliance with organisational and regulatory standards for data protection and cloud operations.
Work closely with cross-functional teams, including data engineers, software developers, and business stakeholders, to align infrastructure with business needs.
Maintain comprehensive documentation for infrastructure, processes, and configurations.
Requirements:
Bachelor's degree in Computer Science, Engineering, or a related field.
5+ years of experience in DevOps or Cloud Engineering roles.
Cloud - Azure: Networking, Key Vault, Az CLI, ADF, Databricks/Snowflake, ADLS, Azure Storage.
DataBricks: Provisioning, Workspace Management, Secret Management, Networking, Pipelines Management, Databricks Bundles.
IAC: Terraform, CI/CD Pipelines on GitHub, Shell/Python Scripting, REST API.
Databricks experience is mandatory.
The candidate will be an individual contributor.
Unity Catalogue, Delta Lake Table Format, PySpark, Py Wheel, PostgreSQL Management.
Strong problem-solving and analytical skills.
Excellent communication and collaboration abilities.

Experience
4-8 yrs