Databricks Architect

About the Organization-
Impetus Technologies is a digital engineering company focused on delivering expert services and products to help enterprises achieve their transformation goals. We solve the analytics, AI, and cloud puzzle, enabling businesses to drive unmatched innovation and growth.
Founded in 1991, we are cloud and data engineering leaders providing solutions to fortune 100 enterprises, headquartered in Los Gatos, California, with development centers in Noida, Indore, Gurugram, Bengaluru, Chennai, Pune, and Hyderabad with over 3000 global team members. We also have offices in Canada and Australia and collaborate with a number of established companies, including American Express, Bank of America, Capital One, Toyota, United Airlines, and Verizon.
Locations: Bengaluru, Pune, Indore, Gurgaon, Noida & Chennai
Experience: 12+ Years
The ideal candidate is a hands-on architect with deep Databricks and PySpark expertise, strong AWS/Azure cloud fluency, and a proven record of setting technical direction on enterprise-scale programs.
Key requirements:
12–14 years of experience in designing, architecting, implementing, and optimizing large-scale (TB/PB) data engineering and analytics solutions and distributed data platforms.
Deep, hands-on architecture expertise with Databricks - Spark/PySpark, Delta Lake, lakehouse and medallion architecture, Unity Catalog, Databricks Workflows, Delta Live Tables, and performance/cost optimization.
Expert-level PySpark and Spark SQL, with strong Python for building reusable, production-grade ETL/ELT frameworks and streaming architectures (batch and real-time).
Strong hands-on experience with AWS and Azure Cloud, including networking, IAM/security, cost optimization, and reliability patterns; comfortable architecting across cloud data services (e.g., AWS S3, EMR, Glue; Azure Data Factory, Synapse, ADLS).
Strong experience architecting solutions with Snowflake and modern data warehousing - data modeling, performance tuning, security, and cost governance - and defining Databricks–Snowflake interoperability patterns.
Strong experience with data orchestration, CI/CD for data, and Infrastructure-as-Code (Terraform or equivalent); exposure to Docker/Kubernetes.
Deep expertise in data governance, data quality, lineage, cataloging, and security across the data lifecycle (e.g., Unity Catalog, RBAC, encryption, audit).
Experience leading cloud/data platform migration and modernization programs - migrating legacy or on-premises platforms to Databricks and the cloud.
Proven ability to set architecture standards, lead design reviews, and provide technical leadership and mentorship across multiple engineering teams.
Strong communication and stakeholder-management skills; able to drive architecture decisions and translate business needs into technical blueprints for both technical and executive audiences.
Good to Have:
Exposure to Generative AI / LLM solution architecture — RAG pipelines, embedding and vector store design, and native Databricks GenAI services (Mosaic AI Model Serving, Vector Search, Agent Framework)
For Quick Response- Interested Candidates can directly share their resume along with the details like Notice Period, Current CTC and Expected CTC at anubhav.pathania@impetus.com

Work arrangement
No