Senior Lead Software Engineer - Java & Databricks

Job Description
Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products.
As a Senior Lead Software Engineer at JPMorganChase within the Digital Communications Compliance team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Drive significant business impact through your capabilities and contributions, and apply deep technical expertise and problem-solving methodologies to tackle a diverse array of challenges that span multiple technologies and applications.
Job Responsibilities
Design and develop scalable, fault-tolerant microservices and APIs that support rule-based and ML-based detection pipelines Model and implement supervision and reviewer workflows using state machines
Build streaming and batch data pipelines that ingest, index, and enrich communications content and alerts
Design data models on Databricks for surveillance, search, retention, and life cycle management at scale
Develop and optimize large-scale batch and streaming data pipelines using Databricks, Apache Spark, and Delta Lake
Build robust unit, integration, and performance tests following Test-Driven Development. Leverage Databricks Workflows, notebooks, and CI/CD pipelines to automate data processing and deployments. Troubleshoot and tune Databricks jobs, clusters, and data pipelines for efficiency and performance Drive adoption of Databricks best practices for data engineering, software engineering, observability, and platform operations
Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
Partner with product management and compliance SMEs to monitor and improve alert accuracy and reliability. Proactively identify hidden problems and patterns in communications data to improve detection quality
Required Qualifications, Capabilities, And Skills
Formal training or certification on software engineering concepts and 5+ years applied experience
Hands-on practical experience delivering system design, application development, testing, and operational stability
Expertise in building resilient, scalable, enterprise-grade cloud-native products, with strong exposure in compliance for the financial industry
Hands-on Databricks experience including development of production workloads using Spark, Delta Lake, and Databricks Workflows. Expert Java/Kotlin and Python programmer with experience building headless, externally consumable APIs. Experience building cloud-native microservices for streaming and batch architectures using Spark and/or Flink
Proficient with AWS services including EC2, ECS, EKS, EMR, S3, and Glacier. Hands-on with Semantic search, Kafka, and PostgreSQL. Experience integrating and operationalizing ML/LLM models and pipelines in production. Experience with observability and monitoring tools such as Prometheus, Grafana, and OpenTelemetry
Experience building CI/CD pipelines using ArgoCD, Helm, Terraform, Jenkins, and GitHub Actions
Prior experience in Test-Driven Development, delivering products with well-defined SLI/SLO/SLAs
Strong ownership mentality with excellent communication skills and a collaborative mindset. Experience mentoring engineers and providing technical leadership at a senior level
Preferred Qualifications, Capabilities, And Skills
Exposure to Unity Catalog, and Databricks Workflows
Exposure to building and optimizing large-scale batch and streaming data pipelines in cloud-native environments
Understanding of distributed systems, data engineering best practices, and operationalizing data and ML workloads in production
ABOUT US

Work arrangement
No