Senior Lead Software Engineer, Big Data & Cloud Engineering — Risk Central (London)

As a Senior Lead Software Engineer in the Risk Central (London) team, you will be a senior hands-on engineer and technical leader responsible for building and operating scalable data and analytics services that support Markets risk use cases. You will work across a modern cloud and big data stack and partner with stakeholders across Front Office, Risk, Product Control, and Finance.
Risk Central is a strategic technology group within JPMorganChase’s CIB that builds and operates a next-generation analytics platform used for critical front office and risk reporting. The platform delivers timely, consistent, and high-quality data and analytics to support risk management and decision-making across global markets. A key aspect of the role is operating across two enterprise data warehouses—Databricks and Amazon Redshift—ensuring data consistency, performance, governance, and reliability across both platforms.
Job responsibilities
Lead end-to-end delivery of complex initiatives across ingestion, transformation, storage, and consumption layers.
Drive engineering best practices (design reviews, code reviews, testing standards, CI/CD, documentation).
Mentor engineers and raise the bar on technical quality, ownership, and operational excellence.
Design and build robust batch and streaming pipelines (high-volume, high-throughput) using distributed compute.
Implement efficient data modeling, partitioning, and performance tuning strategies for large datasets.
Build reusable frameworks/components to accelerate onboarding of new datasets and analytics use cases.
Engineer data products and workflows that span Databricks and Redshift, including ingestion patterns, transformations, and serving layers.
Define approaches to reconciliation/consistency, lineage, and controls across both warehouses.
Optimize query performance and cost across platforms; establish best practices for workload placement.
Build cloud-native solutions on AWS (e.g., S3, EMR, Lambda, Kinesis/MSK, Glue, EventBridge, DynamoDB, Redshift, EKS—depending on team standards).
Leverage Spark-based processing (including PySpark/Scala Spark) and modern lake/lakehouse patterns where applicable.
Own production stability: monitoring/alerting, incident management, root-cause analysis, and preventative engineering.
Define and improve SLAs/SLOs for critical data deliveries and platform uptime.
Stakeholder partnership - work closely with product managers, quants, risk managers, traders, and controllers to translate needs into scalable technical solutions.
Communicate clearly with senior stakeholders on progress, risks, dependencies, and trade-offs.
Contribute to planning, refinement, execution, and retrospectives; help teams deliver predictably with high quality.
Required qualifications, capabilities, and skills
Extensive professional software engineering experience, including delivery of production systems at scale.
Strong programming skills in Python (and/or Java/Scala), with strong CS fundamentals (data structures, algorithms, OO design).
Hands-on experience with distributed data processing (e.g., Spark) and building data pipelines (batch and/or streaming).
Experience with at least one of: Databricks, Amazon Redshift, or equivalent enterprise data warehouse/lakehouse platforms; ability to design and tune performant workloads.
Strong SQL skills and understanding of data modeling and analytics patterns.
Practical experience with cloud engineering concepts (security, networking basics, IAM/access controls, encryption, observability).
Proven ability to troubleshoot production issues and drive operational improvements.
Strong communication skills and experience working with globally distributed teams.
Preferred Qualifications
Experience in Markets technology and familiarity with the trade lifecycle, risk concepts (market risk measures, sensitivities, P&L explain), and common products (FX, rates, credit, equities; derivatives basics).
Streaming technologies (Kafka/MSK/Kinesis) and near-real-time analytics patterns.
Experience with governance, controls, and data quality frameworks (reconciliations, lineage, auditability, entitlements).
Familiarity with lake/lakehouse table formats and tooling (e.g., Iceberg/Delta concepts) and columnar storage formats (e.g., Parquet).
CI/CD, infrastructure-as-code, containerized workloads, and DevOps practices.
Prior experience leading initiatives across multiple teams (tech lead responsibilities, mentoring, cross-team coordination).