Senior Engineer – Batch ETL, Big Data & Workflow Orchestration

Where Ambition Meets Innovation
At LPL’s Global Capability Center, you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We’re proud to be expanding and reaching new heights in Hyderabad.
Join us as we create something extraordinary together.
We are seeking a skilled software developer to help with the development of projects within the Account Lifecycle Management domain, that span different technologies and requirements. You will be working with a motivated team in a fast-paced environment.
A software developer's day involves attending a morning stand-up meeting, reviewing code, working on complex coding tasks, collaborating with team members, and participating in design and architecture discussions. They also attend meetings like sprint planning and retrospectives, monitor project progress, deploy code, and mentor junior developers. By the end of the day, they review their progress, update documentation, and prepare for the next day's tasks.
Responsibilities:
Design, develop, and optimize enterprise-scale batch ETL pipelines supporting LPL’s regulatory, supervision, and analytics platforms.
Lead ETL development using Informatica and PySpark for high-volume and complex data transformations.
Build and maintain Airflow DAGs to orchestrate batch workflows, dependencies, retries, SLAs, and monitoring.
Design scalable solutions for large data processing on distributed and cloud-based data platforms.
Drive performance tuning, resiliency, data quality, and operational stability of batch processing jobs.
Partner with architecture, upstream/downstream system teams, and business stakeholders to translate requirements into robust, compliant data solutions.
Enforce LPL standards for security, data governance, auditability, and operational readiness.
Perform code reviews, mentor Engineer I/II resources, and raise overall engineering maturity of the team.
What are we looking for?
We’re looking for strong collaborators who deliver exceptional client experiences and thrive in fast-paced, team-oriented environments. Our ideal candidates pursue greatness, act with integrity, and are driven to help our clients succeed. We value those who embrace creativity, continuous improvement, and contribute to a culture where we win together and create and share joy in our work.
Requirements:
Strong hands-on experience with batch ETL architectures in regulated environments.
Advanced expertise in Informatica.
Strong proficiency in PySpark and distributed data processing concepts.
Hands-on experience with Apache Airflow (authoring DAGs, scheduling, dependency management, monitoring).
Experience processing large datasets with a focus on performance and reliability.
Strong SQL and data modeling fundamentals.
Ability to own complex problem areas end-to-end and operate with minimal supervision.
LPL Global Business Services, LLP - PRIVACY POLICY