Sr Big Data Engineer

Job Title: Big Data Engineer.
Location: Hyderabad (Hybrid)
Experience: 8+ Years
Job Type: FTE
Key Skills: Big Data, Delta Lake, Hive, Power BI, Python, SQL, Kafka, Spark.
Certifications: Databricks or Data Engineer are required.
Key Responsibilities:
Work closely with technology partners on feature definition, business case development, and overall direction of the enterprise data services platform.
Ensure platform is operating efficiently and effectively
Oversee and lead software development efforts
Provide guidance on routine to highly complex problems to team members
Align to enterprise software development practices
Responsibilities:
Hands on development experience in writing Apache/Pi Spark jobs using Python and Scala
Complete knowledge of the Hadoop ecosystem
Design and development of applications for Ratings Advance analytical, data innovation systems
Participate in multiple software development processes including Coding, Testing, De-bugging & Documentation.
Develop software applications based on clear b usiness specifications.
Work on new initiatives and support existing Ratings applications.
Perform Application & System Performance tuning and troubleshoot performance issues.
Integrate in-house applications with various vendor software platforms.
Setup development environment / sandbox for application development.
Check-in application code changes into the source repository.
Perform unit testing of application code and fix errors.
Effectively interact with customers, business users and IT staff
Qualifications:
Master’s degree in Computer Science, Information Systems or Engineering is required demonstrated equivalence in work experience.
8 + years of IT experience in application development and support.
5+ years of Strong experience with Apache/Pi Spark jobs using Python and Scala
Strong experience with Oracle and related technologies
Strong experience with SQL and no SQL database technologies.
Experience with Unix/Linux Operating System.
Experience using system tools, source control systems, utilities and third-party products.
Experience using scheduling technologies (i.e. Autosys)
Experience with DevOps Tools i.e. Git, Jenkins, Ansible etc…
Experience working in an Agile development environment is MUST
Experience with Cloud Solutions, Design patterns, data models & data architecture & design is a plus.
Experience with Change Management and defect tracking tools
Experience in leading and managing small teams.
Experience with Java Technologies is a plus
Financial experience is preferred

Work arrangement
Hybrid