Senior Data Engineer

Position: Sr. Data Engineer
Location : US Remote (Candidate must be in EST)
Full time role
Skills: Must be 10 years of experience
SQL (expert)
Snowflake - not a roadblock (added advantage)
Cloud - AWS is preferred (exp on any cloud)
Python – intermediate
Databricks - added advantage.
The Data Engineer will be part of our Global Fulfillment team, building enterprise data streams, ingesting data from multiple data sources, and provisioning data as raw data streams and canonical business events. We are working towards integrating our Fulfillment data sources into our federated enterprise data. The engineer would also be responsible for continuously identifying opportunities for automation and reduction of technical debt & manual data manipulations to create efficiencies & optimize data delivery. They should also work with cross-functional business teams, identify data hydration solutions, assess the feasibility, and provide recommendations to other developers on automation opportunities ensuring that delivered frameworks are scalable.
Qualifications
BS in Computer Science or Related
5 to 10 years of data engineering experience
SQL heavy Data Engineering minded and who has been involved performance fine tuning and building ETL pipelines using SQL transformation logic.
Good understanding of modern data platforms such as Snowflake, Data bricks including data lakes, with good knowledge of the underlying architecture, preferably in Snowflake.
· Proven experience in assembling large, complex sets of data that meets non-functional and functional business requirements.
· Experience in identifying, designing, and implementing integration, modelling, and orchestration of complex fulfillment data and at the same time look for process improvements, optimize data delivery and automate manual processes.
Working experience of scripting, data engineering and analytics (SQL, Python, devops CI/CD pipelines, Airflow/ADF, Machine Learning (good to have))
Working experience of performance tuning and optimization, bottleneck problems analysis, and technical troubleshooting in a, sometimes, ambiguous environment.
Working experience of working with cloud-based systems (AWS preferred but GC Azure experience ok to have)
Desired Qualifications:
Experience working with cloud-based systems – AWS/Azure/GC & Snowflake data warehouses.
Expertise in designing data table structures, reports, and queries.
Working knowledge of CI/CD
Working knowledge of building data integrity checks as part of delivery of applications
Experience working with Kafka technologies.
Technical expertise to build code that is performant as well as secure.
Technical depth and vision to perform POC’s and evaluate different technologies.
Experience with Real Time Analytics and Real Time Messaging
Working experience with Microservices is desirable.
Design, implement and monitor 'best practices' for Dev framework.
Experience working with large volume data; retail experience strongly desired.
Possesses an entrepreneurial spirit and continuously innovates to achieve great results.
Communicates with honesty & kindness and creates the space for others to do the same.
Fosters connection by putting people first and building trusting relationships.
Integrates fun and joy as a way of being and working, aka doesn’t take themselves too seriously.
Preferred Tools: Data bricks, Snowflake, Apache Airflow, Microsoft Azure Data Factory, Kafka, Power BI, SSAS
A day in the life:
Co-ordinate with multiple data source teams in getting data published to our Finance Data Warehouse
Working with stakeholders including data, design, product, and executive teams, and assisting them with data related and technical issues.
Building Scalable Data Pipelines for generating training datasets for machine learning deliverables
Mentoring Junior resources and drive end to end design, implementation, and delivery of engineering components.
Communicating with and educate both senior and junior colleagues to further embed data science and analytics across the organization.
Strive for continuous improvement of code quality and development practices.
Willingness to adapt to and self-learn new technologies and deliver on them.
Translating business issues to technical terms
Understanding, leveraging, and applying best practices effectively. Also leads by example and comes up with coding standards and best practices for technology.
Collaborating with cross-functional teams – business stakeholders, engineers, program management, project management, etc. - to produce the best solutions possible.
Anticipating system/application challenges and proposes solutions for the same.
Contributing to story sizing and work estimates for implementation, validation, delivery, and documentation
Reviewing user stories to ensure a quality user experience, well-defined acceptance criteria and thorough test coverage.
Participating in design and code review to ensure quality and testability of feature code.

Work arrangement
No