Big Data Engineer

Our client, one of the largest Amazon Web Services (AWS) partner for data services, is looking for top talent to join their elite team of technologists to build and contribute to large-scale, innovative projects. Our client wants candidates who are eager to learn and grow with them as they work to move their market monitoring applications to modern big data platforms in the cloud where up to 200 billion market events are being processed each day.
If you are a fast learner, have a sharp mathematical and technical mind and are interested in working for one of the biggest data users in the country in a fast-paced, open source environment, our client has an immediate need for a Jr. Big Data Engineer to coordinate changes to computer databases, test, and implement the database applying knowledge of database management systems.
THE PLATFORM
Our client's Data Platform is responsible for the collection, management and dissemination of data in their institution. It's an enterprise capability that services multiple business lines across the entire organization. The platform collects and processes billions of records (up to 100 billion) each day in support of their regulatory mission to protect investors and promote market integrity. They are embarking upon a large, multi-year, enterprise-wide initiative to re-platform major pieces of our existing offering, replacing legacy functionality with a set of modern, highly customer-centric products and capabilities.
The ideal candidate will provide technical leadership and implementation to a team focused on delivering one or more of those modern, customer-centric products and capabilities.
THE ROLE
Jr. Big Data Engineer
Are you passionate about technology? Do you love programming? Do you sometimes pat yourself on the back after writing a particularly elegant function? Are you an aficionado of the art of software design? Do you expect the best from yourself and those around you?
Our client is looking for passionate developers who want to make a difference. If you are a rock star developer, a strong communicator, and a self-starter who can grab hold of a challenge and dive deep with it, then please join our client's delivery-focused team.
Job Responsibilities
Able to grasp complex business requirements
Provide tech expertise and leadership for analysis, design, and modeling work
Design, develop, implement, test, deploy, and support big data processing software in the AWS Cloud
Establish and implement ETL and big data processing best practices to design, test, implement, and support mission critical applications
Set clear expectations with engineering team and peers
Provide timely and regular updates to management
Ensure security and compliance requirements are met
Be delivery-focused
Be quality-focused
Own the outcome
Ship great products!
Experience & Qualifications
1+ years of experience required, including some commercial/non-government work
Hands on experience designing, developing, implementing, testing, and deploying large scale ETL/Data Analytics/Java/J2EE projects
Believes in Scrum/Agile, and has deep experience delivering software when working on teams that use Scrum/Agile methodology
Strong and creative analytical and problem-solving skills
Strong communication skills
Enable long-term implementation, facilitate product vision and strategy.
Required Technical Skills & Knowledge for Lead Data engineer:
SPARK
Hadoop, Hive, Hbase
REST APIs
Write complex SQL queries
Performance tuning and optimization
Familiarity of columnar storage formats ORC, Parquet and various compression techniques
Unit testing with Junit or Scalatest
Git/Maven
Code Reviews
Agile
Testing strategy and implementation
AWS Cloud experience is a plus.
ConsultNet is an Equal Opportunity Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, ancestry, citizenship, disability, age, military or veteran status, and other characteristics protected under federal, state and local law.

Work arrangement
Yes