Ridgefield Park, NJ, United States
May 26, 2023
Apply NowJob description
12 month contract
- Big Data Engineers serve as the backbone of the Strategic Analytics organization, ensuring both the reliability and applicability of the teams data products to the entire organization
- They have extensive experience with ETL design, coding, and testing patterns as well as engineering software platforms and large-scale data infrastructures
- Big Data Engineers have the capability to architect highly scalable end-to-end pipeline using different open source tools, including building and operationalizing high-performance algorithms.
- Big data engineers implement complex big data projects with a focus on collecting, parsing, managing, analyzing, and visualizing large sets of data to turn information into actionable deliverables across customer-facing platforms
- They have a strong aptitude to decide on the needed hardware and software design and can guide the development of such designs through both proof of concepts and complete implementations
- Translate complex functional and technical requirements into detailed design
- Design for now and future success
- Hadoop technical development and implementation
- Loading from disparate data sets by leveraging various big data technology
- Pre-processing using Hive, Impala, Spark, and Pig
- Design and implement data modeling
- Maintain security and data privacy in an environment secured using Kerberos and LDAP
- High-speed querying using in-memory technologies such as Spark
- Following and contributing best engineering practice for source control, release management, and deployment
- Production support, job scheduling/monitoring, ETL data quality, data freshness reporting
- SQL complex queries
- Hive
- Python development
- Hadoop
- Data modeling
