Senior Big Data Engineer
Grainger is a broad line, business-to-business distributor of maintenance, repair and operating (MRO) supplies and other related products and services. More than 3.2 million businesses and institutions worldwide rely on Grainger for products such as safety gloves, ladders, motors and janitorial supplies, along with services like inventory management and technical support. These customers represent a broad collection of industries including commercial, government, healthcare and manufacturing. They place orders online, on mobile devices, through sales representatives, over the phone and at local branches. Approximately 5,000 suppliers provide Grainger with more than 1.6 million products stocked in Grainger’s distribution centers and branches worldwide.
The Sr. Big Data Engineer is a key team member who will work with multiple data scientists to solve complex analytics use cases. He/she will be technically involved with identifying, analyzing, obtaining, understanding and moving big data sets through the Grainger Big Data ecosystem. He or she is responsible for moving large data sets from multiple sources and ingest this data into the Grainger’s Data Science lab environment and ultimately deliver ‘information sets’ to data scientists. The role includes hands on data acquisition and integration work using the full Hadoop stack including Sqoop, HBase, Hive, Oozie, Flume, NiFi, etc. as well as other Big Data technologies.
The ideal candidate is a technologist with strong business acumen. He/she must have passion for data and big data technologies to solve complex business problems.
Principal Duties and Responsibilities:
- Perform detailed analysis/design of functional and technical requirements and translate them to solutions in the Big Data and Data Science Lab environments
- Conduct and participate in solution review with principal stakeholders
- Develop robust and reusable data acquisition and processing routines to ingest data into the Data Science Lab environment
- Maintain data platforms and environments (Hadoop, private cloud instances, public cloud services, GPU-based development server for advanced computing) that are part of the Data Science Lab
- Perform advanced data discovery, profiling, and assessment on required data
- Provide subject matter expertise on data
- Manage the full lifecycle of assigned data projects from requirements to technical design (platform, data, and automation routines), to project deployment of the proposed lab solution.
- Develop generalized routines to automate the ingestion of data into the Data Science Lab
Preferred Education and Experience:
- BS/BA or equivalent preferred
- 7-10 yeas of Technical Business experience – ETL experience
- 4-7 years of solid Hadoop experience – Scoop, Pig, Hive…
- Bachelor’s Degree in Computer Science or related area, or equivalent experience
- Experience with Teradata, SQL, IBM DataStage, Hadoop, Unix Scripting
- Technical experience with big data visualization applications
- Ability to clearly articulate pros and cons of various technologies
- Ability to document use cases, solutions and recommendations
- Excellent written and verbal communication skills
- Ability to support program and project managers in the planning, estimation and implementation of projects
- Ability to quickly develop business acumen and data subject matter expertise
- Ability to perform detailed analysis of business problems and technical environments
- Ability to work creatively and analytically in a fast paced and agile environment;
- A self-starter with the ability to work in cross-functional teams
Grainger is an Equal Opportunity Workplace and an Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.