Big Data Engineer, Senior

| Chicago

 

Summary

The Big Data Engineer is responsible for building scalable data platforms, and large-scale processing systems that enable advanced analytics and support data teams across the enterprise. This hands-on role requires some experience working with AWS cloud platform in addition to expertise in a variety of technologies. The Big Data Engineer will develop and manage the enterprise data warehouse while sourcing data from various databases/applications & web APIs using stream and batch processing architectures.

 

Responsibilities

  • Responsible for designing, building & managing the advanced analytics platform to support downstream data science and the business intelligence teams
  • Work with the product owner, data scientist and various internal data users to understand the business requirements and implement optimal data solutions
  • Keen eye towards configuration driven approach to automate repeatable processes and tasks
  • Build and operate stable, scalable and highly performant data pipelines that cleanse, structure and integrate disparate big data sets into a readable and accessible format for end user analyses and targeting.
  • Develop data quality and governance framework that supports data lineage and ensures delivery of high-quality data to internal and external stakeholders
  • Using analytical and problem-solving skills to take complex business requests and transform them into clean, simple data solutions.
  • Implement/improve version control, deployment strategies, notifications to ensure product quality, agility and recoverability.
  • Understand and work with technology & IT teams to support database procedures, such as upgrade, backup, recovery, migrations, etc.

 

Role Specific Skills

  • Designing, building and operationalizing large-scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, S3, EC2, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake etc.
  • Hands-on experience analyzing, re-architecting and re-platforming on-premise data warehouses to data platforms on AWS cloud using AWS/3rd party services.
  • Knowledge about other NoSQL databases, such as MongoDB, Cassandra, HBase, etc.
  • Building and migrating the complex ETL pipelines on Redshift and Elastic Map Reduce to make the system grow elastically
  • Hands-on knowledge in using advanced SQL queries (analytical functions), experience in writing and optimizing highly efficient SQL queries
  • Proficiency with Python scripting
  • Experienced in testing and monitoring data for anomalies and rectifying them.
  • Advanced communication skills to be able to work with business owners to develop and define key business uses and to build data sets that address them.
  • Experience in working with Data visualization tools such a Tableau

 

   Professional Skills

These are the professional skills we would expect from an individual fully established in this role.

  • Customer Service - Advanced
  • Verbal Communication - Advanced
  • Written Communication - Advanced
  • Teamwork - Advanced
  • Relationships – proficient
  • Learning Agility - Expert
  • Analysis - Expert
  • Problem Solving - Expert
  • Process Orientation - Expert
  • Prioritization - Proficient

 

Qualifications

  • Bachelor’s degree or equivalent in an engineering or technical field such as Computer Science, Information Systems, Statistics, Engineering, or similar.
  • 5-10 years of quantitative and qualitative experience in building ETL data flows in Big Data Ecosystem.

 

­­­­­___________________________________________________________________________________

Please note, this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities and activities, and schedule may change at any time with or without notice.

 

SMS Assist is an Equal Opportunity Employer (EOE) that welcomes and encourages all applicants to apply regardless of age, race, color, religion, sex, sexual orientation, gender identify and/or expression, national origin, disability, veteran status, marital or parental status, ancestry, citizenship status, pregnancy or other reasons prohibited by law.

Read Full Job Description

Location

875 North Michigan Avenue , Chicago, IL 60611

What are SMS Assist Perks + Benefits

Health Insurance & Wellness Benefits
Flexible Spending Account (FSA)
Disability Insurance
Dental Benefits
Vision Benefits
Health Insurance Benefits
Life Insurance
Pet Insurance
Wellness Programs
Onsite Gym
Retirement & Stock Options Benefits
401(K)
Performance Bonus
Child Care & Parental Leave Benefits
Generous Parental Leave
Flexible Work Schedule
Family Medical Leave
Vacation & Time Off Benefits
Paid Volunteer Time
Paid Holidays
Paid Sick Days
Perks & Discounts
Casual Dress
Commuter Benefits
Company Outings
Professional Development Benefits
Job Training & Conferences

Additional Perks + Benefits

Competitive Salary. Flexible Spending Plans. Prescription Drug Program. Free Fitness Center membership.

More Jobs at SMS Assist24 open jobs
All Jobs
Data + Analytics
Dev + Engineer
HR
Internships
Marketing
Operations
Product
Project Mgmt
Operations
new
Chicago
Legal
new
Chicago
Developer
new
Chicago
Data + Analytics
new
Chicago
Data + Analytics
new
Chicago
Internships
new
Chicago
Project Mgmt
new
Chicago
Operations
new
Chicago
Operations
new
Chicago
Operations
new
Chicago
Operations
new
West Suburbs
Operations
new
Chicago
Product
new
Chicago
Data + Analytics
new
Chicago
Data + Analytics
new
Chicago
Data + Analytics
new
Chicago
Internships
new
Chicago
Project Mgmt
new
Chicago
Project Mgmt
new
Chicago
Marketing
new
Chicago