Summary

The Big Data Engineer is responsible for building scalable data platforms, and large-scale processing systems that enable advanced analytics and support data teams across the enterprise. This hands-on role requires some experience working with AWS cloud platform in addition to expertise in a variety of technologies. The Big Data Engineer will develop and manage the enterprise data warehouse while sourcing data from various databases/applications & web APIs using stream and batch processing architectures.

Responsibilities

• Responsible for designing, building & managing the advanced analytics platform to support downstream data science and the business intelligence teams
• Work with the product owner, data scientist and various internal data users to understand the business requirements and implement optimal data solutions
• Keen eye towards configuration driven approach to automate repeatable processes and tasks
• Build and operate stable, scalable and highly performant data pipelines that cleanse, structure and integrate disparate big data sets into a readable and accessible format for end user analyses and targeting.
• Develop data quality and governance framework that supports data lineage and ensures delivery of high-quality data to internal and external stakeholders
• Using analytical and problem-solving skills to take complex business requests and transform them into clean, simple data solutions.
• Implement/improve version control, deployment strategies, notifications to ensure product quality, agility and recoverability.
• Understand and work with technology & IT teams to support database procedures, such as upgrade, backup, recovery, migrations, etc.

Role Specific Skills

• Designing, building and operationalizing large-scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, S3, EC2, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake etc.
• Hands-on experience analyzing, re-architecting and re-platforming on-premise data warehouses to data platforms on AWS cloud using AWS/3rd party services.
• Knowledge about other NoSQL databases, such as MongoDB, Cassandra, HBase, etc.
• Building and migrating the complex ETL pipelines on Redshift and Elastic Map Reduce to make the system grow elastically
• Hands-on knowledge in using advanced SQL queries (analytical functions), experience in writing and optimizing highly efficient SQL queries
• Proficiency with Python scripting
• Experienced in testing and monitoring data for anomalies and rectifying them.
• Advanced communication skills to be able to work with business owners to develop and define key business uses and to build data sets that address them.
• Experience in working with Data visualization tools such a Tableau

   Professional Skills

These are the professional skills we would expect from an individual fully established in this role.

• Customer Service - Advanced
• Verbal Communication - Advanced
• Written Communication - Advanced
• Teamwork - Advanced
• Relationships – proficient
• Learning Agility - Expert
• Analysis - Expert
• Problem Solving - Expert
• Process Orientation - Expert
• Prioritization - Proficient

Qualifications

• Bachelor’s degree or equivalent in an engineering or technical field such as Computer Science, Information Systems, Statistics, Engineering, or similar.
• 5-10 years of quantitative and qualitative experience in building ETL data flows in Big Data Ecosystem.

­­­­­___________________________________________________________________________________

Please note, this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities and activities, and schedule may change at any time with or without notice.

SMS Assist is an Equal Opportunity Employer (EOE) that welcomes and encourages all applicants to apply regardless of age, race, color, religion, sex, sexual orientation, gender identify and/or expression, national origin, disability, veteran status, marital or parental status, ancestry, citizenship status, pregnancy or other reasons prohibited by law.