Master Data Engineer

Discover. A more rewarding way to work.At Discover Financial Services, you’ll find yourself in the company of some of the industry’s smartest and most reliable professionals. And at a company that rewards dedication, values innovation and supports growth.Thrive in an environment that promotes teamwork and shared success. Build on a foundation of mutual respect. Join the company that understands rewarding careers like no other, with this exceptional opportunity: Job Description:
As a Master Data Engineer you will provide engineering leadership to create and enhance data solutions enabling seamless integration and flow of data across our data ecosystem which includes AWS. Additionally you will provide senior level technical consulting to peer data engineers during design and development for highly complex and critical data projects. Some of these projects will include designing and developing data ingestion and processing/transformation frameworks leveraging open source tools such as NiFi, Lambda, Step Functions, Java, Python, Spark, etc. Additionally, you will work on real time processing solutions using tools such as Flink, Storm, Kafka, and Kinesis. You will deploy application code using CI/CD tools and techniques and provide support for deployed data applications and analytical models.

This role will sit in our state of the art, downtown Chicago location at 350 N. Orleans. Responsibilities:

  • Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.
  • Ability to quickly identify an opportunity and recommend possible technical solutions.
  • Utilize multiple development languages/tools such as Python, SPARK, HBase, Hive, Microsoft R, Java to build prototypes and evaluate results for effectiveness and feasibility.
  • Work heavily within the AWS ecosystem, using AWS services
  • Operationalize open source data-analytic tools for enterprise use.
  • Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, NIFI, Python, HBase, Kinesis, and EMR.
  • Custom Data pipeline development (Cloud and locally hosted)
  • Work heavily within the Hadoop ecosystem and migrate data from Teradata to Hadoop.
  • Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
  • Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.
  • Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
  • Promote a risk-aware culture, ensure efficient and effective risk and compliance management practices by adhering to required standards and processes.

Skills:
Skills Required:

  • AWS data services (Lambda,Glue, EMR, Kinesis, Step Functions, Data Pipeline)
  • Deep understanding of the Hadoop technology stack
  • Building custom NiFI processors
  • Data pipeline development
  • Experience in developing Python / R applications
  • Spark application coding in Scala / Python (pySpark)
  • Deep knowledge and very strong in SQL, and Relational Databases
  • Strong in Unix / Shell scripting
  • Experience in creating very efficient HiveQL and SparkQL queries and can educate peers on the topic
  • Bachelors degree in Computer Science, related field, or equivalent work experience

Leadership Skills:

  • 7+ years of experience of being a lead engineer and able to coach/provide guidance to peer and junior engineers.
  • Excellent written and verbal communication, presentation and professional speaking skills
  • Collaborative individual who excels in working within a team and with business partners to identify, develop and deliver innovative data solutions
  • Influencing skills/ability. Must be able to work with effectively with different levels of management and all business areas.
  • Ability to demonstrate leadership to managers, and peer level staff.
  • Ability to build and leverage external relationships.
  • Decision making abilities while gathering information and then put your decisions into action.
  • Passionate learner who enjoys education through class room training and self-discovery on a variety of emerging technologies

#LI-KE We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race, color, sex, age, national origin, religion, sexual orientation, gender identity, status as a veteran, and basis of disability or any other federal, state or local protected class.

Read Full Job Description

Technology we use

  • Engineering
  • Product
  • Sales & Marketing
    • JavaLanguages
    • PythonLanguages
    • ScalaLanguages
    • SqlLanguages
    • AngularJSFrameworks
    • HadoopFrameworks
    • SparkFrameworks
    • SpringFrameworks
    • CassandraDatabases
    • HiveDatabases
    • Microsoft SQL ServerDatabases
    • MongoDBDatabases
    • OracleDatabases
    • TeradataDatabases
    • Google AnalyticsAnalytics
    • IllustratorDesign
    • PhotoshopDesign
    • AgileManagement
    • SafeManagement
    • LeanManagement
    • SalesforceCRM

Location

We also have a downtown Chicago which is known as the“The 606”, which is a flexible contemporary work space for employees.
2500 Lake Cook Road, Riverwoods, IL 60015

What are Discover Financial Services Perks + Benefits

Discover Financial Services Benefits Overview

Start enjoying great benefits Day 1 — We support you with the same dedication we bring to all of our customers.Our comprehensive benefits package features first-class insurance, financial planning support and excellent perks designed to help you reach your goals and live a rich, healthy life.

Check out more of our amazing employee benefits at mydiscoverbenefits.com

Health Insurance & Wellness Benefits
Flexible Spending Account (FSA)
You can open a separate Health Care FSA (HCFSA) and contribute up to $2,650 tax-free from your paycheck to reimburse yourself for eligible out-of-pocket expenses.
Disability Insurance
Employees receive Short-Term Disability Insurance at no cost.
Dental Benefits
Discover offers two dental plan options — Standard and Premier — both are administered by MetLife.
Vision Benefits
Discover offers two vision plan options — Standard and Premier through VSP.
Health Insurance Benefits
Discover offers a variety of medical plans for you and eligible family members, so that you can choose the benefit plan that suits your needs.
Life Insurance
As a Discover employee, you receive Basic Life Insurance of one times your HWEE (up to $500,000) at no cost to you.
Pet Insurance
Purchase medical coverage at a discounted rate for your beloved family pet. The more pets you insure, the greater the discount.
Wellness Programs
Help balance your work and personal life with a wide variety of free and discounted resource and referral services including family and relationship counseling and financial guidance.
Onsite Gym
Discover has fitness centers and Weight Watchers® programs at all five major locations.
Retirement & Stock Options Benefits
401(K)
You may elect to contribute 1% to 30% of your eligible base salary, commissions and bonus on a pre-tax basis, up to IRS limits every year.
401(K) Matching
Discover matches up to 6% of the pre-tax contributions you make to the 401(k) Plan.
Employee Stock Purchase Plan
The ESPP provides eligible employees with an opportunity to purchase shares of Discover common stock through payroll deductions at a 5% discount.
Performance Bonus
Child Care & Parental Leave Benefits
Generous Parental Leave
Flexible Work Schedule
Remote Work Program
Family Medical Leave
Adoption Assistance
Discover helps eligible employees and their families with the costs of adoption by reimbursing certain expenses.
Vacation & Time Off Benefits
Generous PTO
Discover has a Paid Time Off of 4 to 5 Weeks of paid time per year.
Paid Volunteer Time
Paid Holidays
Discover provides 7 paid holidays.
Perks & Discounts
Casual Dress
Commuter Benefits
When you enroll in the Commuter Benefits Program at WageWorks, you’ll save on taxes on mass-transit passes, parking and other eligible expenses.
Company Outings
Game Room
Parking
Recreational Clubs
Relocation Assistance
Professional Development Benefits
Job Training & Conferences
Tuition Reimbursement
Discover provides tuition reimbursement and a full-ride bachelor's degree program for select online degree programs.
Diversity Program
More Jobs at Discover Financial Services53 open jobs
All Jobs
Finance
Data + Analytics
Dev + Engineer
Operations
Project Mgmt
Developer
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Operations
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Operations
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Operations
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Operations
new
Discover Financial Services
Operations
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Developer
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Finance
new
Discover Financial Services
Operations
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Developer
new
Discover Financial Services
Operations
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Developer
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Project Mgmt
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Data + Analytics
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services
Developer
new
Discover Financial Services