Senior Data Engineer

| Chicago
Apply Now
By clicking continue you agree to Built In’s Privacy Policy and Terms of Use.
Company Description
Job Description
Epsilon is the leader in outcome-based marketing. We enable marketing that's built on proof, not promises. Through Epsilon PeopleCloud, the marketing platform for personalizing consumer journeys with performance transparency, Epsilon helps marketers anticipate, activate and prove measurable business outcomes. Powered by CORE ID®, the most accurate and stable identity management platform representing 200+ million people, Epsilon's award-winning data and technology is rooted in privacy by design and underpinned by powerful AI. With more than 50 years of experience in personalization and performance working with the world's top brands, agencies and publishers, Epsilon is a trusted partner leading CRM, digital media, loyalty and email programs. Positioned at the core of Publicis Groupe, Epsilon is a global company with over 8,000 employees in over 40 offices around the world.
For more information, visit epsilon.com. Follow us on Twitter at @EpsilonMktg.
Love cutting-edge tech? We do too.
At Epsilon, we do more than collect and store data. We help some of the world's biggest brands discover real opportunities inside the data types, delimiters and decimals. At Epsilon, we can analyze anonymized, privacy-safe data at internet scale and handle 300B+ of online interactions a day and have $3.8 trillion in multichannel purchases in our database.
We are looking for a Senior Data Engineer with extensive experience in the big data ecosystem to work in our Data Pipeline team. As part of the Epsilon DMS Data Organization, the Data Pipeline team's responsibilities include but is not limited to maintaining streaming and batched data pipelines collecting hundreds of billions of data rows daily, near real time aggregation of a variety of data sets (using Spark Structured Streaming) and maintaining system of record raw and aggregate data sets (HDFS and HIVE). Additionally, as part of the Data Pipeline team work closely with Real Time Bidding (RTB), data warehousing, ETL and decision science teams in building, maintaining and optimizing solutions.
The candidate must be proficient in Scala or Java, with the ability to be a key contributor in building and maintaining Spark jobs and AirFlow DAGs. This role will be hands on in code development and data architecture to optimize the platform for growth and maximum efficiency. The person in this role will need to be able to work both independently and as part of a team to meet required specifications of solution delivery.
Responsibilities
  • You will design and code data pipelines connecting Real Time Bidding platform, HDFS, Elastic, MPP and Postgres databases utilizing Flume, Kafka, Spark, Cassandra/Scylla and other technologies as needed.
  • Maintain and extend Spark framework used by DMS Data Organization to support various aspects of the business.
  • Build and maintain AirFlow DAGs for job management.
  • Work closely with infrastructure teams in capacity planning, hardware procurement and build outs.
  • Build and maintain metrics collection to help with identifying production issues, optimizing job performance and alerting on error conditions (Pager Duty).
  • Develop test cases to demonstrate new code meets functional requirements.
  • Ideal candidate can lead in one or more areas of: design, code development, data modeling, cross team communication, application maintenance and identifying opportunities for improving code quality or performance through refactoring and/or incorporating new technologies.

Required qualifications
  • Bachelor's Degree in Computer Science or equivalent degree is required.
  • 4+ years of experience developing in Java and/or Scala
  • Spark experience a big plus
  • Strong experience in SQL, bash and Python
  • Experience with Hadoop Stack (HDFS, Hive, YARN, HBase)
  • Experience with Kafka and the Kafka producer and consumer APIs (Kafka Connect, K-streams and ksql a plus)
  • Experience with Postgres and Cassandra/Scylla a big plus
  • Docker and Kubernetes experience a big plus
  • ELK stack experience a plus
  • Experience with scheduling applications with complex interdependencies
  • Good experience in working with geographically and culturally diverse teams
  • Excellent written and verbal communication skills
  • Excellent analytical and problem-solving skills
  • Ability to diagnose and troubleshoot problems quickly

Qualifications
Additional Information
When you're one of us, you get to run with the best.
For decades, we've been helping marketers from the world's top brands personalize experiences for millions of people with our cutting-edge technology, solutions and services.
Thanks to the work of every employee, Epsilon products and services have been recognized as industry-leading by the Forrester Wave™. We process more than 500 billion consumer actions each day and hold many patents of proprietary technology, including real-time modeling languages and consumer privacy advancements. And our CORE ID® is the industry's most accurate, stable and scalable identity solution that helps brands recognize and reach 200M+ U.S. consumers in a privacy-safe way.
Positioned at the core of Publicis Groupe, Epsilon is a global company with more than 8,000 employees around the world. For more information, visit epsilon.com
Great People Deserve Great Benefits
We know that we have some of the brightest and most talented associates in the world, and we believe in rewarding them accordingly. If you work here, expect competitive pay, comprehensive health coverage, and endless opportunities to advance your career.
Epsilon is an Equal Opportunity Employer. Epsilon's policy is not to discriminate against any applicant or employee based on actual or perceived race, age, sex or gender (including pregnancy), marital status, national origin, ancestry, citizenship status, mental or physical disability, religion, creed, color, sexual orientation, gender identity or expression (including transgender status), veteran status, genetic information, or any other characteristic protected by applicable federal, state or local law. Epsilon also prohibits harassment of applicants and employees based on any of these protected categories.
Epsilon will provide accommodations to applicants needing accommodations to complete the application process.
#LI-AM1
REF134146K
Read Full Job Description
Apply Now
By clicking continue you agree to Built In’s Privacy Policy and Terms of Use.

Technology we use

  • Engineering
    • C#Languages
    • C++Languages
    • JavaLanguages
    • JavascriptLanguages
    • PerlLanguages
    • PHPLanguages
    • PythonLanguages
    • RLanguages
    • ScalaLanguages
    • SqlLanguages
    • MapRLanguages
    • D3JSLibraries
    • jQueryLibraries
    • ReactLibraries
    • Juptyer NotebookLibraries
    • AngularJSFrameworks
    • Backbone.jsFrameworks
    • CakePHPFrameworks
    • Ember.jsFrameworks
    • HadoopFrameworks
    • LaravelFrameworks
    • Node.jsFrameworks
    • SparkFrameworks
    • TensorFlowFrameworks
    • HbaseFrameworks
    • Kafka Frameworks
    • StormFrameworks
    • CassandraDatabases
    • HBaseDatabases
    • HiveDatabases
    • Microsoft SQL ServerDatabases
    • MySQLDatabases
    • OracleDatabases
    • PostgreSQLDatabases
    • GreenplumDatabases
    • NoSQLDatabases

Location

101 N Wacker Drive, Chicago, IL 60606

An Insider's view of Epsilon

What's something quirky about your company?

We have a company band, made up of anywhere from 30 to 40 employees at a given time. The band has a large following and performs at charity events around Chicago. They also perform a full set at Epsilon’s holiday party each year at the House of Blues. My jaw hit the floor when I saw my peers up on stage rocking out.

Paul

Associate Automation & Infrastructure Engineer

How do you collaborate with other teams in the company?

We collaborate through storytelling. It’s not ‘how’ we build it, but rather ‘why’ we build it. We need to tell stakeholders a story about a consumer pain point, solving it in a differentiated manner and the larger vision. Then, collaboration occurs as we incorporate feedback and have a motivated team building towards a shared vision.

Matt

VP, Product Management:

How would you describe the company’s work-life balance?

Conversant has a very good work-life balance, including unlimited PTO. As a young adult, traveling around the world is one my top priorities and working with a company that aligns with that is a huge plus. I have had the flexibility to visit 7 countries. That flexibility is one of many things that has kept me at Conversant for five years.

Alba

Team Lead, Engineering

What are Epsilon Perks + Benefits

Epsilon Benefits Overview

At Epsilon, we offer benefits and perks that enable our associates to thrive and foster a culture of innovation.

Culture
Volunteer in local community
Epsilon participates in local community activities such as the Greater Chicago Food Depository, Off the Street Club, Chicago Cares Serve-A-Thon, Habitat for Humanity and Ronald McDonald House.
Partners with Nonprofits
Friends outside of work
Eat lunch together
Intracompany committees
Epsilon has a Social Committee that organizes social, charitable and sports activities.
Daily sync
Open door policy
Group brainstorming sessions
Open office floor plan
Diversity
Dedicated Diversity/Inclusion Staff
Unconscious bias training
Health Insurance & Wellness Benefits
Flexible Spending Account (FSA)
Disability Insurance
Dental Benefits
Vision Benefits
Health Insurance Benefits
Life Insurance
Pet Insurance
Wellness Programs
Onsite Gym
Team workouts
Epsilon team fitness initiatives include intramural softball, volleyball and soccer.
Retirement & Stock Options Benefits
401(K)
401(K) Matching
Performance Bonus
Match charitable contributions
Child Care & Parental Leave Benefits
Child Care Benefits
Epsilon provides back-up care benefits.
Flexible Work Schedule
What matters is what you achieve, not how many hours you clock. So take time off to enjoy your family and friends when you need it.
Remote Work Program
Family Medical Leave
Adoption Assistance
Vacation & Time Off Benefits
Unlimited Vacation Policy
Generous PTO
Paid Volunteer Time
Paid Holidays
Paid Sick Days
Perks & Discounts
Beer on Tap
Casual Dress
Commuter Benefits
Company Outings
Epsilon hosts an annual summer boat party on Lake Michigan and a holiday party at the House of Blues, where our employee-run band performs the live entertainment all night.
Game Room
Happy Hours
Happy hours are hosted at team's discretion.
Parking
Recreational Clubs
Fitness Subsidies
Professional Development Benefits
Job Training & Conferences
Epsilon offers employees professional development opportunities like on-site training courses and the ability to attend job-related conferences and seminars.
Tuition Reimbursement
Lunch and learns
Epsilon hosts lunch and learn meetings on occasion.
Promote from within
Mentorship program
Our mentorship program includes 1-to-1 program and career mentoring.
Time allotted for learning
Epsilon encourages employees to complete 48 hours of professional development training per year.
Online course subscriptions available

More Jobs at Epsilon

Apply Now
By clicking continue you agree to Built In’s Privacy Policy and Terms of Use.
Save jobView Epsilon's full profileSee more Epsilon jobs