Lead Data Engineer
Job Description
At Discover, be part of a culture where diversity, teamwork and collaboration reign. Join a company that is just as employee-focused as it is on its customers and is consistently awarded for both. We’re all about people, and our employees are why Discover is a great place to work. Be the reason we help millions of consumers build a brighter financial future and achieve yours along the way with a rewarding career.
Do you enjoy developing highly available, low latent micro-services that interact with large data sets? Are you interested in helping advanced analytics propel the business forward? Are you interested in learning about data sciences libraries like H20 and others?
If you answered Yes to these questions, you might be a great addition to our team! You’ll be working in the Enterprise Decision Technology organization to bring forth solutions for the data science teams to build, integrate and productize real time and batch applications for all product lines including card and bank. You’ll be working on a dynamic team who is constantly flexing to meet the needs of the business and next generation cloud technologies.
You’ll be an advocate for different technologies within the cloud, preferably AWS, and know when to use certain technologies to efficiently build solutions.
Responsibilities
- Designs and develops complex and critical data projects.
- Designs and develops real-time processing solutions.
- Creates and enhances data solutions that enable seamless integration and flow of data across the data ecosystem.
- Designs and develops data ingestion frameworks, leveraging open-source tools and data-processing frameworks.
- Develops, implements, and supports application codes and analytical models.
- Develop, coach and mentor other software engineers on the best practices providing domain and technical best practice guidance.
- Own the run time performance of your platform ensuring it’s behaving as expected.
- Technical point of contact for product teams as it relates to automation, CI/CD, and DevOps.
- Developing secure design patterns for analytics architectures deployed in the public cloud and/or orchestrated container environments using a ‘security first’ mindset.
- Build automation to actively audit the infrastructure for misconfigurations.
- Develop container image lifecycle management systems.
- Ability to adapt communication for effectiveness with business partners and other technical teams.
- In a team setting with other DevOps engineers, design and build tools and automation scripts that enable developers, data scientists and business analysts to easily consume cloud-native services.
Minimum Qualifications
At a minimum, here’s what we need from you:
- Bachelor’s Degree in Computer Science, or related field
- 4+ years of experience in Data Platform Administration/Engineering
Preferred Qualifications
If we had our say, we’d also look for:
- 5+ years’ experience developing services in Java, Python or Scala in a highly available low latent environment using proper design patterns.
- Experience in working in a cloud environment such as AWS, GCP or Azure.
- Working familiarity of Docker and other containerization technologies providing horizontal scaling such as Kubernetes or Openshift.
- Knowledge of working with CI/CD pipelines to ensure automated build, testing, and deployment using Gradle, Jenkins, Nexus and SonarQube.
- Understanding of messaging systems like MQ, Rabbit MQ, Kafka, or Kinesis.
- Experience unit and service level tests to ensure adequate code coverage.
- Experience with relational databases such as MySQL or Postgres and understanding of columnar data stores such as Redshift or Snowflake.
#LI-KE
Discover Financial Services is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran status, among other things, or as a qualified individual with a disability.