Lead Data Engineer
Discover. A brighter future.
With us, you’ll do meaningful work from Day 1. Our collaborative culture is built on three core behaviors: We Play to Win, We Get Better Every Day & We Succeed Together. And we mean it — we want you to grow and make a difference at one of the world's leading digital banking and payments companies. We value what makes you unique so that you have an opportunity to shine.
Come build your future, while being the reason millions of people find a brighter financial future with Discover.
Job Description
At Discover, be part of a culture where diversity, teamwork and collaboration reign. Join a company that is just as employee-focused as its customers and is consistently awarded for both. We’re all about people, and our employees are why Discover is a great place to work. Be the reason we help millions of consumers build a brighter financial future and achieve yours along the way with a rewarding career.
As a Lead Data Engineer, you will provide technical (design & development) leadership to the Enterprise Technology Team in development of Extract/Transform/Load (ETL) applications that will interface with all key Discover applications.
Position requires excellent communication skills for understanding business vision and the ability to translate the vision into technical artifacts. Strong technical analysis and design background is also a must-have to ensure that technical deliverables are providing flexible, architecturally sound infrastructure with the ability to reuse in future Data Stores, analytical and operational in AWS.
Responsibilities
- Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.
- Ability to quickly identify an opportunity and recommend possible technical solutions.
- Develop application systems that comply with the standard system development methodology and concepts for design, programming, backup, and recovery to deliver solutions that have superior performance and integrity.
- Contribute to determining programming approach, tools, and techniques that best meet the business requirements.
- Understand and follow the PDP process to develop, deploy and deliver the solutions.
- Be pro-active and diligent in identifying and communicating design and development issues.
- Provide business analysis and develop ETL code and scripting to meet all technical specifications and business requirements according to the established designs.
- Offer system support as part of a support rotation with other team members.
- Utilize multiple development languages/tools such as Python, SPARK, Hive to build prototypes and evaluate results for effectiveness and feasibility.
- Operationalize open source data-analytic tools for enterprise use.
- Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, Python and AWS based solutions.
- Custom Data pipeline development (Cloud and locally hosted)
- Work heavily within the Cloud ecosystem and migrate data from Teradata to AWS based platform.
- Designs and develops data ingestion frameworks leveraging open source tools and data processing frameworks.
- Designs and develops complex and critical data projects.
- Develops, implements, and supports application codes and analytical models.
- Creates and enhances data solutions that enable seamless integration and flow of data across the data ecosystem.
- Designs and develops real time processing solutions
Qualifications You’ll Need
The Basics
- Bachelor's degree in Information Technology, or related field
- 4+ years of experience in data platform administration/engineering , or related
Bonus Points If You Have
- 6+ years of experience in data platform administration/engineering
- Experience in migrating ETL processes (not just data) from relational warehouse databases to AWS-based solutions
- Hands-on experience with Amazon Web Services (AWS)-based solutions such as Lambda, Dynamodb, Snowflake and S3
- Knowledge and experience using query languages (SQL, Cypher) for relational and graph databases
- Willingness to continuously learn & share learnings with others
- Capability to collaborate with stakeholders and project leaders to understand requirements, deliverables, and set expectations on tasks that you will be responsible for
- Ability to work in a fast-paced, rapidly changing environment
- Experience working in an agile and collaborative team environment
- Excellent written and verbal communication, presentation and professional speaking skills
- Passion for learning and interest in pursuing classroom training and self-discovery on a variety of emerging technologies
- Experience within the financial services industry
#Remote
#BI-Remote
#LI-KE
What are you waiting for? Apply today!
The same way we treat our employees is how we treat all applicants – with respect. Discover Financial Services is an equal opportunity employer (EEO is the law). We thrive on diversity & inclusion. You will be treated fairly throughout our recruiting process and without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status in consideration for a career at Discover.