Lead Data Engineer at Discover
Discover. A brighter future.
With us, you’ll do meaningful work from Day 1. Our collaborative culture is built on three core behaviors: We Play to Win, We Get Better Every Day & We Succeed Together. And we mean it — we want you to grow and make a difference at one of the world's leading digital banking and payments companies. We value what makes you unique so that you have an opportunity to shine.
Come build your future, while being the reason millions of people find a brighter financial future with Discover.Job Description
The Lead Data Engineer participates in data solution design, strategy, including resolving strategic data integration issues, such as data quality and stewardship, real time or event-based data integration, and crafting a vision for data integration working with the architecture organization. The Lead position leads the technical design, development, integration, and testing efforts for ETL to create quality solutions that meet the business requirements. Other responsibilities include: mentoring other Data Engineers, preparing detailed and creating technical design plans
Responsible for working with data asset-related technology that has been approved by the architecture organization to move, transform, and validate data between systems, both internal to Discover Financial Services and with external data sources that provide value to Discover Financial Services. Responsible for writing extract, transform, and load (ETL) scripts, data maps, or integration programs to capture data from source systems and to transform it and load it into a destination system to enable data mining, business intelligence (BI) capabilities for the organization. Translates business functional detailed design requirements into technical solutions through the design, development, integration, and testing (unit and system) of ETL solutions.
Develops and maintains full stack solutions to fit business needs. Full stack solutions require one or more of the following: front-end (user interfaces), back-end (APIs), database and DevOps development. Works directly with business partners to understand business requirements. Works independently or with own team to innovate on and advocate for best practices within the team. Designs complex solutions and leads them from inception to production within the agile team.
- Optimize the performance of ETL processes and scripts by working with other technical staff as needed and document data solution and processes
- Designs and develops data ingestion frameworks leveraging open source tools and data processing frameworks.
- Designs and develops complex and critical data projects.
- Develops, implements, and supports application codes and analytical models.
- Creates and enhances data solutions that enable seamless integration and flow of data across the data ecosystem.
- Designs and develops real time processing solutions.
- Works with key stakeholders to design complex solutions and lead from inception top production
- Creates and maintains DevOps processes, application infrastructure, and utilizes cloud services (including database systems and models)
- Innovates on and advocates for best practices and improved team processes; mentors junior team members
- Supports live systems to ensure business continuity
- Lead data technology strategy, including resolving strategic data integration issues, such as data quality and stewardship, real time, event-based data integration, and crafting a universal vision for data integration, plan and document succession planning within scope of responsibility
- Design, develop, test, and implement data-driven solutions to meet business requirements
- Perform the technical design, development, integration, and testing efforts for data using various development languages and tools to create quality solutions that meet the business requirements
- Lead code review sessions & create technical design plans/STTs
Qualifications You’ll Need
- Bachelor’s degree in information technology, or related
- 4+ years of experience in Data Platform Administration/Engineering, or related
- 6 or more years of work experience in BI lifecycle, from strategy to ETL to report implementation
- Experience in an Agile-based environment
- Big Data stacks/ecosystem including Kafka, Python, Spark, NoSQL
- Expertise in the concepts, technology, and practices of building data solutions
- Expertise with open-source, ETL scripting and processes and related tools
- Expertise with the design and development of ETL data integration solutions
- Expertise with relational databases and experience with Cloud-based technologies
- Expertise in enabling business intelligence solutions through data integration
- Demonstrated experience with integration technologies and how to leverage them into data mapping between systems
- Demonstrated experience with transforming business needs into data design and solutions
- Extensive experience with writing and performance-tuning complex SQL queries
- Experience in agile process and technology
- Experience with Ab Initio or other ETL tools and components
- Experience with the following database and Cloud-based technologies:
- Teradata technologies or similar
- Python, Spark
- Knowledge with AWS, Snowflake and other Cloud-related technologies is a plus
- Strong team player with willingness to collaborate
- Strong analytical and problem solving skills
- Strong capability to execute tasks with quality
- Strong communication and decision-making skills
What are you waiting for? Apply today!
The same way we treat our employees is how we treat all applicants – with respect. Discover Financial Services is an equal opportunity employer (EEO is the law). We thrive on diversity & inclusion. You will be treated fairly throughout our recruiting process and without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status in consideration for a career at Discover.