Principal Data Engineer
At Discover, be part of a culture where diversity, teamwork and collaboration reign. Join a company that is just as employee-focused as it is on its customers and is consistently awarded for both. We’re all about people, and our employees are why Discover is a great place to work. Be the reason we help millions of consumers build a brighter financial future and achieve yours along the way with a rewarding career.
Job Description
At Discover, be part of a culture where diversity, teamwork and collaboration reign. Join a company that is just as employee-focused as it is on its customers and is consistently awarded for both. We’re all about people, and our employees are why Discover is a great place to work. Be the reason we help millions of consumers build a brighter financial future and achieve yours along the way with a rewarding career.
The Customer Engagement value stream at Discover is looking for an experienced Principal Data Platform Engineer. We are modernizing our Marketing Technology and our foundation is customer data. On this team you will lead and collaborate transforming our customer data platform. The transformation will enable real-time data which provides Discover the capability to have relevant, timely, and personalized conversations with our customers.
The candidate will be involved and working with our Agile delivery teams. This includes Product Managers, Product Owners, Scrum Masters, Security, Data Analytics, and Infrastructure teams to drive data transformation. The candidate must be capable of demonstrating advanced and strong Thought and Results Leadership skills, with a view that supports the vision and mission of our team, department, and company.
The Principal Data Platform Engineer will lead our data needs with other project teams and mentor technical staff members on data best practices. The candidate will understand the data frequency requirements (batch, streaming, real-time, near real-time) and employ the right technologies to accomplish the goals.
Responsible for using experience and hands-on approach with next-generation technologies to contribute to the team that delivers the latest data-driven platforms and next-generation analytic technologies.
Responsibilities
- Drives optimal data architecture that efficiently and effectively presents an integrated view of customer marketing and service data
- Defines and articulates the purpose for each data layer in our architecture (e.g. ODS, EDW, Customer Data Platform, Data Marts)
- Documents current state data flows covering interfaces, data stores, transport technologies and the source and destination of data
- Creates target state data flows, alignment to reference architecture, alignment to sources of truth master data
- Responsible for understanding vendor service-level requirements such as availability, data volume, and performance
- Responsible for leading data governance, data privacy, and data security in scope data
- Provide guidance to application development areas by applying knowledge of data warehouse, extract-transform-load (ETL) design and development
- Support the creation and implementation of data strategy and future state architecture
- Determine approach and methods to expose customer marketing and customer service data to other applications and data systems
- Builds and leads a high-performing, Agile team focused on next-generation data and analytic technologies.
- Provides senior-level technical consulting to create and enhance analytic platforms and tools that enable state of the art, next-generation Big Data capabilities to analytic users and applications.
- Provides senior-level technical consulting to application development teams during application design and development for highly complex and critical data projects.
- Be part of teams delivering all data projects, including migration to new data technologies for unstructured, streaming, and high-volume data.
- Develops and deploys distributed computing Big Data applications using Open Source frameworks.
- Utilizes programming languages like Java, Python, and NoSQL databases like Cassandra.
- Utilizes Hadoop modules such as YARN & MapReduce, and related Apache projects such as Hive, Hbase, Pig.
- Leverages DevOps techniques and practices like Continuous Integration, Continuous Deployment, Test Automation, Build Automation, and Test Driven Development to enable the rapid delivery of end-user capabilities.
- Builds and administers large scale data platforms such as Hadoop Data Lake, integrating next-gen data analytic tools into the Big Data ecosystem. Expertise in real-time data technologies.
Minimum Qualifications
At a minimum, here’s what we need from you:
- Bachelor's Degree in Computer Science or related field
- 4+ years of experience in Data Platform Administration, Engineering, or related field
- In lieu of a degree, 6+ years of experience in Data Platform Administration, Engineering, or related field
Preferred Qualifications
If we had our say, we’d also look for:
- 7+ years of relevant hands-on data architecture experience
- 2+ years of relevant hands-on exposure to building data and analytics solutions on AWS
- Experience with one of the ETL tools (Abinitio, Informatica or Data Stage) and tools/frameworks within the Big Data ecosystem
- Solid understanding of Big Data technology trends, with knowledge of technologies such as Kinesis, Kafka, Spark, Hive, pySpark.
- Experience with variety of databases e.g. Snowflake, Teradata, Oracle
- Built architectural patterns at scale, including thoughtfully-designed data solutions, repeatable data pipelines, and efficient software engineering principles
- Experience with MarTech and/or Customer Data Platforms and the integration of these tools
Leadership skills:
- Strong analytical and problem solving skills
- Strong collaboration skills, responsive to service needs
- Passion for learning
- Strong collaborator
- Self-motivated
What are you waiting for? Apply today!
The same way we treat our employees is how we treat all applicants – with respect. Discover Financial Services is an equal opportunity employer (EEO is the law). We thrive on diversity & inclusion. You will be treated fairly throughout our recruiting process and without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status in consideration for a career at Discover.