Senior Manager Data Engineering
At Discover, be part of a culture where diversity, teamwork and collaboration reign. Join a company that is just as employee-focused as it is on its customers and is consistently awarded for both. We’re all about people, and our employees are why Discover is a great place to work. Be the reason we help millions of consumers build a brighter financial future and achieve yours along the way with a rewarding career.
The Enterprise Payments Platform (EPP) team at Discover is looking for a Sr. Manager, Data Engineering. In this role you are responsible for managing a team that oversees and executes the Engineering and Administration of the Hadoop Big Data Platform & integration, ensures overall resiliency, release management, and ongoing operations of applications hosted on the Hadoop Clusters. You will lead a global team dedicated to pushing the limits of continuous improvement and innovation. If you are equally adept at managing and mentoring engineers, architecting and implementing cutting edge technology solutions, leading through change, and managing large scale production operations, this role is for you.
To be successful (and we know you can be), you will need to have a strong IT understanding with work experience in off- and on-premise cloud based and virtual system infrastructure and peripheral services including network, firewall, and database management. We also need you to understand the application development and quality assurance ends of the spectrum as you will need to interface with that crew as well. During problem escalations you are the driver of the team that finds the root cause, restores functionality, and proposes the long term solution.
Responsible for managing a team that delivers data-driven platforms and analytical technologies.
Responsibilities
- Develops and enhances technology solutions to enable seamless integration and flow of data across the data ecosystem.
- Designs and develops data ingestion and processes frameworks leveraging open-source tools.
- Designs and develops real-time processing solutions using open-source tools.
- Develops and manages a team that delivers data and analytic technologies.
- Consults team members during the design and development of projects.
- Lead a highly focused team and work in a fast paced agile environment who will be responsible for daily operations, management, maintenance, production support and triage of the Horton Works Hadoop – Enterprise Payments Data Platform.
- Hadoop administrator using Hortonworks, Administers Hadoop technology and systems responsible for backup, recovery, architecture, performance tuning, security, auditing, metadata management, optimization, statistics, capacity planning, connectivity and other data solutions of Hadoop systems
- Responsible for installation and ongoing administration of Hadoop infrastructure.
- Propose and deploy new hardware and software environments required for Hadoop and to expand existing environments as the need arises. Provision new users, groups and roles and setting up Kerberos principals.
- Ensure the High Availability of Hadoop clusters. Responsible for data movement in and out of Hadoop clusters.
- Addition and deletion of nodes from Hadoop clusters. Monitoring Hadoop eco-system. Log Files and file systems management. HDFS, storage management and capacity planning.
- In depth working knowledge and hands on experience with and not limited to Hbase, SPARK, Kafka, HDFS, Ranger, Ambari & Grafana etc.
- Leverage DevOps techniques and practices like CI/CD, Test Automation, Build Automation and Test Driven Development to enable the rapid delivery of working code utilizing tools like Jenkins, Maven, Nexus, Chef, Ansible, Git, and Docker
- Utilize tools and programming languages like Ab Initio, Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud-based data warehousing services such as Snowflake and Redshift
- Raise the bar and lead a team that delivers results that matter most to our company
- Expertly balance delivery innovation with operational excellence in a data driven way
- Be a force for technological and organizational change as we push to differentiate our company with data
Minimum Qualifications
At a minimum, here’s what we need from you:
- Bachelor’s Degree in Computer Science, or related field
- 6+ years of experience in Data Platform Administration/Engineering
Preferred Qualifications
If we had our say, we’d also look for:
- 8+ years of experience in Data Platform Administration/Engineering( (Hortonworks and/or Cloudera)
- Proven ability to champion an idea and bring a group to consensus
- Proven ability to translate functional needs into technical solutions
- Excellent problem-solving skills involving complex and ambiguous issues
- Experience work in an Agile Scrum delivery framework
- At least 3 years of experience managing tactical delivery teams
Discover Financial Services is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran status, among other things, or as a qualified individual with a disability.
So, what are you waiting for? Apply today!