Lead Data Platform Engineer
Job Description
Do you want to be on the forefront of the next BIG thing in advancing the analytic technology landscape? Are you driven by the thought of being involved at the ground level of an enterprise wide transformation? Do you enjoy solving complex business problems in a fast paced, collaborative environment?
Discover Financial Services is seeking a bright, driven, and talented individual to join a team of passionate and innovative technologists.
*** THIS POSITION WILL BE BASED IN OUR HEADQUARTERS IN RIVERWOODS ILLINOIS WITH THE FLEXIBILITY OF WORKING A FEW DAYS A WEEK FROM OUR DOWNTOWN CHICAGO OFFICE ***
In this role, you will use your experience and hands on approach with next generation technologies to contribute to a team that delivers the latest streaming & next generation data technologies.
Responsibilities:
- Contributing member of a high performing, agile team focused on next generation data & analytic technologies.
- Create and enhance analytic platforms & tools that enable state of the art, next generation Big Data capabilities to analytic users and applications.
- Assist application development teams during application design and development for highly complex and critical data projects.
- Code and integrate open source solutions into the data analytic ecosystem.
- Develop fast prototype solutions by integrating various open source components.
- Operationalization of open source data analytic tools for enterprise use.
- Be part of teams delivering all data projects including migration to new data technologies for unstructured, streaming and high volume data.
- Building efficient data platform for structured and unstructured data.
- Developing and deploying distributed computing Big Data applications using Open Source frameworks like Apache Spark, Apex, Flink, Storm and Kafka
- Utilizing programming languages like Java, Spark, Python with an emphasis in tuning, optimization and best practices for application developers
- Provide solutions for data engineers such as NIFI processors and workflows
- Developing data management and governance tools on an open source framework
- Leveraging DevOps techniques and practices like Continuous Integration, Continuous Deployment, Test Automation, Build Automation and Test Driven Development to enable the rapid delivery of end user capabilities
- Developing data solutions on cloud deployments such as AWS
- Industry experience in Financial Services or other regulated industries
- Hands on experience leading delivery through Agile methodologies
- Management practices by adhering to required standards and processes
Minimum Qualifications
At a minimum, here’s what we need from you:
- Bachelor’s Degree in Computer Science, or related field
- 5+ years of experience in Data Platform Administration, Engineering, or related field
- In lieu of a degree, 6+ years of experience in Data Platform Administration, Engineering, or related field
Preferred Qualifications
If we had our say, we’d also look for:
- 1+ years Hadoop administration
- 1+ years experience with various tools and frameworks that enable capabilities within the data ecosystem (Hadoop, Kafka, NIFI, Hive, YARN, HBase, NoSQL)
- Experience developing software solutions to build out capabilities on a Big Data and other Enterprise Data Platforms
- Experience with solving problems and delivering high quality results in a fast paced environment
- Experience with NIFI specifically with developing custom processors and workflows
- Experience managing NOSQL databases such HBASE , Casandra, MongoDB
- Expertise in coding in Python, Hive, R, Spark, Java with emphasis on tuning / optimization
- Experience developing data solutions on AWS
- Experience designing, developing, and implementing ETL and relational database platforms
- Experience working with automated build and continuous integration systems (Chef, Jenkins, Docker)
- Experience managing code on Github
- Experience with data mining, machine learning, statistical modeling tools or underlying algorithms
#LI-KE
The same way we treat our employees is how we treat all applicants – with respect. Discover Financial Services is an equal opportunity employer (EEO is the law). We thrive on diversity & inclusion. You will be treated fairly throughout our recruiting process and without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status in consideration for a career at Discover.