Senior Data Engineer
Job Description
At Discover Financial Services you can make an impact. Whether it’s developing corporate strategy, innovating new services or supporting IT needs, every employee has the opportunity to be a vital part of our business and make a real difference. We’re all about people, and our employees are why Discover is a great place to work. We’re a company that is just as employee-focused as it is on its customers and we are consistently awarded for both.
We’re a direct banking and payment services company built on innovation and customer service. Our employees have always played a big part in our success. We support, challenge and inspire employees to continually develop their skills, advance their career and help grow our business.
Discover Financial Services is seeking a Senior Data engineer to join our Data Operations team. Successful candidates will partner with our business partners to understand their data needs, and build data pipelines using cutting edge technologies. Additionally, you will provide engineering expertise to create and enhance data solutions enabling seamless integration and flow of data across our data ecosystem. Projects will include designing and developing data ingestion and processing/transformation frameworks leveraging AWS and open source tools such as Java, Python, Spark, Scala, Hive, Python, etc.
Responsibilities:
In this role, you will have the following duties:
- Be part of Data Engineering team translating business and technology requirements into our ETL/ELT architecture.
- Collaborate with cross functional teams such as AWS cloud & platform engineering, DBAs and business teams.
- Design and develop real-time and batch data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, Kinesis, Java, NoSQL DBs, AWS EMR.
- Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.
- Ability to quickly identify an opportunity and recommend possible technical solutions.
- Develop Custom Data pipeline (Cloud and locally hosted).
- Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
- Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.
- Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
- Provide system support as part of a support rotation with other team members.
Minimum Qualifications
At a minimum, here’s what we need from you:
- Bachelor’s Degree in Information Technology, or related field
- 2+ years of experience in Data Platform Administration/Engineering
Preferred Qualifications
If we had our say, we’d also look for:
- Experienced in Agile/Lean/SAFe methodologies.
- Proficiency in one of the scripting languages such as Shell/Python/Scala/Java.
- Good understanding of Big Data technology trends, with knowledge of technologies such as Kinesis, Kafka, Spark, Hive, and Python.
- Experience with one of the ETL tools Abinitio, Informatica, Data Stage or Pentaho.
- Hands on experience with AWS based solutions such as RDS, Lambda, DynamoDB, Snowflake, Redshift or S3.
- Experience in building & utilizing tools and frameworks within the Big Data ecosystem including Hadoop, Kafka, Spark, HBase, NoSQL.
- Experience with relational databases such as MySQL or Postgres and understanding of columnar data stores such as Snowflake or Redshift, and expert skills in writing complicated SQL statements.
- Ability to work in a fast-paced, rapidly changing environment.
- Experience within the financial industry
- AWS Cloud Services certification 3+ years of experience of being a data engineer amongst a team of equal or junior level developers
- Collaborative individual who excels in working within a team and with business partners and identify, develop and deliver innovative data solutions.
- Strong ability to build and leverage external relationships.
- Ability to deliver presentations in a thorough and concise manner to a cross functional or senior level audience.
- Passionate learner who enjoys education through class room training and self-discovery on a variety of emerging technologies.
#LI-KE
The same way we treat our employees is how we treat all applicants – with respect. Discover Financial Services is an equal opportunity employer (EEO is the law). We thrive on diversity & inclusion. You will be treated fairly throughout our recruiting process and without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status in consideration for a career at Discover.