Senior Data Engineer
Discover. A more rewarding way to work.
At Discover Financial Services, you’ll find yourself in the company of some of the industry’s smartest and most reliable professionals. And at a company that rewards dedication, values innovation and supports growth.
Thrive in an environment that promotes teamwork and shared success. Build on a foundation of mutual respect. Join the company that understands rewarding careers like no other, with this exceptional opportunity:
Job Description:
As a Senior Data Engineer, you will provide technical (design & development) leadership to the Enterprise Data Warehouse Team in development of Extract/Transform/Load (ETL) applications that will interface with all key Discover applications.
Position requires excellent communication skills for understanding business vision and the ability to translate the vision into technical artifacts. Strong technical analysis and design background is also a must-have to ensure that technical deliverables are providing flexible, architecturally sound infrastructure with the ability to reuse in future Data Stores, analytical and operational in AWS.
Job Responsibilities:
- Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.
- Ability to quickly identify an opportunity and recommend possible technical solutions.
- Develop application systems that comply with the standard system development methodology and concepts for design, programming, backup, and recovery to deliver solutions that have superior performance and integrity.
- Contribute to determining programming approach, tools, and techniques that best meet the business requirements.
- Understand and follow the PDP process to develop, deploy and deliver the solutions.
- Be pro-active and diligent in identifying and communicating design and development issues.
- Provide business analysis and develop ETL code and scripting to meet all technical specifications and business requirements according to the established designs.
- Offer system support as part of a support rotation with other team members.
- Utilize multiple development languages/tools such as Python, SPARK, Hive to build prototypes and evaluate results for effectiveness and feasibility.
- Operationalize open source data-analytic tools for enterprise use.
- Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, Python and AWS based solutions.
- Custom Data pipeline development (Cloud and locally hosted)
- Work heavily within the Cloud ecosystem and migrate data from Teradata to AWS based platform.
- Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
- Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.
- Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
Technical Skills / Experience
- Hands on experience with Amazon Web Services (AWS) based solutions such as Lambda, Dynamodb, Snowflake and S3.
- Experience in migrating ETL processes (not just data) from relational warehouse Databases to AWS based solutions.
- Experience in building & utilizing tools and frameworks within the Big Data ecosystem including Hadoop, Kafka, Hive, Spark, HBase, NoSQL.
- Deep knowledge and very strong in SQL, and Relational Databases
- Knowledge of Data Warehouse technology (Unix/Teradata/Ab Initio)
- Willingness to continuously learn & share learnings with others
- Ability to work in a fast-paced, rapidly changing environment
- Very strong verbal & written communication skills
- Experience within the Financial industry
Leadership Skills
- 4+ years of experience of being a lead engineer amongst a team of equal or junior level developers
- Excellent written and verbal communication, presentation and professional speaking skills
- Collaborative individual who comfortable working on agile-based teams, excels in working with business partners identify, develop and deliver innovative data solutions
- Capable of collaborating with stakeholders and project leaders to understand requirements, deliverables, and set expectations on tasks that you will be responsible for.
- Ability to demonstrate leadership to managers, and peer level staff.
- Strong ability to build and leverage external relationships.
- Decision making abilities while gathering information and then put your decisions into action.
- Passionate learner who enjoys education through class room training and self-discovery on a variety of emerging technologies
- Promote a risk-aware culture, ensure efficient and effective risk and compliance management practices by adhering to required standards and processes.
Skills:
Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.Ability to quickly identify an opportunity and recommend possible technical solutions.Develop application systems that comply with the standard system development methodology and concepts for design, programming, backup, and recovery to deliver solutions that have superior performance and integrity.Contribute to determining programming approach, tools, and techniques that best meet the business requirements.Understand and follow the PDP process to develop, deploy and deliver the solutions.Be pro-active and diligent in identifying and communicating design and development issues.Provide business analysis and develop ETL code and scripting to meet all technical specifications and business requirements according to the established designs.Offer system support as part of a support rotation with other team members.Utilize multiple development languages/tools such as Python, SPARK, Hive to build prototypes and evaluate results for effectiveness and feasibility.Operationalize open source data-analytic tools for enterprise use.Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, Python and AWS based solutions.Custom Data pipeline development (Cloud and locally hosted)Work heavily within the Cloud ecosystem and migrate data from Teradata to AWS based platform.Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.Technical Skills / ExperienceHands on experience with Amazon Web Services (AWS) based solutions such as Lambda, Dynamodb, Snowflake and S3.Experience in migrating ETL processes (not just data) from relational warehouse Databases to AWS based solutions.Experience in building & utilizing tools and frameworks within the Big Data ecosystem including Hadoop, Kafka, Hive, Spark, HBase, NoSQL.Deep knowledge and very strong in SQL, and Relational DatabasesKnowledge of Data Warehouse technology (Unix/Teradata/Ab Initio)Willingness to continuously learn & share learnings with othersAbility to work in a fast-paced, rapidly changing environmentVery strong verbal & written communication skillsExperience within the Financial industryLeadership Skills4+ years of experience of being a lead engineer amongst a team of equal or junior level developersExcellent written and verbal communication, presentation and professional speaking skillsCollaborative individual who comfortable working on agile-based teams, excels in working with business partners identify, develop and deliver innovative data solutionsCapable of collaborating with stakeholders and project leaders to understand requirements, deliverables, and set expectations on tasks that you will be responsible for.Ability to demonstrate leadership to managers, and peer level staff. Strong ability to build and leverage external relationships. Decision making abilities while gathering information and then put your decisions into action.Passionate learner who enjoys education through class room training and self-discovery on a variety of emerging technologiesPromote a risk-aware culture, ensure efficient and effective risk and compliance management practices by adhering to required standards and processes.
Discover Financial Services is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran status, among other things, or as a qualified individual with a disability.
So, what are you waiting for? Apply today!