Software Engineer—Big Data
3 months ago
Design, build, and maintain processes and components of a streaming data/ETL pipeline to support real-time analytics (from requirements to data transformation, data modeling, metric definition, reporting, etc.).
Ensure the security, confidentiality, and integrity of client information for yourself and team.
Develop code using Big Data Technologies such as Hadoop and Storm/Spark to write MapReduce and In-memory applications using Java, SQL, Pig, Hive, Impala, etc.
Collaborate with data scientists & solution developer to design and develop processes to further the company's common data platform initiatives.
Develop compelling PoC’s for data solutions using emerging technologies for real-time and big data ingestion and processing.
Help define standards and best practices for enterprise usage.