About CCC
CCC Intelligent Solutions Inc. is the technology platform for the P&C insurance economy. CCC technology, insights, and support connect industries - insurers, automotive manufacturers, collision repairers, parts suppliers, lenders, fleet operators and more - to advance decision-making, productivity, and customer experiences for thousands of clients worldwide. Clients leverage CCC's network management, data management, AI, operational workflows and customer experience solutions to efficiently scale, interact, transact and achieve their unique business objectives. CCC was ranked a best mid-sized company to work for by Forbes (2019). BuiltIn Chicago, Austin and LA named CCC a top place to work in 2020 & 2021. Diverse perspectives and experiences are core to CCC's success and award-winning culture of more than 2,000 employees worldwide. We hold inclusion as a core value and are committed to celebrating and cultivating the diversity of our team. With a 40+ year track record of innovation, CCC's tenacious spirit and growth mindset turn next generation technology into real world solutions and empower team members to expand their knowledge and potential. Headquartered in Chicago, CCC has 7 locations worldwide and is listed on the New York Stock Exchange (NYSE: CCCIS). Find out more about CCC Intelligent Solutions by visiting cccis.com.
Job Description Summary
The Enterprise Analytics team at CCC has an open position for a Senior Data Engineer. Analytics team builds platforms to provide insights to internal and external clients of CCC businesses in auto property damage and repair, medical claims and, telematics data. Our solutions include analytical applications against claim processing, workflow productivity, financial performance, client and consumer satisfaction, and industry benchmarks.Our data engineers use big data technology to create best-in-industry analytics capability. This position is an opportunity to use Spark ecosystem tools and technologies on AWS EMR environment for micro-batch and streaming analytics. Data behaviors include ingestion, standardization, metadata management, business rule curation, data enhancement, and statistical computation against data sources that include relational, XML, JSON, streaming, REST API, and unstructured data. From technology point of view, this role has responsibility to understand, prepare, process and analyze data to drive operational, analytical and strategic business decisions. We expect the candidate to manage and mentor a team of Level 1 and Level 2 engineers. Having people management skills and cross functions working experience for at least for two years is a must. The Senior Data Engineer will work closely with product owners, information engineers, data scientists, data modelers, infrastructure support and data governance positions. We look for experienced engineers who have designed, built, maintained, and supported complex data flows while managing and growing a team.
Job Duties

• Own and be responsible for data engineering projects start to finish with a few other data engineers who are reporting to you.
• Be a mentor and be supportive to growth of Engineers reporting to you.
• Modify, maintain, and support existing data pipelines to provide business continuity and fulfill product enhancement requests.
• Provide technical expertise to diagnose errors from production support teams.
• Participate as both leader and learner in team tasks for architecture, design and analysis.
• Coordinate within on-site/local teams as well as work seamlessly with the team in European Development Center (in Ukraine).

Qualifications

• Master's or bachelor's degree with Engineering/Programming/Analytics Specialization
• 5 + years' experience with building, maintaining, and supporting complex data flows with structural and un-structural data
• 2 Years or more experience with managing and mentoring other engineers.
• Experience and ability work with cross functional team (Pos, data modelers, Upstream system developers, and internal stake holders).
• Proficiency in Python and pyspark
• Hands on experience in HDFS, HIVE, and SQOOP
• Experience in building data-pipelines/microservices with Apache Kafka
• Experience in Apache Airflow to orchestrate and schedule complex data flows
• Ability to use SQL for data profiling and data validation
• Experience in Unix commands and scripting
• Practical knowledge on AWS components such as EMR and S3

Preferred Qualifications:

• Experience and understanding of Continuous Integration and Continuous Delivery (CI/CD)
• Knowledge in performance tuning in distributed computing environment (such as Hadoop cluster or EMR)
• Familiarity with BI tools (such as Tableau and MicroStrategy) and commonly use data modeling techniques.