Arity
Chicago, IL

Senior Hadoop Systems Engineer at Arity

| Chicago
!Sorry, this job was removed at 4:25 p.m. (CST) on Wednesday, March 27, 2019

Founded by The Allstate Corporation in 2016, Arity is a data and analytics company focused on improving transportation. We collect and analyze enormous amounts of data, using predictive analytics to build solutions with a single goal in mind: to make transportation smarter, safer and more useful for everyone.

At the heart of that mission are the people that work here—the dreamers, doers and difference-makers that call this place home. As part of that team, your work will showcase both your intelligence and your creativity as you tackle real problems and put your talents towards transforming transportation.

That’s because at Arity, we believe work and life shouldn’t be at odds with one another. After all, we know that your unique qualities give you a unique perspective. We don’t just want you to see yourself here. We want you to be yourself here.
 

 

Job Description

 

Arity is a new technology company that uses advanced predictive analytics to help partners evaluate risk and make sense of everyday experiences. Our proprietary algorithms analyze billions of data points, distilling the flood of information into simple, actionable insights. Chief among these is the Arity Driving Score, which models billions of miles of driving and claims data to assess risk, gauging each driver’s likelihood of an event.

Founded by The Allstate Corporation, an organization synonymous with protection and preparation, Arity is born from a long-standing commitment to deep customer understanding and future-focused innovation. Fueled by the curiosity of a startup, our expertise begins with connected car, providing applied insights that meet insurer and transportation/mobility company needs and enhance driver safety, connectivity and value.

Don’t miss this opportunity to join a team where your innovative thinking and technology skills can improve the safety of those on the road and further enhance the car-ownership experience.

 

 

 

Job Responsibilities

 

1.    Manage large scale multi-tenant Hadoop cluster environments residing on AWS.

2.     Handle all Hadoop environment builds, including design, security, capacity planning, cluster setup, performance tuning and ongoing monitoring.

3.     Perform high-level, day-to-day operational maintenance, support, and upgrades for the Hadoop Cluster.

4.     Research and recommend innovative, and where possible, automated approaches for system administration tasks.

5.     Creation of key performance metrics, measuring the utilization, performance and overall health of the cluster.

6.     Deploy new/upgraded hardware and software releases and establish proper communication channels.

7.     Work with appropriate stakeholders to ensure we have solid capacity planning and can manage our TCO.

8.     Ability to collaborate with product managers, lead engineers and data scientists on all facets of the Hadoop Eco-System.

9.     Ensure existing data/information assets are secure and adhering to a best in class security model.

 

 

 

 

Job Qualifications

 

Technical Experiences Required

  • Possess at least 3 years of managing a multi-tenant production Hadoop environment;
  • A deep understanding of Hadoop internals, design principals, cluster connectivity, security and the factors that affect distributed system performance;
  • Proven experience with identifying and resolving hardware and software related issues;
  • Experience with Hortonworks DataPlatform;
  • Knowledge of best practices related to security, performance, and disaster recovery;
  • Expert experience with at least two of the following languages; SQL, Python, Java, Scala, Spark or Bash.
  • Experience with CloudFormation, Git, Jenkins, Ansible
  • Experience with complex networking infrastructure including firewalls, VLANs, and load balancers.

Additional Experiences Preferred

  • Experience managing Cloud Services (IaaS, PaaS).  AWS Certification is preferred;
  • Experience with a number of different real-time integration tools such as Apache Flink, Kafka, Storm or Flume.  Leveraging Hortonworks DataFlow is preferred;
  • Experience as a DBA or Linux Admin

Personal Experiences Required

  • Proven ability to lead multiple high priorities initiative with aggressive timelines leveraging an agile framework.
  • Comfortable performing in a fast paced, dynamic and ambiguous business environment.
  • Ability to concentrate on a wide range of loosely defined complex situations, which require creativity and originality, where guidance and counsel may be unavailable.
  • Demonstrated a product mindset with an ability to set forward thinking and direction.
  • Excellent listening and communication skills. 
    • Must be able to communicate technical information clearly, able to ‘translate’ between diverse groups of technical and non-technical individuals.
    • Ability to synthesize large amounts of complex data into meaningful conclusions and present recommendations to a vast array of individuals.
    • Partner with business units in order to improve the effectiveness of business decisions.
    • Ability to structure documents to effectively communicate with senior leadership and drive alignment and decision making.
  • Strong problem solving skills with the ability to offer why one technical solutions is more preferable than another.
  • Ability to analyze and interpret complex concepts.
  • Ability to lead a group of employees (directly or indirectly).
  • Ability to work in a team environment in a collaborative manner.
  • Proven track record of sound, effective decision making.
  • Bachelor’s Degree in Computer Science / Business / Data Sciences required, Master’s Degree Preferred.
  •  

Heads up: when you click “apply now” you’ll be directed to the Allstate careers site. You’re still looking at an Arity job, but because we were founded by Allstate, we share the same application system.

See More

Where we are

Arity's located in the Merchandise Mart, the heart of Chicago's tech scene. Mobility is what we do & you can find us using all forms to get to work.

Technology we use

  • Engineering
  • Product
  • Sales & Marketing
    • JavaLanguages
    • JavascriptLanguages
    • KotlinLanguages
    • PythonLanguages
    • RLanguages
    • ScalaLanguages
    • SqlLanguages
    • SwiftLanguages
    • ReactLibraries
    • ReduxLibraries
    • PandasLibraries
    • Sci-KitLearnLibraries
    • NumpyLibraries
    • MatplotlibLibraries
    • HadoopFrameworks
    • SparkFrameworks
    • SpringFrameworks
    • PysparkFrameworks
    • HiveFrameworks
    • ImpalaFrameworks
    • KafkaFrameworks
    • NiFiFrameworks
    • CassandraDatabases
    • MongoDBDatabases
    • OracleDatabases
    • Google AnalyticsAnalytics
    • AxureDesign
    • InVisionDesign
    • SketchDesign
    • MuralDesign
    • SmaplyDesign
    • ConfluenceManagement
    • JIRAManagement
    • Pivotal TrackerManagement
    • TrelloManagement
    • TrackerManagement
    • WordpressCMS
    • SalesforceCRM
    • PardotCRM
Previous
Next

What are Arity Perks + Benefits

Arity Benefits Overview

Arity was founded by The Allstate Corporation, but you’ll be working for—and at—Arity. It’s the best of both worlds. You’ll get access to the full suite of Allstate benefits and work in a fast-paced startup culture. That’s more than just free breakfasts, brain breaks and ping pong. It’s a culture that encourages you to be you.

Health Insurance & Wellness Benefits
Flexible Spending Account (FSA)
Disability Insurance
Dental Benefits
Vision Benefits
If enrolled, you have coverage for certain periodic vision exams, eyeglass lenses and frames, or contact lenses, up to a designated dollar amount.
Health Insurance Benefits
Life Insurance
Wellness Programs
Retirement & Stock Options Benefits
401(K)
We also provide a cash balance pension benefit, at no cost, to eligible employees based on their eligible compensation and years of service with the company. Employees are vested after 3 years.
401(K) Matching
Child Care & Parental Leave Benefits
Family Medical Leave
Arity offers up to 12 weeks of unpaid, job-protected leave per year for any employee eligible under the Family and Medical Leave Act (FMLA).
Adoption Assistance
Vacation & Time Off Benefits
Generous PTO
Paid Holidays
Paid Sick Days
Perks & Discounts
Casual Dress
Commuter Benefits
Company Outings
Free Daily Meals
Game Room
Stocked Kitchen
Recreational Clubs
Fitness Subsidies
Professional Development Benefits
Job Training & Conferences
Tuition Reimbursement
Diversity Program

Additional Perks + Benefits

Looking for more info? You can find details here: https://www.allstategoodlife.com.

An Insider's view of Arity

What’s the vibe like in the office?

At Arity, you’re encouraged to see yourself here and be yourself here. When you walk through the door, you are part of something bigger than yourself and are empowered to be better than when you first start here. The space and culture facilitate collaboration & are designed to spark innovation.

Erica, Steph & Rachel

Culture Team

What's the biggest problem your team is solving?

The mission of our team is to derive meaningful events and features from raw telematics data, with a mix of physics, signal processing, machine learning, & experimentation. Derived from big data, we're developing best-in-class solutions for collision and distracted driving detection which is disrupting the mobility space.

Venu & Kyle

Data Scientists, Advanced Sensor Insights Team

How has your career grown since starting at the company?

Arity supports my career growth by not only having a multitude of resources at my fingertips but by providing the right support structure to make it easy for my manager and me to have development conversations. I’m able to make succinct goals and really follow through.

Baily

People Strategy Consultant

How do your team's ideas influence the company's direction?

At Arity, we solve problems that impact society. Arity Engineering plays a critical role in developing products that incorporate capabilities our customers want with solutions they need. When a customer has a challenge, we leverage the group’s collective learning to work towards an optimized solution. We rarely come across something we can’t solve.

Emad

Vice President, Engineering

How do you collaborate with other teams in the company?

Cross-functional collaboration is built into how we work and develop products on a daily basis. At Arity, we are one team coming together to achieve our goals, continuously innovate and drive the industry forward.

David

Product Owner, Insurance

Arity Jobs in Chicago
14

All Jobs
All Jobs
Finance
Data + Analytics
Dev + Engineer
HR
Product
Project Mgmt
Sales
Tap to close

Arity-Hadoop Systems Engineer

Arity
Chicago
19 hours ago
+22
Manage large scale multi-tenant Hadoop cluster environments residing on AWS. Handle all Hadoop environment builds, including design, security, capacity planning, cluster setup, performance tuning and ongoing monitoring. Perform high-level, day-to-day operational maintenance, support, and upgrades for the Hadoop Cluster. Research and recommend innovative, and where possible, automated approaches for system administration tasks. Creation of key performance metrics, measuring the utilization, performance and overall health of the cluster. Deploy new/upgraded hardware and software releases and establish proper communication channels. Work with appropriate stakeholders to ensure we have solid capacity planning and can manage our TCO. Ability to collaborate with product managers, lead engineers and data scientists on all facets of the Hadoop Eco-System. Ensure existing data/information assets are secure and adhering to a best in class security model.

Arity-Product Manager

Arity
Chicago
19 hours ago
+22
The Product Manager on the Mobility Intelligence team manages strategic product vision, product discovery, roadmaps, and backlog to maximize the customer value provided by their product. Successful Product Managers help to identify assumptions and validate those assumptions through the use of product discovery techniques. The Product Manager is responsible for working closely with key stakeholders and partners, ensuring the successful definition and execution of go-to market and communications strategies for their products. Product Managers are also responsible for actively participating in our learning culture; including coaching and mentoring staff, stakeholders and peers.

Arity-iOS Engineer

Arity
Chicago
5 days ago
+22
The iOS Mobile Engineer creates detailed technical specifications, develops system code, participates in code reviews and module testing, and supports ongoing maintenance. Having a strong understanding of mobile sensor component engineering, the iOS Mobile Engineer develops code for intermediate to complex modules, participates in design reviews, conducts code reviews, creates test plans and participates in testing reviews, and resolves moderate defects. This position designs, documents, reviews, and maintains components of software solutions for intermediate complex projects.

Arity-Rating Services Consultant

Arity
Chicago
5 days ago
+22
The Rating Services Consultant is responsible for delivering and supporting the use of all regulated scoring programs associated with Arity's growing and revolutionary product portfolio. This role will have both strategic and operational responsibility. This position contributes to the development of filing strategy and owns execution and support of subscription services to maximize the customer and enterprise value provided by the regulated scoring programs.

Arity - Director, Mobile Engineering

Arity
Chicago
5 days ago
+22
At Arity, our mobile capabilities are at the core of what we do, powering live telematics connections and the collection of billions of miles of driving data. As the Director of Mobile Engineering, you will report to the VP of Engineering and help define and support our mobile engineering strategy. You will lead the engineering teams that develop our iOS, Android mobile applications and SDKs. Your team will be responsible for the engineering of mobile products that interface with users, and accurately collect/ingest data from smartphone devices. The ideal candidate will combine excellent technical and business skills to create value for our customers and build upon our mobile offerings.

Arity-Software Engineer

Arity
Chicago
6 days ago
+22
As a talented engineer, you love to translate strategic vision into something tangible whether that be a story, a design or code. You will be an innovative thinker, an expert communicator, and an agile problem solver looking to join an exciting team. You will utilize cutting edge technologies like Kafka, Flink, and Cassandra to deliver software solutions within Arity's cloud environment. As an ideal candidate, you can learn and adapt quickly and are able to use every tool at your disposal-software, algorithms, statistical models, and beyond-to understand and effectively tackle hard problems. You appreciate and can navigate the path from ideation through selecting tools, exploring solutions, architecting etc. to an actual implementation of the idea.

Arity - Sr. Scrum Master

Arity
Chicago
6 days ago
+22
The Scrum Master is accountable for monitoring the scrum processes and scrum meetings and for leading a matrixed team through software development life cycle processes. The Scrum Master increases team efficiency, motivates the team, and advocates for changes that will ensure quality and timeliness. The Scrum Master ensures observance of the "Definition of Done". This role will receive demand from multiple functions (product management, marketing, sales, finance, engineering, customer success and operational excellence) and partner with multiple development partners to build scalable software. This includes leading teams through agile scrum software development with cross functional communication, software technical project management, requirements traceability, source code control, test case management and defect tracking. This position may lead other scrum masters of multiple agile teams and report status of multiple agile features to leadership.

Arity-Data Steward Manager

Arity
Chicago
1 week ago
+22
The Data Governance and Quality team works across the Enterprise to manage the strategy, integration and governance of data at Arity. Members of Arity's Data Governance and Quality team have responsibility for ensuring consistency and clarity of our data across our product portfolio. The team works very closely with product partners to convert business needs into data solutions that power our businesses. The Lead Data Steward is responsible for leading a team of Data Stewards that manage and govern core internal and external analytical data for the organization. In this role, you will help establish, train, and execute capabilities that will advance the Data Stewardship program. Through this work you will provide 'fit for use' data to enable timely generation of sound end-to-end business insight by ensuring our data is timely, accurate, complete, and most importantly, 'fit for purpose' for our customers' use cases.

Arity-Data Scientist-Sr Manager

Arity
Chicago
1 week ago
+22
As a Senior Manager on the Data Science team, you will be responsible for leading and inspiring a world-class group of over twenty data scientists, data analytics engineers and business analytics consultants spread across several product development teams. You will empower data scientists to detect sophisticated driving behaviors on the edge, enrich driving routes with meaningful attributes, and develop an in-depth understanding of users' mobility patterns. You will help us recruit and lead top talent and have a key voice in the analytics operations at Arity. This includes deploying models to mobile and cloud platforms. You will also help create and support our business strategy by being a strong voice of the customer from an analytics lens.

Arity-Sr Data Scientist

Arity
Chicago
1 week ago
+22
As a Senior Data Scientist at Arity, you will help lead the use of data to drive decisions at scale. You will collaborate with product owners, data & software engineers, and designers to develop and deploy robust, scalable services that enhance our understanding of user mobility. You will create new ways of finding meaning from our unique combination of driving and mobility data along with insurance, shared mobility, and mobile app data that help us understand transportation and the risk of behaviors on the road. You will also help shape and grow our culture we have worked hard to establish – promoting recognition of good work, continuous learning, winning together, and having fun along the way.

Arity - Android Engineer

Arity
Chicago
1 week ago
+22
The Route Report Android Software Engineer is responsible for the delivery of a market leading mobile app leveraging cutting edge technologies, creative solutioning and results driven customer focus to further enhance our user experience. Be part of a small, agile cross-functional team of energetic developers and product specialists in a company with great culture and start-up environment, situated in the heart of downtown Chicago in the Merchandise Mart.

Arity-Cloud Cost Optimization Analyst

Arity
Chicago
1 week ago
+22
The Cloud Cost Optimization Analyst is responsible for monitoring and auditing AWS resources for the platform team to improve the operational efficiency of Arity's cloud infrastructure. This role will have both strategic and operational responsibility. This position is responsible for estimating product development costs and forecasting growth, while also considering third party support licensing agreements, labor costs, and financial statement impact. The Cloud Cost Optimization Analyst will support Arity's engineering teams by creating and executing plans to reach our financial objectives. You will interact regularly with the Arity engineering team and Arity finance team and partnering to take actions to reduce cloud spending.

Arity-Sr Employee Engagement Consultant

Arity
Chicago
1 week ago
+22
In this role, this individual will be responsible for helping to create and advocate for an open innovative culture of thinking and doing business at Arity by assisting in developing, managing and executing projects supporting these values. Areas of focus will include optimizing the Mart work environment and seating, as well as supporting employee engagement and retention. To accomplish these objectives, this employee will become knowledgeable of the business objectives and employee strategies of the various areas at Arity. This employee will utilize subject matter knowledge to be a resource for team members on employee culture specifically in the tech space, researching trends, best practices and processes that contribute to supporting the business objectives of attracting and retaining the best talent, supporting a collaborative, innovative work environment, and supporting employees who create cutting edge products and services for Arity.

Arity-Sales and Business Development -Sr Mgr - Insurance Solutions

Arity
Chicago
1 week ago
+22
Gain a thorough understanding of Arity, its products and strategy. Lead the creation and execution of short and long-term sales strategy for assigned product segment or segments. Identify new markets or customers within product vertical. Beginning and advancing sales discussions within target markets on Arity's products. Representing Arity at industry conferences and events. Identifying and leveraging subject matter experts within Arity for sales discussions. Contributing feedback and direction on sales messaging to Product Marketing. Training and supporting other Arity teams on product segment messaging.