Testing and Optimization Manager at ShopRunner

| Chicago
Sorry, this job was removed at 12:14 p.m. (CST) on Tuesday, May 28, 2019
Find out who's hiring in Chicago.
See all Developer + Engineer jobs in Chicago
Apply now
By clicking continue you agree to Built In’s Privacy Policy and Terms of Use.


We connect customers to the brands they love by way of a thriving marketplace and a members-only service that provides benefits across 140+ of the best online stores. Our members save time and money with benefits including free 2-day shipping, easy 2-click checkout, and free returns all while we help them stay up on the latest trends that appeal.

For merchants, we’re driving eCommerce business, producing insights with a growing data analytics practice to boot. With a powerful two-sided network and a robust data platform, we’re creating an eCommerce win-win, helping retailers compete. The landscape of retail is changing and we’re here to empower retailers to take their place in that exciting evolution.

We have people in offices around the world: Headquartered in Chicago, with offices in Conshohocken, PA (Philly area), New York, San Mateo and Krakow, Poland. We hustle to get things done, creating wins for customers, merchants and each other.


The ShopRunner Analytics team drives insights in every aspect of our business including marketing, product, and partner relationships. The Testing Optimization Manager will report to the head of Analytics and be a critical part of the ShopRunner Analytics team, overseeing test analytics and growing the test-and-learn program. This person will work directly with business stakeholders, be the expert on testing strategy, and ensure the quality of test analysis.


  • You’ll partner with Marketing, Partner Success and Product teams to understand experimentation requirements, devise test plans, and advise on testing best practices.
  • Own the company-wide view on site testing efforts by driving testing program metrics and communicating about the impact and activities of the testing program
  • Work closely with the Data Platform product and engineering teams to develop and improve testing tools.
  • Be the expert on test methodologies and analyses for the organization. Establish analysis standards and oversee the quality of test results, both those you’ll analyze yourself and those done by others in the Analytics team.
  • Identify opportunities for tests by exploring data
  • Become an expert on our internal data and systems as well as our partner’s overall business as it relates to ShopRunner
  • Identify data issues and drive improvements in data hygiene
  • Present your analyses and insights to all levels of the company, clearly and concisely


  • 7+ years of experience of relevant work experience
  • BS in a relevant technical field such as Computer Science, Engineering, Mathematics, or Statistics
  • Expert in the statistics and methodologies used in A/B and multivariate testing
  • Understanding of experimental design for the purpose of thinking strategically about the interaction effects of tests with internal (e.g. marketing campaigns) and external factors (e.g. weather, seasonality)
  • Familiar with Optimizely, Adobe Target, or other site testing tools
  • Familiar with tools and strategies for testing on mobile devices e.g. Taplytics
  • Solid foundation with analytics, deep understanding of the details of solving data problems
  • Expert at SQL and Excel for querying and manipulating large volume of data for analysis and visualization
  • Excited about the opportunities in a startup and unfazed by the ambiguity and fast-paced culture that come with it
  • Proactive with a problem-solving mindset and attention to detail


  • Experience working across multiple business functions
  • Understanding of eCommerce industry, product, and marketing concepts
  • Understanding of agile development practices
  • Experience with creating dashboards/reports using Business Intelligence tools such as Tableau, Qlikview, Looker, etc.

We want you to bring your whole human self to work every single day. We accept you for who you are and consider everyone on an equal opportunity basis without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Read Full Job Description
Apply now
By clicking continue you agree to Built In’s Privacy Policy and Terms of Use.

Technology we use

  • Engineering
    • JavaLanguages
    • JavascriptLanguages
    • KotlinLanguages
    • PythonLanguages
    • RLanguages
    • SqlLanguages


The beauty of our office is you can get everything you need, without ever having to leave the building. Food, post office, gym.... take that, winter!

What are ShopRunner Perks + Benefits

ShopRunner Benefits Overview

From generous PTO to weekly work-from-home, amazing parental leave, and volunteer time off, we've crafted the kind of benefits plan that supports a stellar employee experience! When you join ShopRunner, you get best-in-class healthcare for you and your family, 401k matching that vests immediately, an annual educational budget, and meaningful company options that give all Shoprunners some skin in the game. It’s all part of our culture of creating an environment that allows you to be your best self at work.

Partners with Nonprofits
Friends outside of work
Eat lunch together
Intracompany committees
Daily stand up
Open door policy
Team owned deliverables
Group brainstorming sessions
Open office floor plan
Documented equal pay policy
Diversity manifesto
Mean gender pay gap below 10%
Health Insurance & Wellness Benefits
Flexible Spending Account (FSA)
Disability Insurance
Dental Benefits
Vision Benefits
Health Insurance Benefits
Life Insurance
Onsite Gym
Retirement & Stock Options Benefits
401(K) Matching
Company Equity
Child Care & Parental Leave Benefits
Generous Parental Leave
Flexible Work Schedule
ShopRunner values what our employees can do, not what time they sit down at their desks. We work hard and we show up, but we don't punch in and out.
Family Medical Leave
Vacation & Time Off Benefits
Generous PTO
Paid Volunteer Time
Our employees receive 8 hours per year of paid volunteer time.
Paid Holidays
Perks & Discounts
Casual Dress
Commuter Benefits
Company Outings
Baseball games, holiday parties, darts, trivia, parties on our patio -- we make time to get to know one another and have fun.
Game Room
No game room, but we do have board game and trivia nights!
Stocked Kitchen
Some Meals Provided
We provide free breakfast on Monday. Employees get free lunch on Thursday.
Fitness Subsidies
Professional Development Benefits
Job Training & Conferences
We sponsor conferences, our teams speak at conferences, and we offer an annual reimbursement program for continuing education.
Tuition Reimbursement
Lunch and learns
Our lunch and learns run the gamut, and have included everything from product demos, to financial advising, trends in ecommerce, and interview training. We make time to learn from one another.
Promote from within
Mentorship program
Our mentorship program includes 1-to-1 program, Cross-department program, Career mentoring, Leadership mentoring.
Continuing Education stipend
Time allotted for learning
ShopRunner sets up quarterly hack weeks, where we focus on new ideas and special projects.
Online course subscriptions available
More Jobs at ShopRunner2 open jobs
All Jobs
Project Mgmt
Project Mgmt
Apply now
By clicking continue you agree to Built In’s Privacy Policy and Terms of Use.
Save jobView ShopRunner's full profileSee more ShopRunner jobs