While I might not be a data scientist, a majority of my colleagues are. I’m a product manager, and it’s my job to make sure the work our data scientists develop makes its way into the products our engineers build. Yes, that’s not surprising, I know. But what drives me everyday is the ability for our team to take incredibly complex machine learning and statistical algorithms and deliver them in our software.
As a product guy, I can’t help but talk about our products. We use the Civis Platform, our primary product—designed for data scientists—to solve our own data problems. And it’s at the core of how we build our Decision Applications, software built for business users, powered by data science.
The real magic though is the science that’s baked into what seems like simple applications. Let’s take a deep dive into a new feature in Civis Explore, a Decision Application that allows you to upload a list and explore how it compares to national demographic breakdowns.
Enhance your data through our Bayesian matching
Understanding how your list—maybe of your top 5 percent most profitable customers—compares with the national average can help you find gaps in the people you are trying to reach and create better targets for outreach. The first step is automatically comparing your list to our national file and using our Bayes Net matching methodology to enhance and add demographic data to your customers.
Delivering automatic insights about how your lists compare
After enhancing your lists, we want to provide automatic insights on what the data uncovered. We don’t do this just through simple arithmetic or a gussied up “index.” We use science.
We conduct a series of statistical tests to identify where there is a significant difference between the distribution of individuals by category in the user list compared to the national file or another list you’ve uploaded. We handle the multiple testing problem that arises by choosing to control for the family-wise error rate.
Then, we rank order insights by the magnitude of the difference between the two groups being compared. And finally, we apply filters to eliminate duplicate insights—like “more men than women” and “fewer women than men”.
All of these statistical workflows are built directly into the application and you reap the benefits without any special training or statistical background needed. All you have to do is upload your list, match a few fields, and scientific insights appear automatically. So while it may seem simple, data science is the core of everything we deliver.
