It seems fitting that Networked Insights, a leader in large-scale, real-time analytics, ran an experiment to decide where to place its headquarters. Upon taking his company’s first round of institutional funding, founder and CEO Dan Neely set up offices in both Chicago and New York City.
“The goal was to let the actual markets determine where to build the company,” Neely said. “Chicago grew a lot faster, and the talent was better, so we decided to do it here.”
We spoke with Neely about Networked Insights’ technology, what sets his team apart and how he recruits for the brightest minds in tech.
NETWORKED INSIGHTS AT A GLANCE
WHAT THEY DO: Networked Insights uses artificial intelligence to help brands understand and reach potential customers.
WHERE THEY DO IT: Chicago.
THE STACK: Apache Storm, Kafka and Spark, Elasticsearch, Postgres, JVM and React.
ASK FOR FORGIVENESS: Not permission.
BIG DATA: Networked Insights analyzes more than half a billion social media posts daily.
BIG AMBITIONS: The goal of it all is to understand how humans interact.
What does Networked Insights do?
I started the company because of one key frustration: the amount of time it took to get information about consumers. And when I did get it, it was usually stuff that I already knew. So we set out on a path to learn more things about people.
About six years ago, we discovered that 93 percent of interaction between people is more implicit than explicit. For example, if my wife asks me whether we have Diet Coke in the fridge, what she really means is that she wants me to go get more Diet Coke. For us, that meant we had to build a machine capable of doing something that makes us uniquely human: understanding inference.
That’s a tall order.
We created a machine capable of building the world’s richest consumer profiles, with information ranging across every aspect of the consumer’s life. Many companies track people’s purchases and clicks. We seek to understand why people do those things.
What you click only represents 1 percent of who you are. The other 99 percent is represented in the data you create and share every day. That information can be used to figure out what content to put in front of a person, based on what they care about. It can also be used to put that content in the right place and in the right context.
We had to build a machine capable of doing something that makes us uniquely human: understanding inference.
What kinds of insights do your machine learning models offer?
One of the most recent ones that we're working through is taking a population of people and finding out how many of them just had a child, which in turn means they were pregnant for the past nine months. What we want to do with that data is train the machine to identify a pregnant consumer at the earliest point possible, without them telling us.
In the past, the way people have done this is by looking at whether people are adding pregnancy-related items to their shopping carts. But that could just mean they have friends that have baby showers coming up.
From a tech perspective, what sets you apart from the competition?
Networked Insights has two core assets. One is the machine learning platform we have developed. The other is our massive training data set, with classified data about roughly 2.3 billion people around the world.
Has your vision for the company changed over the past 11 years?
Our vision was always to make it simpler to gain knowledge about people, but when we started 11 years ago, we had to go through the painful process of building our own data center to do that. The cloud simply wasn’t powerful enough at the time. Today, we can do analytics that even some of the cloud vendors aren’t capable of doing.
In that time span, you’ve also pivoted from a managed services business to a SaaS company. What’s that been like?
When we first built this, it was an industrial-grade product that we had to use for our customers. About three and a half years ago, we went to our board and said we were going to become a SaaS company. Deciding to kill a business that was a known entity, generating a large amount of revenue and profits, is one of the hardest things I’ve ever done in my career. Fortunately, we had good investors who recognized why we had to do it and went down that path with us.
What are the biggest challenges to continuous learning, as opposed to storing data and having the option to go back for a second look?
There are three steps to machine learning. The first step is training the model. The second step is testing it against a data set that matters. The third step is continuous training. You can't get to continuous training unless you’ve addressed some of the latent storage challenges that exist. Some of the big cloud vendors can’t do this yet at massive scale.
Just for some context, a high-frequency trading system does about 10,000 transactions per second. We do about 8 billion. But it’s important to look at the data in real time, because people change their minds every second of every day. And, ultimately, we’re building models about how to look at people’s lives.
How have you built a culture that's appealing to people capable of doing this work?
There are two elements. The first is that you have to be working on the next problem, rather than the problems people are trying to solve today. It’s one thing to implement machine learning in your business, and another to democratize machine learning so anyone can do it. It’s a different paradigm and a very different challenge.
The second thing is that when you reach a certain scale and you’re solving really interesting problems, your team starts to recruit itself.
So if your challenge is interesting enough, the right people will find you?
If we were a Silicon Valley company, every single person here could walk two steps out the door and have another job. Working here has nothing to do with having a job — it’s about the challenges. We employ some of the smartest people in the world, some of whom are former academics who’ve worked on the theories and now want to put them into practical use.
In scaling the company, how do you ensure that the culture you’ve built stays in place?
One of the biggest parts of our culture is to make sure we’re not just reacting to the things in front of us every day. Collaboration is also important to us, as is an entrepreneurial spirit. We’re more about asking for forgiveness than permission.
The underpinnings for our newest product, audience.ai, were developed out of a hackathon. A group of colleagues got together, saying our current way of processing data seemed like it took more time than it should. What they learned as they took on that challenge became the foundation of audience.ai.
What's your favorite thing about the Chicago tech ecosystem today?
My favorite thing is that it is still young. I've been part of the Silicon Valley and the New York ecosystems, but it’s exciting to be part of a system that we get to shape, add to and learn from. It’s a little bit like living through an industrial revolution.
There are challenges that come with that as well. In Chicago, there haven’t been as many big outcomes that have massively changed the lives of developers in the community. If our success continues, you’ll see more engineers go out to start their own companies. That happened in Silicon Valley because financial success spread beyond the C-suite.
The interview has been edited for length and clarity.
