This Built In Chicago series profiles the craftsmanship at Chicago’s digital startups that are solving the toughest technology problems from the ground up. This is not a series about the C-level execs or venture capitalists behind these companies, but rather a zoomed-in view of the tech and the products that are making Chicago history. Yes, that’s right, these stories will dive into the nitty, gritty, techie projects that you’ve always (or never) wanted to know about. Read part 1 on how kCura is solving big problems on an even bigger scale here.
[ibimage==37950==Large==none==self==null]
Even a drive to automate requires a human element.
Just ask Channel IQ, the Chicago-based tech company that provides real-time online retail intelligence solutions for manufacturers, distributors and retailers through its massive online product catalogue. The company aims to give its customers a better, more inclusive worldview of online products (this allows them to gather intelligence about their competitors and manage their own supply/sales channels). Collecting the data means scouring the web to find any and all pertinent information on a product (ranging from makeup to electronics), then sorting and making sense of that data before delivering it to clients.
But perhaps even more than solving the problem above, the company is all about figuring out automation and machine learning. In their business, automation means algorithms. And natural language processing. It’s the only way the company can effectively scale towards its own ambitions.
[ibimage==37949==Large==none==self==null]
Scrubbing Data
Channel IQ pulls data from all over the web – including from sites like Amazon, eBay, Wal-Mart and Alibaba (China's Amazon of sorts), which are all massive data centers in their own right – and cleanly puts that data in one place. The data needs to be complete and accurate. Anything less could cause problems for their clients in addition to their business model.
Channel IQ uses two different technologies to scrub the data. For their data acquisitions team, the single biggest challenge is to normalize a semi-structured world. In other words, taking information that is presented differently across these websites and making it comparable.
Human eyes now still largely determine the business process – what stays in, what needs changing and what the algorithms properly sorted themselves. Having the human element is a necessary step, but one CTO Will Hansmann hopes to mostly eliminate.
The algorithm developed so far sorts through the data and tries to make sense of it. Their current algorithm understands names, IDs and other variances. If it gives a match of over 99 percent (i.e. product X on Alibaba is the same as product #3 on eBay) it's unlikely the item is flagged for human review. But anything less – the company is being conservative in its approach to ensure quality control – means it goes in front of human eyes. The review process sounds simple: it could be as easy as identifying that two products are indeed the same thing, but just presented differently by Amazon and Target. Easy enough for humans, harder for computers.
Automating this process takes some hefty algorithms, refined over and over. (Their current match factors were built a year ago, but the team is still refining the algorithms today). It’s a process Hansmann categorizes as machine learning. Their engineers engage in natural language processing, teaching a computer how to identify similar items and what things to throw out.
[ibimage==37951==Large==none==self==null]
Scaling and Automation – a Perfect Match
The engineers behind the algorithms are vital to the business plans. Scaling for Channel IQ isn’t just a buzzword. If the company had a buzz phrase, it would probably be “fire hose of data” because that’s what the team deals with on an hourly basis. Right now the company has catalogued (and monitors) millions of items, but Hansmann said he expects that will breach a billion products by the end of the year.
The road to automation is paved with sinkholes, but if managed properly, it could provide the company and their clients with quite possibly the world’s best information on their competition and their own supply chain in real time without much effort on the client’s part. Imagine being a manufacturer and understanding where, how and for what price your item is being sold around the world and by who. All with the click of a button.
Boosting the match rate – that 99 percent confidence marker to skip the human element – up to 40 or 50 percent would make the team happy, according to Hansmann but in the long-term, the company realistically needs that match rate closer to 95 percent.
The problem – as they see it – is that each product on a website is described differently, mainly for marketing purposes. What one website might catalogue as the “color” for a lipstick, another might use the word “hue.” For Channel IQ, replicate that over millions of different product types, and all the variations in how a product can be described (was that lipstick or lip gloss, makeup or beauty care, red or vibrant red?). Highschool math tells us as their product depository grows, the variations in description grows exponentially so: “As volumes go up, the dependency on [automation] goes up,” said Hansmann. For Channel IQ, the algorithm is key to growth.
Still, “there is always going to be a human element to this,” said Hansmann. As long as marketers use creativity to try and create an allure for what would otherwise be an identical product, mistakes in algorithms will exist. Channel IQ is pushing forward, but realizes “That last 20 percent is always the most difficult to get,” said Hansmann.
The CORE
Channel IQ's engineering team has 35 employees, broken up into a group that collects data, another one that builds data services and figures out how to manage a billion plus data offerings and a team that builds products. The second team – the data services engineers – is in charge of auto matching. Internally, they are referred to as CORE, and it’s easy to see why.
The company plans to expand the team to 60 people by the end of the year, after raising $12 million in funding last September.
.jpeg)
