Thought Vector Blog

The Last Analytics Company

Posted October 8, 2015

The analytics market is crowded - there are countless companies offering nearly identical services. What’s worse, the technical task of recording analytics has become easy: many technologies throughout the stack make collecting, storing, and analyzing granular data accessible to layman software engineers. Somehow, though, a clear winner hasn’t emerged in the analytics space.

The stage is set for the last analytics company.

The promise of analytics tools is to help gather and analyze data recorded from products and services to yield invaluable business insights. Only with analytics can you answer questions like “Does this copy convert better?” or “Do users actually know how to use feature X?” The broad availability of analytics in the past 5 years has created fast growing companies keenly in-touch with their customers’ needs, and has helped large organizations keep pace in quickly evolving markets.

All of this is built on top of event-level reporting allowing marketers, product managers, and developers alike to slice and analyze user behavior in different ways. Inevitably, though, the ideas have to come from the human - the analytics user often has to know what question they want to answer before the data is useful to them, lest they spend hours wandering without coming to any concrete results. Said differently, the human still had to be the source of insight, with the analytics helping them validate.

Data science tools are automatically generating insights normally only provided by humans.

The expanding prevalence of machine learning and other data science techniques, though, is starting to change this. Now, topic modeling, decision trees, logistic regression, and many other techniques are granting insight in much more vague situations. Using these techniques, product managers no longer have to come with ideas about why they think customers are churning - a linear regression model will show them the biggest churn contributors in many-faceted scenarios they’d never be able to find by slicing analytics data themselves.

In other scenarios, these same product mangers can lean on topic modeling techniques to quantify product needs quickly, without laborious feedback tagging efforts. What’s better, these topic models perform better with more data, where as manual feedback analysis becomes worse. Many of the more tech-savvy companies are already using custom built solutions like these, and the market is waiting for a consolidated set of these services to be available.

The last analytics company will be the first insights company.

The last data analytics company will be the one that provides these insight tools on top of the analytics offerings of today. They will be last not necessarily because they will drive their competitors out of business, but because they will create a new segment, data insights, of which they will be the first. And if they can keep acquiring top talent and building out their portfolio of insight generators, they will keep their differentiation and continue to lead that market.

These tools will greatly benefit their customers as well: their marketers and product managers will be able to spend more time on creative tasks and less on number crunching and analysis, giving them a better bang for their buck in personnel spend. Customers will also be able to iterate faster, getting to substantial answers more quickly and helping them maintain agility in the face of challenges or uncertainty.

Companies have long been sold on the value of big data, and now the technology to make these promises reality is commonly available. Scalable, open source databases, stream processing frameworks, standardized data structures (a’la Segment), and increasingly common data scientists and engineers make these tools well within reach.

Many companies are poised to make this move - now someone just has to do it.


Python 3 on Spark - Return of the PYTHONHASHSEED

If you’re anything like me, you’ve been stuck using Python 2 for the last 10 years, and for 8 of them you’ve been trying to switch to 3. Since the release of Spark and PySpark 1.4, Apache has started...

author Stuart AxelbrookeBy Stuart Axelbrooke, who does data science and text analytics. You should follow him on Twitter