The “Competing on Analytics” panel at the SDForum Conference on “The Analytics Revolution” included people from companies using analytics to “compete at the highest level” according to the five stage maturity model in the book “Competing on Analytics: The New Science of Winning.” The panelists (Amr Awadallah, Cloudera; Joshua Klahr, Yahoo!; James Phillips, Northscale; Joydeep Sen Sarma, Facebook) represented a good mix from the relatively new Twitter to the larger, older, more established eBay. David Steier, PriceWaterhouseCoopers, moderated the panel.
David began by asking: how do the panelists use analytics to get a competitive advantage? What is an example of something they found that surprised them?
Ken Rudin and Zynga use analytics to achieve their goals of increasing revenue, improving user retention and increasing viral spread for their online social games including Farmville. They collect 3-4 terabytes of data from their 40-50 million daily users. Initially they were reactive: producing reports in response to requests. Now they use analytics as an integral part of game design: they use AB testing and experiments in development just as QA is an integral part of development. Analysts are part of the design team. An example of something that surprised them occurred when they analyzed Mafia game players. There are two groups of players, the “crime jobbers” and the “fighters.” They discovered that fighters spend twice as much because they are trying to compete with their friends so they purchase weapons to arm their mafia members so they can fight and defeat their friend’s mafia gang. Since they discovered this, they have changed the game to encourage players to fight more instead of just doing crime jobs.
Kevin Weil mentioned that a key accomplishment of the analytics team at Twitter has been to get everyone to consider data (and to make it possible for them to do so) in all important decisions. He described an example of how analytics surprised them that involved the social network underlying Twitter. Many people use Twitter largely as an information source and their network of “following” links says something about their interests. But the bidirectional links can be ignored when trying to determine their interests. The unidirectional links (e.g., the people they follow who don’t follow them) are the ones that carry the most important incoming information.
David Steier asked the panelists: How do you organize people to achieve the company’s goals? The panelists companies all have analytics teams and a team responsible for the company’s analytics platform and analysts who work with other teams but the way they work within the companies is different and several companies are adapting innovative approaches.
DJ Patil and LinkedIn started out by looking at other companies such as Google and Yahoo. Yahoo had analytics in a separate research organization and it was difficult or impossible to get it into products. Google is driven by technology and bolts products on top (see also http://techcrunch.com/2010/05/15/facebook-google/). So to ensure that analytics is integrated into products at LinkedIn, the Analytics team is a substantial part (1/4th) of the product team and has its own designers and developers so it is easier to go straight to production or to integrate with existing products.
Neel Sundaresan (eBay) claimed that “everybody should be an analytics scientist.” eBay has an analytics platform team that provides data to the rest of the company and “the data tells you what the product should be.” With 200 million users and a billion searches per day, eBay gets tremendous amounts of data and product managers and developers and even some machine learning scientists need to learn to look at the data.
Ken Rudin (Zynga) argued that the whole idea of having an “analytics team” is flawed. He asked: ”Does Microsoft have an internet division?” Like the internet, analytics is or should be fundamental to everything in the company. So his goal is to work himself out of a job by putting analysts in development and product teams and by training almost everyone in the company in analytics starting with product managers, then engineers, and then quality assurance.
In summary, one of the key themes of the panel was that companies competing on analytics are finding innovative ways to integrate analytics throughout their companies. One of the biggest problems companies face these days is that it is difficult to find good analytics people or data scientists. Ken Rudin is looking for different kinds of people now as compared to five years ago. Now he is looking for people with analytics and business abilities. So for example, instead of just taking data that is given to you and looking for interesting patterns, you should be able to take a business goal like “increase player retention” and figure out how to do it, what data you need, and so on.
A recording of this panel is available at dyyno.com.