Big Data: The Management Revolution

Big Data: The Management Revolution

1. What’s New Here?

   • Business executives sometimes ask us, “Isn’t ‘big data’ just another way of saying ‘analytics’?” It’s true that they’re related: The big data movement, like analytics before it, seeks to glean intelligence from data and translate that into business advantage. However, there are three key differences
     • Volume
       • As of 2012, about 2.5 exabytes of data are created each day, and that number is doubling every 40 months or so. More data cross the internet every second than were stored in the entire internet just 20 years ago
     • Variety
       • For many applications, the speed of data creation is even more important than the volume. Real-time or nearly real-time information makes it possible for a company to be much more agile than its competitors
     • Velocity
       • Big data takes the form of messages, updates, and images posted to social networks; readings from sensors; GPS signals from cell phones, and more. Many of the most important sources of big data are relatively new

2. How Data-Driven Companies Perform

   • The second question skeptics might pose is this: “Where’s the evidence that using big data intelligently will improve business performance?” The business press is rife with anecdotes and case studies that supposedly demonstrate the value of being data-driven. But the truth, we realized recently, is that nobody was tackling that question rigorously

3. Improved Airline ETAs

   • Minutes matter in airports. So does accurate information about flight arrival times: If a plane lands before the ground staff is ready for it, the passengers and crew are effectively trapped, and if it shows up later than expected, the staff sits idle, driving up costs. So when a major U.S. airline learned from an internal study that about 10% of the flights into its major hub had at least a 10-minute gap between the estimated time of arrival and the actual arrival time—and 30% had a gap of at least five minutes—it decided to take action.

4. Speedier, More Personalized Promotions

   • A couple of years ago, Sears Holdings came to the conclusion that it needed to generate greater value from the huge amounts of customer, product, and promotion data it collected from its Sears, Craftsman, and Lands’ End brands. Obviously, it would be valuable to combine and make use of all these data to tailor promotions and other offerings to customers, and to personalize the offers to take advantage of local conditions. Valuable, but difficult: Sears required about eight weeks to generate personalized promotions, at which point many of them were no longer optimal for the company. It took so long mainly because the data required for these large-scale analyses were both voluminous and highly fragmented—housed in many databases and “data warehouses” maintained by the various brands.

5. Big data’s power does not erase the need for vision or human insight.

   • New roles
     • s. Executives interested in leading a big data transition can start with two simple techniques. First, they can get in the habit of asking “What do the data say?” when faced with an important decision and following up with more-specific questions such as “Where did the data come from?,” “What kinds of analyses were conducted?,” and “How confident are we in the results?” (People will get the message quickly if executives develop this discipline.)
   • Talent management
     • As data become cheaper, the complements to data become more valuable. Some of the most crucial of these are data scientists and other professionals skilled at working with large quantities of information. Statistics are important, but many of the key techniques for using big data are rarely taught in traditional statistics courses.
   • Technology
     • The tools available to handle the volume, velocity, and variety of big data have improved greatly in recent years. In general, these technologies are not prohibitively expensive, and much of the software is open source
   • Decision making
     • An effective organization puts information and the relevant decision rights in the same location. In the big data era, information is created and transferred, and expertise is often not where it used to be.The artful leader will create an organization flexible enough to minimize the “not invented here” syndrome and maximize crossfunctional cooperation. People who understand the problems need to be brought together with the right data, but also with the people who have problemsolving techniques that can effectively exploit them
   • Company culture
     • The first question a datadriven organization asks itself is not “What do we think?” but “What do we know?” This requires a move away from acting solely on hunches and instinct. It also requires breaking a bad habit we’ve noticed in many organizations: pretending to be more data-driven than they actually are. Too often, we saw executives who spiced up their reports with lots of data that supported decisions they had already made using the traditional HiPPO approach