Big Data, Big Tensions


Washington DC

We're drowning in an unrelenting tsunami of data. Still, no industry from manufacturing to restaurants to higher education can - or should - resist the call of Big Data. The real challenge is to develop the wisdom to discern, however, between data that are useful in a specific context and those that are useless or, worse, misleading. And the greater number of new statistical measures and their fierce advocacy make the pursuit of wisdom even more elusive these days.

I've cautioned in the past about accepting the sometimes piously intoned fiction that, "The data never lie." Data will always have the capacity to lie, mislead or fail to illuminate as long as human beings are involved in their collection, interpretation, communication and utilization. Just consider for a moment the accuracy of political polls, which are reported as hard news and yet so often wrong.

Enter the world-renowned MIT Sloan Sports Analytics Conference, which starts in one month. I’ll be there as your roving reporter to convey the latest from Malcolm Gladwell, Mina Kimes, Michael Lewis, Jackie MacMullan, Daryl Morey, Rachel Nichols, Adam Silver, Nate Silver, Bill Simmons and many others.

Sports have been driving the seeming unquenchable thirst for new and better analytics for several decades now, starting with the emergence of Bill James and his baseball Sabermetrics in the 1980s. I remember reading a lengthy James’ piece in The Atlantic in the mid-80s and being impressed by what the “new” statistics could offer. Over time, of course, OPS, OBP, Runs Created (RC), Isolated Power (IsoP), Secondary Average (SecA) and the ever-present Wins Above Replacement (WAR) have come to rival if not exceed traditional measures of RBIs, Home Runs and Batting Average in hitting circles.

As with most things in life, however, the effective use of data analytics requires balance. Some in the young, data-driven crowd seem to think that data are the Holy Grail, the sine qua non for assessing talent and performance. They seem to put little weight on the so-called intangibles such as players’ character, experience, judgment, instincts and emotional intelligence. On the other hand, older data-phobes who did not grow up with Sabermetrics resist the essential role of data in talent assessment and in-game decision making. Michael Lewis’ book “Moneyball: The Art of Winning an Unfair Game” and the subsequent 2011 Brad Pitt-Aaron Sorkin movie played out this generational cliché to extremes, though it unfortunately has more than a grain of truth to it in the real world.

My unending respect for Red Sox skipper Alex Cora’s leadership owes in part to his comfort with integrating pristine data analytics and messy human behavior into his decision making. Indeed, becoming and remaining an effective leader means developing yourself in both realms. Hunches based on long experience are welcome, but they’re even better when supported by data.

Some observers wondered why Yankees’ GM Brian Cashman named Aaron Boone manager last year. After all, Boone had no big-league managerial experience whatsoever. Skeptics suggested that Cashman and his young analytics team wanted to use Boone’s lack of experience to exert greater, data-dominant power over in-game choices, usurping his prerogative by removing too much human nature from his decision making.

I’ll be exploring these big tensions at MIT in a month. You can count on it. 

Image courtesy of The Data Way.