Monday, October 21, 2013
Big Data and Causation
The new book on Big Data by Viktor Mayer-Schonberger and Ken Cukier is a good intro to the age of big data, with a slightly optimistic telling of its benefits and a fair estimation and treatment of its risks. One thing the book points to remains troubling to me, weeks after reading: "Most strikingly, society will need to shed some of its obsession for causality in exchange for simple correlations: Not knowing why but only what. This overturns centuries of established practices and challenges our most basic understanding of how to make decisions and comprehend reality". Yes, it is a revolutionary approach but is the prediction correct? The point is certainly well taken when you examine, for example, how marketing works in an era of big data. Is the same thing going to be true for cutting edge cancer research? Or has Mayer-Schonberger picked up on something else, that statistical analysis cannot demonstrate causation, and the more data we have (and the more time we spend crunching it), the less time we will have for causal analysis? However, less time for causal analysis should not mean abandoning it. I still find this concept troubling and struggle with giving up the notion of causation - isn't that what the entire scientific process is about? I suspect that M-S is at leats partially correct in his factual view of the rise of correlation, but must causation therefore be diminished? Can it? I'm just not sure.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment