Jonah Lehrer looks at the limitations of empiricism:
My sole point is that our newfound reliance on data and statistics naturally leads to their misapplication. Because we’re so enamored with the numbers, we tend to undervalue what can’t be compressed into numerical form, even as we pay lip service to the lingering importance of intangibles. This is a cognitive bias we all need to watch out for.
He may be talking about the use of data and statistics in sports, but this echos my concerns that an emphasis on empiricism creates a bias for what can be measured and that even when we are relying on data, we knowingly or unknowingly use values to determine what is important and what is not.
He expands on this in another article:
I’m thinking here of a Philip Roth metaphor. When asked by David Remnick, in a 2000 New Yorker profile, how he felt about a cramped literary interpretation of one of his novels, Roth busted out a sports analogy. He imagined going to a baseball game with a little boy for the very first time. The kid doesn’t understand what’s happening on the field, and so his dad tells him to watch the scoreboard, to keep track of all the changing numbers. When the boy gets home someone asks him if he had fun at the game:
“It was great!” he says. “The scoreboard changed thirty-two times and Daddy said last game it changed only fourteen times and the home team last time changed more times than the other team. It was really great! We had hot dogs and we stood up at one point to stretch and we went home.”
If that little kid were around today, he’d be obsessed with sabermetrics. He’d almost certainly win his fantasy league, but he’d miss the point of the game. Sure, he wouldn’t have squandered center field on Rowand, but he also wouldn’t have started Barea or bet on the Mavs. His car would have way too much horsepower and shitty seats.
Here’s my problem with sabermetrics — it’s a useful tool that feels like the answer. If we were smarter creatures, of course, we wouldn’t get seduced by the numbers. We’d remember that not everything that matters can be measured, and that success in sports (not to mention car shopping) is shaped by a long list of intangibles. In fact, we’d use the successes of sabermetrics to focus even more on what can’t be quantified, since our new statistical tools take care of the stats for us. We are finally free to think about how those front seats feel.
But that’s not what happens. Instead, coaches and fans use the numbers as an excuse to ignore everything else, which is why our obsession with sabermetrics can lead to such shortsighted personnel decisions.