I don't like arguments that seem to assume there is some kind of analysis that only deals with statistical data without additional reasoning, interpretation and imagination.
See the old debate between Ta-Nehisi Coates and Jonathan Chait. I think Coates, for all his great points, is too quick to insist that the debate stop using speculation and stick to research. It's not that he's wrong about the value of research, but that he's wrong that it would do much to clarify his and Chait's debate. The deep problem is not that they have much specific disagreement, but that they are playing for different teams. I wonder if it is Chait's hitting close to the bone, rather than his missing the mark, that raises Coates's ire; cogent arguments by someone who disagrees with you are welcome when you feel you can safely allow the other side to score points, but they are panic-inducing when you feel you cannot give ground. Then, entertaining thought exercises feels unsafe.
In a response to Stephen Pinker's The Better Angels of Our Nature and its argument that violence has declined, Nassim Taleb recently commented that "You can't easily extract statistical information from fat tailed data". I think that phrasing suggests a basic misunderstanding of statistics and data and reasoning... there is no pure information devoid of context that can be extracted from data. (And Pinker stays away from misleadingly clean math in his argument for a decline in violence.) There is no point, in collecting evidence, at which you suddenly have enough data to justify interpretation.
Think of the decisions we make in our lives and how little data we have available. Can I point to rigorous statistical studies that tell me my daughters will have a better outcome if they go to our local public school, where the administration and the teaching staff strike me as incompetent and unprofessional, versus if she goes to her actual current school, which is well run and staffed by people who seem, anecdotally, to love children? Of course not. Is my sense of confidence that this is a better choice a worthless illusion? No, it's not--I could well be wrong, and I should push myself to reduce bias in many ways, but my confidence is built on an understanding that is at least somewhat sound. Taleb would probably acknowledge that if asked in isolation, but he doesn't recognize that he precludes this sort of soundness when he dismisses the value of cogent reasoning from limited data. (Something which he himself does plenty of.)
There is no magic shortcut we can use in our analysis, statistical or not, that avoids the need for reasoned and impressionistic interpretation, nor any method that avoids the possibility of our reasoning leading us to being wrong. There is only the actual quality of our reasoning, the carefulness of our interpretation, the multifaceted understanding that we bring to bear, the clarity of our process, our willingness to consider conflicting evidence, and the intellectual honesty to keep score without regard to whether our side or the other has scored each point.
I have read plenty of work by those who bullshit the reader and lie to themselves to promote a point. They have certain consistent characteristics that act as tells: they use ridiculing phrasing to belittle opponents, even when mentioning something that is perfectly normal; they treat their perspective on complex topics as certain, rather than provisional and tentative; they treat their conclusion as so obviously correct as to shame those who have denied it. These are signs that thinkers are not merely our humble allies in processing the complexity of the world, but instead need to enlist our support in nursing their grudges and giving them back the power they deserve.
Pinker is not free of all of this; I would have preferred more recognition of his limitations and the limitations of the data available to him, and more humility in his asides about culture and identity politics. But Taleb's writing shows all of these signs of dishonesty and resentment. (Gary Taubes is another.) When I read his writing--I, one reader, because there are so many other valid readings--I perceive viscerally that the writer feels cornered and that there is danger in the air. An ally in interpreting the world, secure in the soundness of his reasoning, doesn't write publicly that "Problem with Pinker boy is that he made statistical claims without knowing what he is talking about." A bully, who prefers being respected to promoting depth of understanding, does.
When I read polemical writing--which the world needs!--I try to ask myself if the writer is trying to grow my imagination, or constrict it; to seed ideas plentifully, or to poison them selfishly. It can be very hard to notice this through talented prose; read Michael Lewis with this lens, and his romps through tricky topics look quite different.
Labels: criticism, education, epistemology, history, parenting, politics, statistics