Thomas Jefferson once encountered a jaw-dropping claim.
The claim that confronted Jefferson was from eyewitnesses who said that they had seen rocks fall from the sky. They even claimed to retrieve fragments from them.
And here’s the kicker: the witnesses, under normal circumstances, would be considered pretty darn good ones: two Harvard professors.
But Jefferson did not believe their claims. Insofar as Jefferson knew, rocks don’t fall from the sky, and so the witnesses, in his estimation, were either lying or simply mistaken.
Concerning the incident, he said the following: “I could more easily believe that two Yankee professors would lie than that stones would fall down from heaven.”
Jefferson, in other words, possessed a Weltanschauung—a worldview, a paradigm, a model for the way the world is and works—that could not accommodate such a claim at face value. What Jefferson thought he already knew simply did not match what the witnesses said. He saw no way to make the claim and his existing model cohere, and so he dismissed it and did not question his model. He concluded that it was physically impossible for such a thing to literally occur. We now know, of course, that rocks can and do sometimes fall from the sky—as meteors.
In this instance, Thomas Jefferson’s error was the following: he did not bring critical reflection upon both the new claim and his existing view of the world. It was good that he sought coherence between the claim and his worldview (coherence is crucial to critical thinking: what we think we already know about the world should match new claims about the world). But Jefferson failed to reevaluate what he took for knowledge in the light of a new and emphatic claim from otherwise reliable sources: two Harvard professors.
There are three lessons here: (1) worldviews—your own and others—should be made explicit in the evaluation of claims, for they profoundly color interpretations of evidence; (2) when confronting a new claim, bring as much critical skepticism to what you think you already know as you do to the new claim; and (3) seek coherence: new claims should fit into a larger framework of knowledge.
For critical thinking tip #3, see here.
Prometheus Unbound 
Pingback: Critical Thinking Tip #1: Make Explicit Your Weltanschauung (Worldview) | Prometheus Unbound
Jefferson’s response was very much a Humean one. It’s not entirely unreasonable given that he was essentially saying, “Anecdote is not evidence,” – that is a good rule.
However(!), what he should have also asked himself was, (1) “Are these two witnesses likely to lie?”, and, (2) “If they are telling the truth, what could be a possible mechanism to make rocks do this?”
If he had asked the second question, he could very well have made the necessary pioneering science efforts into astronomy and became famous for quite another reason than he is. Now that is a good extra lesson: Don’t be afraid to do good science on unusual hypotheses.
Incidentally, I don’t wholly agree with your third lesson. Sometimes paradigms change. If we didn’t have the courage to allow new discoveries to break our belief systems Doctors would still be using leeches to cure humors and our observatories would be manned by Astrologers.
Iain,
You make good points, and I agree that “don’t be afraid to do good science on unusual hypotheses” is good advice.
As for the third lesson, perhaps I was just unclear. What I meant to say is that all new knowledge should cohere with all of our background knowledge. Truth is not a buffet. If, for example, we believe in ESP we must also admit that it does not cohere with our current background knowledge of physics (what we think we know right now about how information is communicated across space and time). At some point our particular claims to knowledge and our general claims to knowledge must come into alignment (either our view of psychic phenomena or our view of our current background knowledge must change).
Do you disagree with that?
I’m also thinking of the researcher who claimed recently to have found arsenic-based life at Mono Lake. If other biologists conclude that she is probably right, our background knowledge must change: biologists must come up with a theory that accounts for the phenomenon. They might also discount or dismiss the claim (as Jefferson did with the rocks falling to earth claim). Either way, the particular claim must be coherent with what we think our background knowledge surrounding the claim is.
—Santi
I agree, sometimes what they call a “consilience of evidence” is good enough to be “fact” (take evolutionary biology, for example). We need a coherent web of ideas to make sense of the whole picture. A lone data point doesn’t do anyone any good.
I can explain the arsenic bacterium thing. My explanation will be vague since I read about it a while ago now but I’ll provide a link with a good explanation. It’s not quite what The News made it out to be. Basically, life uses certain specific elements. Arsenic is functionally close to the Phosphorus that life normally uses. In a restricted environment, the bacteria can use arsenic to temporarily do the same job (in a more limited way). The bacteria is still “normal” life, with the same biological origins as all other life, it’s just able to be a little bit more flexible and stay alive (in a slightly weaker state) in harsh environments. It isn’t designed (so to speak) to be optimal using arsenic. At least, that’s how I understand it.
I believe this podcast has a better explanation of the details. http://www.theskepticsguide.org/archive/podcastinfo.aspx?mid=1&pid=284
Still, your point is good. People need to study examples like this and do good science rather than jumping to conclusions and either making hasty news reports affirming wacky claims OR rejecting the idea outright.
Iain,
Thanks for linking here. I read your article on the upending of astrology; I only just heard about it yesterday through a friend. I didn’t know the details. Your article inspires me to look into it some more. You’re right that it’s an excellent representative sample of “woo.”
As for arsenic based life, it sounds like you also hit the nail on the head there; that’s my understanding as well. But there are some additional tests to be done this year that should (supposedly) clarify this issue. We’ll see.
—Santi
I just referred to these two posts in the middle of my own newest post. Thought you might want to know 🙂
http://phrenicphilosophy.blogspot.com/2011/01/anatomy-of-pseudoscience.html
Iain – I followed your link, but then first checked your ‘profile’. (I like to have some idea of a blogger’s Weltanschauung). So I almost didn’t read your post. (If someone asks me what my star sign is, or, as you do, volunteers theirs to me, I assume they have mush for brains).
I persisted, on this occasion, as your comments to Santi contradicted the above assumption, as did your post, which I liked, thank you – although a bit didactic.
Colin
Haha thanks, Colin. I think the star sign thing on the profile is a function of the blogger template rather than my own beliefs… but it’s quite ironic given my latest post.
I think you’re right about the didactic nature of the post. I wasn’t trying to be generous. That’s one of my failings, sometimes. The other failing is when I also don’t try to be concise!
Oh I see, perhaps you mean when I mention Ophiuchus? That is merely an example. The changes represent, to me, a symbolic destruction of the whole notion of the zodiac. So when I say, “I’m Ophiuchus,” I really mean nyah nyah astrology sucks and they don’t know what ‘evidence’ means 😉
Sorry for the confusion!
Iain – I didn’t miss the Ophiuchus irony. However, try to overide the blog template as regards its volunteering the ‘saggitarius’ information.
Colin.