| In 2000 Michael Shermer co-produced and co-hosted a 13-hour television series for the Fox Family Channel called Exploring the Unknown. The opening line of every episode was: "Things are not always what they seem when you are exploring the unknown!" Things are not always what they seem because we do not live in a black and white world of unambiguous yeses and noes. We are faced here with what philosophers of science call a "boundary problem"—where do we draw the boundary between orthodox science and heretical science, or between science and pseudoscience, or science and nonscience? The boundary is the line of demarcation between these geographies of knowledge, the border defining these countries of claims. The problem with this geopolitical analogy is that where rivers and mountain ranges, deserts and seas help geographers demarcate the boundaries, knowledge sets are fuzzier. It is not always clear where to draw the line. Whether a particular claim should be put into the set labeled science or pseudoscience will depend on both the claim and the definition of the set. But when we encounter a particular claim there are some questions we can ask. - 1. How reliable is the source of the claim?
- Scientists are usually reliable; pseudoscientists unreliable. As Daniel Kevles showed so effective in his book The Baltimore Affair, in investigating possible scientific fraud there is a boundary problem in detecting a fraudulent signal within the background noise of mistakes and sloppiness that is a normal part of the scientific process. The investigation of a particular set of research notes in a laboratory affiliated with Nobel laureate David Baltimore (and run by Imanishi Kari) by an independent committee established by Congress to investigate potential fraud, revealed a surprising number of mistakes. But science is messier than most people realize. Errors and sloppiness in raw data happen. The question is: can a distinction be made between intentional and unintentional distortion of the data and interpretations? Baltimore and Kari were exonerated when it became clear that there was no purposeful data manipulation.
- 2. Does this source often make similar claim?
- Pseudoscientists have a habit of going well beyond the facts, so when one individual makes numerous such claims it is a sign that they are more than just iconoclasts. Again, this is a matter of quantitative scaling, since some great thinkers often go beyond the data in their creative speculations. Cornell scientist Thomas Gold is notorious for his radical ideas, but he has been right often enough that other scientists listen to what he has to say, and those same scientists are also testing these ideas for their validity. Gold's book, The Deep Hot Biosphere for example, proposes the heretical idea that oil is not a fossil fuel at all, but the by-product of a massive subterranean colony of bacteria living in rocks. Hardly any earth scientists I have spoken with take this thesis seriously, yet they do not consider Gold a crank. Why? Because he plays by the rules of the game of science. What we are looking for here is a pattern of fringe thinking that consistently ignores or distorts data.
- 3. Have the claims been verified by another source?
- Typically, nonscientists and pseudoscientists will make statements that are unverified, or verified by a source within their own belief circle. We must ask who is checking the claims, and even who is checking the checkers. The biggest problem with the cold fusion debacle, for example, was not that Stanley Pons and Martin Fleischman were wrong; it was that they announced their spectacular discovery before it was verified by other laboratories (at a press conference, no less), and, worse, when cold fusion was not verified anywhere they continued to cling to their belief in the phenomenon despite the lack of evidence.
- 4. How does this fit with what we know about the world and how it works?
- An extraordinary claim must be placed into a larger context to see how and where it fits. When pseudoarchaeologists claim that the sphinx was built over 10,000 years ago by an advanced race of humans (because the sphinx shows signs of water weathering that could not have happened after the end of the last ice age), they are not presenting any context for that earlier civilization. Where are the rest of the artifacts of those people? Where are their works of art, their weapons, their clothing, their tools, and their trash? This is simply not how history or archaeology works.
- 5. Has anyone gone out of their way to disprove the claim or has only confirmatory evidence been sought?
- This is the confirmation bias, or the tendency to seek confirmatory evidence and reject or ignore disconfirmatory evidence. The confirmation bias is powerful and pervasive and is almost impossible for any of us to avoid. It is why the methods of science that emphasize checking and rechecking, verification and replication, and especially attempts to falsify a claim, are so critical. There is so much evidence against cold fusion, for example, that all but a handful of die-hard physicists, chemists, and hopelessly optimistic futurists long ago gave up conducting further research. Yet the purveyors of "infinite energy" (there is even a magazine of this title) cling to the slimmest of experimental results and blithely sweep the disconfirming evidence under the rug of conspiracy theories where, for example, oil and electrical conglomerates are said to be preventing the positive evidence from reaching the American public.
- 6. Does the preponderance of evidence converge to the claimant’s conclusion, or a different one?
- The theory of evolution, for example, is proven through a convergence of evidence from a number of independent lines of inquiry. No one fossil, no one piece of biological or paleontological evidence has "evolution" written on it; instead there is a convergence of evidence from tens of thousands of evidentiary bits that adds up to a story of the evolution of life. Creationists conveniently ignore this convergence, focusing instead on trivial anomalies or currently unexplained phenomena in the history of life.
- 7. Is the claimant employing the accepted rules of reason and tools of research, or have these been abandoned in favor of others that lead to the desired conclusion?
- UFOlogists suffer this fallacy in their continued focus on a handful of unexplained atmospheric anomalies and visual misperceptions by uninformed eyewitnesses, while conveniently ignoring the fact that the vast majority (I estimate 90 to 95 percent) of UFO sightings are fully explicable with prosaic answers.
- 8. Has the claimant provided a different explanation of the observed phenomena, or is it strictly a process of denying the existing explanation?
- This is a classic debate strategy—criticize your opponent and never affirm what you believe in order to avoid criticism. But this stratagem is unacceptable in science. Proponents of the pyramids as being built by pre-Egyptians offer no evidence of just who these people are, and instead just pick at anomalies in the work of Egyptian archaeologists. Critics of the Big Bang ignore the convergence of evidence of this cosmological model, focus on the few flaws in the accepted model, and have yet to offer a viable cosmological alternative that carriers a preponderance of evidence in favor of it
- 9. If the claimant has proffered a new explanation, does it account for as many phenomena as the old explanation?
- The HIV-AIDS skeptics argue that lifestyle (drug use or promiscuity, coupled to a correlation with a naturally-weakened immune system), not HIV, causes AIDS. To make this argument they must ignore the convergence of evidence in support of HIV as the causal vector in AIDS, and simultaneously ignore such blatant evidence as the significant correlation between the rise in AIDS in hemophiliacs just after HIV was inadvertently introduced into the blood supply. On top of this, their alternative theory does not explain nearly as much of the data as the HIV theory.
- 10. Do the claimants' personal beliefs and biases drive the conclusions, or vice versa?
- All scientists hold social, political, and ideological beliefs that could potentially slant their interpretations of the data. The question is: how do those biases and beliefs affect the research? It is true that even the most well-intentioned scientists may find themselves searching for facts to fit their preconceptions. But at some point, usually during the peer-review system, such biases and beliefs are rooted out, or the paper or book is rejected for publication. This is why one should not work in a vacuum. Intellect stumbles and falters without critical feedback. If you don't catch your biases in your research, someone else will.
|
Operation Clambake present: