5X5 Episode 100

5X5 Episode 100
Testable
13th January 2011
5X5 99	5X5 101
Skeptical Rogues
S: Steven Novella
R: Rebecca Watson
B: Bob Novella
J: Jay Novella
E: Evan Bernstein
Links
Download Podcast
Show Notes
Forum Topic

Testable

Voice-over: You're listening to the Skeptics' Guide 5x5, five minutes with five skeptics, with Steve, Jay, Rebecca, Bob and Evan.

S: This is the SGU 5X5 and tonight we're talking about testable versus untestable claims in science. The concept of whether or not a claim or a belief, a theory, a hypothesis if you will, is testable is very important, essential in fact, to the scientific method. "Essential" refers to the fact that in order for something to be scientific, it needs to be subject to either observations or experimentations that can distinguish it from other claims or other alternative hypotheses or that can be used to say that a hypothesis or claim is probably true or is probably not true. Karl Popper, the philosopher, took this to the point of saying that in order for a theory to be scientific it not only has to be testable, it has to be falsifiable, meaning that a test has to be able to prove that the theory is false. There doesn't necessarily have to be a test that can by itself demonstrate or prove that a hypothesis is true, but it definitely needs to be possible to prove that it is false. So while this is definitely a necessary criterion for a theory or claim being scientific, it is not sufficient and you know there are other criteria that are also important, but that does remain a very critical component, that is essential to the very concept of science and the scientific method.

E: Yeah, a good example of a untestable claim is the old creationist argument that God created the world to appear exactly as if it had evolved naturally over 4 billion years with fossils and everything else. This claim is certainly consistent with the evidence but it makes no predictions that can be tested against any future observations. In fact it's designed to eliminate any observable distinction between an evolved and a created world. It's therefore important to identify such claims as untestable and therefore non-scientific because such claims are worthless to the advancement of knowledge.

R: Now that's not to say that all supernatural claims aren't testable. There are plenty of supernatural claims that are perfectly testable. Anything you can think of in terms of psychics and ghosts and UFOs can be tested so long as it has some impact on the real world. So if you consider a ghost, for instance, that might be haunting your house. If it makes a sound, or if you see something out of the corner of your eye, that means that it's effecting the everyday real world and that means that we do have a chance to examine it in a scientific manner. Anything that interacts with the real world can be tested scientifically.

B: Often it can be thoroughly easy to determine if a claim is testable or not. It's ultimately a matter of determining if a claim has any consequences to the real world. If 'yes' then it should be testable; if 'no', then it's not testable and therefore not science. Sometimes though it can be very difficult to devise a test for a specific claim. Sometimes you can mistakenly conclude that a claim is untestable and outside the realm of science. The Big Bang Theory, for example, describing the beginning of the universe, was once considered to be untestable. In fact some scientists in the 1940s likened it to the religious creation myths in this regard. It was theorized though that the radiation released by the Big Bang would have certain predictive characteristics and it took a while but the technology was eventually capable of testing these claims and as we all know showed it to be true. So be wary of claims that seem untestable but in reality are. Of course the flip-side is to not let pseudo-scientists use that as an out for their truly untestable claims.

J: The details of a legitimate test must be extremely specific in order for legitimate information to be produced. The more vague a test is the less likely it will produce any usable information. So, as an example, when defining what type of study or test you're going to conduct, your terminology has to be predicted and defined. Every term that's used, you even need to go as far as try to predict all the possible outcomes in order to even understand what those outcomes mean.

S: I think an important point to bring up is that not all notions which are untestable are untestable in the same way though. Bob, you brought up the example of the Big Bang, for example, whereas Evan brought up the example of the earth is created in the recent past but made to look exactly as if it's billions of years old. There are some claims that are inherently untestable. They are formulated in such a way that there is no possible observation that can be made for or against it, and therefore it is forever outside the realm of science. There are other hypotheses or theories or notions which are just practically untestable. We don't have a way now to test it, but there's nothing inherent to the claim itself that makes it untestable. There's just no practical way that anyone has figured out so far. We just don't have the technology at the present time in order to be able to test it. And that's where you get into a little bit of a gray area. Are such notions scientific? I think that they are, but they're just at a preliminary stage of science. They have not yet closed the loop if you will, until somebody actually figures out a way to make an observation or some experiment that can in fact test the hypothesis. But that distinction I do think is important to make and I also want to emphasize what Bob said and that is sometimes claims are made specifically to be unfalsifiable and often those who promote fairly dubious claims will do this whether consciously or unconsciously in order to insulate their claim from falsification, so they fail to meet Popper's minimal criteria of falsifiability. The now classic literary reference to this is Carl Sagan's invisible floating dragon that breathes heatless fire. There maybe such a creature in my garage but if you formulate the claim in such a way that no possible test can falsify its existence, then you have simply placed it outside the realm of science. That doesn't make it wrong. It makes it not even wrong, which is worse in the eyes of science.

S: SGU 5x5 is a companion podcast to the Skeptics' Guide to the Universe, a weekly science podcast brought to you by the New England Skeptical Society in association with skepchick.org. For more information on this and other episodes, visit our website at www.theskepticsguide.org. Music is provided by Jake Wilson.