My theory is scientific, but don't expect me to do science

Back in the Dover trial when Intelligent Design advocates had to square off against some actual biologists, one of the biggest hurdles the IDer's had to face is that they hadn't actually produced any scientific research. Here they were, claiming that they had a scientific theory that deserved equal time with evolution in public school classrooms, and when they were challenged on what empirical evidence they had for their theory, they produced a whole lotta nothing. Kenneth Miller, the savvy evolutionary biologist from Brown who served as the key witness in the Dover trial (and whom, I ought to mention, is a Christian), has frequently taken IDer's to task over their failure to produce research. He did it at the trial, much like he did here while discrediting one of the key concepts in ID:

The point about research is a big one. Science is a methodology for understanding reality. It's the most successful such methodology we've ever developed, and its tremendous success is evident simply in the fact that you're reading this on a computer screen instead of hand-written parchment.

The scientific method is empirical, and it must be replicable. From the almighty Wikipedia:

To be termed scientific, a method of inquiry must be based on empirical and measurable evidence subject to specific principles of reasoning.^[2] The Oxford English Dictionary defines the scientific method as: "a method or procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses."^[3]

And...

Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, giving them the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established (when data is sampled or compared to chance).

I thought that was all pretty uncontroversial. We all learned some basic outline of the scientific method in grade school, but it goes something like this:

• Observe data
• Develop theory to explain data
• Use theory to generate testable hypothesis
• Record data from experiment
• If the hypothesis is successful, let other scientists replicate the process

This topic arose today in another discussion on Randal Rauser's blog following his review of a book extolling near-death experiences, and it's frustratingly typical of theists. They want it both ways – to use words like "evidence" and "science", but then squirm at the suggestion that scientific claims need to follow the scientific method.


A word on near-death experiences (NDEs)

The fact that people have NDEs is uncontroversial. The data, if we follow the method above, is that people have these experiences and describe them in specific ways. The theory that follows attempts to explain the data. In this case, the theory is that people experience NDEs because at the moment of death their souls become disembodied and they remain conscious. We can call this the "disembodied soul theory of NDEs", which I'll call soul-theory for brevity.

Next, we use the theory to generate a testable hypothesis. How might the soul-theory be tested? Well, your guess is as good as mine. One thing's for sure: those who advocate it have utterly failed to postulate any testable hypothesis. And guess what? If it ain't testable, it ain't science.

Now, if on the other hand NDEs can be explained by brain activity, there are certain things we might expect. We might expect cultural influences to provide stark differences in the nature of NDE reports; we would not expect to find any evidence of consciousness when a person's brain was completely inactive; and we would expect to be able to reproduce some or all of the phenomena associated with NDEs under laboratory conditions. Well, we've done all of those things!


What is science? Baby don't hurt me. Don't hurt me no more.

This all brings me to Randal's assertion that the soul-theory of NDEs ought to be taken seriously as a science. His evidence consists of "popular anecdotes of a light at the end of a tunnel and one’s life flashing before their eyes". Anecdotal evidence has another name: hearsay. The reason why case studies, anecdotal reports and hearsay are not taken seriously as science is because such evidence is prone to a litany of very human biases and errors. Data can be unintentionally misrepresented, patients can form false memories, and details may be filled in by others as the story is told – not unlike a psychic doing a "cold reading".

Randal gives the example, from the book, of Pamela Reynolds – a relatively famous case. Randal claims that "This is one of many powerful cases that are particularly resistant to naturalistic analyses". I think Randal's suffering from a limited imagination. But even the most charitable account of Reynolds' case must admit that her story is purely anecdotal. Staffers could have "helped" her fill in details; she could have confused various time frames, and even formed false memories. While her story may be superficially compelling, there is simply not enough data – nor any controls – that enable her story to constitute scientific evidence of soul-theory.

When I challenged Randal on these matters, he responded as follows:

Whether you call it "science" or not is a matter of social conventions as to how "science" is defined. The real question is whether NDEs provide evidence for the survival of the person apart from the body which would in turn falsify naturalism (or at least materialism; naturalism is sufficiently amorphous to survive just about anything).

I am not sure what you call a "pseudoscience". The careful documentation of medical case histories? That's pseudo-science? Or just the careful documentation of medical case histories that differ with your received view of the world?

What I am calling "pseudoscience" is the posturing of anecdotal evidence, case studies, and hearsay as compelling evidence that NDEs are best explained by disembodied souls experiencing consciousness and the continued evasiveness in suggesting that this "disembodied mind theory of NDEs" need not produce falsifiable hypotheses or replicable empirical data – yet for some reason, we should take it seriously as a science anyway.

Even worse is Randal's retreat to a semantic argument over the definition of science. Here's a tip: when you have to fuss over the definition of science just to get your theory taken seriously as a science, you're off to a really bad start. Science is empirical. It requires data, observation, and measurement. Theories must be falsifiable, and to do so they must generate testable hypotheses. Once hypothesis are tested, the research must be replicated in order to account for the possibility that the data was skewed by the types of discrepancies, mentioned above, that can occur with anecdotal evidence. This is uncontroversial – the definition of science is as well-established as it is a successful methodology.


Conversations doomed from the start

Randal's sloppy defense and poor understanding of science was frustratingly typical of those advocating his position, judging by some of the other comments on the post:

JohnM: "I'm not making any theories. I'm just looking for the best explanation. And so far you have offered no alternative."

Me: "A theory is by definition an attempt to explain data."

JohnM: "No. An explanation is an explanation. And a theory is a theory. Those are 2 different words, not the same word."

And then, on the subject of replication, we have this gem from commenter "Tim":

Does that mean we'll have to scientifically replicate the phenomena of life arising from inert matter before we can believe that's how life began on this planet?

At a certain point, it's evident that I'm dealing with people whose scientific literacy is painfully lacking. Granted, I'm no scientist myself, but I study enough of it – though popular science books and my trusty long-running subscription to Scientific American – to know the basics of how it works. Randal wants to call into question the definition of science in order to elevate anecdotes to the level of controlled research. JohnM wants the word "theory" to mean something different than what it does. Tim is unaware that we can test theoretical explanations of past events not by replicating the events, but by predicting what data will be unveiled with future research and replicating the steps that led to that data.

And yet, through all of this, I'm supposed to take the disembodied soul theory of NDEs as serious science? Color me unimpressed.