The everyday scientific method

I have never been entirely convinced that humans are an innately rational species.  We are all too easily fooled by the world around us.  For the best part of human history it was obvious that the sun went round the earth and those early pioneers of a heliocentric solar system certainly paid a high price for saying otherwise.  It takes some considerable effort to question the things around us and at times it takes considerable resilience to continue to question in the face of those that are just utterly certain. About a year ago I had an experience of how the scientific method could be applied to a simple domestic situation. I thought I would share it as there are parallels between science and pseudoscience in this everyday story.

I rented a house in Lincoln (UK) whilst I was an academic at the University’s School of Pharmacy.  Water in the house was heated through an electric immersion tank that was fitted with a standard ballcock valve, which opened to replenish the tank as the hot water was used-up. I noticed however, that there seemed to be a constant trickle of cold water into the tank irrespective of hot-water usage.

A lot of scientific discovery starts with pure curiosity, the “that’s interesting, I wonder what that is?” type question. A perfect example of this was when a molecular geneticist named Richard Jorgensen became curious about stippling on petunia petals. It was at first just a curiosity but his investigations led to the discovery of a whole new branch of genetics and won him the Nobel Prize in 2006.

I wasn’t going to win any prizes for it, but the constant trickle of water was something that needed to be investigated. At the offset it was just a superficial observation and in science these things can just vanish upon more robust inspection.  For example, the trickle of water may have only occurred under particular circumstances when I was coincidently in the house. One of the chracteristics of pseudoscience is to take the first superficial item of evidence that supports some preconceived idea and then never let go of it as proof of the position. The scientific approach on the other hand is to form a hypothesis and then test it. Richard Feynman used to say that first came a guess - if you have time then this video of Feynman is really worth watching.

So my hypothesis (guess) was that there was a leak somewhere along a hot water pipe and as the water slowly drained away, the ballcock value allowed a steady trickle of cold water to flow. I was fully aware however, that I could be wrong and so I needed to gather some data. I traced all the pipework to see if I could find a telltale pool of water.  There was none and hence no evidence for a leak. 

At first attempt my hypothesis seemed to be wrong and the constant trickle of water into the tank might have been quite normal.  The investigation so far was only cursory however, and so I decided further study was required. I therefore found the main valve feeding the tank’s cold water supply and turned it off for a day. The next morning the tank was completely empty; so where had the water gone?  I found the valve for the tank’s hot water outflow which Iturned off and the constant trickle of inflowing cold water stopped. With both the cold water feed valve and hot water outflow valve off, the tank remained full. Clearly that constant trickle of water was not normal.

Based upon my updated experimental evidence I needed to refine my hypothesis.  Although there were no visible signs of a water leak, the tank was still emptying in a matter of hours. The water had to be going somewhere but so far I could find no sign of it. Perhaps the water was leaking out somewhere I couldn’t see? I knew that one hot water pipe went under the floor to feed the kitchen tap.  It was a concrete floor and so I couldn’t get to the pipe to observe if there was a leak or not. 

Scientists figure things out all the time without necessarily making direct observation. Electrons, for example, have never been observed but the resulting actions of their flow is what we call electricity. When we switch the light on we know there's a flow of electrons along the wire because of the glowing lightbulb. In my case, I couldn't see the pipe under the floor and so in order to investigate a possible leak, I had to get a little more inventive. The underfloor pipe branched off from the main pipe at a T-junction. One end of the T went to the kitchen tap (under the floor) the other went into the bathroom.  So I came up with an idea that involved the flow of heat, rather than water. 

The pipes were made of copper which is a good conductor of heat. By opening or closing the kitchen and bathroom taps, I could follow the flow of the water simply by feeling the heat running along the pipes. In technical terms, the heat was a surrogate for the water flow.  With all the taps off there should be no flow of hot water but when I did the experiments I found that the pipe that went under the floor remained warm.  The pipe on the bathroom side of the T-junction remained cold. The conclusion was that there was a flow of hot water (detected from the heat of the pipe) going under the floor, even when there should be none.  The most likely explanation therefore, was that the pipe was leaking somewhere along its length under the floor. To be fair if I was going to be really scientific about it I would have measured the temperature of the pipe in various locations and then plotted out the heat-flow data. The ability to measure some effect is a cornerstone of the scientific method. I am a nerd, I admit, but it would have taken things to new nerdy heights in this case if I had got out a thermometer and notebook. I admit therefore, I left that particular experiment out. 

On the weight of evidence, my hypothesis could now be promoted to a theory that there was a leak in the hot water pipe under the concrete kitchen floor.  The theory was consistent with the known laws of physics - for example, water flows downwards under gravity, copper conducts heat etc. In science, any theory has to be consistent with what is already known.  If some theory in biology has some aspect that breaks a law of physics, then the biological-theory is likely to be wrong. If my theory had demanded that water flowed upwards, for example, then my theory would obviously be wrong.  A hallmark of a pseudoscientist is when they demand that the whole textbook of scientific understanding has to be re-written to support their own ideas, typically based upon the most flimsy of evidence. In my own case however,  it was time to call the landlord and get a plumber.

I explained all the evidence of my diagnosis to the plumber but he did not like the outcome.  I could tell that the thought of having to uproot the floor did not appeal to him. He therefore said that he would replace the ballcock valve, which he said must be leaking (hence the constant trickle of water). 

I explained to the plumber that the ballcock valve could not be the problem because turning off the hot-water outflow stopped the constant trickle of inflowing cold water. When the tank was full, the ballcock valve did its job perfectly well. The plumber however, really didn't want to hear it and discarded this evidence from his diagnosis. He told me that he’d been a plumber for 20 years and so he knew what he was doing. An appeal to authority however, was not consistence with the actual evidence. The problem was that he didn’t want to accept where the evidence was leading simply because of the personal consequences to himself.  Science frequently encounters such self-motivated objections. The economic consequences of reducing carbon emissions, for example, are so profound that the theory of climate change must be wrong!  Getting back to the reluctant plumber, after a little discussion, I got him thinking and he said he would check something out.

He went outside the house and dug down by a wall next to the kitchen. It didn’t take long before he hit mud.  What he was hoping for was dry soil which would be counter-evidence for a water leak. In fact, dry soil under these circumstances would not have been particularly diagnostic one way or the other because the leaking water could have been flowing in any direction under the house. In science, some evidence is more robust and other evidence is more circumstantial. It’s a common fallacy to confuse the two as being equal.  The wet ground in this case, was supporting evidence for the leak-theory but the plumber then rationalised the presence of the mud by saying that it had been a wet year and it could just be ground water.

"An unsinkable rubber duck" is a term used by rational thinkers when someone finds a way round every item of evidence presented to them in order to support their own preconceived position. Young earth creationists who believe the universe is less than 10,000 years old are masters of the unsinkable rubber duck. Point out that light from distant stars has taken many millions of years to reach the earth and they will claim that the speed of light has changed over time. Likewise, no matter what the outcome from the plumber’s test-dig by the kitchen wall, it was either going to reinforce his own viewpoint or at least not change it. The muddy ground by the outside wall was far from conclusive but put this together with everything else and the convergence of evidence all pointed one way.

I had to be careful however, because although the presence of the mud seemed to support my leak theory, the possibility that is was coincidental ground water could not be ruled out. Scientifically, I could have added a tracer dye to the hot water to see if it ended up in the mud. Even for me however, this seemed a little over-the-top and so instead, I suggested that he dig another test hole around the other side of the house to see if that was wet due to the water table. He thought about it but declined.  To him, unwillingness to further test his theory seemed to reinforce his own position. In science it is only when a theory stands up to attempts to disprove it, that it really becomes robust. Pseudoscientists rarely understand this. Instead of trying to disprove their beliefs, they do what this plumber did and simply select those items of evidence that support their view. There is then a reluctance to do further testing using any method that might illicit any doubt.

From the plumber’s perspective I was probably appearing to be a very difficult customer.  I stayed polite the whole time and even made light of some it, just to keep things convivial. The plumber however, was getting more and more irate. I was just following the evidence, albeit that it was with somewhat unpleasant consequences. Irateness and sometimes downright offensiveness is another hallmark of the pseudoscientist. Just question a homeopath on the validity that greater dilution leads to greater therapeutic potency and then wait for the torrent of abuse. 

Scientifically, my water-leak theory would have been tested by others. Someone may have come up with some new ingenious method and others would have confirmed or failed to reproduce my own data. Nothing would be taken for granted until tested and retested. Then those tests would be challenged and retested and so on, with the results either reinforcing the theory, leading to modifications of the theory or even to discard it and start again.  Pseudoscientists on the other hand are content in accepting the first whiff of flimsy evidence that might vaguely support their point of view and ignore far more robust evidence that is against them. The case of my hot water was a little more domestic and anyway, I was getting tired of having to shower at work. 

So what happened?  The plumber and I did not pass on particularly good terms.  I got on well with the landlord and so he let me find another plumber for a second opinion. The second plumber had a very different attitude. His curiosity was refreshing and he took great interest in the way I had diagnosed the problem.  He challenged my assumptions on several occasions, which led to some interesting discussions. Once he agreed that there was very likely a leak under the floor, he came up with a straightforward solution and simply put in a new pipe above floor level, bypassing the one with the leak. It took him about an hour and afterwards the problem was solved. The constant trickle of water to the tank stopped. Over a cup of tea to celebrate a job well done, the plumber looked down at the pipes, thought for a moment and said, "lucky there weren't two leaks in different places, because that would have really messed up your experiments".  This possibility had not occurred to me and of course he was right, two separate leaks could have made the results of my experiments very confusing. A scientific theory has to be repeated by others who may look at things a very different way. Our understanding is built piece my piece progressively until a coherent picture emerges. Of course, in science this whole affair would have opened other questions such as, exactly where was the leak and what was the cause? Even without knowing this level of detail however, the theory of an underfloor leak in a hot water pipe had been confirmed. This illustrates another trait of pseudoscience - any gap in knowledge of an opposing theory is evidence that the whole theory is wrong. "You can't show me every intermediate animal from a dinosaur to a bird, therefore the theory of evolution is wrong," for example.  The gaps can also sometimes be filled with any old nonsense, such as aliens that lived under my house were stealing the water - that type of thing.  Well, you can’t disprove it, can you? And it fits the evidence that you have. 

And just a final note. My persistence with the first plumber was not about being right, it was about the evidence.  All too often there are those that evoke the name of science to try and prove themselves right at any cost. Science is the exact opposite of this.  For my own part in the situation with the first plumber, if he could find better evidence to support a different theory then I would have been very pleased to change my view. Even to the point if an alien had suddenly appeared out of the test-hole!