The reasons why experiments fail have nothing to do with the outcome, and everything to do with how you achieve the outcome. Unfortunately, too many people are under the false impression that an experiment that doesn’t turn out the way you wanted is a failure.
We invest a lot of time and energy into experiments. When done right, they can change our worlds like nothing else. Since they are so powerful, you need to know why experiments fail.
Why It’s Not About the Outcome
Experimentation can prove that the best course of action is to maintain the status quo. When you get this result, you should revel in the success. You’ve avoided making a change that won’t work.
But when it happens personally, we might feel like we should have known the outcome before investigating it. We ruminate over the experiment design and execution. We certainly don’t want to tell anyone that our potential insight was not proven. The failure to publish happens so often; there is a name for it – publishing bias.
However, great minds think alike. How many great minds does it take to be led into thinking something, test it, and find the illusion before one of them stops the insanity?
If you don’t realize your experiment has failed, you might end up doing the exact wrong thing or convince others to do it too.
In 2015, a report found that fewer than half of nearly 100 published psychology findings held true in follow-up studies. The lack of replication is a genuine failure of experimentation. Psychologists are pushing to change how their field works to reduce these failures.
A failed experiment is when you cannot draw a valid conclusion, or you unknowingly draw the wrong conclusion. Here are five common reasons why experiments fail.
An inappropriate test is one in which does not directly relate to or cover the hypothesis.
Doctor Stubbins Ffirth wanted to prove Yellow Fever was not contagious. Glossing over the details, he used fluids from patients in various ways and then spread the word of his ability to resist the disease. However, the absence of evidence is not evidence of absence. We also know that Yellow Fever is contagious in the early stages, but Ffirth’s samples all came from late-stage patients. Make that a double whammy of a failed experiment due to an inappropriate test.
You are exposed to uncontrolled boundaries messing up your experiment when you don’t understand the system fully enough to isolate it.
As Alexander Fleming found, that isn’t always the case. He is famous for the discovery of penicillin. A culture of a flu virus become contaminated. He noticed the area around the contamination was clear of infection. That critical observation led to the discovery.
When you can’t control a variable and know it will have an impact, that variable is called noise.
Mother nature is a usual suspect of noise variables, including things like temperature, humidity, and atmospheric pressure. You can correct for noise variables in an experiment, when you know about them and plan for them. Forget to record these vital pieces of information during the experiment, and your collected data will be useless, and you will have to repeat the experiment.
Ineffective Measurement System
For a measurement system to be effective, it must be capable of several functions. It must be capable of discerning the change with accuracy and precision, and like your experiment, do it time and again.
Experiments that rely on surveys are notoriously questionable. As Daniel Gilbert explains in Stumbling on Happiness, we think much more highly of our future selves, and therefore when asked about what we will do in the future, it’s more positive than accurate.
In one experiment, researchers wanted to prove movie theaters would be able to sell healthy snacks if they offered them. They asked moviegoers “if they would” purchase, prefer or choose a healthy snack. Ask them what they want right now, though, and that’s a different story.
When numbers show up in black and white, the data might look solid, but don’t be fooled with less than the facts.
Equipment can malfunction without your knowledge.
It can lead you down the wrong garden path, but how about when experimenters think their equipment is malfunctioning when it isn’t?
As Isaac Asimov said, “the most exciting phrase to hear in science, the one that heralds discoveries, is not ‘Eureka’ but ‘that’s funny.’” In 1978, Arno Penzias and Robert Woodrow Wilson experimented with the Holmdel Horn Antenna. As they eliminated all interference, they were unable to detect the source of a low, steady, noise that seemed to come from everywhere and anywhere. Finally, they accepted the equipment was functioning properly, and this was the first recognition of cosmic microwave background radiation.
Experiments Gone Right
A successful experiment is one in which a researcher can formulate and test a hypothesis in a repeatable manner, such that other researchers can execute the same experiment and achieve the same result. Knowing what doesn’t work helps other people from wasting time and energy to investigate. Do you have any experiments which you had previously thought of as failures and now realize were successes?