Defending the hypothesis as a framework for scientific experimentation
Scientific experiments help us learn more about reality.
The design and interpretation of experiments is guided by specific frameworks, each with its own underlying philosophy. But most scientists aren't taught this. It's vital we understand experimental frameworks so that we can be truly aligned with knowledge growth.
Here, I'll cover two frameworks: the hypothesis and the question. And I'll argue that the hypothesis is the ideal framework for experimental science.
Frameworks for experimentation
Scientists often use hypotheses to frame experiments. A hypothesis is simply a statement of the expected outcome of an experimental test. And based on the results of a carefully designed experiment, we accept or reject our hypotheses, generating knowledge as a result.
The hypothesis framework has been criticized for its potential to cause confirmation bias 1. This occurs when researchers anticipate a specific outcome and may selectively report or analyze data in a way that supports their hypothesis, leading to poor, irreproducible science.
It has also been argued that the hypothesis is incompatible with experiments that generate large amounts of data. For example, exploratory experiments that use 'Omic' technologies, such as proteomics, transcriptomics or metabolomics, cannot practicly be framed with hypotheses. Rather, these so-called 'fishing expeditions' are hypothesis-generating in nature 2, meaning that follow-up work is needed to test hypotheses formed from the initial data gathered.
To address issues of bias, some scientists have turned to initiatives within the scope of open science, such as registered reports and preregistration, which aim to improve the openness, integrity, and reproducibility of research. These strategies have proven to be effective so far 3, but they haven't been fully adopted by the scientific community.
Another proposed solution to the hypothesis is the question framework 4. Here, an experiment starts with a question, which is asked in order to derive data that can be used to build a model. And the model is intended to be held up for verification — its value apparently lying in its predictive power.
Because experiment outcomes aren't specified with a question, the scientist is somewhat operating in a state of ignorance. This may help to reduce bias. Moreover, using a question is arguably more practical when exploratory research is being performed with Omic-technologies.
Philosophies of experimental frameworks
The question framework relies on a philosophical approach called inductivism: the idea that scientific theories are derived from observations. According to this approach, we can form general conclusions about the world based on specific observations, and as the number of affirming observations increases, our theories become more justified.
Let's use a basic example to illustrate inductive reasoning. Imagine that you see a white swan for the first time. As time passes, you continue to see white swans everywhere you go. Having accumulated many observations of this kind, you generalize that "all swans are white". And you become more confident in your theory each time you see a white swan.
But inductivism has several flaws. For example, it wrongly assumes we can derive theories directly from observations. There's a logical gap that cannot be bridged here; we can't deduce that observations made under specific conditions will hold true in other similar situations.
Inductivism also mistakenly views prediction as the aim of science. Although our best theories contain accurate predictions, their predictive power is only secondary to their explanatory content. The generalized predictions (so-called theories) made via induction aren't analogous to new scientific theories, because they don't contain explanations that answer why and how observations come to be. Indeed, it's the ability of scientific theories to explain the physical world that is fundamental 5. Yet under inductivism, two or more theories with different explanatory contents, that make the same valid predictions, are considered just as good as each other. Clearly, these theories would have vastly different utility in the real world (as was emphasized by Deutsch in the Fabric of Reality).
"Suppose that one day the farmer starts bringing the chickens more food than usual. How one extrapolates this new set of observations to predict the farmer's future behaviour depends entirely on how one explains it. According to the benevolent-farmer theory, it is evidence that the farmer's benevolence towards chickens has increased, and that therefore the chickens have even less to worry about than before. But according to the fattening-up theory, the behaviour is ominous — it is evidence that slaughter is imminent." The Fabric of Reality.
Although prediction isn't the purpose of science, it's a central component of the scientific method. The outcomes of experimental tests help us choose between two competing theories, which ultimately depend upon predictions made by the theories. If the predictions of a theory don't come true then we reject it. This is the main source of error in thinking that there is nothing more to a scientific theory than its predictions.
Aside from philosophy, another critique of the question framework is that it can't completely protect us from bias. We would only ever frame an experiment with a question if our current understanding or explanations seemed inadequate in the first place. This means that our pre-existing knowledge and assumptions would influence our judgement of the experimental outcomes, regardless of the method used to frame the experiment.
Unlike the question, the hypothesis is built on critical rationalism, which is a theory of knowledge that emphasizes criticism 6. This approach seeks to falsify hypotheses through experimental tests. Falsified theories are rejected as poor explanations, whereas those that survive criticism are considered better approximations of the truth. Here, there is an asymmetry between our ability to falsify and support scientific theories. While no number of experiments can prove a theory (though we can tentatively accept the best available explanation until it is superseded), a single reproducible experiment can refute one. For example, no matter how many times you observe a white swan, you can never confirm your theory that all swans are white. But observing one black swan can refute your theory.
Admittedly, it's frustrating to falsify our grand hypotheses. But as critical rationalists, we needn't waste precious resources trying to confirm our hypotheses. Instead, we can create new, more refined theories and subject them to further criticism. This is the beauty of critical rationalism. It encourages us to improve existing theories in a never-ending cycle of conjecture and criticism.
Observation is important under critical rationalism, but unlike inductivism, it doesn't serve as a source of new theories. Rather, observation helps us criticize theories. In other words, it helps us eliminate one of two theories subjected to a crucial experimental test.
This raises an important question. What is the source of scientific theories if not observation? The answer is human creativity. As scientists, we make bold guesses — conjectures — in response to real-world problems. And problems can arise when our observations or predictions conflict with our expectations and reveal deficiencies in our knowledge.
A framework aligned with the aim of science
Given the opposing views of critical rationalism and inductivism, we must ask ourselves: which experimental framework is better aligned with the aim of science?
I'd argue that the hypothesis framework — championing falsification and seeking good explanatory theories — is fundamental for experimental science. Whilst a question may be preferrable for exploratory research intent on producing vast datasets (hypothesis-generating), the hypothesis should be the cornerstone of the scientific method.
But we must acknowledge the philosophy of hypothesis testing. Hypotheses are designed to be falsifed. Not confirmed or verified.
To be truly aligned with the aim of science, we must strive to explain the world through conjecture and criticism.
Summary
The hypothesis and its underlying philosophy of critical rationalism should be the bedrock of experimental science.