From Climate Scepticism
By JOHN RIDGWAY
For the most part, the argument against climate change sceptics fails to aspire to anything greater than insult and contempt. However, every now and then, the debate takes on a more nuanced form of condescension, since it revolves around whether or not the average sceptic can tell the difference between science and what scientists get up to in their free time. Such debate often gets dressed up using philosophical talking points such as the difference between normal and Post-Normal Science (PNS). Sceptics cite climate science as a classic case of PNS and their critics usually respond by deriding the very concept. Lengthy and often tedious discussions follow, peppered with references to the likes of Kuhn, Popper, Funtowicz and Ravetz. It’s a recipe for unresolvable dispute and, quite frankly, it leaves me cold.
So, today I want to steer clear of the philosophy of science and, instead, present a distinction that I feel is far more to the point. It really doesn’t matter whether or not one sees a difference between a normal and post-normal version of science, all one really needs to see is that the climate change issue is not actually about science and a search for the truth. It’s about decision-making under uncertainty and the search for a rational decision. As such, the relevant distinction is the one existing between hypothesis testing, Signal Detection Theory (SDT) and decision theory.
Searching for truth
I won’t dwell too long on hypothesis testing because I suspect most readers will already be familiar with the concept. However, the following points need emphasising.
Hypothesis testing seeks to ascertain the statistical significance of the data, and for that purpose a so-called p-value is determined and compared against an agreed threshold. Here is a classic representation of the hypothesis testing problem:
The p-value tells you the probability of observing the data, in the event that the null hypothesis were true.1 And although an attempt to discount the null hypotheses is intended to determine the truth, the real outcome of the test is a statement of statistical significance, that is to say it is a statement regarding the adequacy of the data. If it isn’t sufficient to help you decide between your treasured hypothesis and the null hypothesis, then you have a decision to make: keep on collecting data in the hope that a strong enough signal emerges or abandon your hypothesis. Amongst other things, that decision depends upon how much noise you think you are dealing with. Which brings me on to Signal Detection Theory.
A common way of illustrating what is going on in an hypothesis test is to represent the posited and null hypotheses using two overlapping probability distributions, one for the noise (the null hypothesis) and the other for the signal (the alternative hypothesis). Hence:
This is a useful representation because it highlights an important point: for any given choice of ‘critical value’, there are two basic categories of error that can be made.
The first is the so-called Type I error, indicated by the area under the null hypothesis curve to the right of the critical value. This represents the probability that noise could be misread as signal, thereby causing the alternative hypothesis to be accepted in error. The second is the Type II error, indicated by the area under the alternative hypothesis curve to the left of the threshold. This represents the probability that a signal could be misread as noise, thereby causing the alternative hypothesis to be rejected in error. To reduce the likelihood of Type I errors one can readjust the critical value used to delineate statistical significance, but that will only increase the likelihood of Type II errors.2 Furthermore, both types of error come with a cost that boils down to value-laden judgement.
And this is the point where the scientific pursuit of truth merges into decision theory.
Making a decision
Although we are still talking about hypothesis testing and signal detection here, by introducing the concept of value we cannot avoid the topic of decision-making under uncertainty, in which the rationality of a decision is as much determined by notions of utility and risk as it is by data and statistical significance. It is for this reason that I suggest that decision theory provides the most appropriate framework for thinking about what is going on in the climate change debate.
Actually, it is perfectly natural that Signal Detection Theory (SDT) and decision theory should be mentioned in the same breath, since the former is just a sub-branch of the latter. SDT was first developed by radar researchers seeking to provide a mathematical basis for the detection of signals, in which there are obvious trade-offs to be made between missing a significant signal, on the one hand, and raising false alarms, on the other. Every time a radar operator interprets a blip on the screen, he or she is making a decision under such uncertainty. However, whereas SDT can be used to codify Type I and Type II errors, it cannot, on its own, tell you which to prefer. The radar operator’s dilemma is a classic case of a decision to be made under uncertainty, where values are disputed and time is of the essence. You can call that post normal science if you want, but that would be quite unnecessary; it is simply decision theory in practice.
Once the utility theory aspect of decision theory is added into the mix, it opens up the possibility of determining the correct decision according to a rationale, i.e. that of maximising utility. Take, for example, a decision made on the basis of a radar blip that may or may not indicate a threat. Here we have the following set of possibilities, in which ‘value’ can be a measure of a payoff or a penalty:
- The value of threat correctly detected (TCD)
- The value of threat correctly rejected (TCR)
- The value of threat missed (TM)
- The value of a false alarm (FA)
According to the decision theoretic approach, the correct decision-making strategy is the one that maximises utility by setting the critical value for believing a blip (β) as follows:
β = ((TCR – FA) x prob(noise)) / ((TCD – TM) x prob(signal))
Of course, in reality it can be very difficult to make such a calculation, particularly when the impacts of missing a threat and a false alarm are both potentially catastrophic, such as when the blip is possibly caused by an incoming salvo of ICBMs. Nevertheless, recognising and accepting the general form of the equation is an important first step towards understanding the nature of the problem. We are now a long way from simply testing a null hypothesis.
Tracking the climate change blip
Applying the β equation to the issue of climate change illustrates a number of points.
The first thing to understand is that there is no universal, correct value of β to be had. The equation shows that there is a relationship between the strength of evidence required and the imperative for taking alternative actions, but there is no reason to assume that exactly the same calculus applies to all people, or indeed at all levels of community. This effect is often described as a sign of motivated reasoning, as if that were a bad thing. The fact is, all reasoning is motivated, and it may or may not be rational depending only upon the extent to which the direction it takes is in keeping with the equation. When it comes to climate change, one person’s denial is another person’s precaution.
Secondly, it can be seen from the equation that two people may have very similar views regarding the signal to noise ratio and yet still differ profoundly regarding where they draw the line for action. This would be due, of course, to their differing evaluations of TCR, FA, TCD and TM. Both parties may accept the existence of climate change as a phenomenon because they both see a clear and distinct blip on their radar, and yet there are still factors to take into account before deciding the appropriate reaction. And it is this decision, based upon a personal calculation of β, that determines whether or not one is branded a denier. Consequently, seeing and accepting the blip on the screen is not good enough to escape accusations of blip denial.
The next point to appreciate is that we are not all looking at the same screen. When climate scientists look at their screen, the noise to signal ratio has a very physical interpretation. It is the analysis of data that enables them to discern weather from climate, to infer meaning from sea level measurements and temperature records, or read significance into the frequency or severity of extreme weather events. However, for the rest of us, the much more relevant signal to noise ratio is determined by observing the signal given out by a much vaunted scientific consensus, and comparing that with the noise represented by dissenting scientists. How reliable this second signal to noise ratio is depends very much upon one’s views regarding the filters used to achieve it. Yes, the first ratio plays a significant role in determining the second, but it certainly isn’t on its own in being an influence. This matters because it is the second ratio that enters into the β calculation that determines policy and the public’s level of support for it.
Finally, when it comes to the issue of climate change, there isn’t a single element of the β calculation that should be beyond disputation. And yet this debate is difficult to maintain when there isn’t even a universal acceptance of the equation to be applied. Talk to Extinction Rebellion, and you would think that β = (TCR – FA) x prob(noise), where prob(noise) = 0. Put another way, whatever anyone says about the benefits of correct rejection, or the cost of a false alarm, believe the blip!
But despite all of the above, whatever one may think about the equation, it is all rather academic now. The blip has appeared on the radar and panic has ensued. Great efforts have been made to alter people’s calculation of β to suit the mood music. In light of the harmony, it is getting increasingly difficult to get people to understand that the decision-making is actually a wicked problem resonant of cacophony. Even if one accepts that the science is closed, the calculation of β should still be an open issue. But that is not how it is. The blip was seen. It was interpreted as a salvo of incoming ICBMs. And the retaliatory response is already on its way.
Notes:
[1] Unfortunately, statements regarding the truth of the posited hypothesis are Bayesian posteriors that can’t actually be calculated, because null hypothesis testing doesn’t provide the necessary Bayesian prior.
[2] Altering the critical value is referred to as a response bias adjustment. Another way of influencing the number of Type I and Type II errors is to simply separate the probability distributions for the noise and the signal, i.e. improve the experimental setup.
Further Reading:
Utility and rational decision-making
SDT and utility in an integrated framework
You must be logged in to post a comment.