0.3% consensus, not 97.1%…‘Quantifying the consensus on global warming

Christopher Monckton of Brenchley:

Capsule

The latest paper apparently showing 97% endorsement of a consensus that more than half of recent global warming was anthropogenic really shows only 0.3% endorsement of that now-dwindling consensus.

Abstract

Cook et al. (2013) stated thatabstracts of nearly all papersexpressing an opinionon climate change endorsed consensus,which, however, traditionally has no scientific role; usedthreeimprecise definitions of consensus interchangeably; analyzed abstracts only;excluded67%expressing no opinion;omitted some key results; misstated others; and thus concluded that97.1% endorsed the hypothesisas defined in their introduction,namely that the “scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)”.The authors’ own data file categorized 64 abstracts, or only 0.5% of the sample,as endorsing the consensus hypothesis as thus defined.Inspection shows only 41 of the 64, or 0.3% of the entire sample, actually endorsed their hypothesis. Criteria for peer review of papers quantifying scientific consensus are discussed.

Introduction: no role for consensus in science

Though Cook et al. (2013) reviewed abstracts of 11,944 papers on climate change and concluded that 97.1% of those expressing an opinion supported consensus, the philosophy of science allows no role for head-count.Aristotle, in his Sophistical Refutations, (c. 350 B.C.E.),identified the argument from consensus as one of the dozen commonest logical fallacies in human discourse.

Al-Haytham, the astronomer and philosopher of science in 11^th-century Iraq who is recognized as the father of the scientific method, wrote that “the seeker after truth” – his phrase for the scientist – does not place his faith in any mere consensus, however venerable. Instead, he checks. “The road to the truth,” said al-Haytham,“is long and hard, but that is the road we must follow.”

In 1860 T.H. Huxley said: “The improver of natural knowledge absolutely refuses to acknowledge authority, as such. For him, skepticism is the highest of duties: blind faith the one unpardonable sin.”

Albert Einstein, when told that 100 Nazi scientists had published a book rejecting his theory of special relativity, responded that a single paper would have sufficed to refute his hypothesis. His own single paper of 1905 on the electrodynamics of moving objectshad demonstrated why Newton’s laws, till then universally accepted as true, incompletely described the motion of celestial objects.

Popper (1934) formalized the scientific method as an iterative algorithm starting with a general problem (GP₀), to address which a scientist proposed a falsifiable hypothesis or tentative theory (TT₀). Thereupon others would either demonstrate during the error-elimination phase (EE₀) that the hypothesis was false, in which event it was rejected, or, more rarely, demonstrate that it was true.

By far the commonest outcome, however, especially in the physical sciences, is that error elimination will fall short of demonstrating the hypothesis but will fail to disprove it, in which event it gains some credibility. The statement of the general problem may then be modified accordingly (GP₁), and a new tentative theory (TT₁) may later be advanced to address the modified problem; and so on. Pedetemptim, and if necessary ad infinitum, science iteratively converges upon the truth (Fig. 1).Consensus adds no value to this process.

In the scientific method, then, there is no place for mere consensus. A hypothesis that is demonstrated – such as Pythagoras’ theorem – needs no consensus, for it is objectively true. A hypothesis that is disproven needs no consensus, for it is objectively false. A hypothesis that is neither demonstrated nor disproven gains credibility,and not because a dozen or even 12,000 papers endorse it but because – and to the extent that – it has not been demonstrated to be false. Science is not a belief system.A priori, then, head-countsare inappropriate tests of scientific results.

Problems in defining the climate consensus

Al-Haytham’s scientific method required observing nature, stating a problem precisely, formulating a hypothesis to address it, testing the hypothesis, analyzing the results, drawing conclusions, and publishing the findings. It is essential to the scientific method that a tentative theory be stated in rigorously precise terms.

In particular, the definition of the hypothesis should be expressed quantitatively. An imprecisely defined hypothesis, especially if it is not quantitative, may be insufficiently rigorous to be testable. If it beuntestable, then, stricto sensu, it is not of interest to science. It is a mere curiosity.Yet Cook et al. do not confine themselves to a single definition of the hypothesis to which their consensus is said to adhere. Three definitions of climate consensus coexist in the paper –

Definition (1): “the consensus position that humans are causing global warming” (abstract);

Definition (2):the “scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)” (introduction);

Definition (3):that our enhancement of the greenhouse effect will be dangerous enough to be “catastrophic”; (implicit in the introduction, in discussion of the need to raise awareness of scientific consensus to justify a “climate policy”, and explicit in Table 2 of Cook et al., citing a paper opposing “the catastrophic view of the greenhouse effect”).

President Obama was among those who thought Cook et al. had shown a consensus endorsing definition (3). The following tweet was posted in his name:

“Ninety-seven percent of scientists agree: #climate change is real, man-made and dangerous.” [Emphasis added]

Definitions (1, 3) fall short of the criteria for definition of a Popper-falsifiable hypothesis, and definition (2) could have been clearer. Not only do Cook et al. adopt the definitions interchangeably, but each definition is imprecise and insufficiently quantified to allow rigorous Popper-falsification.None of the definitionsspecifiesthe period to which it applies, or how much global warming was observed over that period, or whether the warming is continuing, or, if so, at what rate, or whether that rate is considered dangerous, or what rate if any is considered dangerous.

Additionally, definitions (1) and (3) do not specify what fraction of warming was considered anthropogenic, and definition (2) assignsno quantitative value to the term “very likely”. Such imprecisions render the hypotheses unfalsifiable and hence beyond the realm of legitimate scientific inquiry.

Definition (2) is akin to, but less precise than, that of the Intergovernmental Panel on Climate Change (IPCC, 2007, p. 665):

“Greenhouse gas forcing has very likely [90% confidence] caused most of the observed global warming over the last 50 years.”

Definition (3) is implicit in the opening words of Cook et al.:

“An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al., 2011). Communicating the scientific consensus also increases people’s acceptance that climate change is happening (Lewandowsky et al., 2012).”

The implication is that the authors of all abstracts endorsing definitions (1) and (2) also endorse the catastrophist definition (3). However, a hypothesis to the effect that humans cause some warming, or even that most current global warming is very likely to be anthropogenic, is not – and need not imply – a hypothesis that current warming, if continued, might prove sufficiently damaging to justify a climate policy.It is by this unwarranted extension that President Obama’s Twitter erroneously assumed that the survey indicated 97% endorsement of catastrophic global warming.

The use of multiple imprecise and ill-quantified definitions of climate consensus has some precedents in the literature. Cook et al.cite two instances:

“Surveys of climate scientists have found strong agreement (97-98%) regarding AGW amongst publishing climate experts (Doran & Zimmerman, 2009; Anderegg et al., 2010).”

Doran and Zimmerman (2009)

The two authors sent a 2-minute online survey to 10,257 earth scientists at universities and government research agencies. Only 5% ofthe 3,146 respondents identified themselves as climate scientists;90% believed mean global temperatures had generally risen compared with pre-1800s levels; and 82% believed human activity was a significant contributing factor in changing mean global temperatures.Only 79 of the respondents listed climate science as their area of expertise and had also published more than half of their recent peer-reviewed papers on climate change. Of these, 98% believed human activity was a significant contributing factor in changing mean global temperatures. However, the sample size was insufficient to deliver a statistically reliable result, and the respondents were not asked whether they believed the anthropogenic influence on temperature might become sufficiently damaging to require a “climate policy”.

Anderegg et al.(2010)

From publication and citation data, the authors selected 908 of 1372 climate researchers, defined as people who had published at least 20 climate papers and had either signed petitions opposing or supporting the IPCC’s positions or had co-authored IPCC reports. Of these, 97-98% believed that “anthropogenic greenhouse gases have been responsible for ‘most’ of the ‘unequivocal’ warming of the Earth’s average global temperature over the second half of the 20th century”.The definition of the consensus in Anderegg et al. is less imprecise than definition (2) in Cook et al. Yet, like Cook et al., Anderegg et al. did not seek to determine how many researchers considered global warming to be actually or potentially damaging enough to require a climate policy.Nevertheless, the two surveys are often cited as demonstrating a near-unanimous scientific consensus in favor of a climate policy, when in fact, like Cook et al., neither survey had asked any question either about whether and to what extent the anthropogenic component in recent warming might be dangerous or about whether a “climate policy” should be adopted in attempted mitigation of future warming.

In Cook et al., the definition of consensus hypothesis that comes closest to those of the IPCC and of other head-count papers is definition (2). Table 1 lists some of these definitions.

Source	Quantitative definition of climate consensus
IPCC (2001), Ch. 12 (attribution), p. 699	“… most of the observed warming over the last 50 yearsis likely to have been due to the increase in greenhouse gasconcentrations.”
Oreskes (2004)	As IPCC (2001), supra.
IPCC (2007), Ch. 9 (attribution), p.665	“Greenhouse gas forcing has very likely [90% confidence] caused most of theobserved global warming over the last 50 years.”
Schulte (2008)	As IPCC (2001), supra.
Doran & Zimmerman (2009)	“… human activity is a significant contributing factor in changing mean global temperatures.”
Anderegg et al. (2010)	“… anthropogenic greenhouse gases have been responsible for ‘most’ of the ‘unequivocal’ warming of the Earth’s average global temperature over the second half of the 20th century.”
Cook et al. (2013), definition (2)	“…scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW).”

Table 1. Successive quantitative definitions of a climate consensus.

Incomplete statement of the survey results

None of the seven “levels of endorsement” by which Cook et al. categorize their selected abstracts provides evidence that any of the 11,944 abstracts encompassesthe catastrophist definition (3):

1                    “Explicitly states that humans are the primary cause of global warming”
2                    “Explicit endorsement without quantification”
3                    “Implicit endorsement”
4                    “No opinion, or uncertain”
5                    “Implicit rejection”
6                    “Explicit rejection without quantification”
7                    “Explicit rejection with quantification”

The first endorsement level, “Explicitly states that humans are the primary cause of global warming”, reflects definition (2) and is akin to the other definitions in Table 1. The second and third levels, “Explicit endorsement without quantification” and “Implicit endorsement”, reflect definition (1) in that, like it, they are not quantitative.Yet the first three levels of endorsement are treated as one in the results:

“To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit: categories 1-3) …”.

Results of an inspection of the Cook et al. data file

It is not possible to discern either from the paper or from the supplementary information what fraction of all abstracts endorse definition (2).A file of raw data was supplied, though it was only posted online some weeks after publication. This comma-delimited text file was dowloaded and the abstracts allocated by Cook et al. to each level of endorsement were counted.Results are given in Table 2:

Table 2. Abstracts in the seven levels of endorsement specified in Cook et al. (2013). Only 64 abstracts, according to the authors’ data file, explicitly endorsed definition (2), the quantitative hypothesis. NB: “+quant” indicates “with quantification”; “–quant” indicates “without quantification”; “% all” indicates the percentage of all 11,944 abstracts that fell in each level of endorsement; “% opin” indicates the percentages of all 4014 abstracts, excluding the 7930 that expressed no opinion but including the 40 that expressed uncertainty (1% of all papers). These 40 are not shown separately in the datafile or in the table. Therefore, the percentages of papers expressing an opinion sum to 99%, not 100%.

Definition (1): The count confirmed the authors’ count that 3896 of the 11,944 abstracts (i.e., 32.6%) fell in their endorsement levels 1-3, indicating that fewer than one-third of all abstracts indicate implicit or explicit support even for the limited definition (1) hypothesis that humans cause some warming. It was only by excluding those 7930 endorsement-level-4 abstracts that expressed no opinion (but retaining the 40 level-4 abstracts expressing uncertainty) that Cook et al. were able to conclude that 97.1% endorsed consensus.

Definition (2): The count, in line with an earlier result published by Friends of Science in Canada, showed only 64 papers, or just 0.5% of the sample, explicitly endorsing the quantitative hypothesis to the effect that humans are the primary cause of current warming. This value was independently verified by a separate inspection of the data file to identify occurrences of the search term “,1” at the end of each data record using the search facility in Microsoft Notepad, whereupon 64 such occurrences were indeed found.However, of the 64 abstracts to which Cook et al. assigned an endorsement level of 1 (“explicit endorsement with quantification”: Annex 1), 23 do not in fact endorse definition (2). Only 41 papers (0.3% of the sample: Annex 2) endorse definition (2).

The conclusion of Cook et al., as expressed in their abstract, is as follows:

“Among [4014] abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.”

A 97% consensus is also asserted in the closing words of the paper:

“Among [4014] papers expressing a position on AGW, an overwhelming percentage (97.2% based on self-ratings, 97.1% based on abstract ratings) endorses the scientific consensus on AGW.”

In the introduction. Cook et al.define “AGW” as the “scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)”. However, the authors’ own analysis shows that only 0.5% of all 11,944 abstracts, and 1.6% of the 4014 abstracts expressing a position, endorsed “AGW” as they had defined it. Taking into account that more than one-third of the 64 abstracts do not in fact endorse the quantitative hypothesis in Cook et al., the true percentages endorsing that hypothesis are 0.3% and 1.0% respectively.

Evidence that the climate consensus is declining

Oreskes (2004)published an essay in Science alleging that not one of 928 abstracts she had reviewed had disagreed with the consensus as defined in IPCC (2001): “Most of the observed warming over the last 50 years is likely to have been due to the increase in greenhouse gas concentrations”. Her adopted definition of consensus, then, was similar to but more precise than definition (2) in Cook et al. It, too,falls well short of stating that manmade warming may prove catastrophic.

Her essayconcluded that three-quarters of her sampleendorsedthe “consensus” either explicitly or, by evaluating impacts or proposing mitigation, implicitly. A quarter took no view. None, she said, disagreed with the consensus as defined in IPCC (2001). Yet the fact that a paper evaluates impacts or proposes mitigation does not necessarily imply that the authors endorse the notion that more than half of the past 50 years’ global warming was anthropogenic.

Schulte (2008) reviewed 539 papers published in the three years following the period studied by Oreskes, using the same search termand the same IPCC definition of consensus.He found that “the proportion of papers that now explicitly or implicitly endorse the consensus has fallen from 75% to 45%.”Only 2% of the papers reviewed “offer new field data or observations directly relevant to the question whether anthropogenic warming has prevailed over natural variability in the past half-century”.Only one paper mentioned the possibility of catastrophic climate change, but without providing any evidence for it. No paper provided quantitative evidence for the consensus hypothesis.Schulte concluded: “There appears to be little basis in the peer-reviewed literature for the degree of alarm on the issue of man-made climate change which is being expressed in the media and by politicians.”

The 75% consensus reported by Oreskes in 2004 exceeds the 45% found by Schulte in 2008, which in turn exceeds the 0.5-1.6% (0.3-1.0% after correction) found but not reported by Cook et al. in 2013.

Accordingly, the undisclosed results in Cook et al. indicate either that earlier head-count papers had not (as Cook et al. in their published results had not) drawn any distinction between quantified and unquantified consensus, or that over the past decade the consensus has dwindled.

Has there been any ‘current’ warming?

Figure 2. No global warming for 12 years 6 months, according to the HadCRUt4 dataset.

Cook et al.’s definition (1) is that “humans arecausing global warming”. Definition (2) is that humans are the chief cause of “the current warming”. Though the least-squares linear-regression trend on the monthly HadCRUt4 dataset (Morice et al., 2012) shows surface warming over the half-century 1956-2005 at a rate equivalent to 1.2 Cº/century, trends on the same dataset over shorter periods show no warming exceeding the upper bound of the published measurement uncertainties for more than 17 years, no warming at all over the 12 years 6 months November 2000 to May 2013 (Fig. 2), and cooling at 0.5 Cº/century over the decade May 2003-April 2013.

In the usual sense of the word “current”, then, global warming is not currently occurring, rendering definitions (1) and (2) untestable: for no scientist could legitimately endorse a consensus to the effect that global warming is currently occurring when, currently, it is not.

Discussion

The defects identified in the surveys of climate consensus by Cook et al. and by the authors of some of the papers they cite follow a pattern to whose existence peer-reviewers should be alert. First, any argument from consensus on a question such as the extent to which anthropogenic global warming may prove dangerous is defective a priori and ought really to be rejected without further review.

Secondly, no survey of opinion for or against a consensus hypothesis ought to be regarded as scientific where it is not made clear which hypothesis is under test, or where the hypothesis under test is not clearly and precisely formulated. A fortiori, a survey paper that exhibits multiple definitions of the consensus hypothesis and fails to state clearly the identity and definition of the hypothesis on the basis of which the survey was actually conducted should surely be rejected.

Thirdly, the consensus hypothesis under test ought to be expressed in quantitative terms. Mere qualitative definitions of any scientific hypothesis run the risk of appearing more political than scientific in their formulation, and papers based on such definitions may also prove more political than scientific in their effect.

Fourthly, if several “levels of endorsement” are specified, then the number of abstracts, papers, or scientists considered to have supported each level of endorsement ought to be explicitly stated in the paper under review. Cook et al. specified three levels of endorsement that supported the notion of anthropogenic warming (however defined); yet,on the stated ground of simplifying the analysis, the number of papers allocated to each of the three levels of endorsement – a key result on any view – was not stated. The analysis would indeed have been simpler if one endorsement level supporting one definition of climate consensus had been adopted.

Fifthly, all data files and programs should be archived at the time of submission to the journal and included at the time of publication as part of the supplementary material. Reviewers should ask for the datafiles and programs if they are not available.

Conclusion

The non-disclosure in Cook et al. of the number of abstracts supporting each specified level of endorsement had the effect of not making available the fact that only 41 papers –0.3% of all11,944 abstracts or 1.0% of the 4014 expressing an opinion, and not 97.1% – had been found to endorsethe quantitative hypothesis, stated in the introduction to Cook et al. and akin to similar definitions in the literature, that “human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)”.

References can be found here [PDF]