Strangest Chart Ever Seen – The 97% Climate Expertise

Brandon Shollenberger:

I think I may have found the strangest chart I have ever seen. You can see it below, taken from the newly published paper on the supposed “consensus on the consensus” on global warming:

Now, I discussed this paper a bit yesterday, and there are probably a lot of things more important to discuss than this chart. Those other things aren’t as funny though. You see, this chart is complete nonsense. Look at the x-axis. See how it says “Expertise”? Tell me, what scale do you think that’s on?

You’re wrong. It doesn’t matter what your answer might have been; it’s wrong. It’s wrong because there is no scale for the x-axis on this chart.

Seriously. This is what the authors of the paper had to say about the chart:

Figure 1 uses Bayesian credible intervals to visualise the degree of confidence of each consensus estimate (largely a function of the sample size). The coloring refers to the density of the Bayesian posterior, with anything that isn’t gray representing the 99% credible interval around the estimated proportions (using a Jeffreys prior). Expertise for each consensus estimate was assigned qualitatively, using ordinal values from 1 to 5. Only consensus estimates obtained over the last 10 years are included.

For today, let’s ignore the part about the “coloring” and “credible intervals.” Let’s just focus on the part where it says the expertise values were “assigned qualitatively.” What that means is there was no rigorous method to how they assigned these values. They just went with whatever felt right. That’s why there is no rubric or guideline published for the expertise rankings.

Kind of weird, right? Well that’s not too important. What is important is… there are five categories. Look at the chart. Where are they?

To answer this question, I did a quick tabulation of the table presented in the paper. I found the number of papers in each category is:

1 – 2
2 – 2
3 – 3
4 – 0
5 – 9

I was able to match the coded entries in the chart to those in the table to confirm this. Based on that, I was able to add lines to the chart showing where the categories are. Take a look:

I have no idea what to call that kind of scale. The 1 and 2 values on the x-axis each have two items, the 3 value on it has three items, the 4 value doesn’t exist and the 5 value covers more than half the chart. If you divided the chart in half, splitting “Higher” and “Lower” evenly, one category would fall in both halves. How does one even begin to interpret that?

Here’s what you would get if you just plotted each point by its category:

Read more