Secret Archive Exposes NOAA’s Skewed Disaster Data

by Roger Pielke Jr.

Shortly after my paper Scientific integrity and U.S. “Billion Dollar Disasters” was accepted for publication, I was tipped off to a public but unnamed and well-hidden directory on the website of the National Oceanic and Atmospheric Administration (NOAA) that contained 17 (now 18) of the most recent versions of the “billion dollar disaster” (BDD) tabulation, dating to March 2020.

Today, I reveal the archive and what it tells us about the problematic methods underlying NOAA’s billion dollar disaster tabulation.

First, some brief background — Last January, upon submitting my paper for publication, in parallel I submitted a formal request to NOAA asking for a “correction,” based on my paper’s conclusions that NOAA had failed to follow its own Information Quality and Scientific Integrity policies.

NOAA first told me they would respond to my request by mid-May, then later told me that it would be mid-June, and then last week they said a response would come mid-July. I take that as a positive sign that NOAA is taking this issue seriously and needs more time to address the issues that I raise in my paper. I had planned to write this post in mid-June with NOAA’s response in hand, but discussing NOAA’s response will have to wait.

 
In the absence of methodological transparency, the hidden archive of the most recent 18 versions of NOAA’s “billion dollar disasters” allows an unprecedented opportunity to reverse-engineer NOAA’s methods that the agency employs for creating and updating its widely cited and highly influential tabulation.1 As you will see below, what the 18 versions reveal is highly concerning.

I document in my recent paper that NOAA releases neither its methodology nor the input data that underlies the various iterations of the BDD tally — only the final tabulation, and even then NOAA does not document changes to loss estimates between versions or maintain a public archive of previous tabulations.

In recent weeks, as I have been analyzing the 18 versions of the hidden directory, I contacted NOAA’s National Center for Environmental Information (NCEI) with a few very basic questions about their data.

In response, NOAA/NCEI refused to share any methodological details or data, telling me curtly:

“Much of the core asset level data are from private sector data sources and proprietary in nature, which cannot be shared.”

NOAA/NCEI did not respond to further inquiries.

With this post I report some of my initial findings after looking carefully at the 18 most recent versions of the BDD tabulation.2

It is clear that NOAA is risking a significant science scandal — simply because the agency has implemented flawed methodological choices with the resulting effect, in each instance, of artificially boosting counts of billion-dollar disasters, and there is no methodological or data transparency that would allow NOAA to justify these choices.3

We should of course expect disaster losses to increase over time due to population growth and increasing wealth and development — however the methodological choices revealed by the hidden NOAA archive indicate improper increases in recent estimated disaster losses and disaster counts independent of any of these factors, which NOAA acknowledges that it does not consider.

The juicing of the counts of billion dollar disasters matters not because the tabulation is of significant scientific importance — it is not — but because NOAA has promoted the tabulation so heavily as being scientifically important. The list of disaster counts has also variously been adopted as a scientifically-meaningful dataset by the U.S. National Climate Assessment, the U.S. president, in the peer-reviewed scientific literature, across the major media, and by decision makers in many public and private sector settings.

The hidden archive reveals that NOAA has improperly inflated trends in the counts of billion dollar disasters. I won’t speculate on why NOAA — one of the nation’s preeminent scientific agencies, full of smart and thoughtful people — has taken this route, but I do hope that the ongoing investigation ensures that this sort of thing can not happen again. I have a lot of confidence in the scientists in NOAA.4

Below, I discuss four methodological issues reveled by the hidden archive:5

• The recent increase in mini-disasters,
• Inflation adjustments that increase loss estimates for recent disasters more than for historical disasters,
• Additions of historical disasters using unknown methods that depart from NOAA’s described methods,
• Events with loss totals that cannot be replicated using public or private information.

Let’s consider each in turn.

Many Mini-Disasters

 
One of the problems with maintaining a time series of “billion dollar disasters” is that the meaning of a billion dollars changes over time. The right panel in the figure above shows that $1 billion in 2024 equates to about $260 million in 1980, when the BDD tally starts, based on the CPI Index used by NOAA. Similarly, the left panel above shows that $1 billion in 1980 equates to about $3.8 billion in 2024.

In my paper, I document the recent rapid and puzzling increase in disasters with estimated losses of $1 billion to $2 billion. Since 2020, across the 18 versions, NOAA has added 91 disasters to the tally — 61 are new events and 30 are from before 2020 (and discussed in detail below).

Of these 91, there are 55 — more than 60% — that would not have met the $1 billion threshold in the 1980s or 1990s, simply due to CPI adjustments. That means that the overall trend would be cut by about in half if the billion dollar threshold applied for inclusion were inflated to its 2024 value.

The inclusion of many recent mini-disasters that would not have appeared in the 1980s or 1990s boosts the recent count of billion dollar disasters.

Inflation Adjustments Affects Recent Disasters More than Past Disasters

In principle, a CPI adjustment for inflation should apply equally to each disaster. For instance, if inflation from last year to this year is 3% then a 3% adjustment should apply equally to the inflating of each disaster’s estimated losses, whether that disaster occurred in 1981 or 2021.

NOAA does not apply an equal application of CPI adjustments to all events across the different versions.6

Specifically, across the 18 versions — from March 2020 to April 2024 — the average inflation increase in losses for disasters of 1980 to 1989 was about 18%, but for disasters from the most recent decade — 2014 to 2023 — it was more than 26%.7

Because NOAA does not provide details on its methods it is impossible to know why the inflation adjustment is differentially applied, and why that application is a function of the recency of the disaster.

What we can say with certainty is that the result of the adjustment protocol is to make more recent events increase in losses faster that those of the past, again with the effect of boosting recent counts of billion dollar disasters.

Additions of Past Disasters is Arbitrary

NOAA claims:

“[W]e introduce events into the time series as they “inflate” their way above $1B in costs in today’s dollars. Every year, this leads to the introduction of several new events added from earlier in the time series.”

The hidden archive’s 18 different tabulations show undeniably that this claim is false.

The table below shows the reverse engineering of the inclusion of 30 historical disasters (pre-2020) which were added to the tabulation over the 18 versions from 2020 to 2024.8

The table shows, in its columns from left-to right (you can click on the image to enlarge it):

Read more