Why have the model’s forecasts dropped so much?

The explanation is really quite simple
=============

Pundits are jumping all over the modelers – specifically, the team at Washington University’s IHME – accusing them of being inept or politically motivated.

I don’t think either of those accusations are true.

Here’s why…

Let’s start with the numbers.

The initial IHME projection, made on March 25, was 81,114 “deaths from COVID-19 over the next 4 months in the US.”.

At that time, the 95% confidence interval (in layman’s terms: the range of uncertainty) was 38,242 to 162,106.

For more detail on the IHME Model, see our previous post:
What you need to know about the IHME Model

Note that the confidence level was explicit … and admittedly very large.

The IHME forecast has been revised at least 5 times since March 25.

image

Note that he current projection (60,415) is well within the original confidence interval (38,242 to 162,106) … it’s not some sort of wildly whacky outlier.

Still, let’s drill down on the initial forecast (81,114) and the current projection (60,415).

=============

First, it helps to understand the initial forecast.

To their credit, the IHME modelers were quite explicit about their model’s design and its limitations.

See the IHME Model Documentation  for details

The model was initially developed to provide a “quick & dirty” estimate of the likely draw on hospital capacity and resources in Washington state.

Note: “Quick & dirty” are my words, not their’s, and they are not meant to be derogatory. The modelers were working with limited time and sketchy data.

In fast order, the team was coaxed to go national.

Arguably, the team might have been better served at that time  to go back to the drawing boards rather than stretching their model beyond its original, narrower intent. But, they were obviously under a lot of external pressure with time of the essence.

=============

Back to the numbers ….

The model has a lot of underlying math and relational formulae.  But, the complexity is mostly built-in to spread the estimates over time and to translate patient loads into hospital requirements.

The model’s estimate of deaths was (and is) very direct and straightforward.

In a nutshell, the modelers projected US deaths based on the virus’ death rates in other earlier infected countries (e.g. Italy).

Appropriately, the modelers did not work from ‘”reported confirmed cases”, recognizing (as we have) that that number conflates infection incidence and the level of testing.

Rather, they based their estimates on the ratio of deaths to population (not by reported confirmed cases).

=============

OK, let’s cut to the chase, using Italy as an example.

Note: From this point forward, I’m trying to reverse engineer the modeler’s’ process and thinking.  While I don’t have any direct information, I would (humbly but confidently) bet on my analysis.

At the time of the initial IHME forecast, Italy had already suffered about 10,000 deaths …  and IHME was projecting that number would grow to about 15,000.

Italy’s population is about 60.5 million, so the 15,000 translates to about 250 deaths per million Italians.

Multiply that number (250 per million) times the U.S. population (about 325 million) and you get 81,250.

Does that number look familiar?

If not, glance back at the chart above: It’s almost exactly the initial IHME forecast for the US.

The takeaway: IHME’s initial forecasts appear to have been initially based on the Italian experience — the best available data at the time.

============

The current US forecast (60,415) translates to about 185 deaths per million … down from 250 in the initial forecast.

Does this sound reasonable?

Here’s a recap from our daily tracker:

image

The US has already accumulated about 17,000 deaths … which translates to about 50 DPM (Deaths per Million).

While the game isn’t over, the countywide results seem to be running far better than the wildfire-like experiences in, say, Spain and Italy.

Note: The IHME team explicit disclosed that their model directly consider the potential impacts of better therapeutic protocols that could materially reduce hospitalizations and deaths.

Is hospital care in the US better than it is in Italy (at least during this crisis)?

I’ll let you decide …

There’s a lot of room between the current death rate (50 DPM) … and the IHME’s adjusted forecast (185 DPM).

My bet: that number will continue to go down …. and we’ll see the forecast cut a few more times … probably drastically.

============

So, finally, to answer the question: “Why have the model’s forecasts dropped so much?”

Surely, the IHME modelers are adjusting their model to place more emphasis on the actual US experience … and less on the experience reported in other earlier infected countries. Period.

It’s that simple.

==============
Follow on Twitter @KenHoma

>> Latest Posts

#HomaFiles

2 Responses to “Why have the model’s forecasts dropped so much?”

  1. April 11: COVID-19 Tracker | The Homa Files Says:

    […] News & Views on Marketing, Economics & Politics « Why have the model’s forecasts dropped so much? […]

  2. April 12: Covid-19 UPDATE | The Homa Files Says:

    […] Why have the model’s forecasts dropped so much? […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s