The Homa Files

Yesterday, we reached into our toolkit and pulled out behavioral economics and Bayesian Inference.

Our big conclusion in that post was that if C-19 tests are 90% accurate and 5% of the people in our reference group are walking around infected, then roughly 2/3’s of all people who get positive test results are not infected … they’re so-called false positives.

• See our prior post If I test positive for COVID, am I infected? for an explanation of the method and a walk-thru of the analysis leading to that conclusion.

Now, let’s change one of our assumptions.

In the prior post, we assumed that we were asymptomatic, have been sheltering-in-place (i.e. minimal social contacts outside of our homes) and don’t work in a COVID-prevalent environment … and we used 5% as our base rate (of virus prevalence among our reference group).

Now, let’s assume that the reference group we’re working with is elderly, has a comorbid medical history of respiratory and heart problems and is experiencing COVID-like symptoms (high fever, persistent cough), have had contact with an infected person.  That’s essentially the only group that initially qualified for coronavirus testing.  Lets, assume that 75% of the people in that reference group are, in fact, infected with the virus.

Here’s the Bayesian results chart would look like:

The question: what is the likelihood that the people who fit this profile are correctly diagnosed as having the virus (or not)?

=============

We plugged in our key assumptions: 90% testing accuracy (the yellow box above Colum 2)   and 75% infection prevalence for this specific reference group (the yellow box in Column 3, Row 3).

Again, refer back to our prior post for a walk-through of the calculations done to construct the table.

The answer to our headline question: “I tested negative, so I’m not infected, right?” is “maybe but…”

Since the test is 90% accurate, 90% of the infected people will, by definition, test positive … but there’s a statistically significant number of infected people who get a negative test reports (Column3, Row 2) … i.e. they are false negatives.

In fact, 25% of the people from this high infection reference group who get negative test results are infected.

=============

Note that in Column 5, 70% of the sample get positive test results.

But, we assumed that the reference group being tested has a 75% infection rate.

Where did the other 5% go?

Again, it’s those 75 people who are infected but get negative test results – the false negatives.

These people — depending on the severity of their symptoms —  would quite possibly be diagnosed as having pneumonia (or some other infliction), prescribed some medicine and sent home to recuperate.

The good news: a hospital bed wouldn’t be allocated to them (which was a very high priority in the early days of the pandemic).

The bad news: there’s always the risk that their condition will deteriorate … and, anybody who comes in contact with them during their recovery would be put at risk of getting infected.

The simple solution to avoid inadvertently putting infected people back into the population is to give people from a highly likely to be infected group (think: those with symptoms) who test negative a second test to confirm the diagnosis.

Trust me, if the retest is done by a different medical crew on a different day using a different test kit, it’s statistically very unlikely that they would get another false negative test result. If they test negative again, they’re virtually certainly not infected.

==============

Takeaways:

Bayesian Inference tells us to always consider  specific observations (i.e. test results, witness accounts) in the broader context (i.e. the base rates  of the reference group being evaluated.

The base rate depends on virus prevalence in the relevant local area (town, workplace, hangouts), mitigation behavior and symptomology.

For COVID, that means, at a minimum, sorting people into at least 2 categories: asymptomatics who have a relatively low probability of being infected and those presenting with symptoms who have a relatively high probability of being infected.

For the former group, the asymptomatics, the biggest risk lies in false positives that potentially waste test & track resources on wild goose chases.

The risks for the symptomatics are false negatives — failing to provide appropriate, timely treatment and returning infected people back into the population, quite possibly spreading the virus.

90% testing accuracy is good. but not good enough.

To mitigate the risks (false positives for asymptomatics; false negatives for those with symptoms):

• Retest positive testing asymptomatics before contact tracing based on their test result, and
• Retest those with symptoms who test negative before returning them to the general population).

This entry was posted on May 28, 2020 at 11:59 am and is filed under Coronavirus - Covid 19, Covid-19 Testing, COVID-19 Tracking. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.