Making sense of COVID19 test results is no easy task. To do it, we’ll need to dive into statistics. I have delayed getting this blogpost out because, like many others of you out there, statistics was one of my least favorite subjects in school and I have been fighting with my inner procrastinator on this one. However, my childish “I don’t wanna” excuse is finally being overtaken by my more mature “But it is oh so important” reasoning, and here we are.
Understanding statistics, as we aim to make sense of our COVID19 tests and test results, has never been more important. Yet, statistical concepts are tough and can be difficult for even us science-y folks to grasp. So, I will attempt to make these often complex ideas more palatable. I will break them down into easy(er) to understand language and use examples that feel relevant to all of us. I will also post some videos in coming days that will reinforce these valuable lessons, so continue to tune in. Let’s buckle up for this wild statistics ride! Yeehaw!
Why do statistics matter?
As requests for COVID19 antibody tests come pouring in (was there an announcement on the news that I missed?), we need to have a way to judge the accuracy of these tests. And we need a way to interpret the test results. For, as I said in my last blogpost, we never say “never” and never say “always” in science. A positive test for a particular disease doesn’t always mean that the person has the disease. And a negative test doesn’t always mean that they don’t have the disease. Frustrating and confusing, I know. So, how can we have confidence in our tests? How can we tell how accurate they are? Enter Statistics.
To start, let’s get familiar with the statistician’s favorite visual aid – the 2 x 2 table – and a few common statistical terms. We will need to come back to these again later! On the horizontal axis, we have the population (those with disease and those without disease). On the vertical axis, we have the test result (a positive test result or a negative test result).
Adapted from Medmastery
If a person has the disease and their test is positive for that disease then they would be counted as a True Positive (TP). If a person does not have the disease, but their test comes back positive, they would be counted as a False Positive (FP). Conversely, if a person has the disease but their test for the disease comes back negative, that would be considered a False Negative (FN). Finally, if the person being tested does not have the disease and their test comes back negative, that would be a True Negative (TN).
Looking at the population
When testing a population for disease, we want to have a reliable test. A test that misses half the cases and, in the case of SARS-CoV-2, turns positive because of cross reactivity with other coronaviruses would not be useful. In an ideal world, our test would be highly sensitive (it would detect all cases of the disease in question) and highly specific (it would detect only the disease of interest and not other similar diseases).
- Sensitivity
- This test characteristic starts with the assumption that the population we are dealing with DOES HAVE THE DISEASE in question. The sensitivity of a test tells us how likely the test is to show a positive result (to detect the disease) when disease is present. So, it counts the true positives out of all those who have the disease. The more sensitive the test, the more true positives and the fewer false negatives.
- Sensitivity = TP/TP + FN
Adapted from Medmastery
- Specificity
- This test characteristic starts with the assumption that the population we are dealing with DOES NOT HAVE THE DISEASE in question. The specificity of a test tells us how likely the test is to show a negative result when the disease is absent. Specificity counts the true negatives out of all those who don’t have disease. The more specific the test, the more true negatives and the fewer false positives.
- Specificity = TN/TN+ FP
Adapted from Medmastery
Looking at the test results
When we are looking at our test results, we need to know how confident we can be that our results are accurate. And we need to be really confident. Imagine treating a person with chemotherapy (which can have significant side effects) based on a test for cancer that had a high rate of false positives. We could do real harm. Or imagine a test for a contagious disease that had a high rate of false negatives. We could be reassuring people that they were fine and unwittingly contribute to the spread of illness. Our tests need to predict, with a high degree of accuracy, both positive and negative outcomes.
- Positive Predictive Value
- This test characteristic starts with the test result and asks, “If I have a positive test result, what is the chance that I actually have the disease?” The positive predictive value (PPV) is the true positives out of all the positive test results. A test with a high PPV means that it has a high number of true positives and a low number of false positives.
- PPV = TP/TP + FP
Adapted from Medmastery
- Negative Predictive Value
- This test characteristic starts with the test results and asks, “If I have a negative test result, what is the chance that I truly don’t have the disease?” The negative predictive value (NPV) is the true negatives out of all the negative test results. A test with a high NPV means that it has a high number of true negatives and a low number of false negatives.
- NPV = TN/TN + FN
Adapted from Medmastery
Viewing test results in context
Test results have to be viewed in context. Sometimes this means viewing a particular lab result, let’s say a person’s blood sugar, in context of the person’s history and of their prior labs. If normal fasting blood sugars range from 60-100, a blood sugar of 110 may seem worrisome. Indeed, that may signal a march toward Diabetes. But if we know the patient’s history (that they already carry a diagnosis of Diabetes) and we know the prior lab values (145, 163, etc.), then suddenly the 110 value looks really good! What first made us nervous, now makes us happy – all with just a change of context.
When we are talking about interpreting test results for illnesses like SARS-CoV-2, we still have to view the test result in the context of the person’s medical history. For example, is the person immune suppressed and therefore unable to mount a full immune response to infection? If they don’t develop antibodies like someone with a healthy immune system, will we be able to detect those antibodies on our antibody test? If they have worrisome symptoms and their test comes back negative, do we trust the test?
Prevalence
Likewise, we must also view the results in the context of a person’s environment. This is where prevalence comes in. Prevalence of a disease is defined as how common that particular disease is in the community. For example, SARS-CoV-2 is likely more prevalent in New York, New York than it is in Spokane, Washington. We can’t know this with 100% certainty because we are unable to test every single person in each of these locations. But we can presume this with great confidence based on the rates of illness and death in New York compared to those in Spokane.
Just like medical context as discussed above, prevalence becomes important when we are trying to interpret our test results. If you test two people with suspicious symptoms, one from Spokane and one from New York, and both test negative, which test result are you more likely to believe? Will it be the person, with suspicious symptoms, who comes from a place where there is a high prevalence of COVID19 in the community? Or will it be the person who comes from a place with a low prevalence of COVID19 in the community? Before we even ordered the test, our clinical suspicion should have told us that the New Yorker more likely had the illness. Our pre-test probability was high (we’ll talk more about pre-test probability below). Consequently, a false negative is more likely to occur in the person from New York than it is in the person from Spokane.
Clear as mud? Feel free to refer back to that 2 x 2 square. This is tricky stuff.
Pre-test probability
This is where clinical intuition and experience really come into play. As clinicians, whenever we order a test, we should always be asking ourselves two questions. Based on the answers to these questions, we either will or will not (or should or should not) order a test.
- How suspicious am I that “X” disease process or illness is occurring?
- What will I do with the test results when I have them?
Let’s use influenza as an example. Unlike COVID19 (which seems to have a widely variable way of showing up in people), influenza has pretty classic symptoms – sudden onset of high fever, severe body aches, fatigue, sore throat, headache and cough. And, other than looking like death warmed over, a person’s physical exam is pretty normal (assuming they are not having complications of their illness). If someone comes into my office with the exact symptoms and exam findings as described above, and it is smack dab in the middle of flu season (meaning, a high prevalence in the community), my pre-test probability (my level of suspicion) that this is the flu is very high. That answers question #1.
But what about question #2? What will I do with the information if I decide to test? If I do a test and the result is positive, confirming my suspicion, will I do anything different with that information than I would have done if I didn’t test? Probably not. And if I do a test and the result is negative, what then? Knowing that the person has classic symptoms, is in a time where prevalence of influenza in the community is high, and knowing that the sensitivity of the flu test is actually pretty poor (50-70% sensitive, meaning it misses 50-30% of positive cases), I’m not likely to believe that result. This then begs the question, why test in the first place? And many of us don’t test for the flu during flu season. Instead, we rely on our clinical suspicion.
Now, let’s say it’s not the symptomatic person sitting in front of me whose health I’m as worried over, it’s his immune compromised wife at home that I have concerns about. If I am on the fence about whether this is the flu, then I may want to test. A positive result may make me act more aggressively (prescribe an antiviral, for example, to both the patient and his wife) to protect them both.
Or, if we were in a COVID19 pandemic (which we are) and if that illness could mimic the flu (which it can), then I may be more inclined to do a test. Let’s put aside for the moment that it is possible that the flu and COVID19 can coexist and infect one person at the same time. If the test was positive for flu, let’s say, then this information could be quite useful in helping me determine which of the two illnesses the person has. And this could then impact our instruction on quarantine and required time away from work, for example.
All of this is going through your clinician’s head when they are thinking about conducting tests. Whew. Exhausting.
A look at COVID19 tests
There are two types of tests commonly available for COVID19. There are viral tests (detecting the virus itself) and antibody tests (testing for the body’s immune response to the virus). A viral test only comments on whether you have a current infection. An antibody tests more often comments on whether you have had a previous infection. Antibody tests are not ideal for telling us about current infection because it can take generally 1-3 weeks for the body to start to make antibodies.
COVID19 viral test
The viral test is collected by inserting a swab into the patient’s nasopharynx (that part where the back of your throat meets the back of your nose) to collect fluid and cells. The genetic material collected is then put through a process called a reverse transcription polymerase chain reaction (RT-PCR) which allows amplification of small quantities of the virus’ genetic material. If SARS-CoV-2 virus is present, the test will “light up” and give a positive result.
- False positives can occur in the unlikely occasion that there is cross contamination between samples in the lab. This might lead someone to think they were positive for the virus when they, in fact, had not been exposed.
- More worrisome, though, are the false negatives. This could provide false reassurance to someone and may lead them to be less cautious about isolation, leading to infection of others. False negatives can occur if the swab sample is not collected correctly (that sucker really has to go way back there) or if someone is tested too early in the disease course when the viral load (the number of viral particles) is low.
COVID19 antibody tests
Antibody tests are serology tests, meaning they are taken from a blood sample. There are currently numerous antibody serology tests on the market. These have not yet been approved by the FDA but several have been accepted for emergency use authorization (EUA). I will speak specifically to the Antibody test that is being used at the University of Washington, as this is what we are using in Spokane. To look at the characteristics of other tests used across the country, check out this website for more information.
- The COVID19 IgG antibody serology test looks only for IgG. Other tests on the market look at both IgM (the antibody that is formed during an active infection) and the IgG (antibody whose presence indicates prior infection) antibodies. However, the production of IgM antibodies is variable and delayed compared to infectivity, so this testing is not felt to be as helpful. The UW COVID19 IgG serology test is produced by Abbott Laboratories and was evaluated for accuracy by testing it against known positive COVID19 cases. It shows a very high degree of sensitivity, detecting nearly 100% of the known cases. It’s specificity is also quite high, between 99.6-99.8% (meaning only 0.2-0.4% of positives were false positives). The cause of false positives is not yet known and the possibility of cross-reactivity with other Coronaviruses is under study.
- The value of antibody testing, however, is currently limited. It is probably best used for epidemiological studies of communities to determine the prevalence of disease or to look at gaps in protection of groups of people. Looking ahead to a possible resurgence of illness in the fall, antibody testing could help determine if a prior infection helps provide protection against re-infection. It could also be used to help determine possible cause of illness, and help guide decision making, in people who have unusual presentations of COVID19 – such as strokes or blood clots.
- The COVID19 IgG serology test is NOT indicated for diagnosing acute infection. Indeed, it must be drawn at least 14 days AFTER the onset of symptoms, as it takes time for antibodies to develop. Also, we can’t use a negative result to rule out infection. There could be a SARS-CoV-2 infection but not enough antibody to detect on the study. This could be the result of a patient’s lack of a fully functioning immune system, a test that was drawn too early when not enough antibodies had yet been produced, or a test that was drawn too late when antibody levels were waning. So, a negative test doesn’t always reassure us.
- And a positive test result, showing prior exposure to SARS-CoV-2, cannot be used to provide reassurance that we are no longer infectious. Nor that we will not be at risk of developing COVID19 again in the future. We don’t know enough about our immune response to this virus to know at what level of antibodies, or what time away from symptoms, we are no longer contagious. We also don’t know how long our antibody levels will last and offer us protection. Will they fade over time? Only time will tell as we monitor the virus and our immune response to it.
- And remember that how meaningful a positive result is depends on the prevalence of disease in the community. Because we haven’t tested everyone in the community, we don’t know the exact prevalence of SARS-CoV-2. However, the greater the prevalence, the greater the test’s positive predictive value (PPV). For PPV based on possible prevalence in the community and a 99.6% test specificity, see the information below.
-
- Prevalence PPV
1% 71%
2% 83%
3% 88%
4% 91%
5% 93%
- Prevalence PPV
What can I expect if I call to request a test?
I will respond to this by saying that every clinician is different. But don’t be surprised if you are asked to have a visit (likely virtual or by phone) to discuss the specifics of your request. The antibody test is not appropriate for everyone and we have to have realistic expectations about what we can do with the information once we have it. It may take some discussion to figure out if this test is right for you.
Until next time…