Here’s what’s been on my mind for the past several weeks: Are our COVID tests too sensitive? Have they been too sensitive all along? Are we overcounting positives, lumping in defeated COVID fragments which are not dangerous with those full viruses of infectious patients? Are we throwing away or deliberately obscuring some of the most useful signal data we have when we simply report “positive” or “negative?”
I don’t think we are doing enough to distinguish between “worrisome” positives and “less concerning” positives. I think in several years’ time, we will look back on this period and say we made a major mistake in the way we lumped them all together, when it was possible not to do so.
There is considerable evidence that in the “Positive” case counts, we are also counting those who pose no infectious spread risk and in fact are successfully processing or have already processed the virus, which helps toward herd immunity. From a public policy standpoint, given that so many of our “reopen” plans hinge upon “case counts must be below X%”, it is vital that we do as much as we possibly can to separate “worrisome” positives from “non-worrisome” positives.
And, though less urgent: Are our tests and thresholds even consistent between nations? Between public health departments within the US? Between college campuses? We should seek better standardization, so that counts can be better compared.
Let’s Dive Into “Positives”
When we look at “positive test cases”, we first need to understand what that number means.
First, does every positive case represent someone who is contagious or poses substantial risk of spread?
No. And evidence is growing that in many cases, perhaps even the majority of cases, a positive test does not mean they are necessarily infectious. That’s because “positive” counts will also include those whose immune system is well-functioning, who have low viral load, and even those who have inactive viral fragments and have “beaten” the virus, with or without their knowledge. Positives will also detect infectious people, as well as those who have low viral loads but are at the beginning stages of their infection.
It’s remarkable we are able to see into the molecular level and do this. PCR technology is incredible.
But each of these positive cases are very different circumstances which I would argue deserve different public health responses. And the ambiguity in a “positive” arises from the technology we use, which detects fragments or markers of the virus, not necessarily an entire active virus itself. It will pick up fragments of a defeated virus, extremely rare and tiny ones if something called the “Cycle Threshold” is high enough.
Some scientists are now suggesting ways to better distinguish between these very different types of positives (explained below.)
But importantly, for what must be various public policy reasons and inertia, we appear not to presently want to embrace or even much explore it with the urgency I think it deserves.
At a minimum, I feel it’s vital that we know and report hospitalization counts along with positives, but we also need to know something called the Cycle Threshold of viral detection, as well as both the aggregate and individual trends in that number.
Impact
Given the importance we place in the positive test counts now that we’ve moved from “flattening the curve so hospitals aren’t overloaded” to today’s I-don’t-really-know-what-but-sort-of “no positive test growth must be observed”, I’m surprised this issue of what constitutes a “positive” isn’t getting much more attention. Positive test rates now impact everything from school and campus closures to quarantine and isolation to individual mandatory quarantining to business and retail closures and cancellations and more.
We label a large number of positive test cases an “outbreak.” But in reality, a high cluster of positives might not represent an “outbreak” at all, it might in fact represent a cluster of, say, young people who have each successfully defeated the virus. A cluster of positives is a cluster of positives — reason for caution, dynamic, preventative action until more is known (and it generally can be known within days or weeks), and also, deeper investigation.
Shouldn’t we get this number right, and work hard to disambiguate between worrisome and not-yet-worrisome cases? Can adjustments to our testing protocols be made to better distinguish between those who might be on the early end of infection from latter tail? I think yes, below, see “Recommendations.”
What percentage of us actually knows what “positive” specifically means?
Shouldn’t we be reporting some kind of broken-out number that clearly distinguishes between those with strong enough viral load to be concerning versus those with a miniscule level of viral load which may only be partial? Shouldn’t we be establishing new protocols for those who come in with “high viral load signals” versus low viral load signals (e.g., testing the latter soon thereafter, to detect directionality — whether it’s worsening, stable or improving?)
By boiling down our reporting into a simple binary, we are losing a lot of valuable information, and causing a lot of unnecessary harm and restriction in our public response.
Background To Know
The main test technology we use in the US, polymerase chain reaction (PCR), is an exceedingly powerful technology. It works by duplicating the genetic material in a sample to the point where a specific genetic sequence is detected. Samples are put in a machine. Reagents are added, and multiple passes, or “cycles” are run, doubling the molecules each time. Ct is the “cycle threshold”, the number of amplification cycles needed to detect a particular genetic strand.
Note that PCR machines aren’t generally trying to detect the entire, active virus, but rather fragments. In the case of COVID-19, it’s key RNA markers. They may or may not be parts of a complete viral protein.
If the virus is detected in fewer cycle thresholds, a high viral load is likely to be in the sample. The more cycles (Ct) the test runs, the more likely it is to detect a small or even miniscule viral load, such as fragments of the virus that are not viable or contagious.
The cycle threshold (referred to as the Ct value) is the number of amplification cycles required for a fluorescent signal to cross a certain threshold. This allows very small samples of RNA to be amplified and detected. Each pass represents an approximate doubling of molecules in the sample – so there is a huge difference even between Ct 30 and 31.
A very important thing to know is that a person who has RECOVERED, with or without their knowledge, will have a small amount of viral debris and fragments in their body, and generally, from what I’ve read, evidence is emerging that 30 cycle passes (Ct==30) might not tend to detect those tinier and lower-viral-load fragments of the “already recovered or recovering”, but (Ct>= ~40) probably would, because the amplification is enormous at high ends.
A Harvard University epidemiologist recently suggested that positives produced with more than 30 cycles are unlikely to find infectious patients and some studies are confirming that conclusion (some of which are below.), But this is controversial, and is at the intersection of both longstanding public policy practices and agency/manufacturer recommendations and more. A leader of Minnesota’s public health lab said there is “no convincing proof” and that her lab is confident in the federally required cycle threshold (CT) of 38 for its COVID-19 test. Of course, the difference between 38 and 30 cycles is eight doublings — an enormous degree of sensitivity difference.
The WHO’s guidelines appear to be 45 cycles (Ct==45!)
The FDA appears to recommend Ct values of 40, as of late July 2020 (page 34-35.)
Worse, different manufacturers have varying recommendations on the “right” number Ct’s by which to run their machine. A growing number of researchers are recommending the Ct value be more like 30, and that any fragments picked up beyond 30 passes tend to NOT yield a patient who is contagious (studies linked below), but rather a person who has too little of the fragments to be a public health concern. Is it time to change the guidelines for these tests?
The FDA apparently has regulations or guidelines which prevent the Ct value from being reported by labs, only “positive” or “negative.” Why are we deliberately throwing away this potentially useful information? I am still trying to find a rational reason why we are deliberately mapping a fairly big continuum to a simple 1 or 0. That’s like taking a beautiful photograph and downsampling and grey-scaling it to a 16x16px black and white icon. You can never get that information back, once filed away and fragmented.
Related Evidence and Readings
Predicting Infectious SARS-CoV-2 from Diagnostic Samples: In this study, scientists were unable to demonstrate any viral growth in “positive” samples with Ct > 24, but could do so with 24 and below. “SARS-CoV-2 Vero cell infectivity was only observed for RT-PCR Ct < 24 and STT < 8 days. Infectivity of patients with Ct >24 and duration of symptoms >8 days may be low.” As a layman, I read that and say, “well, scientists tried hundreds of times and seem to be unable fostering any reproduction of the virus in Ct levels below 25. Probabilistically, it would seem to suggest that we really ought to pay attention to low-Ct positives and maybe less so those positives which are over some buffer beyond that level.”
To Interpret the SARS-CoV-2 Test, Consider the Cycle Threshold Value: “…[H]igh sensitivity for viral RNA can be helpful for initial diagnosis. However, reporting as a binary positive or negative result removes useful information that could inform clinical decision making. Following complete resolution of symptoms, people can have prolonged positive SARS-CoV-2 RT-PCR test results, potentially for weeks, as Xiao et al report. At these late time points, the Ct value is often very high, representing the presence of very low copies of viral RNA [5–8]. In these cases, where viral RNA copies in the sample may be fewer than 100, results are reported to the clinician simply as positive. This leaves the clinician with little choice but to interpret the results no differently than for a sample from someone who is floridly positive and where RNA copies routinely reach 100 million or more.”
Strong Inverse Correlation Between SARS-CoV-2 Infectivity And Cycle Threshold Value: “Correlation between successful isolation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in cell culture and cycle threshold (Ct) value of quantitative reverse transcription polymerase chain reaction (RT-PCR) targeting E gene suggests that patients with coronavirus disease 2019 (COVID-19) with Ct above 33 to 34 are not contagious and can be discharged from hospital care or strict confinement, according to a brief report published in the European Journal of Clinical Microbiology & Infectious Diseases.”
Are you infectious if you have a positive PCR test result for COVID-19? Oxford University. Noted in this meta-review is the following chart, which shows how the probability of infectiousness is greater (red bars) when the cycle threshold is lower (blue line.) It also shows that when symptoms to test time is shorter, it’s also more infectious:
In the above study of 32,480 subjects in Massachusetts nursing homes (residents and staff), it was observed that those with Cts less than or equal to 23 harbored 99.9% of the cumulative viral load, and those with Cts less than or equal to 26 harbored 99.99% of the population cumulative viral load.
Rethinking Covid-19 Test Sensitivity — A Strategy for Containment, New England Journal of Medicine (just published yesterday), focuses on the logistical and time delay involved in using PCR testing because of its high sensitivity, arguing that it would be more effective to go with cheaper (but less accurate) technology which can be done on-site. This article is peripherally related to the main issue of this blog post, but provides added caution about overemphasis on PCR and general goals of the upper ends of high-sensitivity.
Backgrounder on PCR and Cycle Thresholds from Public Health England
Very good backgrounder. But note their summary at the end — is it excessively cautious given the cost of lockdown?
Understanding_Cycle_Threshold__Ct__in_SARS-CoV-2_RT-PCR_Ct is a semi-quantitative indicator of the concentration of viral genetic material in a
patient sample.From a laboratory perspective, Ct values should only be reported and applied for clinical
interpretation and action where the linearity, limit of detection and standard quantification
curves are assured.Ct values are not directly comparable between assays and may not be reported by
some RT-PCR platforms in use. Interpreting single positive Ct values for staging
infectious course, prognosis, infectivity or as an indicator of recovery must be
done with context about the clinical history.Low Ct values (high viral load) more likely indicate acute disease and high infectivity.
High Ct values (low viral load) can be attributed to several clinical scenarios whereby the
risk of infectivity may be reduced but interpretation requires clinical context.
Serial Ct values have greater utility for interpretation but are generally only undertaken in
hospital settings for the purpose of clinical management rather than infection control
purposes.
CDC Guidelines
As of August 4, 2020, the CDC is displaying this Q&A with respect to cycle threshold values, which seems to be contradicted by several of the studies above:
Other Parts of the World
- Spain also seems to have changed their guidelines this week about Ct’s, recommending that they stay in the 30-35 range when reporting positives
- India also has now declared that COVID-19 test reports must now state cycle threshold
There are many more such studies and meta-reviews emerging, and I’ll update this list as I discover them.
Twitter Threads On the Subject
In the United States, one of the key scientists raising some of these questions is Michael Mina, an epidemiologist, immunologist and physician and Harvard Public Health/Medical School. His thread below is well worth reading. Click through on the link, and read each of the 17 posts in the thread:
Media Coverage
Given what’s at stake — millions of people in lockdown and quarantine, introduction of enormous anxiety for those who test “positive”, many of whom might not have major cause for alarm — this issue has gotten fairly light coverage. It has been drowned out by a host of other stories, from the national election to wildfires to other urgent crises. But a few reports have broken through:
- In mid-September, the New York Times highlighted this issue with Your Coronavirus Test is Positive. Maybe It Shouldn’t Be.
- Minneapolis Star Tribune: Minnesota’s COVID-19 Testing Under the Microscope
- The DailyMail: Experts: US COVID-19 Positivity Rate High Due to ‘Too Sensitive’ Tests
Here’s a good interview with Dr. Michael Mina, Assistant Professor of Epidemiology at Harvard and director of molecular virology about the Ct issue, and why institutional practices on reporting have been slow to adapt:
Cases vs. Hospitalizations
A trend I’ve been noticing in many US dashboards is that hospitalizations used to follow cases after about 14 days or so, but often no longer do.
For instance, my son is a Northwestern University student, and so I follow the dashboard of Suburban Cook County IL closely. Let’s take a look at case counts vs. hospitalizations since this crisis began:
What’s going on here? Why did hospitalizations follow cases in April & May, and yet are virtually unaffected by positive cases starting in July? There are several possible explainers, which I’ll delve into in a future post, and it’s unlikely that there is just one and only one contributor. But one contributing factor is very possibly that a “positive” is also counting in those who have successfully processed the virus and/or who pose considerably less risk to public health. And yet, our public policy is essentially to treat all positives the same.
Key Questions
Are high positive test counts always a sign of the problem worsening within a community?
It doesn’t seem that this conclusion would naturally follow. It could also indicate clusters of “herd immunity / passed virus” forming, since a “positive” at high Ct can also indicate someone who is well past processing of the virus. We should ask this question particularly among student populations on campuses, since that age cohort has extremely low mortality from COVID.
Could overly-sensitive “positives” be a major contributing factor in why we’ve not seen the expected rise in hospitalizations and mortality following positive-counts? Back in March, spikes in positives would lead to strong spikes in hospitalizations and deaths. In late September, hospitalizations and mortality largely do not follow as closely, at anywhere close to the same order of magnitude as back in wintertime.
Could the way we report positives be improved to better differentiate “worrisome” positives from “less worrisome” positives?
Given the fact that people whose immune system has functioned well and fought off the virus will still test “positive”, should we still be using the term “outbreak” when we observe high positive test counts, or should we look more toward either low-Ct values or hospitalizations for such a descriptive term?
Most important, perhaps, what steps can or should we take to better distinguish between positives-likely-to-be-infectious and positives-likely-not-infectious in our public health reporting and public health response?
Recommendations
I am not alleging any kind of conspiracy or malicious intent to hide the truth. I am also continuing to mask-up, particularly in confined spaces like retail, etc. I am also not intending to suggest that COVID and associated risks are not serious — it is, and they are.
But I am also reminding us that we as humans are learning about this virus every day, and shouldn’t have our initial implementation of policies ossify just because they’re were established as such in emergency conditions. Public health officials are doing their best and have crafted responses. But some of those responses I think need to be adjusted, some perhaps dramatically. Certainly I think we could use more dynamism in the policies that are implemented — I’m not sensing a lot of movement or dynamic revision. That’s understandable — institutions and agencies don’t turn on a dime — but I don’t think it’s adequate, given the high costs being imposed.
Specifically, from what I’ve read on this subject, I don’t quite understand why we aren’t now altering our public health response in the following ways:
- Move beyond a simple binary “positive/negative.” Sure, continue to report positives and negatives, but also: report Ct values on every test, and aggregate them into a histogram-style chart, as well as trends in that Ct value: medians, modes, and standard deviations over time. Set a benchmark for what constitutes a “high” viral load. (Several scientists now seem to coalesce around the number 30 or so.) How many subjects were detected with “high” viral loads? “Low” viral loads? What is the trend in that signal? Get a better picture of the “severity and infectiousness” of the virus.
- Always report hospitalizations, whenever “positives” are reported. Hospitalization counts are very important disambiguating metrics. Every dashboard needs to also report hospitalizations and mortality. Yet I’m not seeing hospitalization counts on many community and college campus dashboards. Just showing “positive test cases” creates alarm where it might not need to, because it is also including those in the population whose immune system is functioning well and have defeated the virus and are not at risk of infectious shedding.
- Shift from a “one and done” testing model to situational immediate followup testing for directionality, particularly among those with “low” viral load. That is, if you test positive but with Ct over some potentially “less concerning” threshold value (e.g., 34, or whatever health officials determine), yes, go ahead and isolate/quarantine but also test again in 18-24 hours to disambiguate between those who might be early in their infection versus those who might be beyond it. If the Ct of detection in the second or third test continues to be high (or higher), perhaps consider loosening quarantine protocols. Consider less rigorous quarantine or lockdown for those with high Ct values per the research above. If that number drops, quarantine and isolate, as it appears the course of the virus might be growing.
I think in a few years, we may look back on this time and say we made a major mistake overcounting positives, and considering that a high number of positives is an “outbreak.” It is vital that we at least take a moment to examine whether we should be making changes to the way we are reporting COVID positives.
DISCLAIMER
I am not a physician. I’m not a scientist. Nor do I have any medical training. I’m simply Very Online, and I read a lot. Like a lot of us, I enjoy building, I plan, and I invest. I generally find I need to have a mental model of what the next 0-24 month time horizon looks like in order to build and plan and invest successfully, and I work pretty hard at trying to have a good guess about that near to mid-term horizon.
