Accuracy and cost description of rapid antigen test compared with reverse transcriptase-polymerase chain reaction for SARS-CoV-2 detection

Rapid and accurate detection of SARS-CoV-2 infection is essential in limiting the spread of infection during the COVID-19 pandemic. The cornerstone of SARS-CoV-2 testing is real-time reverse transcriptase-polymerase chain reaction (RT-PCR) of an upper-respiratory specimen. RT-PCR relies on centralised laboratory capacity and complex logistics, and scalability may be difficult if test demand is increasing. Rapid antigen (AG) tests can better provide scalability and buffer capacity. AG tests detect protein antigens from SARS-CoV-2 and may be performed onsite, are easy to administer and results are available within minutes. This enables faster tracing of infected individuals. However, the accuracy of the AG tests compared with the gold standard RT-PCR is questioned, which may limit their use despite relatively low costs per test. Studies have reported a sensitivity down to 55%; yet, other studies report a sensitivity of nearly 100% [1-3]. Specificity seems to be consistently high. However, peer-reviewed data on the sensitivity and specificity in larger public settings is sparse and cost evaluation has not been addressed. The aim of this study was to determine the accuracy of the World Health Organization (WHO) Emergency Use Listed (EUL)-approved STANDARD Q COVID-19 AG test (SD BIOSENSOR) by comparison with RT-PCR in a public setting and to describe their costs.

METHODS

RT-PCR testing is available free of charge for all citizens in Denmark at public test centres. Individuals aged 18 years or older who had booked an appointment for a RT-PCR test on 26-31 December 2020 at Testcenter Taastrup in Copenhagen, Denmark, were invited to participate. An oropharyngeal swab was collected for RT-PCR analysis immediately followed by a nasopharyngeal swab examined by the STANDARD Q COVID-19 AG test (SD BIOSENSOR). Using oropharyngeal swabs for RT-PCR testing is common practice for COVID-19 testing in Denmark. The AG test was performed as part of this project and was not a standard practice. Participants were asked to complete an online questionnaire regarding symptoms before leaving the test area.

This study was conducted in accordance with the guidelines of the Declaration of Helsinki and all participants provided informed consent. Approval for conducting the study was obtained from the Regional Committee on Health Research Ethics (case no. H-20083631) and from the Danish Data Protection Agency (P-2020-1222).

Reverse transcriptase-polymerase chain reaction

Detection of SARS-CoV-2 was performed by single-target RT-PCR at TestCenter Danmark, Statens Serum Institut. Oropharyngeal swabs were collected by the personnel at Testcenter Taastrup and eluted in phosphate-buffer saline, and ribonucleic acid was extracted using RNAdvance Blood (Beckman). One-step RT-PCR to detect SARS-CoV-2 was performed using Luna Universal Probe One-step RT-qPCR kit (New England Biolab) [4]. The followingprimers and probe binding to the E-gene were used:

E_Sarbeco_F (ACAGGTACGTTAATAGTTAATAGCGT), E_Sarbeco_R (ATATTGCAGCAGTACGCACACA), E_Sarbeco_P1 (FAM-ACACTAGCCATCCTTACTGCGCTTCG-BHQ1). Samples with viral cycle threshold (Ct) values between ten and 38 were considered positive. The results of the RT-PCR test were considered the gold standard.

Antigen test

The WHO EUL-approved STANDARD Q COVID-19 AG test produced by SD BIOSENSOR was performed by personnel fromCopenhagen Medical A/S according to SD BIOSENSOR’s instructions. No participants would leave the test facility before a conclusive test result had been obtained. No inconclusive tests were found for AG tests, and there was no need for re-testing. Participants received the result of the rapid AG test by individual links on their phones.

Questionnaire

The participants’ mobile phone number was registered and a link to an online questionnaire was sent to them by SMS. The questionnaire was developed in REDCap and the participants’ answers were collected here. The questionnaire included questions about whether or not the participant had symptoms of COVID-19 and – in case of symptoms - a specification of which symptoms (i.e. fever, throat pain, cough, shortness of breath, headache, loss of taste and/or smell, tiredness, general soreness, skin rash, conjunctivitis, and diarrhoea). Participants were categorised as having symptoms if they answered “yes” to having symptoms, regardless of how many and which symptoms they had in the next question.

Cost description

Costs were calculated using the ingredients method [5]. All costs associated with the sampling and analysis for the two tests were calculated from the perspective of the Danish health authorities. We included costs for test kits, personnel needed for obtaining the samples, personal protective equipment and price for renting test facilities, including depreciation of equipment over a five-year period. We cross-checked these costs against the operating costs registered with the Copenhagen Emergency Medical Services responsible for testing and with Statens Serum Institut to ensure the validity of our estimates. We calculated the cost per true positive test result for both test types, including total costs and test kit costs only. We used official exchange rates to convert costs from DKK into USD.

Statistical analysis

Sensitivity, specificity, positive and negative predictive values of the AG test were calculated using the test results from the RT-PCR as a reference. A boxplot depicting difference in Ct values between participants with true positive and false negative AG tests, including analysis for statistical difference by Wilcoxon Rank Sum test, was performed in R statistics (version 3.6.1).

Trial registration: Clinicaltrials.org: NCT04716088.

RESULTS

Overall, 4,811 paired conclusive results from the RT-PCR tests and AG tests were accessible, corresponding to 4,697 separate participants as 196 participants were tested twice or more, with a minimum of one day between tests. The majority were females (n = 2,456, 53.3%) and the median age was 45 (interquartile range: 30-56). Among the 4,811 paired conclusive test results, 221 (4.6%) RT-PCR tests were positive.

A total of 66 RT-PCR results were missing, and 31 RT-PCR results were inconclusive (i.e. Ct > 38). These tests were excluded from the analysis.

Among the positive RT-PCR tests, 154 had a paired positive Ag test, corresponding to a 69.7% sensitivity. Among the 4,590 negative PCR tests, 4,567 had a paired negative AG test, corresponding to a specificity of 99.5%. With 23 false positive results and 67 false negative results of the AG test, the positive and negative predictive values were 87.0% and 98.5%, respectively (Table 1).

Changing the criteria of positive RT-PCR to Ct ≤ 33 increased the sensitivity of the AG test to 76.9%. At a Ct ≤ 30, the sensitivity was 81.1%.

A total of 3,713 (77.2%) participants answered the questionnaire. Among participants with self-reported symptoms and paired conclusive test results, the accuracy of the AG test was higher, at a sensitivity of 78.8% and a specificity of 98.8%. For participants without self-reported symptoms, the accuracy of the AG test was lower, at a sensitivity of 49.2% and a specificity of 99.6% (Table 1).

Ct values were significantly higher among participants with false negative AG tests than among participants with true positive AG tests, both for participants with self-reported symptoms (median Ct value: 30 and 27, p = 0.005) and participants without (median Ct value: 32 and 27, p = 0.0002) (Figure 1).

Cost description

At 1,000 daily tests, the AG test cost 2.8-4.8 times less than the RT-PCR test depending on whether only equipment costs or total costs were included (Table 2). However, as the RT-PCR tests become less expensive with increasing volume, this difference lessened for a scenario with 100,000 daily tests, where the costs per positive sample was 1,459.0 DKK (236.1 USD) for the RT-PCR test and 1,093.0 DKK (176.8 USD) for the AG test (Table 3).

DISCUSSION

This study found a 69.7% sensitivity of the AG test in a public testing setting with a 4.6% prevalence of SARS-CoV-2 infection. For participants with self-reported symptoms, the sensitivity was 78.8%, whereas it was 49.2% for participants without self-reported symptoms. In agreement with the recommendation from the Centers of Disease Control and Prevention (CDC) on the use of AG testing, the sensitivity of the investigated rapid AG test indicates that it should not replace RT-PCR in diagnosis and surveillance of SARS-CoV-2 infection [6]. However, the AG tests might still play an essential role in screening and containment strategies. As has been argued, the relatively lower sensitivity of the AG tests compared with RT-PCR testing is of less importance if testing frequency is increased [7, 8]. Thus, it has been shown that effective screening depends on the frequency and speed of testing, whereas effective screening is only marginally improved by a high sensitivity. Further, infrequent testing with a sensitive test will result in isolation and quarantining of individuals in the recovery period who have detectable virus but are not at risk of infecting others as their virus load is below the infectious threshold [8, 9]. So far, the test strategy in Denmark has focused on easy access to test facilities for all citizens. In December, the AG test was implemented as part of the publicly funded test offer to enhance test capacity [10].

Participants with false negative results of the Ag test had significantly higher Ct values corresponding to a lower viral load. This suggests that participants with false negative AG tests may be less infectious in general. Further, RT-PCR tests may be positive for several weeks in the course of infection when no culturable virus can be detected [11]. Frequent testing combined with the fact that test results from the AG test are available within minutes means that the AG tests might be effective at identifying infected individuals when they enter the transmissible stage. RT-PCR testing would instead detect the infected individual at a lower viral load, but would also have a one-two-day delay before asymptomatically infected individuals with a high viral load can be detected and isolated.

The positive and negative predictive values of the AG test were high, especially among participants with symptoms. The predictive values are, however, highly dependent on the prevalence of infection in the population and the values are most reliable when the prevalence is high. Individuals undergoing testing must therefore be informed that a negative test gives no certainty of a true negative result.

The costs of the AG test were 2.8-4.8 times lower than those of the RT-PCR test. Despite the lower sensitivity, the AG test costs about two-thirds of the RT-PCR test per positive case detected in our study context when applying the assumptions that produced the lowest costs of the RT-PCR test (100,000 tests per day, equipment and analysis costs only). These findings have implications for policy decisions regarding he choice of test method for regular mass testing as more than twice the number of individuals may be tested for the same price when using the AG test rather than the RT-PCR tests. The consequences of false negatives are important to consider. False negative tests most likely result in failure to self-isolate with the risk of infecting others [12]. The economic costs of unintended viral spread due to false negative testing should be considered, although estimating its exact magnitude is challenging.

The study was performed in a public setting with a relatively low prevalence of SARS-CoV-2. Both the rapid AG test and the RT-PCR test were performed as a screening for SARS-CoV-2 infection in the general population, and therefore the results may be generalised [13].

A limitation to the study is the comparison of test results from oropharyngeal and nasopharyngeal swabs. However, oropharyngeal swabs are the standard in the public RT-PCR test facilities in Denmark, and two nasopharyngeal samples would have increased test discomfort and decreased inclusion of volunteers. Diagnostic results from RT-PCR of oropharyngeal and nasopharyngeal swabs are comparable [14] and both methods are in accordance with CDC recommendations [15]. Furthermore, our results are in line with results on the sensitivity and specificity reported by another study in which nasopharyngeal sampling was performed for both the AG test and the RT-PCR test [3]. Even though RT-PCR is considered the gold standard for detection of SARS-CoV-2 infection, it is not flawless and the choice of RT-PCR as a reference and the criteria defined for positive results have implications [16, 17]. As seen in this study, changing the criteria for positive RT-PCR from Ct ≤ 38 to Ct ≤ 33 and Ct ≤ 30 increased the sensitivity of the AG test from 69.7% to 76.9% and 81.1%, respectively. An increase in viral load values and thus a lower Ct value is associated with greater risk of transmission and a greater risk of symptoms [18]. Further, error in registration and the need to transport the swab material to laboratories for RT-PCR testing leads to a number of missing test results, in this study 1.4% of the RT-PCR tests. This contrasts with the AG test where all participants had received a conclusive test result before leaving the test facility. The RT-PCR test is very accurate. However, its high sensitivity may result in virus being detected after the infectious period, leading to false positive results [19]. Whether or not participants had symptoms of SARS-CoV-2 infection was based on self-reporting in an online questionnaire. This may be a limitation as some might mistakenly consider a non-relevant symptom to be a symptom of SARS-CoV-2 infection. However, this was limited by presenting examples of relevant symptoms in the questionnaire.

A limitation to the cost evaluation was that the oropharyngeal swabs and RT-PCR analysis were conducted by the health authorities, whereas the nasopharyngeal swab and AG test were performed by a private company. This might impact the price setting of costs per test.

In agreement with the WHO’s recommendation to test for SARS-CoV-2 as intensively as possible, the STANDARD Q COVID-19 AG test and other rapid AG tests with a similar accuracy seem to be a relevant and good supplement to RT-PCR testing and a method for SARS-CoV-2 screening that costs approximately two thirds of the RT-PCR test.

Correspondence Christian von Buchwald. E-mail: Christian.von.Buchwald@regionh.dk
Accepted 17 May 2021
Conflicts of interest none. Disclosure forms provided by the authors are available with the article at ugeskriftet.dk/dmj
Acknowledgements We thank Copenhagen Medical A/Sfor delivering the rapid AG tests and providing test personnel for performing the tests. We would like to thank test Centre Director Andreas Lyman and the personnel at Testcenter Taastrup for their essential assistance in collecting data. Finally, we express our gratitude to Centre Director Martin Magelund Rasmussen, Rigshospitalet, for critical discussion and help with logistics.
Cite this as Dan Med J 2021;68(7):A03210217