Hesti Lina Wiraswati^1,2, Lia Faridah^1,2, Savira Ekawardhani^1,2, Nur Akmalia Hidayati³, Anastasya Fidausi², Neng Rita Nurjannah², Amila Laelalugina², Nisa Fauziah^1,2*

¹Department of Biomedical Sciences, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia

²Care and Control of Infectious Disease, Universitas Padjadjaran, Bandung, Indonesia

³Research Center for Environmental and Clean Technology, The National Research and Innovation Agency (BRIN), Bandung, Indonesia

Corresponding Author E-mail: nisa@unpad.ac.id

DOI : https://dx.doi.org/10.13005/bpj/2986

Abstract

The quality control of the COVID-19 Rapid Diagnostic Test (Ag-RDT) product is regarded as one of the government’s responsibilities. The Indonesian government establishes rules for Ag-RDT post-market validation, where it should be performed by two designated laboratories, using the spiking technique. The usage of this technique raises concerns, especially if it does not represent the precise product quality, due to the sample dilution. In addition, the requisite of using fresh samples that should be prepared for less than 48 hours is considered costly and time-consuming. In response to this, we tested two samples from different age groups on the Ag-RDT brand recommended by the World Health Organization (WHO); Panbio™ Covid-19 Ag Rapid Test (Abbott) and standard Q Ag-RDT (SD Biosensor, Roche). In both Ag-RDT products, the samples observed in the cycle threshold (Ct) values≤25 groups exhibit >80%sensitivity and >97% specificity as in compliance with the WHO recommendation. Meanwhile, as observed in the Ct>25 groups, the sensitivity of the two Ag-RDT products was below 25%, which was not in compliance with the WHO recommendation. Overall, this study indicated that the Spiking technique is eligible to be used for evaluating the performance of Ag-RDT, especially at Ct≤25. Additionally, the samples’ life span of up to 2 weeks of storage at -80^oC can be used for post-market validation of Ag-RDT. Furthermore, the quality control assay for longer sample storage is interesting to be carried out.

Keywords

Ag-RDT; Covid-19; Spiking, Panbio™; SD Biosensor

Download this article as: 

Copy the following to cite this article: Wiraswati H. L, Faridah L, Ekawardhani S, Hidayati N. A, Fidausi A, Nurjannah N. R, Laelalugina A, Fauziah N. The Establishment of the Spiking Method to Evaluate the Rapid Diagnostic Test Antigen (Ag-RDT) Product for COVID-19 Detection. Biomed Pharmacol J 2024;17(3).

Copy the following to cite this URL: Wiraswati H. L, Faridah L, Ekawardhani S, Hidayati N. A, Fidausi A, Nurjannah N. R, Laelalugina A, Fauziah N. The Establishment of the Spiking Method to Evaluate the Rapid Diagnostic Test Antigen (Ag-RDT) Product for COVID-19 Detection. Biomed Pharmacol J 2024;17(3). Available from: https://bit.ly/3ZLosL2

Introduction

The rapid chromatographic immunoassay for the qualitative detection of specific antigens of SARS-CoV-2 can be performed by antigen-detecting rapid diagnostic tests (Ag-RDT)^{1, 2, 3}. The number of Ag-RDTs on the market is recently increased in order to meet the demand for this product ^{4, 5, 6, 7}. This situation forces the government to assess the safety, quality, and performance of distributed Ag-RDT products. Each country holds its own regulation for the implementation of Ag-RDT, including Indonesia. The Indonesian government has established testing rules to ensure the validity of Ag-RDT in the context of contact tracing, diagnosis, and Covid-19 screening⁸. The assay is carried out by two designated laboratories using a predetermined Standard Operational Procedure (SOP) as described in the decree of the Minister of Health of the Republic of Indonesia⁹. The testing result is required to obtain marketing authorization from the Minister of Health^{9, 10}. The established protocol to validate the Ag-RDT kit is urgently needed due to the increasing demand from industries, societies, and laboratories during a prolonged Covid-19 pandemic.

Based on the regulation in Indonesia, the samples used for Ag-RDT kit validation must be fresh, and should be prepared within 2×24 hours⁹. Freshly prepared samples are better for clinical testing in that there is no limitation due to storage or transportation. The tested samples included 30 positive samples with Cycle Threshold Value (Ct) values ≤25, 30 positive samples with Ct>25, and 30 negative samples. However, employing freshly prepared samples was sometimes difficult particularly when the Covid-19 case was declined. Therefore, other alternative protocols, which are highly validated should be established. In appropriate storage conditions at −80^oC or −20^oC, the stored biological samples can still be feasibly used. In previous studies, nasopharyngeal and oropharyngeal swabs stored in Viral Transport Media (VTM) or sterile saline can be feasibly stored at −70 °C for more than 12 days¹¹. Gulec et al. reported that swab samples for both positive and negative samples, can be stored and retained their quality at 4°C for up to 12 days¹². A significant effect on the sample’s Ct value was observed after 10 cycles of freeze-thawed¹³. Within a week of the time limit, not all laboratories were ready to supply the test samples, thus the comparison assay of the Ag-RDT validation was carried out using fresh samples and the spiking method using 2 weeks old samples. In this study, we reported the implementation of the spiking method using swab samples with a longer shelf life on two brands of Ag-RDT kits recommended by WHO, which were Panbio™ Covid-19 Ag Rapid Test (Abbott) and Q Ag-Standard, RDT (SD Biosensor, Roche)^{14, 15}.

Methods

The assay was carried out by the ”Spiking Method” which examined the flock swabs from the nasopharyngeal/oropharyngeal specimens in VTM that were previously dipped into the Ag-RDT buffer kit. As a comparison, an RT-PCR assay was carried out on the same sample using QIAamp Viral RNA Kits (Qiagen) to isolate the viral RNA with N1 and N2 genes became the PCR-targeted genes. The reverse transcription and cDNA amplification were carried out using SuperScript™ III Platinum™ One-Step qRT-PCR Kit (Invitrogen). The Ag-RDT assay was carried out on 2 brands, namely Panbio™ Covid-19 Ag Rapid Test (Abbott) and Standard Q Ag-RDT (SD Biosensor, Roche) using the spiking technique on specimens that were confirmed positive with Ct≤25 and Ct>25, as well as negative specimens. The assay examined 2 different sample sets prepared from different time courses, including 2-week sand <48 hours samples

RT-PCR

Samples were extracted using QIAamp Viral RNA Kits, which were then analyzed by RT-PCR using SuperScript™ III Platinum™ One-Step qRT-PCR Kit and Agilent AriaMx Real-Time PCR system.

Sample Collection

The nasopharyngeal/oropharyngeal specimens in VTM were obtained through the C.29 laboratory (National Reference Laboratory, Covid-19 Testing No. 29), Universitas Padjadjaran-Indonesia. Samples were taken from two different time courses; 90 samples of BBT (2 weeks) were taken on February 25-28, 2021, and 66 fresh samples (<48 hours) were taken on March 24, 2021. The old samples were obtained from several hospitals in Bandung, Indonesia, while the fresh samples were obtained from regular patients in our laboratory.

Spiking Method Assay

The sterile flocked swab from each of the tested Ag-RDT kits was dipped and rotated to make sure all sides of the tip were coated. Then, the swab tip was taken out from the sample and swirled in the buffer fluid. The next step was dropping the spiked sample on the test device following each kit’s reference instructions. The result can be obtained within 15 minutes. 

Sensitivity and Specificity Analysis

The sensitivity percentage was calculated by the number of specimens identified as positive by the Ag-RDT assay divided by the number of specimens identified as positive by the RT-PCR reference assay. The specificity percentage was calculated by the number of specimens identified as negative by the Ag-RDT test divided by the number of specimens identified as negative by the RT-PCR reference assay.

Results and Discussion

Initially, the 2-week-old sample was validated using two brands of Ag-RDT kits, namely Panbio™ and SD Biosensor. The validation assay was carried out by the “Spiking Method”, which resulted in both high sensitivity and specificity values from both kit brands. According to the standard of the sensitivity and specificity values set by WHO, the results shown in Table 1 have passed the standard qualifications. It has been proved that the dilution which performed at the validation assay using the “Spiking Method” did not substantially reduce the quality of the antigen testing.

Based on the sensitivity values obtained at CT<25 and CT>25, the application of the antigen test is more accurate when employed in symptomatic patients with CT<25, while for CT>25 the sensitivity value did not successfully meet the WHO standard. Previous studies suggested that Ag-RDT was not accurate in the samples with Ct value of over 25 as it contained low viral loads^{16, 17}. A high sensitivity value is influenced by a good sample storage condition so that it retained the sample quality. There were no false positives observed from all the negative samples. However, there were some false negative results were obtained in both Ag-RDT tests. Overall, the sensitivity and specificity values for both kit brands still met the required WHO standard for the samples with CT≤25.

Table 1: Ag-RDT performance according to storage time of samples

Characteristics	Fresh Samples (n=66)				2 Weeks Old Samples (n=90)
	Panbio™		Standard™ Q		Panbio™		Standard™ Q
	Ct≤25	Ct>25	Ct≤25	Ct>25	Ct≤25	Ct>25	Ct≤25	Ct>25
Sensitivity	90%	19%	85%	13%	93%	10%	90%	7%
Specificity	100%	100%	100%	100%	100%	100%	100%	100%

Furthermore, a comparison assay was conducted with samples aged <48 hours or fresh samples collected sometime after evaluating the 2-week-old sample. The total of collected samples was 66 samples which was considered as less than the 2-week old sample since collecting the fresh samples at the same time as the desired number of positive and negative samples was quite difficult, especially when the Covid-19 cases began to decline. All samples were treated the same way using the “Spiking Method” in both the Ag-RDT brands. As shown in Table 1, it can be observed that the sensitivity and specificity values were very high at CT≤25 for both kit brands.

There are many factors must be considered when selecting an appropriate method to improve the evaluation of new devices in the clinical laboratory. In previous studies, spiking techniques have been reported to be successful used for a variety of different spiked-samples¹⁸. Dong et al. reported spiking pathogens into whole blood and virus into plasma showed acceptable reproducibility and can be followed by assay developers who are targeting low prevalence pathogens¹⁹. Detection of Mycoplasma pneumoniae in spiked respiratory samples also showed high sensitivity²⁰. The present study and other studies suggest that the spiking method is effective and applicable. The spiking technique can be an alternative method for Ag-RDT evaluation when the fresh samples are difficult to obtain if the samples are well-preserved at 20⁰C or –80⁰C with only one freeze-thaw cycle allowed.

The spiking technique offered advantages, such as the ability to get rapid result due to its ability to test several Ag-RDT brands at the same time, which also required only a set of experiments and a low-cost system. However, this technique raises concerns, especially for the companies,  as the validation that is carried out may not represent the true quality of the product that is being tested. It might be due to the dilution of the viral RNA which decreased the concentration. In addition, the compatibility of the buffer utilized for storing the sample with the Ag-RDT product has been brought into question. It has been noted that the transport medium may comprise guanidine salt, which possesses the property of a protein denaturant²¹. This may lead to a decrease in the activity of the protein or the complete denaturation of the protein due to the strong interaction between guanidine and the catalytic residues of the protein²². Atienzar et al. have reported that 6 of 19 Ag-RDT brands were incompatible with Amies media and the sensitivity decreased up to 2 to 20 times²³. It emphasizes the importance of choosing the appropriate sample matrices and assays for each specific use, particularly when employing Ag-RDT, as it can greatly affect the effectiveness of isolation and tracing measures.

Conclusion

The C.29 Laboratory of Universitas Padjajaran has conducted an assay on two Ag-RDT kits that specifically met the WHO criteria with ≥80% sensitivity and ≥97% specificity, namely Panbio™ Covid-19 Ag Rapid Test (Abbott) brand and Q Ag-Standard, RDT (SD Biosensor, Roche) using the “Spiking Method” on the samples with Ct value ≤25, >25, and RT-PCR-confirmed negative samples. In these studies, Panbio™ and SD Biosensor showed  93% sensitivity for the samples with CT value ≤25 and 90% sensitivity for the 2 weeks-old samples. The evaluation and validation of Ag-RDT using this spiking technique are deemed required as it can gain more benefits when testing several Ag-RDT brands at the same time, as it can utilize the same sample set for multiple validations.

Acknowledgement

This study was supported by the Directorate for Research, Community Service, and Innovation of Universitas Padjadjaran (DRPMI-UNPAD).

Conflict of Interest

The author(s) do not have any conflict of interest

Funding Sources

The author(s) received no financial support for the research, authorship, and/or publication of this article

Data Availability Statement

This statement does not apply to this article.

Ethics Statement

This study utilizes biological material stored without identifiable links to patients, approved by the Research Ethics Committee of the Faculty of Medicine, Universitas Padjadjaran, with registration no. 0720121265.

Informed Consent Statement

This study did not involve human participants, and therefore, informed consent was not required.

Author’s contribution

Conceptualization, HLW; methodology HLW, SE; software, AF, NRN, and AL; validation, HLW, SE; formal analysis, HLW and AF; investigation, AF, NRN, and AL; resources, LF and NF; data curation, AF, NRN, and AL; writing—review and editing, AF, HLW, NAH, LF, SE, and NF; supervision, HLW, LF, NF and SE; project administration, AL. All authors have read and agreed to the published version of the manuscript.

References

1. World Health Organization. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays: interim guidance, 11 September 2020. WHO/2019-nCoV/Antigen_Detection/2020.1.Geneva: World Health Organization; 2020.
2. Mina MJ, Parker R and Larremore DB. Rethinking Covid-19 test sensitivity – a strategy for containment, N Engl J Med., 2020; 383(22):e120.
   CrossRef
3. Brümmer LE, Katzenschlager S, Gaeddert M, Erdmann C, Schmitz S, Bota M, Grilli M, Larmann J, Weigand MA, Pollock NR, Macé A, Carmona S, Ongarello S, Sacks JA, Denkinger CM. Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: A living systematic review and meta-analysis. PLoS Med. 2021; 18(8):
   CrossRef
4. Federal Institute for Drugs and Medical Devices. Antigen-Tests auf SARS-CoV-2. Bonn: Federal Institute for Drugs and Medical Devices; 2021 [cited 2022 Jun 7]. https://www.bfarm.de/DE/ Medizinprodukte/ Antigentests/_node.html.
5. Kohmer N, Toptan T, Pallas C, Karaca O, Pfeiffer A, Westhaus S, Widera M, Berger A, Hoehl S, Kammel M, Ciesek S, Rabenau HF. The Comparative Clinical Performance of Four SARS-CoV-2 Rapid Antigen Tests and Their Correlation to Infectivity In Vitro. J. Clin.Med. 2021; 10: 328.
   CrossRef
6. Berger A, Nsoga MTN, Perez-Rodriguez FJ, Aad YA, Sattonnet-Roche P, Gayet-Ageron A, Jaksic C, Torriani G, Boehm E, Kronig I, Sacks JA, de Vos M, Bausch FJ, Chappuis F, Renzoni A, Kaiser L, Schibler M, Eckerle I. Diagnostic accuracy of two commercial SARS-CoV-2 antigen-detecting rapid tests at the point of care in community-based testing centers. PLoS ONE. 2021; 16(3):
   CrossRef
7. Baro B, Rodo P, Ouchi D, Bordoy AE, Saya Amaro EN, Salsench SV, Molinos S, Alemany A, Ubals M, Corbacho-Monné M, Millat-Martinez P, Marks M, Clotet B, Prat N, Estrada O, Vilar M, Ara J, Vall-Mayans M, G-Beiras C, Bassat Q, Blanco I, Mitjà O. Performance characteristics of five antigen-detecting rapid diagnostic test (Ag-RDT) for SARS-CoV-2 asymptomatic infection: a head-to-head benchmark comparison. J. Infect. 2021; 2021: 82(6) 269-275.
   CrossRef
8. Indonesia’s Ministry of Health no 446. Penggunaan Rapid Diagnostic Test Antigen dalam Pemeriksaan Corona Virus Disease 2019 (Covid-19). Keputusan Menteri Kesehatan Republik Indonesia Nomor Hk.01.07/Menkes/446/2021. Jakarta: Indonesia’s Ministry of Health; 2021.
9. Indonesia’s Ministry Of Health No 477. Laboratorium Penguji Validitas Rapid Diagnostic Test Antigen (Rdt-Ag). Keputusan Menteri Kesehatan Republik Indonesia Nomor Hk.01.07/Menkes/477/2021. Jakarta: Indonesia’s Ministry of Health; 2021.
10. Indonesia’s Ministry Of Health No 3602. Panduan Pelaksanaan Pemeriksaan, Pelacakan, Karantina, dan Isolasi dalam Rangka Percepatan Pencegahan dan Pengendalian Coronavirus Disease 2019 (Covid-19). Keputusan Menteri Kesehatan Republik Indonesianomor Hk.01.07/Menkes/4641/2021. Jakarta: Indonesia’s Ministry of Health; 2021.
11. World Health Organization (WHO). Guidance for laboratories shipping specimens to WHO reference laboratories that provide confirmatory testing for COVID-19 virus. Interim guidance. 31 March 2020. WHO reference number: WHO/2019-nCoV/laboratory_shipment/2020.3. WHO; 2020.
12. Yilmaz Gulec E, Cesur NP, Yesilyurt Fazlioğlu G, Kazezoğlu C. Effect of different storage conditions on COVID-19 RT-PCR results. J Med Virol. 2021; 93(12): 6575-6581.
    CrossRef
13. Dzung A, Cheng PF, Stoffel C, Tastanova A, Turko P, Levesque MP, Bosshard PP. Prolonged Unfrozen Storage and Repeated Freeze-Thawing of SARS-CoV-2 Patient Samples Have Minor Effects on SARS-CoV-2 Detectability by RT-PCR.J Mol Diagn. 2021; 23(6): 691-697.
    CrossRef
14. Panbio™ Covid19 Ag Rapid Test Device. Scale Up Covid-19 Antigen Testing Panbio™ Covid-19 Ag Rapid Testing Device. Abbot; 2020.
15. SD Biosensor Covid-9 Ag Home Test. 2020. Standar Q Covid-19 Ag Home Test Sd Biosensor Device. SD Biosensor; 2020.
16. Veyrenche N, Bolloré K, Pisoni A, Bedin AS, Mondain AM, Ducos J, Segondy M, Montes B, Pastor P, Morquin D, Makinson A, Le Moing V, Van de Perre P, Foulongne V, Tuaillon E.  Diagnosis value of SARS-CoV-2 antigen/antibody combined testing using rapid diagnostic tests at hospital admission. J Med Virol. 2021; 93: 3069–3076.
    CrossRef
17. World Health Organization. Advice on the use of point-of-care immunodiagnostic tests for COVID-19; scientific brief, 8 April 2020.WHO/2019nCoV/Sci_Brief/ POC_immunodiagnostics/2020.1. Geneva: World Health Organization; 2020.
18. Queipo-Ortuño MI, Tena F, Colmenero JD, Morata P. Comparison of seven commercial DNA extraction kits for the recovery of Brucella DNA from spiked human serum samples using real-time PCR. Eur. J. Clin. Microbiol. 2007; 27(2), 109–114.
    CrossRef
19. Dong M, Fisher C, Añez G, Rios M, Nakhasi HL, Hobson JP, Beanan M, Hockman D, Grigorenko E, Duncan R. Standardized methods to generate mock (spiked) clinical specimens by spiking blood or plasma with cultured pathogens. J. Appl. Microbiol. 2016; 120(4): 1119–1129.
    CrossRef
20. Loens K, Ursi D, Ieven M, van Aarle P, Sillekens P, Oudshoorn P, Goossens H. Detection of Mycoplasma pneumoniae in Spiked Clinical Samples by Nucleic Acid Sequence-Based Amplification. J. Clin. Microbiol. 2002; 40(4), 1339–1345.
    CrossRef
21. Wiraswati HL, Gaffar S, Ekawardhani S, Fauziah N, Rinawan FR, Widyatmoko L, Laelalugina A, Arimdayu AR, Kusniati T, Andari CD, Faridah L, Evaluation and Clinical Validation of Guanidine-Based Inactivation Transport Medium for Preservation of SARS-CoV-2. Adv Pharmacol Sci. 2022; 1677621: 1-9.
    CrossRef
22. Biswas B, Muttathukattil AN, Reddy G, Singh PC. Contrasting Effects of Guanidinium Chloride and Urea on the Activity and Unfolding of Lysozyme. ACS Omega. 2018; 3 (10): 14119-14126.
    CrossRef
23. Cubas-Atienzar AI, Kontogianni K, Edwards T, Wooding D, Buist K, Thompson CR, Williams CT, Patterson EI, Hughes GL, Baldwin L, Escadafal C, Sacks JA, Adams ER. Limit of detection in different matrices of 19 commercially available rapid antigen tests for the detection of SARS-CoV-2. Sci Rep. 2021; 11(1):18313.
    CrossRef

(Visited 95 times, 1 visits today)