MSACL 2022 Abstract

Introduction: Targeted mass spectrometry-based platforms have become a valuable tool for the sensitive and specific detection of protein biomarkers in the clinical and research setting. Assay validation criteria adopted from small molecule analyses and applied to MS based peptide measurements have resulted in improved assay reliability, standardization, and inter-laboratory agreement.
Achieving robust performance with targeted MS-based assays designed to detect analyte present at exceedingly low concentrations such as viral antigen detection, however, remains challenging. These assays require extreme sensitivity along with a well characterized limit of detection for confident analyte identification. The presence of background signal and increased measurement variability at the low end of abundance, however, can increase the LOD, and in turn, negatively impact the functional sensitivity of an assay.

Objectives: Here, we set out to develop a targeted parallel reaction monitoring (PRM) based assay for the detection of SARS-CoV-2 from nasopharyngeal swabs. In an effort to improve the functional sensitivity of the assay, we developed a novel machine learning (ML) based framework to directly analyze the fragment intensities of native peptides and discriminate positive from negative samples. The performance of this novel method was compared to that of the traditional PRM method, which used SIL-based quantification for detecting peptides of interest for viral antigen detection.

Methods: For the PRM based assay, two peptides from SARS-CoV-2 nucleocapsid protein were targeted for detection with stabile isotope labeled internal standards (SIL) using an Exploris 480 (Thermo Scientific, Bremen Germany) mass spectrometer coupled to a preformed gradient LC system (EvoSep One, EvoSep Inc., Odense, Denmark). The top 4 most intense fragment ions from each peptide were monitored. An extensive characterization of assay performance was then performed over 20 separate days to determine parameters such as the analytical limit of detection (LOD), linearity and intra/inter-day assay precision for each targeted peptide.

For the machine learning based data analysis, b- and y-ion intensities of the two native peptides described above were directly modeled using an ensemble ML framework that utilized random forest, penalized logistic regression, and generalized logistic regression methods. The three trained models were fused into a “Super Learner” that combined the individual model predictions to produce a probabilistic score of positivity for a sample. The model was trained with the observed intensities from 1276 SARS-CoV-2 NP swabs (positive = 282, negative = 994) acquired using the targeted PRM method. This ensemble model was then validated using an independent set of 350 NP swab samples (positive = 200, negative = 150).

Results: Traditional validation of the assay over 20 separate days demonstrated good analytical performance with an LOD of less than 0.45 pmol/L for each peptide. Additionally, precision at three different control levels ranged from 9.9-21% for intraday precision and 11.5-24.5% for interday precision. Using a traditional PRM-based data analysis approach to analyze the validation samples (n = 350), the assay resulted in a functional sensitivity of 86.5% (81.0 – 90.9) with a specificity of 96.0% (91.5, 98.5) when considering the gold standard as positive results obtained by RT-PCR. When analyzing the same data using the ensemble ML model a superior sensitivity of 92.0% (87.3, 95.4) with specificity of 98.7% (95.3, 99.8) was achieved. Additionally, a separate analysis showed that the ML model is more robust against performance loss when working with noisy data, which stood in contrast to the traditional quantitative PRM method for antigen detection.

Conclusion: These results demonstrate that machine learning can be applied to qualitative PRM assays and results in superior performance when compared to traditional data analysis methods.