Overall Percent Agreement

In the FDA`s latest guidelines for laboratories and manufacturers, “FDA Policy for Diagnostic Tests for Coronavirus Disease-2019 during Public Health Emergency,” the FDA explains that users should use a clinical trial to establish performance characteristics (sensitivity/AAE, specificity/NPA). While the concepts of sensitivity/specificity are widely known and used, the terms AAE/APA are not known. Nor do these statistics support the conclusion that one test is better than another. Recently, a British national newspaper published an article on a PCR test developed by Public Health of England and the fact that with a new commercial test in 35 samples out of 1144 (3%) disagreed. Of course, for many journalists, this was proof that the PHE test was imprecise. There is no way to know which test is correct and which is wrong in any of these 35 discrepancies. We simply do not know the actual state of the subject in unit studies. Only further investigation into these discrepancies would identify the reasons for these discrepancies. To avoid confusion, we recommend that you always use the terms positive agreement (AAE) and negative agreement (NPA) when describing the agreement of these tests.

CLSI EP12: User Protocol for Evaluation of Qualitative Test Performance Protocol describes the terms of the Positive Percentage Agreement (AEA) and the Negative Performance Agreement (NPA). If you have two binary diagnostic tests to compare, you can use an agreement study to calculate these statistics. Although the positive and negative matching formulas are identical to those for sensitivity/specificity, it is important to distinguish them because the interpretation is different. In the next blog post, we`ll show you how to use Analysis-it to perform the contract test with a treated example. Due to COVID-19, there is currently a great deal of interest in the sensitivity and specificity of a diagnostic test. These terms refer to the accuracy of a test for the diagnosis of a disease or condition. To calculate these statistics, the actual condition of the subject must be known, whether the subject has disease or condition. As you can see, these measures are asymmetrical. In other words, changing the methods of testing and comparison, and therefore of the b and c values, changes the statistics. However, they have a natural and simple interpretation when one method is a reference/comparison method and the other is a test method. We have seen that the information produced for a COVID-19 rapid test uses the terms “relative” sensitivity and “relative” specificity compared to another test.

The term “relative” is an erroneous term. This means that you can use these “relative” ratios to calculate the sensitivity/specificity of the new test based on the sensitivity/specificity of the comparative test.