Verification of new AST Systems

Introducing new testing methods into the laboratory is an exciting event but it comes with additional quality process requirements. While the system manufacturer offers a readymade solution, laboratories are often required to verify that the system is performing according to the manufacturer’s specifications in their own laboratory environment. Verification ensures that the system functions correctly and that laboratory personnel can produce accurate and reproducible results.

System verification and validation are often interchanged – so what is the difference? According to Clinical and Laboratory Standards Institute (CLSI) a validation consists of collecting a wide set of performance data, often generated by multiple operators and study sites with a specific type and number of samples, that can be used to seek FDA clearance or IVD/CE marking 1,2. Validation work is done by the method/system manufacturer. If a lab is developing an IVD testing method of their own or is using a commercial product in a way that deviates from the manufacturers specified use, a validation study would need to be performed2.

Verification recommendations

Verification is required by various guidelines, such as ISO 15189 standard3, EUCAST4 and regional recommendations5, when a system is introduced into the laboratory for the first time to either measure an analyte previously unknown to the laboratory or to compare the new system to an existing system already in use. CLSI document M521  describes a detailed protocol on Verification of commercial microbial identification and antimicrobial susceptibility testing systems.

What is required to be tested for an antimicrobial susceptibility testing (AST) system verification? 1,2 Accuracy and precision are the main performance specifications:

CriteriaDefinitionAcceptance limits
Accuracy
  • Categorical agreement (CA): S, I, R are consistent between the methods compared.
  • Essential agreement (EA): MIC results within +/-1 doubling dilution or within +/-2 doubling dilutions with yeast compared to the reference method.
Agreement ≥ 90% of test results, with < 3% very major/major discrepancies or errors
Precision (Reproducibility)
  • S, I, or R interpretation reproducible.
  • MIC value reproducible with an accepted variation of +/- 1 doubling dilution for bacteria and +/- 2 doubling dilutions for yeast.

Agreement ≥ 95% of test results.

≥ 95% of QC strain results within QC specifications.

Comprehensive and limited verification are performed by testing accuracy and reproducibility but the number of isolates tested varies; comprehensive verification would be performed in the event of a new system or change in method while limited verification is sufficient when introducing a new antimicrobial agent within an existing method, for example.

Type of changeAccuracyReproducibility

Comprehensive

  • New system
  • Change in testing method
Minimum of 30 isolatesTest 5 isolates x 3 (QC strains or clinical isolates)

Limited

  • New antimicrobial agent
  • Second instrument of the same model added
Minimum of 10 isolatesTest QC strains 3 x for 5 days or 1 x day for 20 days

Limited

  • New dilutions added for MIC test e.g. breakpoint change
Minimum of 30 isolatesTest QC strains 1 x for 5 days

 

How about reference method or samples?

Reference method for laboratory verification can be selected from one of three options2:

  • IVD labelled and previously verified testing method.
  • AST reference method: broth microdilution or agar dilution MIC testing.
  • Obtaining isolates with known AST results from a verified AST system (from outside source).

Samples tested should cover clinical strains including strains with relevant resistance mechanisms; the selection of organisms for testing should be representative of those clinically indicated for the antimicrobial agents that will need to be verified1. Proficiency testing isolates and strains from other laboratories are also often utilized. In the US the CDC-FDA Antimicrobial Resistance Isolate Bank (AR Bank) offers various isolates that can be used for system verification6 and in Europe EUCAST has started to build recommendations for verification isolate sets7. In addition to testing for accuracy and reproducibility, quality control testing is recommended to be performed every day of testing with a set of various QC strains. See a comprehensive selection of QC strains here: View the full portfolio ›

 

Ready to implement a new antimicrobial susceptibility testing system into your laboratory? Discover our full range of AST solutions here

References

  1. CLSI. Verification of Commercial Microbial Identification and Antimicrobial Susceptibility Testing Systems. 1st ed. CLSI guideline M52. Wayne, PA: Clinical and Laboratory Standards Institute; 2015.
  2. Hindler, J.A. & Prouse A.B. Verification of Commercial Microbial Identification and Antimicrobial Susceptibility Testing Systems. Clinical and Laboratory Standards Institute.
  3. ISO 15189:2012. Medical laboratories – Requirements for quality and competence. Geneva, Switzerland: International Organization for Standardization; 2012.
  4. Matuschek, E. et al. Development of the EUCAST disk diffusion antimicrobial susceptibility testing method and its implementation in routine microbiology Laboratories. Clin Microbiol Infect 2014; 20: O255–O266.
  5. Cattoen, C. et al. QUAMIC 2019, 2nd Edition, La Société Française de Microbiologie (SFM). Antibiogramme, CMI et tests complémentaires pour la recherche de mécanismes de résistance aux antibiotiques. France; 2019.
  6. CDC Centers for Disease Control and Prevention. CDC & FDA Antibiotic Resistance (AR) Isolate Bank, https://www.cdc.gov/drugresistance/resistance-bank/index.html, March 9 2020 
  7. The European Committee on Antimicrobial Susceptibility Testing.  EUCAST panels of phenotypically defined strains https://www.eucast.org/ast_of_bacteria/strains_with_defined_susceptibility/ , January 19 2021 

Have you signed up to our newsletter?

Sign up now