The Predictive Genomics blog series provides perspectives on the use and impact of genetic risk screening & pharmacogenomics in clinical research. Predictive Genomics is powerful capability to study disease risk and understand drug response in order to focus health care resources on improving outcomes and managing costs. To talk with our team on implementing your genetic-based initiatives, please contact us or browse our predictive genomics solutions.
Polygenic risk scores (PRS’s) are a type of genetic test in which dozens to millions of low-impact genetic variants are identified and mathematically combined to produce a “risk” score for a specific health condition (eg, Type 2 Diabetes or Schizophrenia) or phenotypic feature (eg, height). In the scientific literature, the resulting score is typically presented as an odds ratio.
If a PRS is making a health-related claim, it requires all the same evidence that is required of any other clinical test before it can be legally marketed. Similarly, a PRS’s path to widespread adoption will be the same as for other clinical tests. In this blog we will review the data necessary for a clinical test to be legally marketed, the routes to market, and the data needed for payers and providers to make decisions about coverage and use. We will discuss why it is a formidable challenge to apply this evidentiary framework to PRS’s and elucidate why no PRS’s have achieved widespread usage or coverage to date.
What Types of Pre-Market Studies are Required for Clinical Tests?
The two main requirements for legally marketing a clinical test are analytical validity and clinical validity. Tables 1 and 2.
| Analytical Validity | Clinical Validity | |
| Purpose | Quantifies the test’s ability to detect what it says it detects. | Quantifies the test’s ability to detect a disease or condition. |
| Examples | Detection of a nucleic acid base change from A>G in DNA.
Detection of a protein fragment from the HHV-8 virus in blood. |
People with dysfunctional NF1 genes have Neurofibromatosis Type 1.
People with HHV-8 have ____ disease. |
| Definition | ||
| Outputs | Analytical sensitivity
Analytical specificity Reliability Reproducibility Interfering substances Limit of detection
|
Clinical sensitivity
Clinical specificity Positive predictive value Negative predictive value Likelihood ratios |
Study Designs for Establishing Analytical and Clinical Validity
Both are based on the classic 2 x 2 table that compares the performance of the new test (the index test) to the reference standard. The output metrics for each are mathematical calculations from the 2 x 2 table and are well-described in the medical and statistical literature. (REFS)
|
ANALYTICAL VALIDATION STUDY DESIGN |
||
| Samples known to have the analytic change of interest as determined by reference standard | Samples known NOT to have the analytic change of interest as determined by reference standard | |
| Samples positive for the analyte change by NEW TEST | True Positives | False Positives |
| Samples negative for the analyte change by NEW TEST | False Negatives | True Negatives |
|
CLINICAL VALIDATION STUDY DESIGN |
||
| Samples from individuals known to have the condition of interest as determined by reference standard | Samples individuals known NOT to have the condition of interest as determined by reference standard | |
| Samples positive for the condition by NEW TEST | True Positives | False Positives |
| Samples negative for the condition by NEW TEST | False Negatives | True Negatives |
Regulatory Routes in the United States
The more complex question in the United States is which regulatory route a test developer chooses to take to get their test on the market. There are two federal agencies and two separate regulations: The Food and Drug Administration (FDA) regulations are based on the laws set forth in the Tobacco Control Act and the Food, Drug, and Cosmetic Act (FD&C Act) and the Center for Medicare and Medicaid (CMS) with regulations based on CLIA’88.
The US Food and Drug Administration (FDA) claims jurisdiction for all medical devices, including in vitro diagnostics (IVDs). The objective is to offer assurance that a test performs robustly enough for clinical decision-making and that the associated clinical claims are valid. Classic IVDs are test kits (ie, pre-measured reagents and the directions for how to combine them) that are produced in a manufacturing plant and sold to credentialed clinical labs via interstate commerce. Once the manufacturer develops the intended use statement, they are required to submit data showing both the analytical validity of the test and clinical validity of the claim to the FDA before marketing the kit to clinical laboratories. Once purchased, the performance metrics of the test are verified by the local clinical lab and the test is made available for physicians to order. (See verified vs validated). The premarket review process makes it relatively expensive (due to both fees and personnel costs), delays time to market, and stifles innovation and improvement of previously submitted products because amendments are expensive and time-consuming.
However, some laboratories have adopted a different path for clinical tests called a laboratory developed test (LDT). One example is Dutch molecular diagnostics firm SkylineDx that offers laboratory developed tests for multiple myeloma and acute myeloid leukemia, called MMprofiler and AMLprofiler, both of which run on Thermo Fisher Scientific’s microarrays. Their new melanoma test based on Thermo Fisher QuantStudio 5 quantitative PCR platform will also be offered as an LDT in the US, possibly paving the way for a CE-IVD test next year.
According to the FDA, an LDT is a type of IVD that also falls under their jurisdiction. This is contested by many laboratory professional entities. Historically, the FDA has exercised “enforcement discretion” with LDTs, i.e. they only intervene and require pre-market review when the laboratory/manufacturer of the LDT is marketing a product that does not have sufficient analytical and clinical evidence to be on market.
Although CLIA only requires laboratories to demonstrate analytical validity before marketing a test, in practice, all credible laboratories also document the clinical validity of the claim. Well-trained health professionals operate with patient safety as the priority. Besides, why spend the money to develop a test that isn’t clinically valid? Recently, however, there has been an increase in the number of investor-backed clinical laboratories where decision-making authority is given to individuals without proper education and training in clinical laboratory medicine. Investor’s needs rival or supersede patient safety as the guiding principle and tests without evidence to support the clinical claim are marketed to physicians or consumers. This is why some people think that all LDTs should be regulated like IVDs and undergo pre-market review by the FDA. In the end, if the claims about the clinical meaning of your LDT does not have clinical validity, the agency will retract its enforcement discretion, pull your product from the market, and require you to submit the necessary data for your LDT to the FDA prior to returning to the market.
The takeaway is that all clinical tests are required to demonstrate analytical and clinical validity before they are marketed for clinical use, and the same applies to PRS’s making a health claim. The route to market is the more complex question.
How Does a PRS-based Test Achieve Widespread Adoption?
Regulatory bodies care primarily about the analytical and clinical validity of the test. Payers and providers care primarily about the clinical utility and cost-effectiveness of the recommended intervention based on the results of the test.
Clinical validity quantitates the strength of the association between the analytical change detected and the clinical condition of interest. But, is knowing that clinically useful? What will doctors do with that information? Is it more or less useful than the standard of care? This is called clinical utility and payers and providers expect this question to be answered before they cover or adopt a new test. To date, there are no published clinical utility studies for claims made based on a PRS.
Clinical utility studies test the safety and efficacy of an intervention based on the PRS test result versus current standard of care. For example,
- Men over 40 identified by PRS of being in top 3% of CAD event risk who take a statin will have a lower percentage of CAD events than men over 40 identified as being in the top 3% risk by clinical risk score who are put on statins.
- Individuals In the top decile of CAD PRS have the same risk of CAD as individuals with monogenic FH (clinical validity claim) and should be managed the same as FH (clinical utility claim).
What about the economics of the new test and its intervention? Does this cost a huge amount of money for a negligible or no real health benefit? Payers and providers also expect to have an understanding of the economic factors before making a coverage or adoption decision. (Looking into the future)
The types of studies required by regulatory agencies to assess if a test is appropriate for clinical use and the types of studies required by payers and providers to make coverage or adoption decisions are well-known. For polygenic risk scores to achieve widespread clinical adoption, each one will need to undergo the same studies expected of all clinical tests. The choice of regulatory path may also impact the time to market and the ability to iterate to improve both quality and cost of testing.
Leave a Reply