Rapid technologic advances in human genome sequencing in the past 2 decades have resulted in the identification of a wide variety of disease-causing mutations and single-nucleotide polymorphisms (SNPs) that are associated to varying degrees with medical disorders. Many of these DNA "markers" have subsequently been configured as genomic tests and are available to the public, either through health care practices or via direct-to-consumer marketing. In the field of nutrition, much attention focuses on predictive DNA testing, with an expectation that test results will improve both individualized lifestyle management and patient motivation, thereby reducing morbidity and mortality. Genomic testing that is available for such purposes, however, has usually been introduced before completion of definitive studies to determine whether (and to what degree) knowledge about specific genotypes will be more effective in improving health than knowledge gained from existing practice. Single-nucleotide polymorphism panels, for example, usually yield relative risks no higher than 1.5 and no lower than 0.5. When combined with other information, such as family history, serum cholesterol, and blood pressure, the DNA-related relative risks usually become even less predictive, as the excess (or lower) risk is already accounted for by these other factors.1 As a result, expectations for improved health outcomes from genomic testing may only be partially realized, or even not realized at all. This article describes initiatives that have been undertaken to help practitioners and the public decide on how and when to use these new tests.

Development of an Assessment Process

In 2002, the Office of Public Health Genomics at the Centers for Disease Control and Prevention funded a cooperative agreement with our group for developing and testing the feasibility of a framework for collecting, evaluating, interpreting, and reporting data about DNA (and related) testing for disorders with a genetic component.2 A brief description of that project and the topics evaluated in the feasibility study can be found at http://www.cdc.gov/genomics/gtesting/ACCE/FBR/index.htm. The format allows policymakers to access up-to-date and reliable information for decision making. The process builds upon methodology published by Wald and Cuckle3 and uses terminology introduced by a committee established by the Secretary of Health and Human Services (the Secretary's Advisory Committee on Genetic Testing).4 It is designed primarily to guide those carrying out formal evidence reviews, but it is also readily accessible for less formal use by practitioners attempting to gain an overall impression of any given test's performance. The framework considers 4 critical assessment components-Analytic validity, Clinical validity, Clinical utility, and Ethical, legal, and social issues (abbreviated as ACCE)-and is initiated by specifying the clinical disorder, the test(s) to be used for detecting the disorder, and the setting in which the test is to be applied. Then, the 4 assessment components, each with a standardized set of specific questions, are systematically applied, as described below. An important by-product of this process is the identification of gaps in knowledge.

The ACCE Model for Assessing Genomic Tests

Defining the Clinical Disorder Being Sought by Testing

The Figure displays the ACCE model. Each of the ACCE components is labeled in the figure, along with its key elements. The questions contained in these elements were field tested and refined during the feasibility study described in the preceding paragraph. The first step (shown at the center of the target) calls for the clinical disorder to be carefully defined, along with the testing scenario (eg, population screening) and the test that is to be used. Table 1 lists specific questions to be answered to guide correct characterization of the disorder, setting (scenario), and test. Particular attention is paid to this first activity, because successfully assessing the 4 components depends on a clearly defined starting point. Problems have arisen in the past when a "disorder" has been described in terms of the test being used to identify it, rather than in terms of clinical manifestations. One such example from the pre-DNA testing era arises when cholesterol measurement is stated to be for the purpose of identifying "hypercholesterolemia." Hypercholesterolemia is a laboratory finding rather than a clinical disorder. The clinical disorder is coronary artery disease, and the usefulness of cholesterol testing needs to be based on the extent to which test results lead to a reduction in prevalence of that disorder. Similarly, it used to be commonly thought that blood pressure testing was for detecting "hypertension." Hypertension is high blood pressure, and high blood pressure is not a medical disorder. Instead, the disorder being sought is stroke (and cardiovascular disease), and the effectiveness of screening needs to be based on the ability of testing and treatment to reduce the risk of these disorders.

Figure. Graphic representation of the ACCE system for assessing genomic tests. ACCE stands for

Table 1. Questions to Define the Clinical Disorder and Setting in Which DNA Testing Is to Be Applied

Analytic Validity of the Genomic Test

Analytic validity, the first assessment step, defines a test's ability to accurately and reliably measure the genotype of interest. Table 2 lists questions to guide reliable and comprehensive assessment of this component of a genomic test. This aspect of evaluation examines all aspects of laboratory testing, including preanalytic, analytic, and postanalytic performance, but it does not attempt to evaluate clinical implications. Reviews of analytic performance carried out to date using this system indicate that DNA testing available in practice is highly reliable, although not perfect. Key questions include (1) When a mutation (or SNP) is present, how often does the test correctly identify it? (2) When a mutation (or SNP) is not present, how often does the test correctly classify its absence? Four main elements of analytic validity include analytic sensitivity (or the analytic detection rate), analytic specificity (or 1 - the analytic false-positive rate), laboratory quality control, and assay robustness.

Table 2. Questions for Assessing Analytic Validity

Clinical Validity of the Genomic Test

Assessment of clinical validity defines a DNA test's ability to detect or predict the associated clinical disorder. An important distinction exists between analytic and clinical validity. For example, a given DNA test may reliably detect a genotype (analytic validity), but if a serious medical problem occurs only once among every 100 persons who have that genotype, the clinical validity is poor. Clinical validity builds upon analytic validity by assessing 5 more elements (Figure). These include clinical sensitivity (or the clinical detection rate), clinical specificity (or 1 - the clinical false-positive rate), prevalence of the clinical disorder, positive and negative predictive values, and penetrance (including gene and environmental modifiers). Penetrance refers here to the proportion of individuals carrying a disease-causing mutation (genotype) that develops the clinical manifestations (phenotype). One obvious question for a predictive DNA test would be: What proportion of individuals will develop the disorder being sought, when the mutation/genotype is present? For example, about two-thirds of women with BRCA1/2 mutations will develop breast cancer (positive predictive value), whereas one-third will not (clinical false-positive rate). Among women with a BRCA mutation who do not develop breast cancer, the test result is nearly always analytically valid, but penetrance of the mutation is incomplete.5 A second question would be: What proportion of women will develop the disorder being sought, when a mutation/genotype is not present? For example, BRCA mutations occur in only 2 or 3 of every 1000 women. Among the remaining 997 women in the general population with no BRCA mutation, 1 in 10 will develop breast cancer during their lifetime (clinical false-negative rate). This means that no more than 2 of every 100 breast cancer cases are associated with BRCA mutations. A clinical false-negative result is usually not due to laboratory error. Instead, it indicates that the disorder can be caused not only by the mutation(s) being tested for, but also by other mutations or causal agents.5Table 3 lists questions that are helpful in guiding assessment of clinical validity.

Table 3. Questions for Assessing Clinical Validity

Clinical Utility of the Genomic Test

Clinical utility defines the risks and benefits associated with a test's introduction into practice. Specifically, clinical utility focuses on the health outcomes (both positive and negative) associated with testing, as summarized in the Figure. Table 4 shows that the list of questions to be asked about clinical utility is more extensive than for the other 3 components. The natural history of the clinical disorder needs to be understood, so that such considerations as optimal age for testing might be taken into account. It is necessary to determine the availability and effectiveness of interventions aimed at avoiding adverse clinical consequences. If no effective interventions are available, for example, testing may not be warranted, even if clinical sensitivity and specificity are high. Quality assurance assesses procedures in place for controlling preanalytic, analytic, and postanalytic factors that could influence the risks and benefits of testing. Pilot trials assess the performance of testing under real-world conditions, including uptake rates of testing and the psychological and social risks and benefits of testing. Health risks define adverse consequences of testing or interventions in individuals with either positive or negative test results. Economic evaluation helps define and compare the financial costs and benefits of testing. Facilities assess the capacity of resources to manage all aspects of the service. Education assesses the quality and availability of validated informational materials and expertise. Monitoring and evaluation assess a program's ability to maintain surveillance over its activities and make adjustments.

Table 4. Questions for Assessing Clinical Utility

Ethical, Legal, and Social Implications Associated With Testing

Ethical, legal, and social implications (ELSIs) surrounding the testing process for the clinical disorder refer to 2 types of concerns: those inherent in any medical technology and those particularly germane to testing for diseases that have a genetic component. The latter concerns include implications for relatives of the person undergoing testing, the possibility of insurance discrimination, and stigmatization based on genotype (disease risk), rather than on phenotype (actual disease). The precise nature of these risks, however, depends to a great degree on the preceding components. Thus, ELSI concerns are represented in the Figure by a penetrating pie slice. Table 5 lists specific ELSI questions. Although many concerns in this area are common to a variety of medical conditions and diagnostic procedures, implications for relatives are of special concern for genomic testing.

Table 5. Questions for Assessing Ethical, Legal, and Social Implications

Purpose of the ACCE Assessment Process

The ACCE process is designed to serve as an organizing framework for information about genomic tests, as a way to assist professional, government, and other independent groups in developing policy recommendations.2 To ensure that focus is maintained on the rapidly expanding list of new genomic tests at the policy recommendation level, the Office of Public Health Genomics at the Centers for Disease Control and Prevention established an initiative called the "Evaluation of Genomic Applications in Practice and Prevention" (EGAPP), in late 2004. The EGAPP initiative (which has no oversight or regulatory authority), in turn, established a nonfederal, independent Evaluation Working Group (EWG) and incorporated aspects of the ACCE process into its own methods for carrying out evidence reviews, including such features as selective use of unpublished data and "gray literature" (gray literature refers to pertinent, recent information not easily found through conventional channels).6 Electronic search engines now exist for gray literature (E.G., Google, LexisNexis, Gray Literature Report) to gain access to international clinical, regulatory, and health technology assessment Web sites for publications, as well as policy, technology, and quality control assessment reports/white papers. The EWG's purpose is not only to expand the scope and capacity of assessment activities in genomics, but also to develop recommendation statements based on the evidence reviews, as a way to guide practice. It is possible to contact EGAPP directly and request that a specific topic be added to the EWG's existing topic list (http://www.egappreviews.org/contact.htm).

Converting Genomic Test Evaluations Into Recommendations

Several evidence reports have now been commissioned by the EWG and completed by Evidence-Based Practice Centers (http://www.ahrq.gov/clinic/epc/) or by consultants and in-house staff. These can be accessed via http://www.egappreviews.org/workingp/reports.htm and include genetic testing for venous thromboembolism, metastatic colorectal cancer, Lynch syndrome, breast cancer, nonpsychotic depression, and ovarian cancer. In follow-up to these reports, 4 recommendation statements have been published,7-10 with more in process. Most recently, a meta-analysis examined the association between 9p21 genomic markers and heart disease, the findings and implications of which might be of interest to health professionals in the field of nutrition.11 In that analysis, 9p21 SNPs were significantly associated with heart disease, although the magnitude of the association was small. Intervention trials were not found to establish the clinical utility of adding 9p21 testing to traditional risk factors. Three studies evaluated the impact of adding 9p21 but did not modify treatment protocols based on this addition and did not examine long-term health or behavioral outcomes. Thus, the meta-analysis documented clinical validity, but not clinical utility, indicating that more research is needed prior to introduction into practice.

Implications for Health Professionals

Any genomic test that is being introduced into everyday practice should initially be viewed with healthy skepticism, whether its availability is through health care practices or direct to consumer. Often, risk factors with clear clinical validity are mistakenly considered to have clinical utility as screening tests, when, in fact, few are truly suitable.12,13 Tests being advocated for widespread application, such as routine screening, are of special concern, because of the potential for generating false expectations and wasting resources that might be better invested elsewhere. On the other hand, considerable attention needs to be given to designing delivery systems to maximize benefits of tests that have been proven useful. When a published recommendation or comprehensive review is not available to guide use of a test, the usual reason is that performance data are sparse. In such instances, a better understanding may be gained by systematically applying an abbreviated ACCE methodology.14 As with more comprehensive literature reviews, however, a preliminary literature search needs to be preceded by a clear definition of the clinical disorder and health end point to be achieved. In this abbreviated approach, the next step in ACCE assessment of test performance is restricted to the most relevant questions (selected from clinical validity or clinical utility). These initial steps serve as the template for extracting key information and focusing conclusions. If the test's clinical sensitivity, specificity, and/or positive predictive value are determined to be unsatisfactory, there is no need to assess clinical utility. Conversely, a preliminary search may demonstrate only a weak association between the test and the disorder being sought (clinical utility), indicating that there is no point in evaluating clinical validity. In either case, the inevitable conclusion will be that the test is not ready for routine clinical application. The take-away message in this paragraph about maintaining a healthy skepticism cannot be overemphasized, especially in the current environment in which numerous new tests are being actively promoted.

REFERENCES

1. Paynter NP, Chasman DI, Pare G, et al Association between a literature-based genetic risk score and cardiovascular events in women. JAMA. 2010;303:631-637. [Context Link]

2. Haddow JE, Palomaki GE: ACCE: A Model Process for Evaluating Data on Emerging Genetic Tests. Human Genome Epidemiology: A Scientific Foundation for Using Genetic Information to Improve Health and Prevent Disease. Oxford: Oxford University Press; 2003:217-233. [Context Link]

3. Wald N, Cuckle H. Reporting the assessment of screening and diagnostic tests. Br J Obstet Gynaecol. 1989;96:389-396. [Context Link]

4. Enhancing the oversight of genetic tests: recommendations of the SACGT. July 2000. http://oba.od.nih.gov/oba/sacgt/reports/oversight_report.pdf. Accessed February 23, 2010. [Context Link]

5. McClain MR, Palomaki GE, Nathanson KL, Haddow JE. Adjusting the estimated proportion of breast cancer cases associated with BRCA1 and BRCA2 mutations: public health implications. Genet Med. 2005;7:28-33. [Context Link]

6. Teutsch SM, Bradley LA, Palomaki GE, et al. The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group. Genet Med. 2009;11:3-14. [Context Link]

7. Recommendations from the EGAPP Working Group: can UGT1A1 genotyping reduce morbidity and mortality in patients with metastatic colorectal cancer treated with irinotecan? Genet Med. 2009;11:15-20. [Context Link]

8. Recommendations from the EGAPP Working Group: can tumor gene expression profiling improve outcomes in patients with breast cancer? Genet Med. 2009;11:66-73. [Context Link]

9. Recommendations from the EGAPP Working Group: genetic testing strategies in newly diagnosed individuals with colorectal cancer aimed at reducing morbidity and mortality from Lynch syndrome in relatives. Genet Med. 2009;11:35-41. [Context Link]

10. Recommendations from the EGAPP Working Group: testing for cytochrome P450 polymorphisms in adults with nonpsychotic depression treated with selective serotonin reuptake inhibitors. Genet Med. 2007;9:819-825. [Context Link]

11. Palomaki GE, Melillo S, Bradley LA. Association between 9p21 genomic markers and heart disease: a meta-analysis. JAMA. 2010;303:648-656. [Context Link]

12. Wald NJ, Morris JK, Rish S. The efficacy of combining several risk factors as a screening test. J Med Screen. 2005;12:197-201. [Context Link]

13. Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ. 1999;319:1562-1565. [Context Link]

14. Gudgeon JM, Palomaki GE, Williams MS. Rapid, evidence-based reviews of genetic tests. In: Khoury MJ, Bedrosian SR, Gwinn M, Higgins JPT, Ioannidis JPA, Little J, eds. In: Human Genome Epidemiology. New York: Oxford University Press; 2010:482-496. [Context Link]

For additional continuing education articles related to Genetic topics, go to http://NursingCenter.com/CE.