Skip to main content

Advertisement

Comparative analysis of functional assay evidence use by ClinGen Variant Curation Expert Panels

Article metrics

Abstract

Background

The 2015 American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) guidelines for clinical sequence variant interpretation state that “well-established” functional studies can be used as evidence in variant classification. These guidelines articulated key attributes of functional data, including that assays should reflect the biological environment and be analytically sound; however, details of how to evaluate these attributes were left to expert judgment. The Clinical Genome Resource (ClinGen) designates Variant Curation Expert Panels (VCEPs) in specific disease areas to make gene-centric specifications to the ACMG/AMP guidelines, including more specific definitions of appropriate functional assays. We set out to evaluate the existing VCEP guidelines for functional assays.

Methods

We evaluated the functional criteria (PS3/BS3) of six VCEPs (CDH1, Hearing Loss, Inherited Cardiomyopathy-MYH7, PAH, PTEN, RASopathy). We then established criteria for evaluating functional studies based on disease mechanism, general class of assay, and the characteristics of specific assay instances described in the primary literature. Using these criteria, we extensively curated assay instances cited by each VCEP in their pilot variant classification to analyze VCEP recommendations and their use in the interpretation of functional studies.

Results

Unsurprisingly, our analysis highlighted the breadth of VCEP-approved assays, reflecting the diversity of disease mechanisms among VCEPs. We also noted substantial variability between VCEPs in the method used to select these assays and in the approach used to specify strength modifications, as well as differences in suggested validation parameters. Importantly, we observed discrepancies between the parameters VCEPs specified as required for approved assay instances and the fulfillment of these requirements in the individual assays cited in pilot variant interpretation.

Conclusions

Interpretation of the intricacies of functional assays often requires expert-level knowledge of the gene and disease, and current VCEP recommendations for functional assay evidence are a useful tool to improve the accessibility of functional data by providing a starting point for curators to identify approved functional assays and key metrics. However, our analysis suggests that further guidance is needed to standardize this process and ensure consistency in the application of functional evidence.

Background

In 2015, the American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) established standards and guidelines [1] for clinical variant interpretation. These guidelines provided criteria for classifying variants as pathogenic (P), likely pathogenic (LP), variant of uncertain significance (VUS), likely benign (LB), or benign (B) using distinct evidence types, each of which was assigned a level of strength. Additional rules specified combinations of the types and strengths of criteria sufficient to reach a pathogenic or benign classification. In cases of insufficient or conflicting evidence, variants were classified as VUS, which present a challenge in clinical molecular genetic testing as they should not alone be used to define clinical decision-making according to the ACMG/AMP standards. Functional data has considerable potential to aid in variant classification, particularly VUS reclassification [2]. In contrast to the opportunistic nature of many types of evidence (such as the fortuitous discovery of a family with sufficient segregation data to aid interpretation), functional assays are the most amenable to development and therefore the most tractable to be produced in a timely manner after a variant is observed. The ACMG/AMP guidelines state that the results of “well-established” functional studies can qualify as evidence for functional criteria application coded as PS3 or BS3 (an abbreviation for functional evidence in the direction of a pathogenic or benign interpretation, respectively, at a default evidence strength of strong) and that validation, reproducibility, robustness, and ability of the assay to reflect the biological environment should be considered. However, it is unclear how these attributes should be evaluated, and selecting appropriate functional evidence often requires expert-level knowledge of the gene and disease.

The Clinical Genome Resource (ClinGen) has founded Variant Curation Expert Panels (VCEPs) in multiple high-priority disease areas, each tasked with developing adaptations of the ACMG/AMP rules for their disease or gene of interest [3]. These VCEP specifications regarding functional data provided expert interpretations of the qualities required for an assay to be deemed “well-established.”

In this study, we sought to define the characteristics of functional assays that satisfy PS3/BS3 criteria by conducting a comparative analysis of VCEP recommendations for these criteria. Six initial VCEPs that published disease- and gene-specific adaptations to the ACMG/AMP guidelines (CDH1, Hearing Loss, Inherited Cardiomyopathy-MYH7, PAH, PTEN, and RASopathy [4,5,6,7,8,9]) span multiple conditions with different underlying, well-known disease mechanisms; thus, diverse categories of functional assays (biochemical function, protein interaction, expression, model system, etc.) have been applied to test variants in the associated genes. We used these initial functional assay recommendations as a case study, allowing us to assess the validation parameters and evidence strength for each approved assay, as well as the features of assays that were not approved by the VCEPs. We curated instances of assays in the primary literature cited by each VCEP both in their recommendation publication and in the course of their pilot variant classification using consistent criteria. This approach allowed us to assess the extent to which cited instances of assays satisfy VCEP-specified recommendations and how they differed. Results from this work will be critical to identify areas of functional assay interpretation that requires further guidance to ensure uniformity in variant classification, both in the context of VCEP recommendation development and beyond the scope of ClinGen, such as interpretation in clinical laboratories and other curation efforts.

Methods

Evaluation of ClinGen VCEP specifications

We assessed the guidance for the use of PS3/BS3 by six ClinGen VCEPs with approved and published variant interpretation recommendations as of April 2019: CDH1, Hearing Loss, Inherited Cardiomyopathy-MYH7, PAH, PTEN, and RASopathy [4,5,6,7,8,9]. Information about the clinical features of these conditions and the detailed ACMG/AMP rule modifications can be found in each reference. In our initial survey of recommendations, we noticed certain parameters (replicates, controls, thresholds, and validation measures) were identified by more than one group. We evaluated how often these four assay parameters were specified by the VCEPs and whether each VCEP provided recommendations for modifying PS3/BS3 evidence strength to a moderate (PS3_M) or supporting (PS3_P/BS3_P) level.

Literature search and variant identification

To identify relevant primary literature for each VCEP, we cataloged each of the variants classified by the VCEP as part of their pilot variant classification effort and the final classification of each pilot variant (P, LP, VUS, LB, or B). Next, we determined which pilot variant interpretations included PS3/BS3 evidence and the specific instances of assay cited as evidence utilizing information in the VCEP publication, as well as ClinVar (https://www.ncbi.nlm.nih.gov/clinvar/) and the ClinGen Evidence Repository (https://erepo.clinicalgenome.org/evrepo/). In addition to curating primary literature cited as evidence in pilot variant interpretation, we curated the primary literature and reviews the VCEPs cited in their publications in support of their approval or exclusion of a given assay (see the “Literature curation approach” section).

Inclusion and exclusion criteria

We focused our curation efforts on model organism and cellular in vivo and biochemical in vitro assays that determined the function of a gene product. We excluded assays that tested splicing, as these typically evaluated the transcript rather than the encoded protein function. We conducted a limited evaluation of assay instances using cells or tissue derived from affected individuals as the primary experimental material, as the variant in question was not isolated from the individual’s genetic background and, as a result, abnormal gene product function cannot be definitively attributed to the genetic variant.

Literature curation approach

We developed consistent criteria for evaluating classes of functional assays and specific instances of their use in evaluating the impact of a variant by establishing three main domains to describe a given assay. First, we curated the disease mechanism for a given gene-disease pair using the associated Monarch Disease Ontology (MONDO) identifier [10], the functional pathway using Gene Ontology (GO) terms [11,12,13], the molecular etiology using controlled vocabulary (e.g., loss-of-function, dominant negative, or gain-of-function), and the inheritance pattern, also using a controlled vocabulary. Next, we identified the general class of each assay used in the primary literature each VCEP cited using ontology terms from Bioassay Ontology (BAO; http://bioassayontology.org/) [14, 15] and Evidence and Conclusion Ontology (ECO; http://www.evidenceontology.org/) [16]. In some cases, ontologies describing the class of assay were found in only one of the two ontology databases. Finally, we used a structured narrative to describe the specific instance of an assay being performed. We summarized multiple attributes, including PubMed Identifier (PMID), study purpose, the entity performing the assay, methodology (including replicates, controls, thresholds, and validation measures), and assay results. We also cataloged other details specific to the assay, such as experimental material, quantitation measures, and statistical analyses.

Results

Each VCEP approved between one and seven assays for use as evidence for PS3/BS3 application (Table 1), all reflective of the disease mechanism but with widely varying specificity regarding the descriptions of approved assays. These ranged from detailed assays evaluating the myristoylation status of a single residue in a given protein (RASopathy VCEP) to a broader specification of any mammalian variant-specific knock-in model (Inherited Cardiomyopathy VCEP). Two VCEPs (Hearing Loss and PTEN) approved any sufficiently validated assays not explicitly approved in their recommendations if deemed appropriate by the analyst in future variant interpretation efforts. We also noted the variability in the inclusion of guidance for downgrading strength modifications to a moderate or supporting level. We next surveyed the parameters stipulated by each VCEP (Table 2). We also observed variation in the frequency and methods by which these parameters were specified, with most VCEPs detailing a need for one to two of these four parameters to be fulfilled by an individual instance of a functional assay.

Table 1 Overview of Variant Curation Expert Panel (VCEP) disease areas and mechanisms, functional assay classes, and accompanying strength level modifications
Table 2 Summary of PS3/BS3 assay parameter specifications and strength modification recommendations by each Variant Curation Expert Panel (VCEP)

The frequency of functional criteria application in pilot variant interpretation varied widely among VCEPs (Fig. 1a), with the PAH and RASopathy VCEPs using PS3/BS3 at the highest frequency in their pilot variant classification (31/85 variants and 36/103 variants, respectively) while the CDH1 and Inherited Cardiomyopathy VCEPs applied PS3/BS3 less commonly (4/49 variants and 4/60 variants, respectively). Variants that were ultimately classified as VUS rarely included PS3/BS3 evidence codes (Fig. 1b). We noted general agreement between the functional data criteria applied to pilot variants and the overall variant classification (Fig. 1c). Pilot variant interpretations that included PS3 criteria were frequently given an overall classification of LP or P, with very few classified as VUS and none classified as LB or B. Similarly, those that included BS3 criteria were often classified as LB or B, with one interesting exception of a variant with BS3 evidence ultimately classified as P. Given the variation observed in our broad analysis of parameter specification across VCEPs, we used consistent criteria to curate primary literature cited by each of the six VCEPs to assess their application of these parameters (see the “Methods” section).

Fig. 1
figure1

Use of the PS3/BS3 criteria in Variant Curation Expert Panel (VCEP) pilot variant classification. a Comparison of PS3 criterion application at any strength level (purple) and BS3 criterion application at any strength level (orange) in the pilot variant classification of each VCEP. b Comparison of PS3 criterion application at any strength level (purple) and BS3 criterion application at any strength level (orange) to variants ultimately classified as variants of uncertain significance (VUS) in the pilot variant classification of each VCEP. c Comparison of the final classification (P, LP, VUS, LB, or B) of pilot variants with PS3/BS3 criteria (at any strength level). The CDH1 VCEP and the Inherited Cardiomyopathy-MYH7 VCEP did not use BS3 evidence in the interpretation of any pilot variants

CDH1 VCEP

The CDH1 VCEP sets guidelines for functional studies of the E-cadherin protein encoded by CDH1 (Table 1) [4]. Loss-of-function variants in the CDH1 gene have been associated with hereditary diffuse gastric cancer through a loss of cell adhesion and an increase in cell motility [17, 18]. In vitro studies commonly test CDH1 variants for retention of 2 main functions: cell-cell adhesion and invasion suppression, through aggregation assays or collagen invasion assays, respectively (see Additional file 1 for a detailed discussion of biological context of assays). The CDH1 VCEP evaluated 49 variants in their pilot study and assigned the PS3 criterion to 4 [4]. The only approved assays were those that measure abnormal splicing of the CDH1 gene, as this measures one of the main disease mechanisms. For the purpose of this analysis, we only assessed functional studies that evaluate the effect of CDH1 variants on protein function and not those assessing splicing variation (see the “Methods” and “Discussion” sections).

This VCEP also reviewed literature studying the effect of missense variants [19] and identified 14 variants with 2 or more published “abnormal” functional assay results, 6 of which were included in the pilot set. However, this VCEP ultimately decided these assays (aggregation/invasion, wound closure, and proximity ligation) were not sufficient predictors of pathogenicity, in part because none of the 14 variants was found in a large database of CDH1 variants from individuals with disease [4]. To better understand why the VCEP deemed these assays poor predictors of pathogenicity for missense variants, we evaluated each functional assay the VCEP considered (Fig. 2; see also Additional file 2: Tables S1 and S2). We then compared the findings from these functional studies to assertions in ClinVar, both from the VCEP and other clinical labs (Additional file 2: Table S3), to examine if the functional assays that tested CDH1 missense variants could predict pathogenicity. While most clinical lab entries in ClinVar did not specify which rule codes they used in their interpretation, many commented on the functional data. Only 1 of the 14 variants analyzed had a likely pathogenic assertion, while the remaining variants were classified as benign (5), VUS (8), conflicting (1), or not listed in ClinVar (3). We also noted that while each assay instance incorporated wild-type and mock controls, no known pathogenic or benign controls were used to validate the assays. This limited validation coupled with an absence of identified definitively pathogenic missense variants makes it difficult to determine the positive predictive value of these assays and likely contributed to the VCEP not approving any existing functional studies of missense variants.

Fig. 2
figure2

Representative findings from the curation of specific instances of CDH1 functional assays. We assessed the following methods and validation parameters of aggregation/invasion, wound closure, and proximity ligation assays: experimental material, controls, replication, and output. *Assays not approved by the Variant Curation Expert Panel (VCEP), see Additional file 2: Tables S1 and S2 for the full primary literature curation results

Hearing Loss VCEP

The Hearing Loss (HL) VCEP developed disease-specific recommendations for variant interpretation in the context of nine genes commonly associated with hearing loss: CDH23, COCH, GJB2, KCNQ4, MYO6, MYO7A, SLC26A4, TECTA, and USH2A (Table 1; see also Additional file 1) [5]. A single assay type, a variant-specific knock-in mouse model recapitulating the hearing loss phenotype, was the only functional study approved for PS3 application at the strong level. Additional guidelines for PS3/BS3 application at reduced strength levels were given for three genes: COCH, GJB2, and SLC26A4. Given the heterogeneity in disease mechanism underlying the multiple types of hearing loss, each gene was associated with a unique set of approved functional assays (Table 1; see also Additional file 1 for a detailed discussion of biological context of assays). The HL VCEP calculated the positive and negative predictive values of functional assays commonly used to assess variants in these three genes (COCH, GJB2, SLC26A4) by comparing published assay results with ClinVar classifications [5]. For a P or LP ClinVar variant, an “abnormal” assay result compared to wild type was considered a true positive, while an assay result similar to wild type was considered a false negative. Similarly, for a variant classified as B or LB in ClinVar, a wild-type-like assay result was considered a true negative, while an “abnormal” result was considered a false positive.

In the VCEP pilot variant classification of 41 variants, PS3 (at any strength level) was applied to 8 variants and BS3 (at a supporting strength level) was applied to 2 variants. The VCEP did not cite any mouse models in their final variant curations, despite previous reports of mouse models generated for 2 pilot variants (GJB2 c.109G>A and SLC26A4 c.919-2A>G) [20, 21]. Assays testing transport capability (electrical coupling, dye transfer, anion transport) were the most commonly used functional evidence (applied as PS3/BS3 assertions at reduced strength for 8 variants). We assessed each of the 31 specific instances of these assays cited by the VCEP, in which some variants were evaluated more than once, to determine how often the parameters defined by the VCEP were satisfied (Fig. 3; see also Additional file 2: Tables S4 and S5). While all instances [22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48] tested a wild-type control, water-injected or nontransfected controls were less consistently used (24/31), despite the VCEP’s stated requirement. Statistical testing was included in 17/31 cited instances of assay. In particular, no statistical analysis was done for the dye transfer assays, possibly because the results of this test are qualitative rather than quantitative. Finally, the HL VCEP applied PS3_supporting to a variant in an additional gene not given assay-specific recommendations, KCNQ4 c.853G>A p.(Gly285Ser). Two instances of an electrical coupling assay [49, 50] showing little to no electrical current in cells expressing KCNQ4 p.Gly285Ser were used as evidence for PS3_supporting. Although no specific guidance was given for KCNQ4 variant interpretation, functional assays with limited validation were generally approved by the VCEP at the PS3_supporting level for all hearing loss-associated genes.

Fig. 3
figure3

Representative findings from the curation of specific instances of GJB2 functional assays. We assessed the following methods and validation parameters of electrical coupling and dye transfer assays: experimental material, controls, replication, and output. Specific instance of assay cited by the Variant Curation Expert Panel (VCEP) as evidence for the PS3/BS3 criteria in pilot variant classification, see Additional file 2: Tables S4 and S5 for the full primary literature curation results

A variant in the pilot set, SLC26A4 c.349C>T p.(Leu117Phe), was assigned BS3_supporting, but given an overall pathogenic classification (and, post-publication, downgraded to likely pathogenic in the ClinGen Evidence Repository). Although a radioactive anion isotope transport assay [46] found no statistically significant difference in the rate of iodide efflux in cells expressing the SLC26A4 variant, the VCEP reasoned that the functional assay may not assess all ion transport functions of the protein or fully reflect the biological environment, and noted that the benign functional evidence at a supporting level was not considered to be in conflict with other pathogenic evidence leading to the final classification. While the VCEP did not give specific recommendations for handling conflicting criteria, this case suggests that functional assays, even if VCEP-approved, are limited in their ability to test all functions of a protein. Functional evidence, especially evidence supporting BS3 criteria, must be weighed with other types of evidence in making an overall classification determination (see the “Discussion” section).

Inherited Cardiomyopathy-MYH7 VCEP

The Inherited Cardiomyopathy VCEP published recommendations for interpretation of variants in MYH7, encoding α (alpha) cardiac myosin heavy chain, a gene associated with multiple forms of cardiomyopathy (dilated, hypertrophic, and restrictive) (Table 1) [6]. The expert panel reviewed published functional evidence for their 60 pilot variants to determine which assays qualified for PS3/BS3 evidence (see Additional file 1 for a detailed discussion of the biological context of assays). After evaluating in vivo and in vitro functional evidence for 23 of these variants, they approved only in vivo mammalian, variant-specific knock-in models to serve at the strong level and applied this evidence to 4 variants. Given the poor predictive value of the 16 in vitro assays evaluated in their review, no in vitro assays were approved at any strength level and were not cited as evidence for any pilot variants.

When assessing the various functional assays this expert panel reviewed but ultimately did not approve, we noted that the MYH7 c.1208G>A p.(Arg403Gln) variant was tested in many of the functional studies. We used this variant to compare the characteristics of the assays that this VCEP did approve for use as evidence of PS3/BS3 (knock-in mouse model) to those that were not approved (in vitro motility assay and ATPase assay) (Fig. 4; see also Additional file 2: Tables S6 and S7). The first knock-in mouse model of hypertrophic cardiomyopathy introduced the c.1208G>A p.(Arg403Gln) variant into the endogenous murine Myh7 [51]. The mice had a heart phenotype similar to hypertrophic cardiomyopathy that was recapitulated in multiple instances, which reported defective myocyte function and development of cardiac hypertrophy and lethal cardiomyopathy [36, 52,53,54,55,56,57] in mice bearing an p.Arg403Gln Myh7 variant. The VCEP considered this strong evidence for pathogenicity. We also reviewed two classes of in vitro functional assay commonly used to assess the effect of MYH7 c.1208G>A p.(Arg403Gln), but not approved by the VCEP: the in vitro motility assay and the ATPase assay. The in vitro motility assay measures the velocity of actin filament sliding on a surface coated with myosin, a motion required for normal muscle contraction in vivo [58, 59], while the ATPase assay measures the enzymatic function of ATP exchange required for force generation [60, 61]. We examined several instances of each assay type [57, 62,63,64,65,66,67,68,69,70] and noted heterogeneity in the source of myosin used, as well as a general lack of controls with known effect (other than wild type) for comparison to the variant myosin [57, 62,63,64,65,66,67,68,69,70]. Furthermore, distinct instances of this assay examining the c.1208G>A p.(Arg403Gln) MYH7 variant yielded conflicting results, with some studies finding increased actin filament velocity [57, 65,66,67,68,69] or ATPase activity [57, 66, 67] and others reporting decreased actin filament velocity [62,63,64, 70] or ATPase activity [64, 68, 70]. Poor reproducibility of the motility assay has been previously reported [71] and is thought to arise, at least partially, due to technical complications in myosin isolation. Ultimately, this case study demonstrates that poor reproducibility across instances of an assay class complicates the interpretation of the results in aggregate and no evidence from this assay class was approved for application.

Fig. 4
figure4

Representative findings from the curation of specific instances of MYH7 functional assays. We assessed the following methods and validation parameters of variant-specific knock-in mouse models, ATPase activity, and motility assays: experimental material, controls, replication, and output. *Assays not approved by the Variant Curation Expert Panel (VCEP). Specific instance of assay cited by the VCEP as evidence for the PS3/BS3 criteria in pilot variant classification, see Additional file 2: Tables S6 and S7 for the primary literature curation results

PAH VCEP

The PAH VCEP published functional study guidelines for variants in the phenylalanine hydroxylase (PAH) gene associated with phenylketonuria (PKU) (Table 1) [7]. They reviewed the existing literature for functional studies and approved a well-established in vitro PAH enzyme activity assay involving the expression of the variant allele in cultured cells and measurement of the variant enzyme activity in comparison with wild-type activity. In vitro PAH enzyme activity correlates with the severity of the PKU phenotype (see Additional file 1 for a detailed discussion of the biological context of assays) [72]. A threshold of 0–50% residual enzyme activity compared to wild type was recommended for evidence of abnormal activity sufficient for PS3 application [7]. The VCEP assessed 85 variants in their pilot study and assigned PS3 to 29 variants with residual PAH activity values of ≤ 50% compared to wild type. The VCEP did not recommend the use of other assays described in the primary literature measuring PAH expression or protein folding, aggregation, or stability [73].

To assess the use of the enzyme activity assay by the VCEP, we evaluated specific instances of assays measuring PAH activity cited as evidence in their pilot variant classification. We noted several discrepancies in assay methodology among the different research groups (Fig. 5; see also Additional file 2: Tables S8 and S9). In most instances, PAH variants were expressed in COS monkey kidney cells and enzyme activity was measured in cell extracts [73,74,75,76,77,78,79], though some expressed the variant in Escherichia coli and measured enzyme activity of the purified protein [73, 80, 81]. In some instances, a synthetic cofactor 6-MPH4 [73, 74, 76, 79, 82] was used in place of the natural cofactor BH4 [76,77,78, 80, 81, 83]. The method for measuring the conversion of phenylalanine to tyrosine also differed among experiments, with early researchers using paper chromatography or thin-layer chromatography (TLC), visualizing the results with autoradiography, and quantifying via a liquid scintillation counter [73, 75, 79, 83]. As technology advanced, experiments used high-performance liquid chromatography (HPLC) with fluorometric detection [80, 84] or the more sophisticated method of liquid chromatography measured with electrospray ionization tandem mass spectrometer [78].

Fig. 5
figure5

Representative findings from the curation of specific instances of PAH functional assays. We assessed the following methods and validation parameters of enzyme activity and protein folding/stability assays: experimental material, cofactor, detection method, controls, replication, and output. *Assays not approved by the Variant Curation Expert Panel (VCEP). Specific instance of assay cited by the VCEP as evidence for the PS3/BS3 criteria in pilot variant classification. TLC, thin-layer chromatography; HPLC-Fluorescence, high-performance liquid chromatography coupled with fluorescence detection; LC-MS, liquid chromatography coupled with mass spectrometry, see Additional file 2: Tables S8 and S9 for the full primary literature curation results

Given the differences in assay methodologies in instances of the PAH enzyme activity assay, we concluded that PAH activity results may vary among different instances of the assay analyzing the same variant. To test this, we compared enzyme activity results from multiple sources to the final enzyme activity cited by the VCEP as evidence for the interpretation of a given variant. One such source, a meta-analysis of in vitro PAH enzyme activity assays of 87 PAH variants from 49 publications also noted different methodologies for measuring PAH activity, including different cell expression systems, cofactors, temperatures, reaction times, measurement methods, and variation in the final assay result [72]. Of the 29 variants assessed in the VCEP pilot study and ultimately assigned PS3, 18 had more than 1 result reported in the sources we reviewed (Table 3). We observed consistency in the activity levels of more severe variants that were often identified in individuals with classical PKU, but the milder variants had a wide range of reported enzyme activity levels. Of note, 9 variants had at least 1 report of enzyme activity ≥ 50% of wild type, which would exceed the VCEP-established cutoff and conflict with the VCEP-cited evidence in support of a PS3 assertion. The variation in enzyme activity levels may be dependent on experimental design; thus, further guidance is needed on the most appropriate method to measure the activity level and/or on how to resolve the conflicting results.

Table 3 Comparison of PAH enzyme activities reported in a meta-analysis publication Himmelreich et al. [72], the PAH locus-specific database (PAHvdb), and the PAH Variant Curation Expert Panel (VCEP) variant evidence for PS3 criteria

PTEN VCEP

The PTEN VCEP outlined specific recommendations for seven accepted general classes of functional assays testing the effect of variants in this gene associated with hereditary cancer (Table 1; see also Additional file 1 for a detailed discussion of the biological context of assays) [8]. In the VCEP pilot variant classification of 36 PTEN variants, PS3 was applied to 4 variants, PS3_supporting was applied to 1 variant, and BS3_supporting was applied to 1 variant. Phosphatase activity was the most commonly used assay (3 of 4 variants assigned PS3), with a single study [85] testing the ability of purified proteins to dephosphorylate PIP3 in vitro used to support the PS3 assertion for all 3 variants. Each of these variants displayed > 90% reduction in phosphatase activity, well below the VCEP-approved threshold of ≥ 50% reduction in protein activity compared to wild-type PTEN, and were replicated in 3 independent experiments, but the VCEP-specified catalytically inactive control was not included (Fig. 6; see also Additional file 2: Tables S10 and S11). Two variants were classified PS3 or BS3 based on splicing assays [86, 87], which we did not evaluate (see the “Methods” section). The final variant in the pilot set was assigned PS3_supporting based on altered protein localization [88].

Fig. 6
figure6

Representative findings from the curation of specific instances of PTEN functional assays. We assessed the following methods and validation parameters of phosphatase activity, PTEN level, pAKT level, protein localization, cell proliferation, and cell migration assays: experimental material, controls, replication, and output. Specific instance of assay cited by the Variant Curation Expert Panel (VCEP) as evidence for the PS3/BS3 criteria in pilot variant classification, see Additional file 2: Tables S10 and S11 for the full primary literature curation results

The VCEP cited a study that assessed PTEN protein levels in lymphoblast cell lines derived directly from individuals with Cowden syndrome [89]. Further guidance on the use of tissues and cell lines derived from affected individuals in functional assays is needed to inform the application of this type of evidence (see the “Methods” and “Discussion” sections). Additional guidance may also be necessary to interpret transgenic model organism evidence. Although the VCEP specified that this class of functional assay could be used in support of PS3_supporting, no studies of transgenic model organisms were used by the VCEP in pilot variant classification and the exact phenotypes required to use this type of evidence are unclear. Also of note was a high-throughput assay cited by the VCEP, but not used in pilot variant interpretation. This study used saturation mutagenesis to assess the pathogenicity of over 8000 PTEN variants, nearly all possible missense variants, by expressing PTEN in yeast cells and using cell growth rate as a readout for phosphatase activity [90]. The same three variants in the pilot set found to have reduced phosphatase activity in an in vitro phosphatase activity assay [85] were also tested in this high-throughput assay [90]. Fitness scores of all three of these variants were lower than that observed for wild-type or “wild-type-like” variants, suggesting agreement of this approach with small-scale in vitro assays.

RASopathy VCEP

The RASopathy VCEP published recommendations for PS3/BS3 application in interpretation of variants in nine genes linked to RASopathy conditions: BRAF, HRAS, KRAS, MAP 2K1, MAP 2K2, PTPN11, RAF1, SHOC2, and SOS1 (Table 1; see also Additional file 1 for a detailed discussion of the biological context of assays) [9]. Assays measuring MAP 2K1/2 and ERK1/2 phosphorylation [91,92,93,94,95,96,97,98,99,100,101,102,103,104] were the most commonly cited functional evidence in pilot variant classification (24/36 variants). The VCEP indicated that MAP 2K1/2 and ERK1/2 activation should be measured both basally and following receptor tyrosine kinase stimulation, typically via epidermal or fibroblast growth factor addition (EGF and FGF). We noted disparities of assay instances with respect to whether measurements were taken in serum-starved cells, stimulated cells, or both, and in the method of stimulation (serum addition vs. purified EGF or FGF addition) (Fig. 7; see also Additional file 2: Tables S12 and S13). Direct quantification was not required, but was completed in many instances, as were statistical analyses.

Fig. 7
figure7

Representative findings from the curation of specific instances of RASopathy functional assays. We assessed the following methods and validation parameters of MAP 2K1/2 and ERK1/2 phosphorylation assays: cell treatment, controls, replication, and output. Specific instance of assay cited by the Variant Curation Expert Panel (VCEP) as evidence for the PS3/BS3 criteria in pilot variant classification. Specific instance of assay cited by the VCEP as evidence for the PS3/BS3 criteria in pilot variant classification for variants in multiple genes: PMID 18413255 cited as evidence for the PS3/BS3 criteria for variants in BRAF and MAP 2K1/2; PMID 16439621 cited as evidence for PS3/BS3 for variants in KRAS and MAP 2 K1/2; and PMID 21784453 cited as evidence for PS3/BS3 for variants in RAF1 and SOS1, see Additional file 2: Tables S12 and S13 for the full primary literature curation results

Gain-of-function BRAF variants leading to an increase in kinase activity are most commonly associated with cardio-facio-cutaneous syndrome; however, gain-of-function variants that result in reduced kinase activity and impaired stimulation of MAP 2K1/2 and ERK1/2 phosphorylation have also been observed [101, 105]. Although the VCEP specified that increased kinase activity could be used in support of PS3, no guidelines were given for variants that result in kinase impairment, despite their application of PS3 to variants with reduced BRAF kinase activity (e.g., BRAF c.1787G>T p.Gly596Val). Similarly, PS3 was applied to a PTPN11 variant (c.1403C>T p.Thr468Met) with impaired phosphatase activity measured in different instances of the same assay type, but the VCEP only gave recommendations for variants that resulted in increased phosphatase activity. In two instances, the VCEP cited an ELK transactivation assay as evidence for PS3 application. While this assay was not explicitly approved by the VCEP, it appears to reflect the disease mechanism, as it measures the ability of BRAF to activate downstream transcription of ELK transcription factor.

Discussion

Our review of disease- and gene-specific functional assay evidence recommendations by six VCEPs highlighted a general uniformity across VCEPs in the approval of assays reflective of disease mechanism and, in some cases, the explicit exclusion of assays deemed to be poor predictors of variant pathogenicity. This underscores the requirement of a well-characterized disease mechanism to inform the development and application of functional assays for variant interpretation. However, our efforts also identified the major differences among VCEPs in parameter specification and evidence capturing, suggesting the need for a baseline guide for functional evidence assessment and consistent criteria for capturing functional evidence. Along with the evidence curation criteria described in this study, standardized ClinGen criteria for functional assay evaluation should be developed to ensure consistency among VCEPs. The use of standard operating procedures for curating functional assay evidence could also improve transparency by encouraging complete recording of evidence used in variant classification, including documenting any conflicting evidence and whether a given piece of functional evidence for a variant was considered but not deemed appropriate, vs. not evaluated at all.

We noted five recurring points of interest that will likely require further elucidation by the ClinGen Sequence Variant Interpretation Working Group to streamline functional evidence interpretation: (1) methodology for estimating the predictive power of assays, (2) consideration of splicing assays within PS3/BS3 criteria, (3) use of functional data from experimental materials derived from affected individuals, (4) unclear recommendations for creation and interpretation of model organism evidence, and (5) limited guidance for conflicting evidence.

Two VCEPs, Hearing Loss (HL) and CDH1, detailed their approach for estimating the predictive power of assays to determine which assays should be approved for use as PS3/BS3 evidence. The HL VCEP calculated the positive and negative predictive value of functional assays commonly used to assess variants in three genes (COCH, GJB2, SLC26A4) by comparing published assay results with ClinVar classifications [5], while the CDH1 VCEP compared published assay results with data from affected individuals (see the “CDH1 VCEP” and “Hearing Loss VCEP” sections) [4]. Importantly, these estimations were limited by the number of variants assessed, with 10 to 23 variants analyzed per hearing loss-related assay or by its reliance on previous identification of the variant in populations of affected individuals. Furthermore, the HL VCEP used aggregated results of multiple specific instances of a general class of assay, rather than assessing each instance and its validation parameters independently. In our view, the predictive value of a functional assay is most reliably determined using variants of known pathogenic or known benign interpretation (interpreted as such without using functional evidence) in the same instance of the assay, rather than attempting a post hoc calculation across different instances of the same assay. Clearly, additional guidance on appropriate methods for estimating the predictive power of assays is needed.

For the purpose of this analysis, we defined functional assays as systematic experiments (either in vitro or in vivo) used to elucidate the function of a protein in a cellular pathway or biological process [106]. With this in mind, we did not curate splicing assay evidence, despite splicing assessment being explicitly approved by the CDH1, PAH, and PTEN VCEPs and implicitly approved by the HL VCEP (via the use of splicing evidence in the interpretation of a pilot variant). While these assays can provide evidence of abnormal splicing and confirm the results from in silico predictors, it does not directly test the function of the protein, and as a result, we suggest that splicing evidence represents a distinct type of evidence that may require separate interpretation recommendations.

We also observed relatively frequent citation of functional studies using cells or tissue derived from affected individuals in the primary literature used as evidence for PS3/BS3 criteria. It is important to consider both the utility and the limitations of these experimental materials. While analyzing the expression of the expected mRNA in the affected individual can be a useful way to identify unexpected effects of a variant on the RNA abundance or splicing, assessment of protein function using samples derived from affected individuals can be difficult to interpret. As reasoned by Strande et al. [107], studies conducted using tissue or cells from an affected individual can provide high-level information about the clinical phenotype (biochemical or enzymatic dysfunction), but not the variant-level effect, as the variant being tested cannot be isolated from other variants present in the individual’s genome. In general, this evidence may be better suited as evidence for the application of PP4 (supporting evidence of variant pathogenicity based on the individual’s specific phenotype as it relates to a disease).

Knock-in animal models were approved in some capacity by four of the six VCEPs; however, we noted a lack of guidance in their recommendations for model creation and interpretation. Some VCEPs gave no specifications for the number or type of different strains that should be used, the number of individual organisms that should be analyzed, or the features the animal must display to sufficiently recapitulate the disease phenotype. It was also unclear if studies using cells or tissue derived from a model organism for in vitro experiments should be considered model organism evidence. For example, although the Inherited Cardiomyopathy VCEP only approved mammalian variant-specific knock-in models, we noted at least one study used as pilot variant classification evidence that created a mouse model but did not directly assess the phenotype of the organism [69]. Instead, this study used myosin derived from these mice for in vitro assays of myosin motility and ATPase activity. Additional guidance on the interpretation of model organisms is necessary to reduce variability in evidence interpretation and application.

Our primary literature curation uncovered multiple instances of conflicting functional evidence for a single variant, yet only the HL VCEP provided guidance on the interpretation of conflicting evidence from functional studies, suggesting “no criteria should be applied if multiple assay results do not agree” [5]. A striking example of the prevalence of conflicting data can be seen in different instances of the PAH VCEP-approved enzyme activity assay (Table 3). Variation in the study design likely contributed to the wide range of activity levels observed for the same variant; however, this was not addressed by the VCEP, and it is unclear how the VCEP selected which activity level to ultimately use as evidence for the PS3 criteria. This highlights the importance of not solely approving all assays of a given class, but rather evaluating the specific result of an assay in the context of that assay’s validation. In addition to conflicts between functional evidence, we also noted a need for guidance surrounding cases of functional evidence that conflicts with other types of evidence gathered for a variant. The ACMG/AMP guidelines suggest that a variant with conflicting evidence should be classified as a VUS [1]. A Bayesian reinterpretation of the ACMG/AMP guidelines specified a method for weighting combinations of pathogenic and benign evidence [108] that may help to solve this problem. For example, the Hearing Loss VCEP classified the SLC26A4 variant c.349C>T as pathogenic (and later downgraded the variant to likely pathogenic), despite also applying BS3_supporting criteria to the variant. In doing so, the VCEP acknowledged that the anion isotope transport assay cited as evidence may not assess all aspects of protein function. Although not explicitly stated, we infer that the VCEP did not want a “supporting” piece of evidence to call into question the overall interpretation, when other lines of evidence more strongly suggested a pathogenic interpretation. This example also raises the question of how functional assays that examine only one aspect of protein function should be interpreted when a variant demonstrates a normal result and whether some form of combinatorial evidence from more than one class of assay should be required to support a benign interpretation.

Finally, through our curation of primary literature cited by the VCEPs, we observed that studies used as evidence for PS3/BS3 often did not satisfy all of the VCEP-recommended parameters (Figs. 2, 3, 4, 5, 6, and 7). Understandably, many functional assays have been performed in basic science laboratories for the purpose of understanding the gene and not to provide clinically validated evidence of a pathogenic or benign classification. That being said, it is critical for VCEPs and others evaluating variants to approach this data critically and conservatively. While we suggest the development and implementation of criteria that sets baseline quality requirements, we also believe this finding demonstrates a need for the ACMG/AMP guidelines and VCEP recommendations surrounding PS3/BS3 criteria to be conveyed to research laboratories for incorporation into the study design of future research assays. The inclusion of pathogenic and benign controls, assay replication, and statistical analyses, among other practices, have the power to improve the clinical utility of studies conducted in research labs by aiding in clinical variant interpretation.

Conclusions

In summary, our comparative analysis identified both commonalties and discrepancies among the functional assay evidence evaluation recommendations made by six ClinGen VCEPs. We observed multiples areas of discordance that warrant additional guidance, including setting a standard for basic validation parameters that should be fulfilled by functional studies, establishing if assays using experimental material derived from affected individuals are appropriate for PS3/BS3 evidence, and determining how conflicting evidence should be assessed. Although VCEP recommendations are an indispensable tool for the interpretation of functional evidence in a given disease area, more formal methods for assessing the clinical validation of functional assays are needed to take full advantage of this evidence type for the classification of variants [109]. While ClinGen aims to establish VCEPs for many Mendelian disorders, this general guidance will be particularly important for interpreting functional assays of variants in genes and disease areas not yet encompassed by a VCEP.

Availability of data and materials

All data generated or analyzed during this study supporting the conclusions of the article are included in this published article and its supplementary information files.

Abbreviations

ACMG:

American College of Medical Genetics and Genomics

AMP:

Association for Molecular Pathology

ATP:

Adenosine triphosphate

BH4 :

Tetrahydrobiopterin

DFNA9:

Autosomal dominant nonsyndromic deafness 9

DFNB1:

Autosomal recessive nonsyndromic deafness 1

DFNB3:

Autosomal dominant nonsyndromic deafness

DFNB4:

Autosomal recessive nonsyndromic deafness 4

BAO:

Bioassay Ontology

B:

Benign

ClinGen:

Clinical Genome Resource

ECO:

Evidence and Conclusion Ontology

EGF:

Epidermal fibroblast growth

FGF:

Fibroblast growth factor

GO:

Gene Ontology

HL:

Hearing Loss

HPLC:

High-performance liquid chromatography

LB:

Likely benign

LP:

Likely pathogenic

MONDO:

Monarch Disease Ontology

P:

Pathogenic

pAKT:

Phosphorylated AKT

PKU:

Phenylketonuria

PMID:

PubMed identifier

TLC:

Thin-layer chromatography

VCEP:

Variant Curation Expert Panel

VUS:

Variant of uncertain significance

References

  1. 1.

    Richards S, Aziz N, Bale S, Bick D, Das S, Gastier-Foster J, et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med. 2015;17(5):405–23.

  2. 2.

    Brnich SE, Rivera-Muñoz EA, Berg JS. Quantifying the potential of functional evidence to reclassify variants of uncertain significance in the categorical and Bayesian interpretation frameworks. Hum Mutat. 2018;39(11):1531–41.

  3. 3.

    Rivera-Muñoz EA, Milko LV, Harrison SM, Azzariti DR, Kurtz CL, Lee K, et al. ClinGen Variant Curation Expert Panel experiences and standardized processes for disease and gene-level specification of the ACMG/AMP guidelines for sequence variant interpretation. Hum Mutat. 2018;39(11):1614–22.

  4. 4.

    Lee K, Krempely K, Roberts ME, Anderson MJ, Carneiro F, Chao E, et al. Specifications of the ACMG/AMP variant curation guidelines for the analysis of germline CDH1 sequence variants. Hum Mutat. 2018;39(11):1553–68.

  5. 5.

    Oza AM, DiStefano MT, Hemphill SE, Cushman BJ, Grant AR, Siegert RK, et al. Expert specification of the ACMG/AMP variant interpretation guidelines for genetic hearing loss. Hum Mutat. 2018;39(11):1593–613.

  6. 6.

    Kelly MA, Caleshu C, Morales A, Buchan J, Wolf Z, Harrison SM, et al. Adaptation and validation of the ACMG/AMP variant classification framework for MYH7-associated inherited cardiomyopathies: recommendations by ClinGen’s Inherited Cardiomyopathy Expert Panel. Genet Med. 2018;20(3):351–9.

  7. 7.

    Zastrow DB, Baudet H, Shen W, Thomas A, Si Y, Weaver MA, et al. Unique aspects of sequence variant interpretation for inborn errors of metabolism (IEM): the ClinGen IEM Working Group and the Phenylalanine Hydroxylase Gene. Hum Mutat. 2018;39(11):1569–80.

  8. 8.

    Mester JL, Ghosh R, Pesaran T, Huether R, Karam R, Hruska KS, et al. Gene-specific criteria for PTEN variant curation: recommendations from the ClinGen PTEN Expert panel. Hum Mutat. 2018;39(11):1581–92.

  9. 9.

    Gelb BD, Cavé H, Dillon MW, Gripp KW, Lee JA, Mason-Suares H, et al. ClinGen’s RASopathy Expert Panel consensus methods for variant interpretation. Genet Med. 2018;20(11):1334–45.

  10. 10.

    Mungall CJ, McMurry JA, Köhler S, Balhoff JP, Borromeo C, Brush M, et al. The Monarch Initiative: an integrative data and analytic platform connecting phenotypes to genotypes across species. Nucleic Acids Res. 2017;45(D1):D712–22.

  11. 11.

    Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, et al. Gene Ontology: tool for the unification of biology. Nat Genet. 2000;25(1):25–9.

  12. 12.

    The Gene Ontology Consortium. The Gene Ontology Resource: 20 years and still GOing strong. Nucleic Acids Res. 2019;47(D1):D330–8.

  13. 13.

    Carbon S, Ireland A, Mungall CJ, Shu S, Marshall B, Lewis S, et al. AmiGO: online access to ontology and annotation data. Bioinformatics. 2009;25(2):288–9.

  14. 14.

    Abeyruwan S, Vempati UD, Küçük-McGinty H, Visser U, Koleti A, Mir A, et al. Evolving BioAssay Ontology (BAO): modularization, integration and applications. J Biomed Semantics. 2014;5(Suppl 1):S5.

  15. 15.

    Vempati UD, Przydzial MJ, Chung C, Abeyruwan S, Mir A, Sakurai K, et al. Formalization, annotation and analysis of diverse drug and probe screening assay datasets using the BioAssay Ontology (BAO). PLoS One. 2012;7(11):e49198 Cox D, editor.

  16. 16.

    Giglio M, Tauber R, Nadendla S, Munro J, Olley D, Ball S, et al. ECO, the Evidence & Conclusion Ontology: community standard for evidence information. Nucleic Acids Res. 2019;47(D1):D1186–94.

  17. 17.

    Green KJ, Getsios S, Troyanovsky S, Godsel LM. Intercellular junction assembly, dynamics, and homeostasis. Cold Spring Harb Perspect Biol. 2010;2(2):a000125.

  18. 18.

    Gabbert H, Wagner R, Moll R, Gerharz CD. Tumor dedifferentiation: an important step in tumor invasion. Clin Exp Metastasis. 1985;3(4):257–79.

  19. 19.

    Suriano G, Oliveira C, Ferreira P, Machado JC, Bordin MC, De Wever O, et al. Identification of CDH1 germline missense mutations associated with functional inactivation of the E-cadherin protein in young gastric cancer probands. Hum Mol Genet. 2003;12(5):575–82.

  20. 20.

    Chen Y, Hu L, Wang X, Sun C, Lin X, Li L, et al. Characterization of a knock-in mouse model of the homozygous p.V37I variant in Gjb2. Sci Rep. 2016;6(1):33279.

  21. 21.

    Lu Y-C, Wu C-C, Shen W-S, Yang T-H, Yeh T-H, Chen P-J, et al. Establishment of a knock-in mouse model with the SLC26A4 c.919-2A>G mutation and characterization of its pathology. PLoS One. 2011;6(7):e22150 Sham MH, editor.

  22. 22.

    Ambrosi C, Walker AE, DePriest AD, Cone AC, Lu C, Badger J, et al. Analysis of trafficking, stability and function of human connexin 26 gap junction channels with deafness-causing mutations in the fourth transmembrane helix. PLoS One. 2013;8(8):e70916 Koval M, editor.

  23. 23.

    Bicego M, Beltramello M, Melchionda S, Carella M, Piazza V, Zelante L, et al. Pathogenetic role of the deafness-related M34T mutation of Cx26. Hum Mol Genet. 2006;15(17):2569–87.

  24. 24.

    Bizhanova A, Chew T-L, Khuon S, Kopp P. Analysis of cellular localization and function of carboxy-terminal mutants of pendrin. Cell Physiol Biochem. 2011;28(3):423–34.

  25. 25.

    Bruzzone R, Veronesi V, Gomès D, Bicego M, Duval N, Marlin S, et al. Loss-of-function and residual channel activity of connexin26 mutations associated with non-syndromic deafness. FEBS Lett. 2003;533(1–3):79–88.

  26. 26.

    Choi BY, Stewart AK, Madeo AC, Pryor SP, Lenhard S, Kittles R, et al. Hypo-functional SLC26A4 variants associated with nonsyndromic hearing loss and enlargement of the vestibular aqueduct: genotype-phenotype correlation or coincidental polymorphisms? Hum Mutat. 2009;30(4):599–608.

  27. 27.

    Choung YH, Moon S-K, Park H-J. Functional study of GJB2 in hereditary hearing loss. Laryngoscope. 2002;112(9):1667–71.

  28. 28.

    Cirello V, Bazzini C, Vezzoli V, Muzza M, Rodighiero S, Castorina P, et al. Molecular and functional studies of 4 candidate loci in Pendred syndrome and nonsyndromic hearing loss. Mol Cell Endocrinol. 2012;351(2):342–50.

  29. 29.

    D’Andrea P, Veronesi V, Bicego M, Melchionda S, Zelante L, Di Iorio E, et al. Hearing loss: frequency and functional studies of the most common connexin26 alleles. Biochem Biophys Res Commun. 2002;296(3):685–91.

  30. 30.

    Dossena S, Bizhanova A, Nofziger C, Bernardinelli E, Ramsauer J, Kopp P, et al. Identification of allelic variants of pendrin (SLC26A4) with loss and gain of function. Cell Physiol Biochem. 2011;28(3):467–76.

  31. 31.

    Dossena S, Nofziger C, Brownstein Z, Kanaan M, Avraham KB, Paulmichl M. Functional characterization of pendrin mutations found in the Israeli and Palestinian populations. Cell Physiol Biochem. 2011;28(3):477–84.

  32. 32.

    Dossena S, Rodighiero S, Vezzoli V, Bazzini C, Sironi C, Meyer G, et al. Fast fluorometric method for measuring pendrin (SLC26A4) Cl-/I- transport activity. Cell Physiol Biochem. 2006;18(1–3):67–74.

  33. 33.

    Dossena S, Vezzoli V, Cerutti N, Bazzini C, Tosco M, Sironi C, et al. Functional characterization of wild-type and a mutated form of SLC26A4 identified in a patient with Pendred syndrome. Cell Physiol Biochem. 2006;17(5–6):245–56.

  34. 34.

    Fugazzola L, Cirello V, Dossena S, Rodighiero S, Muzza M, Castorina P, et al. High phenotypic intrafamilial variability in patients with Pendred syndrome and a novel duplication in the SLC26A4 gene: clinical characterization and functional studies of the mutated SLC26A4 protein. Eur J Endocrinol. 2007;157(3):331–8.

  35. 35.

    Gillam MP, Sidhaye AR, Lee EJ, Rutishauser J, Stephan CW, Kopp P. Functional characterization of pendrin in a polarized cell system. Evidence for pendrin-mediated apical iodide efflux. J Biol Chem. 2004;279(13):13004–10.

  36. 36.

    Haack B, Schmalisch K, Palmada M, Böhmer C, Kohlschmidt N, Keilmann A, et al. Deficient membrane integration of the novel p.N14D-GJB2 mutant associated with non-syndromic hearing impairment. Hum Mutat. 2006;27(11):1158–9.

  37. 37.

    Ishihara K, Okuyama S, Kumano S, Iida K, Hamana H, Murakoshi M, et al. Salicylate restores transport function and anion exchanger activity of missense pendrin mutations. Hear Res. 2010;270(1–2):110–8.

  38. 38.

    Jang JH, Jung J, Kim AR, Cho YM, Kim MY, Lee SY, et al. Identification of novel functional null allele of SLC26A4 associated with enlarged vestibular aqueduct and its possible implication. Audiol Neurootol. 2014;19(5):319–26.

  39. 39.

    Lee JR, Derosa AM, White TW. Connexin mutations causing skin disease and deafness increase hemichannel activity and cell death when expressed in Xenopus oocytes. J Invest Dermatol. 2009;129(4):870–8.

  40. 40.

    Mani RS, Ganapathy A, Jalvi R, Srikumari Srisailapathy CR, Malhotra V, Chadha S, et al. Functional consequences of novel connexin 26 mutations associated with hereditary hearing loss. Eur J Hum Genet. 2009;17(4):502–9.

  41. 41.

    Meşe G, Londin E, Mui R, Brink PR, White TW. Altered gating properties of functional Cx26 mutants associated with recessive non-syndromic hearing loss. Hum Genet. 2004;115(3):191–9.

  42. 42.

    Palos F, García-Rendueles MER, Araujo-Vilar D, Obregon MJ, Calvo RM, Cameselle-Teijeiro J, et al. Pendred syndrome in two Galician families: insights into clinical phenotypes through cellular, genetic, and molecular studies. J Clin Endocrinol Metab. 2008;93(1):267–77.

  43. 43.

    Pera A, Dossena S, Rodighiero S, Gandía M, Bottà G, Meyer G, et al. Functional assessment of allelic variants in the SLC26A4 gene involved in Pendred syndrome and nonsyndromic EVA. Proc Natl Acad Sci U S A. 2008;105(47):18608–13.

  44. 44.

    Reimold FR, Heneghan JF, Stewart AK, Zelikovic I, Vandorpe DH, Shmukler BE, et al. Pendrin function and regulation in Xenopus oocytes. Cell Physiol Biochem. 2011;28(3):435–50.

  45. 45.

    Scott DA, Wang R, Kreman TM, Andrews M, McDonald JM, Bishop JR, et al. Functional differences of the PDS gene product are associated with phenotypic variation in patients with Pendred syndrome and non-syndromic hearing loss (DFNB4). Hum Mol Genet. 2000;9(11):1709–15.

  46. 46.

    Taylor JP, Metcalfe RA, Watson PF, Weetman AP, Trembath RC. Mutations of the PDS gene, encoding pendrin, are associated with protein mislocalization and loss of iodide efflux: implications for thyroid dysfunction in Pendred syndrome. J Clin Endocrinol Metab. 2002;87(4):1778–84.

  47. 47.

    Yuan Y, Guo W, Tang J, Zhang G, Wang G, Han M, et al. Molecular epidemiology and functional assessment of novel allelic variants of SLC26A4 in non-syndromic hearing loss patients with enlarged vestibular aqueduct in China. PLoS One. 2012;7(11):e49984 Schrijver I, editor.

  48. 48.

    Zhang Y, Tang W, Ahmad S, Sipp JA, Chen P, Lin X. Gap junction-mediated intercellular biochemical coupling in cochlear supporting cells is required for normal cochlear functions. Proc Natl Acad Sci U S A. 2005;102(42):15201–6.

  49. 49.

    Bal M, Zhang J, Zaika O, Hernandez CC, Shapiro MS. Homomeric and heteromeric assembly of KCNQ (Kv7) K+ channels assayed by total internal reflection fluorescence/fluorescence resonance energy transfer and patch clamp analysis. J Biol Chem. 2008;283(45):30668–76.

  50. 50.

    Kubisch C, Schroeder BC, Friedrich T, Lütjohann B, El-Amraoui A, Marlin S, et al. KCNQ4, a novel potassium channel expressed in sensory outer hair cells, is mutated in dominant deafness. Cell. 1999;96(3):437–46.

  51. 51.

    Geisterfer-Lowrance AAT, Christe M, Conner DA, Ingwall JS, Schoen FJ, Seidman CE, et al. A mouse model of familial hypertrophic cardiomyopathy, vol. 272, New Series; 1996.

  52. 52.

    Spindler M, Saupe KW, Christe ME, Sweeney HL, Seidman CE, Seidman JG, et al. Diastolic dysfunction and altered energetics in the alphaMHC403/+ mouse model of familial hypertrophic cardiomyopathy. J Clin Invest. 1998;101(8):1775–83.

  53. 53.

    Fatkin D, Christe ME, Aristizabal O, McConnell BK, Srinivasan S, Schoen FJ, et al. Neonatal cardiomyopathy in mice homozygous for the Arg403Gln mutation in the alpha cardiac myosin heavy chain gene. J Clin Invest. 1999;103(1):147–53.

  54. 54.

    Georgakopoulos D, Christe ME, Giewat M, Seidman CM, Seidman JG, Kass DA. The pathogenesis of familial hypertrophic cardiomyopathy: early and evolving effects from an α-cardiac myosin heavy chain missense mutation. Nat Med. 1999;5(3):327–30.

  55. 55.

    Kim S-J, Iizuka K, Kelly RA, Geng Y-J, Bishop SP, Yang G, et al. An α-cardiac myosin heavy chain gene mutation impairs contraction and relaxation function of cardiac myocytes. Am J Physiol Circ Physiol. 1999;276(5):H1780–7.

  56. 56.

    McConnell BK, Fatkin D, Semsarian C, Jones KA, Georgakopoulos D, Maguire CT, et al. Comparison of two murine models of familial hypertrophic cardiomyopathy. Circ Res. 2001;88(4):383–9.

  57. 57.

    Palmer BM, Fishbaugher DE, Schmitt JP, Wang Y, Alpert NR, Seidman CE, et al. Differential cross-bridge kinetics of FHC myosin mutations R403Q and R453C in heterozygous mouse myocardium. Am J Physiol Circ Physiol. 2004;287(1):H91–9.

  58. 58.

    Honda H, Nagashima H, Asakura S. Directional movement of F-actin in vitro. J Mol Biol. 1986;191(1):131–3.

  59. 59.

    Kron SJ, Spudich JA. Fluorescent actin filaments move on myosin fixed to a glass surface. Proc Natl Acad Sci U S A. 1986;83(17):6272–6.

  60. 60.

    Taussky HH, Shorr E. A microcolorimetric method for the determination of inorganic phosphorus. J Biol Chem. 1953;202(2):675–85.

  61. 61.

    Perry SV. [94] Myosin adenosinetriphosphatase. Methods Enzymol. 1955;2:582–8.

  62. 62.

    Cuda G, Fananapazir L, Zhu WS, Sellers JR, Epstein ND. Skeletal muscle expression and abnormal function of beta-myosin in hypertrophic cardiomyopathy. J Clin Invest. 1993;91(6):2861–5.

  63. 63.

    Cuda G, Fananapazir L, Epstein ND, Sellers JR. The in vitro motility activity of beta-cardiac myosin depends on the nature of the beta-myosin heavy chain gene mutation in hypertrophic cardiomyopathy. J Muscle Res Cell Motil. 1997;18(3):275–83.

  64. 64.

    Fujita H, Sugiura S, Momomura S, Omata M, Sugi H, Sutoh K. Characterization of mutant myosins of Dictyostelium discoideum equivalent to human familial hypertrophic cardiomyopathy mutants. Molecular force level of mutant myosins may have a prognostic implication. J Clin Invest. 1997;99(5):1010–5.

  65. 65.

    Palmiter KA, Tyska MJ, Haeberle JR, Alpert NR, Fananapazir L, Warshaw DM. R403Q and L908V mutant beta-cardiac myosin from patients with familial hypertrophic cardiomyopathy exhibit enhanced mechanical performance at the single molecule level. J Muscle Res Cell Motil. 2000;21(7):609–20.

  66. 66.

    Tyska MJ, Hayes E, Giewat M, Seidman CE, Seidman JG, Warshaw DM. Single-molecule mechanics of R403Q cardiac myosin isolated from the mouse model of familial hypertrophic cardiomyopathy. Circ Res. 2000;86(7):737–44.

  67. 67.

    Yamashita H, Tyska MJ, Warshaw DM, Lowey S, Trybus KM. Functional consequences of mutations in the smooth muscle myosin heavy chain at sites implicated in familial hypertrophic cardiomyopathy. J Biol Chem. 2000;275(36):28045–52.

  68. 68.

    Wang Q, Moncman CL, Winkelmann DA. Mutations in the motor domain modulate myosin activity and myofibril organization. J Cell Sci. 2003;116(Pt 20):4227–38.

  69. 69.

    Debold EP, Schmitt JP, Patlak JB, Beck SE, Moore JR, Seidman JG, et al. Hypertrophic and dilated cardiomyopathy mutations differentially affect the molecular force generation of mouse alpha-cardiac myosin in the laser trap assay. Am J Physiol Heart Circ Physiol. 2007;293(1):H284–91.

  70. 70.

    Di Domenico M, Casadonte R, Ricci P, Santini M, Frati G, Rizzo A, et al. Cardiac and skeletal muscle expression of mutant β-myosin heavy chains, degree of functional impairment and phenotypic heterogeneity in hypertrophic cardiomyopathy. J Cell Physiol. 2012;227(10):3471–6.

  71. 71.

    Lowey S. Functional consequences of mutations in the myosin heavy chain at sites implicated in familial hypertrophic cardiomyopathy. Trends Cardiovasc Med. 2002;12(8):348–54.

  72. 72.

    Himmelreich N, Shen N, Okun JG, Thiel C, Hoffmann GF, Blau N. Relationship between genotype, phenylalanine hydroxylase expression and in vitro activity and metabolic phenotype in phenylketonuria. Mol Genet Metab. 2018;125(1–2):86–95.

  73. 73.

    Pey AL, Desviat LR, Gámez A, Ugarte M, Pérez B. Phenylketonuria: genotype-phenotype correlations based on expression analysis of structural and functional mutations in PAH. Hum Mutat. 2003;21(4):370–8.

  74. 74.

    John SW, Scriver CR, Laframboise R, Rozen R. In vitro and in vivo correlations for I65T and M1V mutations at the phenylalanine hydroxylase locus. Hum Mutat. 1992;1(2):147–53.

  75. 75.

    Okano Y, Asada M, Kang Y, Nishi Y, Hase Y, Oura T, et al. Molecular characterization of phenylketonuria in Japanese patients. Hum Genet. 1998;103(5):613–8.

  76. 76.

    Liang Y, Huang M-Z, Cheng C-Y, Chao H-K, Fwu VT, Chiang S-H, et al. The mutation spectrum of the phenylalanine hydroxylase (PAH) gene and associated haplotypes reveal ethnic heterogeneity in the Taiwanese population. J Hum Genet. 2014;59(3):145–52.

  77. 77.

    Ho P, Huang M-Z, Fwu VT, Lin S-C, Hsiao K-J, Su T-S. Simultaneous assessment of the effects of exonic mutations on RNA splicing and protein functions. Biochem Biophys Res Commun. 2008;373(4):515–20.

  78. 78.

    Trunzo R, Santacroce R, Shen N, Jung-Klawitter S, Leccese A, De Girolamo G, et al. In vitro residual activity of phenylalanine hydroxylase variants and correlation with metabolic phenotypes in PKU. Gene. 2016;594(1):138–43.

  79. 79.

    Marvit J, DiLella AG, Brayton K, Ledley FD, Robson KJ, Woo SL. GT to AT transition at a splice donor site causes skipping of the preceding exon in phenylketonuria. Nucleic Acids Res. 1987;15(14):5613–28.

  80. 80.

    Steventon GB, Mitchell SC, Pérez B, Desviat LR, Ugarte M. The activity of wild type and mutant phenylalanine hydroxylase with respect to the C-oxidation of phenylalanine and the S-oxidation of S-carboxymethyl-L-cysteine. Mol Genet Metab. 2009;96(1):27–31.

  81. 81.

    Cerreto M, Cavaliere P, Carluccio C, Amato F, Zagari A, Daniele A, et al. Natural phenylalanine hydroxylase variants that confer a mild phenotype affect the enzyme’s conformational stability and oligomerization equilibrium. Biochim Biophys Acta. 2011;1812(11):1435–45.

  82. 82.

    Okano Y, Eisensmith RC, Güttler F, Lichter-Konecki U, Konecki DS, Trefz FK, et al. Molecular basis of phenotypic heterogeneity in phenylketonuria. N Engl J Med. 1991;324(18):1232–8.

  83. 83.

    Dworniczak B, Aulehla-Scholz C, Kalaydjieva L, Bartholomé K, Grudda K, Horst J. Aberrant splicing of phenylalanine hydroxylase mRNA: the major cause for phenylketonuria in parts of southern Europe. Genomics. 1991;11(2):242–6.

  84. 84.

    Danecka MK, Woidy M, Zschocke J, Feillet F, Muntau AC, Gersting SW. Mapping the functional landscape of frequent phenylalanine hydroxylase (PAH) genotypes promotes personalised medicine in phenylketonuria. J Med Genet. 2015;52(3):175–85.

  85. 85.

    Han SY, Kato H, Kato S, Suzuki T, Shibata H, Ishii S, et al. Functional evaluation of PTEN missense mutations using in vitro phosphoinositide phosphatase assay. Cancer Res. 2000;60(12):3147–51.

  86. 86.

    Chen HJ, Romigh T, Sesock K, Eng C. Characterization of cryptic splicing in germline PTEN intronic variants in Cowden syndrome. Hum Mutat. 2017;38(10):1372–7.

  87. 87.

    Guénard F, Labrie Y, Ouellette G, Beauparlant CJ, Bessette P, Chiquette J, et al. Germline mutations in the breast cancer susceptibility gene PTEN are rare in high-risk non-BRCA1/2 French Canadian breast cancer families. Familial Cancer. 2007;6(4):483–90.

  88. 88.

    Gil A, Rodríguez-Escudero I, Stumpf M, Molina M, Cid VJ, Pulido R. A functional dissection of PTEN N-terminus: implications in PTEN subcellular targeting and tumor suppressor activity. PLoS One. 2015;10(4):e0119287 Uversky VN, editor.

  89. 89.

    Tan M-H, Mester J, Peterson C, Yang Y, Chen J-L, Rybicki LA, et al. A clinical scoring system for selection of patients for PTEN mutation testing is proposed on the basis of a prospective study of 3042 probands. Am J Hum Genet. 2011;88(1):42–56.

  90. 90.

    Mighell TL, Evans-Dutson S, O’Roak BJ. A saturation mutagenesis approach to understanding PTEN lipid phosphatase activity and genotype-phenotype relationships. Am J Hum Genet. 2018;102(5):943–55.

  91. 91.

    Chen P-C, Yin J, Yu H-W, Yuan T, Fernandez M, Yung CK, et al. Next-generation sequencing identifies rare variants associated with Noonan syndrome. Proc Natl Acad Sci U S A. 2014;111(31):11473–8.

  92. 92.

    Denayer E, Parret A, Chmara M, Schubbert S, Vogels A, Devriendt K, et al. Mutation analysis in Costello syndrome: functional and structural characterization of the HRAS p.Lys117Arg mutation. Hum Mutat. 2008;29(2):232–9.

  93. 93.

    Gremer L, Merbitz-Zahradnik T, Dvorsky R, Cirstea IC, Kratz CP, Zenker M, et al. Germline KRAS mutations cause aberrant biochemical and physical properties leading to developmental disorders. Hum Mutat. 2011;32(1):33–43.

  94. 94.

    Lee BH, Kim J-M, Jin HY, Kim G-H, Choi J-H, Yoo H-W. Spectrum of mutations in Noonan syndrome and their correlation with phenotypes. J Pediatr. 2011;159(6):1029–35.

  95. 95.

    Martinelli S, Nardozza AP, Delle Vigne S, Sabetta G, Torreri P, Bocchinfuso G, et al. Counteracting effects operating on Src homology 2 domain-containing protein-tyrosine phosphatase 2 (SHP2) function drive selection of the recurrent Y62D and Y63C substitutions in Noonan syndrome. J Biol Chem. 2012;287(32):27066–77.

  96. 96.

    Pandit B, Sarkozy A, Pennacchio LA, Carta C, Oishi K, Martinelli S, et al. Gain-of-function RAF1 mutations cause Noonan and LEOPARD syndromes with hypertrophic cardiomyopathy. Nat Genet. 2007;39(8):1007–12.

  97. 97.

    Pannone L, Bocchinfuso G, Flex E, Rossi C, Baldassarre G, Lissewski C, et al. Structural, functional, and clinical characterization of a novel PTPN11 mutation cluster underlying Noonan syndrome. Hum Mutat. 2017;38(4):451–9.

  98. 98.

    Rauen KA, Tidyman WE, Estep AL, Sampath S, Peltier HM, Bale SJ, et al. Molecular and functional analysis of a novel MEK2 mutation in cardio-facio-cutaneous syndrome: transmission through four generations. Am J Med Genet A. 2010;152A(4):807–14.

  99. 99.

    Razzaque MA, Nishizawa T, Komoike Y, Yagi H, Furutani M, Amo R, et al. Germline gain-of-function mutations in RAF1 cause Noonan syndrome. Nat Genet. 2007;39(8):1013–7.

  100. 100.

    Roberts AE, Araki T, Swanson KD, Montgomery KT, Schiripo TA, Joshi VA, et al. Germline gain-of-function mutations in SOS1 cause Noonan syndrome. Nat Genet. 2007;39(1):70–4.

  101. 101.

    Rodriguez-Viciana P, Rauen KA. Biochemical characterization of novel germline BRAF and MEK mutations in cardio-facio-cutaneous syndrome. Methods Enzymol. 2008;438:277–89.

  102. 102.

    Senawong T, Phuchareon J, Ohara O, McCormick F, Rauen KA, Tetsu O. Germline mutations of MEK in cardio-facio-cutaneous syndrome are sensitive to MEK and RAF inhibition: implications for therapeutic options. Hum Mol Genet. 2008;17(3):419–30.

  103. 103.

    Smith MJ, Neel BG, Ikura M. NMR-based functional profiling of RASopathies and oncogenic RAS mutations. Proc Natl Acad Sci U S A. 2013;110(12):4574–9.

  104. 104.

    Wu X, Yin J, Simpson J, Kim K-H, Gu S, Hong JH, et al. Increased BRAF heterodimerization is the common pathogenic mechanism for Noonan syndrome-associated RAF1 mutants. Mol Cell Biol. 2012;32(19):3872–90.

  105. 105.

    Rodriguez-Viciana P, Tetsu O, Tidyman WE, Estep AL, Conger BA, Cruz MS, et al. Germline mutations in genes within the MAPK pathway cause cardio-facio-cutaneous syndrome. Science. 2006;311(5765):1287–90.

  106. 106.

    Simpson JC. Functional Assays. In: Encyclopedic reference of genomics and proteomics in molecular medicine. Berlin Heidelberg: Springer; 2005. p. 617–20.

  107. 107.

    Strande NT, Brnich SE, Roman TS, Berg JS. Navigating the nuances of clinical sequence variant interpretation in Mendelian disease. Genet Med. 2018;20(9):918–26.

  108. 108.

    Tavtigian SV, Greenblatt MS, Harrison SM, Nussbaum RL, Prabhu SA, Boucher KM, et al. Modeling the ACMG/AMP variant classification guidelines as a Bayesian classification framework. Genet Med. 2018;20(9):1054.

  109. 109.

    Brnich SE, Abou Tayoun NA, Couch FJ, Cutting GR, Greenblatt MS, Heinen CD, et al. Recommendations for application of the functional evidence PS3/BS3 criterion using the ACMG/AMP sequence variant interpretation framework. 2019. https://doi.org/10.1186/s13073-019-0690-2.

Download references

Acknowledgements

We are grateful to ClinGen Sequence Variant Interpretation Working Group Chairs Leslie G. Biesecker (NIH/NHGRI) and Steven M. Harrison (Broad Institute) for their critical reading of this manuscript and valuable comments. We also appreciate the helpful discussion with the ClinGen Functional Studies Team at Stanford University (Matt W. Wright, Michael A. Iacocca, Arturo Lopez Pineda, and Hannah Wand).

Funding

This work was supported by the following grants: UNC/ACMG/Geisinger/Kaiser under the award numbers U41HG009650 and 3U41HG009650-02S1. ClinGen is primarily funded by the National Human Genome Research Institute (NHGRI), through the following three grants: U41HG006834, U41HG009649, and U41HG009650. ClinGen also receives support for content curation from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), through the following three grants: U24HD093483, U24HD093486, and U24HD093487. SEB is supported in part by the National Institute of General Medical Sciences grants 5T32 GM007092 and 5T32 GM008719-6. SEB is also a recipient of support from the University Cancer Research Fund as an MD/PhD scholar. JSB is a recipient of the Yang Family Biomedical Scholars Award. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health (NIH).

Author information

DMK, MKJ, SEB, CB, BCP, and JSB designed the study. DMK, SMM, and MKJ conducted the VCEP specification evaluation, variant identification, literature search, and curations. DMK, MKJ, SMM, SEB, CB, BCP, and JSB participated in the data analysis. DMK, SMM, MKJ, and JSB wrote the original manuscript draft. DMK, MKJ, SMM, SEB, CB, BCP, and JSB reviewed and edited the final manuscript draft. All authors read and approved the final manuscript.

Correspondence to Jonathan S. Berg.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1. Summaries of VCEP disease mechanisms and assay specifications.

Additional file 2: Specific instance of assay literature curation results. Table S1. Summary of CDH1 variant-level curation. Table S2. Summary of CDH1 specific instance-level curation. Table S3. Comparison of CDH1 functional assay findings to variant evidence in ClinVar. Table S4. Summary of Hearing Loss variant-level curation. Table S5. Summary of Hearing Loss specific instance-level curation. Table S6. Summary of Inherited Cardiomyopathy (MYH7) variant-level curation. Table S7. Summary of Inherited Cardiomyopathy (MYH7) specific instance-level curation. Table S8. Summary of PAH variant-level curation. Table S9. Summary of PAH specific instance-level curation. Table S10. Summary of PTEN variant-level curation. Table S11. Summary of PTEN specific instance-level curation. Table S12. Summary of RASopathy variant-level curation. Table S13. Summary of RASopathy specific instance-level curation.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kanavy, D.M., McNulty, S.M., Jairath, M.K. et al. Comparative analysis of functional assay evidence use by ClinGen Variant Curation Expert Panels. Genome Med 11, 77 (2019) doi:10.1186/s13073-019-0683-1

Download citation

Keywords

  • Clinical genome resource
  • Clinical exome sequencing
  • Clinical genome sequencing
  • Functional assays
  • Variant interpretation