Define rules for determining 'direction' of evidence provided by Criterion Assessments

[mbrush]

In the ClinGen Interpretation data model that is currently being aligned with SEPIO, CriterionAssessments are assertions based on ACMG Criteria that are used as a first tier of evidence for Variant Interpretations. There are a number of different possibility that can result from a given CriterionAssessment, depending on whether the ACMG criterion used is: (1) determined to be met, unmet, or refuted; and (2) how the class of criterion used (P vs B) aligns with the final call made in the target Variantinterpretation.

In short, we will have to translate the outcome for a CriterionAssessment used as evidence (i.e. whether a B or P criterion is met or unmet for a VariantInterpretation of benign or pathogenic), into a directionality for the relationship used in linking the VariantInterpretation to its CA-supported EvidenceLine (i.e. whether the CA provides supporting, refuting, or inconclusive evidence).

We have had initial discussions on this topic, as documented in the 2-23-17 cmap here.

[mbrush]

On 3-21-17 call, we decided that ClinGen would not assert a direction for evidence in its v1 model.
ClinGen will just use the more generic has_evidence relation. The directionality can be inferred based on information built into the CriterionAssessment description (i.e. its P vs B direction and how this alignes with the direction of the supported VariantInterpretation, and whether the CA was met or unmet or refuted).

For v2 of the model, we will revisit how we want to capture the directionality - as this is an important aspect for SEPIO integration and analysis use cases. e.g. we want to easily query for all evidence lines that support vs refute a target assertion. Ultimately it may not be something that CG tool users assert themselves, but something that is derived/inferred using a set of rules that leverage the information that does get asserted.
Some initial ideas in gdoc here.

[mbrush]

Some additional notes from 4-3-17:

In the ACMG/ClinGen framework, the direction and strength of evidence is quite nuanced - dependent on the specific criteria used, the outcome determined, and the alignment of the 'class' of the criterion and the final VI (i.e. if they are P vs B). I started a matrix to describe all combinations here, which may help us to consider how to assign direction and strength for the trickier cases, as needed.

I think that the evidence direction can be deterministically calculated in all cases - and again, this is what SEPIO is most concerned with. Determination of strength in cases where a criterion is unmet (refuted or insufficient data) is a trickier proposition - but one that I don’t think we need to address now. I think that here too, we can algorithmically assign a strength from basic user input data that includes the criterion used, the asserted outcome, and any modification of the criterion's default strength made by the agent/assessor.

In summary: (for the initial ClinGen release)

1. Direction: we agreed to infer this - will just use agnostic has_evidence property. Direction can be computed automatically in a post-processing step, based on the asserted outcome, and the 'alignment' between the criterion the variant interpretation (i.e. P vs B)

2. Strength:

• for met criteria aligned with VI call - go with criteria default (e.g. strong for BS1) unless overridden
• for met criteria not aligned with VI call - also go with criteria default (e.g. strong for BS1) unless overridden
• for insufficient criteria aligned or not aligned with VI call - explicitly assert strength of 'no support' or 'neutral' or 'n/a', as per Chris recommendation.
• for refuted criteria aligned with VI call - leave blank for now. These provide refuting evidence. They will be rare, and very tricky to assign. The strength of these is very context dependent (i.e. what the criterion is, if it has a paired criterion in opposite class, etc)
• for refuted criteria not aligned with VI call - leave blank for now. These provide supporting evidence. They will be rare, and very tricky to assign. The strength of these is very context dependent (i.e. what the criterion is, if it has a paired criterion in opposite class, etc)