github.com/gemaraproj/gemara@v1.3.0

docs/adrs/0022-evidence-on-assessment-log.md raw

 1---
 2layout: page
 3title: Add Evidence to AssessmentLog
 4---
 5
 6- **ADR:** 0022
 7- **Proposal Author(s):** @jpower432, @eddie-knight
 8- **Status:** Accepted
 9
10## Context
11
12Gemara's measurement layers produce opinions, structured conclusions about the state of a target. An `EvaluationLog` is a tool's opinion: "based on what I observed, this control passes or fails." An `AuditLog` is an auditor's opinion: "based on the evidence I reviewed, this control is effective or not." Both opinions are rooted in evidence, the data that was actually observed that helped inform the actual conclusion. The tool sees it firsthand (an API response, a file, a config dump). The auditor sees it through the tool's record. Different proximity to the raw data, same root.
13
14Currently, the evidence root is invisible. An `AssessmentLog` records a pass/fail result with a message, but not "I consulted this specific resource at this time." The tool's opinion exists without a record of what it was rooted in, limiting downstream consumers of the log.
15
16## Action
17
18Reshape the existing `#Evidence` type in the experimental `auditlog.cue` to serve both the evaluation layer (`AssessmentLog`) and the audit layer (`AuditLog`). 
19The `#Evidence` type captures what was cited to support an opinion for a specific activity, raw data for the tool, evaluation and enforcement artifacts for the auditor.
20
21```cue
22#Evidence: {
23    // id uniquely identifies this evidence
24    id?: string
25
26    // type categorizes the kind of evidence (open enum)
27    type: #EvidenceType
28
29    // collected is the timestamp when the evidence was gathered
30    "collected-at": #Datetime
31
32    // address identifies where the evidence can be found
33    address?: string
34
35    // payload is the raw evidence data collected
36    payload?: _
37
38    // digest is a hash of the evidence content at collection time for integrity verification
39    // Enables verification that mutable-address evidence (S3 objects, HTTP URLs, API responses) has not changed since the tool observed it.
40    // Not needed for content-addressable systems (OCI, git) or inline payloads.
41    digest?: string
42
43    // description explains what this evidence represents
44    description?: string
45}
46```
47`#EvidenceType` remains an open enum. Recommended values include artifact types already known to Gemara (e.g., `EvaluationLog`, `EnforcementLog`) plus categories for common evidence forms like provenance.
48
49**Changes to the Assessment Log**:
50```cue
51#AssessmentLog: {
52    // ... existing fields ...
53
54    // evidence records the raw data cited to support this assessment's opinion
55    evidence?: [#Evidence, ...#Evidence]
56}
57```
58
59The field is optional (one or more when present) to maintain backward compatibility with existing EvaluationLog documents.
60
61## Consequences
62
63### Positive
64
65- Assessment logs can record exactly what the tool consulted, when, and what was observed, making the root of the tool's opinion inspectable.
66- The `address` field enables SARIF converters to populate `PhysicalLocation` and `Region` with file paths and line numbers from evidence data.
67- `address` as a plain string works for file paths, API endpoints, ARNs, container image references, physical locations, and any other evidence source.
68
69### Negative
70
71Embedding raw evidence via the `payload` field can make assessment logs large. 
72It is the evaluator's responsibility to manage payload size, trim sensitive data, and ensure evidence does not cross trust boundaries inappropriately.
73
74## Alternatives Considered
75
76### Separate `#Provenance` type
77
78A standalone `#Provenance` type on `#AssessmentLog` with fields for `address`, `collected`, and `observed`. This was the original proposal in [issue #417](https://github.com/gemaraproj/gemara/issues/417).
79
80**Rejected because:** Both layers produce opinions rooted in evidence. The tool cites raw data to support its opinion. The auditor cites evaluation artifacts to support theirs. Introducing a separate type creates an artificial distinction when the relationship (opinion rooted in evidence) is the same at both layers.
81
82