Evidence collection

Overview

Evidence collection is the automation layer on top of compliance frameworks. Where the frameworks article covers manual attachment (someone clicks "attach evidence" on a control), this covers the rules that auto-discover and auto-attach evidence on a schedule: ComplianceEvidenceQuery rows define a query, a Celery task runs them periodically, and matching rows are attached to the right controls as WorkspaceControlEvidence records (WB.16). Bulk-attach endpoints (T1.12) and per-row actions (PR2) round out the manual side.

The collection page lives at /ws/{slug}/governance/evidence. It shows every active query, its last run timestamp, the count of rows attached this period, and a manual Run now button for ad-hoc execution.

Why it exists

A SOC2 control like "we approve every production change" produces evidence on every change approval — but doing the attach by hand 200 times a quarter is wasted work. A ComplianceEvidenceQuery says "find every ChangeRequest that was approved in the last 24 hours where environment=prod, and attach each one as evidence to control CC8.1". That query runs nightly, the evidence appears on the control without anyone touching it, and the auditor sees a populated control instead of an empty one.

The same pattern works for MFA enforcement evidence, cert renewal evidence, incident postmortem evidence, vendor assessment evidence — any control where the underlying activity is already happening in Vigilo and the only missing step is "tell the framework about it".

ComplianceEvidenceQuery model

Fields:

• workspace — scope.
• name — human-readable label.
• description — what the query is finding and why.
• query_kind — enum: audit_log, change_request, incident, cert_event, vendor_assessment, member_event, playbook_run. Determines which model to query against.
• filters — JSON dict of filter criteria. Shape depends on query_kind; example for change_request: {"status": "approved", "environment": "prod", "approved_within_days": 1}.
• target_controls — many-to-many to Control. The matched rows are attached as evidence to each of these controls.
• auto_link — boolean. When True, matched rows are attached automatically. When False, the query just lists candidates and a human has to click attach.
• is_active — pause/resume without deleting.
• last_run_at, last_match_count, last_attached_count.

The model lives in compliance/models.py.

Collection Celery task

vigilo.compliance.evidence_collect runs every 6 hours by default (configurable via VIGILO_EVIDENCE_COLLECT_INTERVAL_HOURS). For each active ComplianceEvidenceQuery:

1. Translate filters into an ORM queryset against the model implied by query_kind.
2. Restrict the queryset to rows newer than last_run_at (so each row attaches at most once per query).
3. For each matched row, iterate target_controls and upsert a WorkspaceControlEvidence with evidence_kind matching the query kind and evidence_ref = matched row UUID.
4. Set status to accepted if the query has auto_accept=True, else pending_review.
5. Update last_run_at, last_match_count, last_attached_count.

The task is idempotent on a per-row basis — re-attaching the same (control, evidence_kind, evidence_ref) is a no-op.

Bulk attach endpoint (T1.12)

For human-initiated batch attaches, the endpoint POST /ws/{slug}/api/v1/compliance/controls/{control_id}/bulk-attach-evidence/ accepts a list of {evidence_kind, evidence_ref} records and attaches them all in one transaction. The UI uses this from the Attach evidence → Bulk select flow: tick 20 audit log rows in a search result, click Attach to control, choose the control, submit.

Permission required: compliance.attach_evidence (granted to engineer+ by default).

Per-row actions (PR2)

Each evidence row in a control's evidence list has a row-action menu:

• Add evidence — opens the attach modal (same as the top-level attach button).
• Update status — toggle between pending_review, accepted, rejected. Status changes write an audit log entry and notify the control's owner.
• Assign owner — pick a member responsible for reviewing this specific evidence. Useful when a control has many evidence rows from different domains (audit log, change, runbook) and you want to delegate review.
• Remove — detach the evidence. The underlying row (e.g. the ChangeRequest) is not deleted; only the link from this control to it.

These actions exist on every evidence row regardless of how the row was created.

Framework navigation

The collection page links upward to the framework view:

• Click any query's target controls chip to jump to the framework's control list with that control highlighted.
• Click a control's evidence collected by this query count to see which rows are attached via this query specifically (vs manual or other queries).

The framework dashboard tile shows, per control, the breakdown of evidence sources: how many manual, how many from each query. A control whose evidence is 100% from one query is a candidate for review — if the query stops returning matches (because the underlying activity stopped), the control will silently lose all its evidence.

Dashboard

The collection dashboard tile shows:

• Total active queries
• Matches attached in last 24h
• Top 5 queries by attach volume
• Queries that returned 0 matches on their last 3 runs (likely broken or stale)

The last bullet is the most actionable — a stale query is a compliance gap waiting to surface during an audit.

Common workflows

1. Auto-attach every prod change approval to control CC8.1

1. Governance → Evidence → New query.
2. Name: "Prod change approvals".
3. Query kind: change_request.
4. Filters: {"status": "approved", "environment": "prod", "approved_within_days": 1}.
5. Target controls: select CC8.1 (and any other controls about change management).
6. auto_link = True. is_active = True.
7. Save. Within 6 hours, the next sweep attaches every matching change.

2. Bulk-attach historic incidents to a postmortem control

1. Audit log or Incidents page → filter to incidents with postmortems.
2. Tick the rows you want.
3. Bulk actions → Attach to control.
4. Pick the control. Confirm.
5. The bulk-attach endpoint creates the evidence rows in one transaction.

3. Review fresh evidence in the queue

1. Governance → Evidence → Pending review tab.
2. Each row shows the evidence summary, source query, and an inline Accept / Reject action.
3. Use bulk-tick + bulk accept for low-judgment cases (e.g. MFA monitor outputs).

4. Assign a control's evidence review to a specific member

1. Open the control. For each evidence row, click the row menu → Assign owner.
2. Pick the member. They get an in-app and email notification.
3. The control's gap dashboard groups evidence by assignee so the owner sees their personal queue.

Permissions

Troubleshooting

Query attaches the same rows over and over. The last_run_at cursor isn't being updated; check the query's history page for run errors. Most common cause: a filter that doesn't include a date constraint, combined with a model that doesn't have a created_at cursor. Add a created_within_days filter.

Query returns 0 matches but I know rows exist. Filter mismatch. Open the query editor and click Preview matches — this runs the query without attaching, returning the count and the first 10 rows. Tune filters until the preview makes sense.

Manual Run now button is greyed out. You don't have compliance.attach_evidence permission, or the query is is_active=False.

Bulk attach fails with 400. One of the evidence refs is invalid (the underlying row was deleted between selection and submit). Re-tick from the current list and retry.

Auto-accepted evidence is being rejected later. A reviewer doesn't trust the query. Discuss with them; either tune the query or set auto_accept=False so a human eyeballs every attach.

Related articles

• Compliance frameworks — frameworks and controls evidence attaches to.
• Audit log — the most common evidence source.
• Vendor risk — vendor assessments are a common evidence source for vendor-management controls.