Add file annotations to steer detection (inclusion/exclusion regions)

[martsokha]

Context

The runtime engine (nvisy-schema) exposes AnalyzerParams.annotations: AnyAnnotations — caller-supplied inclusion/exclusion regions that steer detection:

• Inclusion — "there may be an entity in this region" → recognizers (esp. LLM) adjudicate it. Carries a location (byte range for text: {start, end, page?}), plus optional label/name/confidence.
• Exclusion — "flag nothing here" → the analyzer drops any entity overlapping the region.
• Bagged per-modality: AnyAnnotations { text, tabular, image, audio }.

We currently pass AnyAnnotations::default() (empty) at detect, so this capability is unused. This issue tracks wiring it up so reviewers can steer detection ("definitely check this paragraph" / "this footer is always a false positive").

A first implementation was built and then removed pending the design decision below (see Open question). The runtime field default remains so the engine compiles.

The open question (must resolve first): cardinality

Are annotations a durable property of the file or a choice made at detection time? This decides the whole data model:

Model	Fits when	Shape
One set per file	Annotations are corrections/facts about the document that apply to every future detect ("this letterhead SSN is a false positive"). No "which set" question.	file_id UNIQUE, detect auto-loads it.
Per-run input	Annotations are a detection-time choice (a strategy, an experiment, a one-off).	Belongs on the detect request (inline, like scope) or as a named strategy on the pipeline — not multiple file-scoped stored sets.
Both	Persistent per-file base + per-run inline override.	One stored set per file + optional inline on CreatePipelineRun.

"Multiple named annotation sets per file that detect picks between" is the awkward middle — it puts a detection-time strategy into file-scoped storage; cases like A/B or conservative-vs-aggressive belong on the run/pipeline layer instead. Decide the producer/use-case before building.

Validated implementation shape (for the one-set-per-file model)

The removed prototype mirrored the run's analyzed_document pattern and worked end-to-end:

• Storage: 1 DB row per file + 1 encrypted blob per file. Row (workspace_file_annotations) holds { id, file_id UNIQUE, account_id, annotations_key, timestamps }; the AnyAnnotations bag is crypto.encrypt_json'd into a dedicated AnnotationsBucket (no TTL), keyed by a deterministic AnnotationKey::from_parts(workspace_id, file_id) so replace overwrites in place (no orphaned blobs).
• API: PUT/GET/DELETE /files/{fileId}/annotations — PUT is full-replace/upsert; GET returns 200 {} when none; DELETE clears row + blob.
• Detect: load the file's annotations (degrade to empty + log if the blob is unloadable — annotations are advisory) into AnalyzerParams.annotations.
• Auth: an AnnotateFiles permission for PUT/DELETE, ViewFiles for GET.

Review findings to carry forward (all fixed in the prototype)

• Orphan-blob leak on re-PUT if using a random key → use a deterministic per-file key so writes overwrite in place. (AnnotationsBucket has no TTL, unlike IntermediatesBucket, so random keys leak forever.)
• GET should return 200 {} for "no annotations", not 404 — empty is a valid state for a full-replace resource.
• A missing blob (torn record) should degrade detect to empty, not hard-fail — steering is advisory.

Notes

• Sensitive data: annotations reveal where PII is/isn't in a document → must be encrypted at rest (workspace-keyed), like the analyzed document and file bytes.
• Distinct from workspace_files.tags (document classification tags → ScopeParams.tags, a separate integration gap) and from the engine's label_catalog (entity types to emit).