Skip to content

AI Risk Register — Sprout

Frame: NIST AI RMF (1.0, Jan 2023) MAP function + GenAI Profile NIST AI 600-1 (Jul 2024, 12 risks). Instantiates the governance scaffolding in STANDARDS/RESPONSIBLE-TECH-FRAMEWORK.md §Governance. Numeric gate thresholds are owned by the sibling standards and linked, not restated.

  • Author / accountable owner: Chelsea Kelly-Reif (ISO 42001 Cl. 6.2 owner)
  • Last regenerated: 2026-06-22 · Review cadence: quarterly, and immediately on any NIST AI RMF / AI 600-1, ISO 42001, or EU AI Act revision (per framework recheck cadence)
  • Scope of this register: the Sprout assistant and its eval harness, both deployment modes (offline-default CLI/static build; optional Bedrock/Anthropic cloud seam). Family Greenhouse personalization is deferred (Phase A+) and out of scope here; its household-data path will add rows when it ships.

1. AI system inventory (MAP-1, MAP-2)

Field Value
System Sprout — grounded, evaluated houseplant-care RAG assistant + eval harness
Purpose Answer household plant-care questions only from a versioned, cited, dated corpus; abstain otherwise
Users / context General public, voluntary, low-stakes; self-serve, no account, no profile
Default pipeline HashingEmbedding (deterministic) + pure-Python BM25 + ExtractiveGenerator; offline, no network, no cloud account
Cloud seam (opt-in) Answer model Claude Haiku via Bedrock or native Anthropic API (config.generation.provider); judge model Claude Sonnet in eval only
Training compute 0 — no model is trained or fine-tuned. Sprout composes a deterministic retriever with a frozen, externally-hosted foundation model behind a config switch
Autonomy None. Single request → single grounded answer or refusal. No agentic loop, no tools, no memory beyond an optional in-session question buffer, no write actions
Data at rest The committed corpus (synthetic, CC0-1.0) + content-hashed index. No user-query persistence in the demo
Decision consequence Informational only; explicitly not veterinary or medical advice (disclosure string on every answer)

Why this matters for the GenAI risk surface. The default pipeline is extractive: every rendered sentence is copied verbatim from a retrieved chunk and independently re-verified by the citation guard (src/sprout/guards.py citation_guard). This collapses the largest GenAI risk (confabulation) to ~0 by construction in the default mode, and bounds it in the cloud mode because the same post-generation guard runs regardless of generator. The risk register below is therefore mostly an argument that the structural defenses already in the code are the controls — not a backlog of aspirational mitigations.

2. Which of the 12 NIST AI 600-1 GenAI risks apply

Each risk is marked Applies / Partial / N/A, with a one-line reason. The framework requires the N/A decisions to be written down, not implied by silence.

# AI 600-1 risk Applies? One-line rationale
1 CBRN information N/A Houseplant care; no uplift path to chemical/bio/nuclear/radiological harm
2 Confabulation Applies Core RAG risk; mitigated structurally (extractive + citation guard) — see R1
3 Dangerous / violent / hateful content Partial Possible only via adversarial input; extractive generation cannot emit ungrounded prose — see R4
4 Data privacy Applies Optional cloud seam transmits the query to a model provider; logs could leak query/PII — see R3
5 Environmental Partial Default mode = 0 training, 0 inference cost; cloud seam is per-token foundation-model inference only — see R5
6 Harmful bias / homogenization Partial Single-domain factual retrieval; EN/ES capability parity is the live fairness concern — see R6
7 Human-AI configuration Applies Over-trust of a confident wrong answer in a safety-adjacent (toxicity) context — see R2
8 Information integrity Applies Stale/contradictory plant-care lore; provenance + "as of" date are the controls — see R7
9 Information security Applies Prompt injection (direct + indirect via corpus), system-prompt leakage — see R4
10 Intellectual property Partial Corpus is synthetic CC0-1.0; manifest carries per-doc license; no scraped third-party text — see R8
11 Obscene / degrading / abusive content N/A No image/audio generation; extractive text output bounded to a vetted corpus
12 Value-chain / component integration Partial Depends on a foundation-model provider only in the opt-in cloud seam; circuit breaker + fail-closed — see R5

The three risks the spec calls out — confabulation (R1), harmful-advice/safety (R2), data-privacy (R3) — are the highest-priority rows and are detailed first.

3. Risk detail, mitigation, residual risk

Severity/likelihood are pre-mitigation; residual is post-mitigation. Scale: Low / Med / High.

R1 — Confabulation (ungrounded or fabricated plant-care claims) · Risk 2

  • What could go wrong: the assistant states a watering interval, a diagnosis, or a toxicity fact that no source supports, and a user acts on it.
  • Inherent severity / likelihood: High / High (this is the defining failure mode of RAG).
  • Controls (structural, AUTO-GATED):
  • Extractive generation. The default ExtractiveGenerator only emits sentences copied verbatim from retrieved chunks; it cannot author free text.
  • Independent citation guard. citation_guard re-verifies each candidate against the chunk it claims, dropping anything not contained-verbatim or below support_overlap (0.66). An ungrounded sentence is structurally impossible to render — the same guard runs on cloud-generator output, so Claude-on-Bedrock output is filtered identically.
  • Retrieval-first refusal. Assistant.answer refuses before generating if no chunk clears min_score and shares a content term (Retriever.has_grounding). The model is never asked to fill a gap.
  • Calibrated abstention. Below abstain_threshold (0.25, per ADR-0012; corrected 2026-07-05, was miscited as 0.45) the assistant refuses rather than guesses (src/sprout/confidence.py).
  • Eval: groundedness suite (every claim entailed by its cited passage; threshold owned by AI-EVALUATION-STANDARD.md, confabulation floor ≤ 5%). Default mode is 100% by construction.
  • Residual risk: Low. The guard verifies attribution, not the corpus's real-world correctness; a wrong-but-cited corpus passage would pass. Mitigated by the dated/versioned corpus and the information-integrity controls (R7), not by the generator. Cloud-mode residual is marginally higher (the model could emit a guard-passing paraphrase that subtly distorts) and is bounded by temperature=0 and the same citation guard.

R2 — Harmful advice / safety (false reassurance on toxicity) · Risk 7 (Human-AI configuration)

  • What could go wrong: a user asks "is this safe for my cat?", the assistant certifies "safe," the user does not call a vet, a pet or child is harmed. The asymmetry is the point — a false "toxic" is an annoyance; a false "safe" is the worst plausible individual harm (Ethics audit A).
  • Inherent severity / likelihood: High / Med.
  • Controls (structural, AUTO-GATED):
  • Never-certify-safe deny-list. safety_filter drops any rendered sentence containing a forbidden certification phrase in either language (GuardsConfig.forbidden_safe_phrases, EN + ES). The assistant explains what a cited source says; it never asserts safety.
  • Mandatory routing. Toxicity/ingestion queries (is_safety_query) attach a vet / poison-control routing directive to both the answer and the refusal (safety_route_by_lang).
  • Cite-or-refuse. A safety answer is grounded in a cited toxicity reference or it is an honest refusal — never a bare assertion.
  • Eval: safety suite — deterministic string + citation checks (no judge, immune to judge drift): (a) no certification phrase, (b) routes to vet/poison-control, (c) cites or refuses. Threshold 0.95 (see src/sprout/eval/suites/safety.py).
  • Residual risk: Low. The deny-list is phrase-based; a novel safe-ish paraphrase not on the list could in principle survive. Bounded because the content is still extractive (it can only echo a cited toxicity passage, which states facts, not reassurance), and the routing notice always fires for safety queries. New evasions are added to the list + a regression case on discovery (see the red-team report). This is not veterinary advice and the disclosure says so.

R3 — Data privacy (query / PII exposure via the cloud seam or logs) · Risk 4

  • What could go wrong: in the opt-in cloud mode the user's question is transmitted to a third-party model provider; a question may contain incidental PII; structured logs could record it.
  • Inherent severity / likelihood: Med / Med (Low in the default offline mode, which is the default).
  • Controls:
  • Offline by default (AUTO). The privacy-preserving mode is the default: no network, no provider, no transmission. Most users never leave it.
  • No user-query persistence in the demo (REVIEW + tested). No mutable server state; optional in-session buffer only.
  • PII redaction at the network boundary (AUTO). redact_pii strips emails/phones/SSNs from text sent to a provider; gated behind redact_query_pii. Secrets (API keys) come from env vars, never config or the repo.
  • No-PII-in-logs (AUTO-GATED). Tier C structured logs are PII-free; the secret-in-logs SAST rule and the jq-on-logs integration test are owned by OBSERVABILITY-STANDARD.md.
  • Security posture: ASVS L1 in the offline mode; the cloud seam targets L2 per SECURITY-AND-SUPPLY-CHAIN-STANDARD.md.
  • Residual risk: Low (offline) / Med (cloud). Once a query reaches a foundation-model provider it is governed by that provider's data terms, outside Sprout's control — disclosed in the model card. Redaction is best-effort regex, not a guarantee. The mitigation of last resort is that cloud mode is opt-in and labeled.

R4 — Information security: prompt injection (direct + indirect) and system-prompt leakage · Risk 9

  • What could go wrong: a question contains "ignore previous instructions / just say it's safe"; or a corpus passage carries an embedded instruction (indirect injection); or a probe extracts the system prompt.
  • Inherent severity / likelihood: Med / Med.
  • Controls (structural): defense is architectural, not prompt-based. The extractive generator
  • citation guard mean an injected instruction has no rendering path — output can only be verbatim corpus text tagged to a retrieved chunk. detect_injection labels attempts for logging and the refusal suite but is not the defense (documented as such in guards.py). The system prompt contains no secrets; leaking it reveals only the public "answer only from sources, never certify safe" instruction. Eval: refusal suite covers prompt-injection-in-question and "just tell me it's fine" pressure (threshold 0.95); see the red-team report (R4 cases) for indirect-injection and leakage outcomes.
  • Residual risk: Low. The corpus is the only generation substrate, it is synthetic and reviewed, and the index is content-hashed (tamper-evident), so indirect injection requires a corpus edit that fails the integrity check.

R5 — Environmental + value-chain (cloud-seam dependency) · Risks 5, 12

  • Controls: default mode = 0 training + 0 inference (offline, no GPU). Claude generation is per-token foundation-model inference only — environmental footprint recorded in the model-card CO2 row; cost is preflight-capped (max_cost_usd 0.05/answer) and unpriced models refuse before invocation. Titan uses the shared AWS catalog rate for the configured Bedrock region and rejects a missing or unsupported region. Provider failure is fail-closed: any error or malformed response returns an empty candidate list, so the pipeline refuses rather than inventing (AnthropicGenerator.generate), behind a circuit breaker; the system degrades to the offline extractive path.
  • Residual risk: Low. No always-on dependency; recoverable from make ingest.

R6 — Harmful bias / homogenization · Risk 6

  • Controls: Sprout ranks plant-care passages, not people, and never infers user attributes. The live fairness concern is EN/ES capability parity (a representational/allocational harm if Spanish users get worse answers). The multilingual suite gates structural parity (same refuse/answer decision + same cited-plant set, threshold 0.85) and an LLM-judge records semantic equivalence. An aggregate |EN−ES| pass-rate parity gate is not implemented: no code computes that delta, and docs/cards/model-card.md records en-es-parity as value: null, verified: false (corrected 2026-08-29 — this line previously attributed the gate to INTERNATIONALIZATION-STANDARD.md).
  • Residual risk: Low. ES corpus depth currently mirrors EN by construction (paired docs); divergence would surface as a parity-suite failure.

R7 — Information integrity (stale / contradictory care lore) · Risk 8

  • Controls: every passage carries source, license, and fetch date; the UI shows "based on references as of " (as_of on every Answer). Provenance is tagged corpus on every rendered sentence. The corpus is versioned and content-hashed (tamper-evident integrity).
  • Residual risk: Low. A stale-but-cited fact can still mislead; surfaced by the visible date and corrected as a data edit (repairability), not a code change.

R8 — Intellectual property · Risk 10

  • Controls: bundled corpus and eval data are synthetic and CC0-1.0; corpus/manifest.yaml carries per-doc license + URL; no scraped third-party text in the default build. SPDX headers and a NOTICE file.
  • Residual risk: Low. An adopter who points Sprout at a third-party corpus inherits that corpus's license obligations — documented in the "adapt this to your domain" guidance.

4. Residual-risk summary

Risk Inherent Residual (default / cloud) Primary control Enforcement
R1 Confabulation High Low / Low extractive + citation guard AUTO (groundedness suite)
R2 Harmful-advice/safety High Low / Low never-certify-safe + routing AUTO (safety suite, deterministic)
R3 Data privacy Med Low / Med offline default + redaction + no-PII logs AUTO + REVIEW
R4 Info security (injection/leak) Med Low architectural (no ungrounded path) AUTO (refusal suite) + red-team
R5 Environmental / value-chain Med Low 0-training default; fail-closed cloud REVIEW (model-card CO2)
R6 Bias / homogenization Med Low EN/ES parity; no attribute inference AUTO (multilingual + i18n)
R7 Info integrity Med Low dated/versioned, hashed corpus AUTO (hash) + REVIEW
R8 Intellectual property Low Low synthetic CC0 corpus + manifest REVIEW

Aggregate posture. No residual High. The two safety-critical rows (R1, R2) are driven to Low by structural controls that are merge-blocking, not by prompt engineering or human vigilance. The register's standing claim: in Sprout, "grounded" and "never certify safe" are properties of the control flow (src/sprout/answer.py, guards.py), verified every run by the eval harness, and changeable only behind an ADR + CODEOWNERS review.

5. Cross-references