--- id: SMH-PLAN-99TPLN9I type: plan title: "Recorded Context and Dispatch Provenance" spec: SMH-SPEC-SPEC0001 status: draft depends_on: [SMH-PLAN-KFFL0_FT, SMH-PLAN-VOETK_DD, SMH-PLAN-PLUG0001] --- # Recorded Context and Dispatch Provenance ## Outcome An edited, resumed, or routed turn exposes exactly the secret-safe context and physical dispatch used by each provider attempt. Raw history remains intact; replay uses recorded inputs; usage follows physical work once. ## Scope Implement the amended [Smith Specification](../../specs/SMH-SPEC-SPEC0001-smith-specification/index.md): Sessions → Effective request context and Context edits; Providers → Dispatch identity; Compaction, cost, and secrets → physical and nested accounting. Shared eval/RPC runtime integration belongs here; plugin authority and lifecycle arbitration belong to [Plugin Tool Authority and Run Settlement](../SMH-PLAN-SLXZ2R9V-plugin-tool-authority-and-run-settlement/index.md). Exclude classifier routing, virtual-model APIs, image operations, cache warming, QuickJS, MCP clients, Brush/Nushell facilities, vendor/OAuth migration, and general TUI work. Do not rewrite existing approved plans. ## Entry contracts - [Paged storage](../SMH-PLAN-KFFL0_FT-borrowed-codecs/index.md) supplies complete-record publication, bounded readers, branch selection, observer enforcement, and inspection. - [Auth and selection](../SMH-PLAN-VOETK_DD-auth-store-and-provider-selection/index.md) supplies explicit selection and secret-safe credential resolution. - [Plugin harness](../SMH-PLAN-PLUG0001-plugin-harness/index.md) supplies ordinary component loading, granted recorded effects, and atomic generations. - A real component provider supplied by the [provider-plugin plan](../SMH-PLAN-AIAGD002-provider-and-oauth-plugins/index.md), or its ordinary conformance fixture, is required for provider-dependent binary proof. Compile and load that component through the ordinary host; a loopback fixture scripts transport bytes, never dispatch identities, generation pins, or provenance records. These are prerequisite contracts, not claims that their plans are complete. Missing contracts block dependent slices; do not replace them with native vendor dispatch or manufactured generation IDs. Current seams: `smith-core/src/session.rs`, `content_codec.rs`, and `store.rs` supply typed records and bounded readers; `agent.rs` constructs requests from resident selected context. `smith-ai/src/models.rs` and `mux.rs` resolve catalog plans and establishment failover, but `smith-harness/src/runtime.rs` selects native transports and does not wire that router into the binary runtime. Current WIT is tool-only; provider, generation, and transform handoffs must be verified before consuming them. Plan prerequisites and cross-spec interface handoffs are distinct. ```mermaid flowchart TB S["KFFL0_FT: bounded storage"] --> C["99TPLN9I: recorded context and dispatch"] A["VOETK_DD: auth and selection"] --> C H["PLUG0001: ordinary plugin harness"] --> C P["AIAGD002: component provider contract"] -.-> C L["SLXZ2R9V B2: pinned loadout"] -.-> C U["SLXZ2R9V B3: causal child usage"] -.-> C ``` ## Behaviors Each slice includes its required codecs, runtime integration, public-interface proof, and fresh scoped gates. Narrow pure tests establish rules; real files, components, sockets, and the built binary establish integration. Fixture creation may use public session APIs; fixture injection is not a production command or tool. ### B1 · Edit future context without rewriting history Append one validated omit/replace batch through the public session mutation API; the next resumed provider request sees the edited content while raw inspection retains the original message. Touch `smith-core/src/session.rs`, `content_codec.rs`, `store.rs`, `agent.rs`, and a narrow new projection module. - Encode one bounded edit batch as one typed semantic record; validate before append, publish only after durable acknowledgement. - Resolve targets by durable identity on the selected ancestry; preserve target role, identity, lineage, raw bytes, effects, and usage. - Validate the resulting complete context, including tool-call/result pairing and ordering; reject unknown or off-branch targets and invalid batches without changing the prior projection. - Use the projection in actual request construction, not only an inspection helper. Proof: public API appends to a real session; built `smith eval --session` dispatches the replacement through a component provider to a scripted loopback endpoint; `sessions get` still exposes original content. Invalid paired-tool edits, oversized batches, read-only access, interrupted append, and persistence failure preserve the previous published projection. ### B2 · Preserve projection across branch, compaction, and reopen Navigate before an edit, fork an unrelated branch, compact retained context, and reopen; each request follows only its selected history. Touch the same projection path plus current compaction behavior in `smith-core/src/agent.rs`, `trace.rs`, and `smith-harness/src/inspection.rs`. An extracted compaction module would be new, not an existing interface. - Apply the latest applicable edit after the compaction boundary; never resurrect discarded targets or regenerate a summary implicitly. - Traverse bounded durable metadata/chunks; decode retained context, not unrelated branches or a mandatory whole-history index. - Carry any projection checkpoint needed across compaction as recorded data; keep it reconstructible from authoritative entries. - Preserve unsupported records verbatim; fail explicitly if required edit semantics are unknown. Proof: branch/fork/compaction fixtures exercised through resumed eval and raw inspection; navigation before the edit restores the earlier projection. Property tests cover ancestry isolation, latest-edit precedence, deterministic reopen, and rejected-batch invariance. Allocation/working-set tests vary irrelevant history while holding effective context fixed; account for retained effective context separately. ### B3 · Inspect the exact request that crossed the provider boundary Before every provider effect, durably capture its effective instructions, ordered messages, advertised definitions, accepted transformations, and generation provenance. Touch `smith/src/provider.rs`, `smith-core/src/agent.rs`, projection/codec/store/trace paths, `smith-harness/src/runtime.rs`, `inspection.rs`, and `smith-cli/src/cli.rs`. - Give the logical provider round and each attempt stable recorded identities; anchor projection to the selected leaf before provenance append. - Record accepted prompt/context outputs and runtime prompt/tool changes; never reconstruct historical requests by rerunning transforms or reading current configuration. - Take advertised definitions and generation identity from the B2 pinned-loadout handoff in the companion plan, not an independently sampled registry. - Persist a bounded self-contained request representation, with references only to immutable recorded inputs sufficient for offline reconstruction. - Redact registered secrets and retain secret references before persistence; credentials cannot appear in reconstruction, export, diagnostics, or replay. - Add bounded request selection to existing `sessions get` human/JSONL inspection; distinguish raw transcript from effective request context. - Keep current configuration unchanged; later runtime changes affect later requests only. Proof: change prompt/tool configuration and reload between requests; inspect and ordinarily replay the earlier request with no credentials or network and obtain its original context and definitions. A stalled-provider signal lets a test submit a later change and prove the already-started request is unchanged. Intent/provenance append failure prevents dispatch; unknown required provenance fails explicitly. Ordinary context reconstruction does not execute prompt, context, or routing handlers; provider byte-decoder replay retains its existing pure-codec contract. ### B4 · Record selection separately from every physical attempt Resolve a selected alias/group offline, validate each candidate's physical capabilities, and record bounded retry/failover decisions before dispatch. Touch `smith-ai/src/models.rs`, `mux.rs`, `smith/src/provider.rs`, `smith-harness/src/runtime.rs`, and request/provenance paths. - Wire existing deterministic resolution and establishment-fault policy into the shared runtime; keep vendor transport/codec ownership in ordinary provider components. - Record selection, physical provider/model, account-bucket provenance, effective thinking level, routing source, catalog capabilities, and generation for every dispatched attempt. - Capture routing inputs/results before dispatch; external routing information must arrive through granted recorded effects. - Keep the initial round's generation, advertised definitions, and executable handlers through retries, failover, response, and complete tool-result batch. - Later attempts record their own physical decisions; a later round may acquire a replacement after preceding work settles. - Do not replay a partially consumed stream or completed/uncertain effects automatically. Proof: two scripted loopback endpoints behind a real provider component; primary rate-limit/transient faults, backup success, physical capability mismatch, auth/invalid-request faults, and cancellation expose ordered attempt records. Reload during establishment retry and during tool execution cannot change that round's handlers; the next round can see the replacement. The selected label remains unchanged; unsupported physical-model options fail before its transport effect. Replay preserves recorded decisions without network resolution or current catalog/configuration. ### B5 · Attribute physical and child usage once Inspect totals for direct, retried, routed, compaction/summary, and nested work without charging parent reports twice or inventing zero for missing usage. Touch current accounting in `smith-core/src/agent.rs`, session/provenance codecs, `smith/src/stream.rs`, and existing eval/RPC result serialization. An extracted accounting module would be new, not an existing interface. - Associate each usage observation with its recorded operation/attempt, causal parent, physical model, and pricing inputs actually used. - Count observed work even when a later attempt or parent fails; do not infer usage for an attempt that reported none. - Consume companion B3 child-work attribution; parent summaries reference child observations rather than creating duplicate observations. - Preserve unknown usage and pricing explicitly through aggregates, inspection, replay, eval JSON, and RPC results. Proof: scripted component provider and child effects report known-zero, missing, partial, and repeated referenced observations; arithmetic and order-independent deduplication properties hold. Binary eval/RPC/inspection agree with replayed totals after catalog price changes; physical backup pricing wins over alias pricing. Context edits and parent failure cannot erase recorded usage. ## Behavior dependencies Solid edges order local behavior; dashed edges identify companion interface handoffs, not whole-plan cycles. ```mermaid flowchart LR B1["B1: durable edit in next request"] --> B2["B2: branch and compaction projection"] B2 --> B3["B3: recorded effective request"] B3 --> B4["B4: physical attempt provenance"] B4 --> B5["B5: once-only usage"] L["SLXZ2R9V B2: pinned loadout"] -.-> B3 U["SLXZ2R9V B3: child attribution"] -.-> B5 B1 -.-> E["SLXZ2R9V B4: context-edit actions"] ``` ## Interface ownership Shared vocabulary carries immutable request/attempt identities and snapshots; core owns projection, append validation, reconstruction, and accounting. AI owns catalog resolution and physical capability validation; harness owns recorded plugin/effect execution and feeds ordered decisions into core. Frontends serialize the same core outputs; RPC never acquires a dependency on the harness crate. New projection/provenance modules are implementation candidates, not claims about existing APIs. Library arrows point from consumer to dependency; core receives shared records, never harness types. ```mermaid flowchart TB CLI["smith-cli: eval and request inspection"] --> H["smith-harness: ordinary plugin and effect runtime"] CLI --> R["smith-rpc: shared result serialization"] H --> C["smith-core: projection, records, accounting"] H --> A["smith-ai: resolution and validation"] R --> C C --> V["smith: request and attempt vocabulary"] A --> V R --> V ``` ## Verification and stop conditions - Every changed test file names `SMH-SPEC-SPEC0001`; companion policy tests name their owning spec. - Run scoped `cargo x check`, `cargo x e2e`, `cargo x doc`, and architecture gates after each integrated slice; run full relevant gates before completion. - Pin human/wire output with reviewed snapshots; typed codec roundtrips and projection/accounting laws use property tests. - Extend exact type/allocation bounds and run scoped mutation tests over projection, attempt classification, and deduplication. - New golden records require a deliberate frame-version decision; no silent migration or regeneration. - Keep existing instruction pins; profile regressions before optimizing; never silently re-pin or claim speed from implementation shape. - Stop if exact historical reconstruction needs current configuration, fresh transforms, secret plaintext, unrelated payload residency, or an undocumented semantic assumption. - Stop if a provider effect can start before provenance/intent publication, retries lose the round pin, or uncertain effects are replayed. - Stop if required plugin contracts are missing; do not broaden this plan into the harness/provider migration or introduce privileged fallback. Draft creation proves no implementation state.