Luigit
repositories / smith

smith

There are many coding harnesses - but this one is fast

owned by admin

.system/plans/SMH-PLAN-99TPLN9I-recorded-context-and-dispatch-provenance/index.md

Raw
Rendered preview

id: SMH-PLAN-99TPLN9I type: plan title: "Recorded Context and Dispatch Provenance" spec: SMH-SPEC-SPEC0001 status: draft depends_on: [SMH-PLAN-KFFL0_FT, SMH-PLAN-VOETK_DD, SMH-PLAN-PLUG0001]

Recorded Context and Dispatch Provenance

Outcome

An edited, resumed, or routed turn exposes exactly the secret-safe context and physical dispatch used by each provider attempt. Raw history remains intact; replay uses recorded inputs; usage follows physical work once.

Scope

Implement the amended Smith Specification: Sessions → Effective request context and Context edits; Providers → Dispatch identity; Compaction, cost, and secrets → physical and nested accounting. Shared eval/RPC runtime integration belongs here; plugin authority and lifecycle arbitration belong to Plugin Tool Authority and Run Settlement.

Exclude classifier routing, virtual-model APIs, image operations, cache warming, QuickJS, MCP clients, Brush/Nushell facilities, vendor/OAuth migration, and general TUI work. Do not rewrite existing approved plans.

Entry contracts

  • Paged storage supplies complete-record publication, bounded readers, branch selection, observer enforcement, and inspection.
  • Auth and selection supplies explicit selection and secret-safe credential resolution.
  • Plugin harness supplies ordinary component loading, granted recorded effects, and atomic generations.
  • A real component provider supplied by the provider-plugin plan, or its ordinary conformance fixture, is required for provider-dependent binary proof. Compile and load that component through the ordinary host; a loopback fixture scripts transport bytes, never dispatch identities, generation pins, or provenance records. These are prerequisite contracts, not claims that their plans are complete. Missing contracts block dependent slices; do not replace them with native vendor dispatch or manufactured generation IDs.

Current seams: smith-core/src/session.rs, content_codec.rs, and store.rs supply typed records and bounded readers; agent.rs constructs requests from resident selected context. smith-ai/src/models.rs and mux.rs resolve catalog plans and establishment failover, but smith-harness/src/runtime.rs selects native transports and does not wire that router into the binary runtime. Current WIT is tool-only; provider, generation, and transform handoffs must be verified before consuming them.

Plan prerequisites and cross-spec interface handoffs are distinct.

flowchart TB
  S["KFFL0_FT: bounded storage"] --> C["99TPLN9I: recorded context and dispatch"]
  A["VOETK_DD: auth and selection"] --> C
  H["PLUG0001: ordinary plugin harness"] --> C
  P["AIAGD002: component provider contract"] -.-> C
  L["SLXZ2R9V B2: pinned loadout"] -.-> C
  U["SLXZ2R9V B3: causal child usage"] -.-> C

Behaviors

Each slice includes its required codecs, runtime integration, public-interface proof, and fresh scoped gates. Narrow pure tests establish rules; real files, components, sockets, and the built binary establish integration. Fixture creation may use public session APIs; fixture injection is not a production command or tool.

B1 · Edit future context without rewriting history

Append one validated omit/replace batch through the public session mutation API; the next resumed provider request sees the edited content while raw inspection retains the original message. Touch smith-core/src/session.rs, content_codec.rs, store.rs, agent.rs, and a narrow new projection module.

  • Encode one bounded edit batch as one typed semantic record; validate before append, publish only after durable acknowledgement.
  • Resolve targets by durable identity on the selected ancestry; preserve target role, identity, lineage, raw bytes, effects, and usage.
  • Validate the resulting complete context, including tool-call/result pairing and ordering; reject unknown or off-branch targets and invalid batches without changing the prior projection.
  • Use the projection in actual request construction, not only an inspection helper.

Proof: public API appends to a real session; built smith eval --session dispatches the replacement through a component provider to a scripted loopback endpoint; sessions get still exposes original content. Invalid paired-tool edits, oversized batches, read-only access, interrupted append, and persistence failure preserve the previous published projection.

B2 · Preserve projection across branch, compaction, and reopen

Navigate before an edit, fork an unrelated branch, compact retained context, and reopen; each request follows only its selected history. Touch the same projection path plus current compaction behavior in smith-core/src/agent.rs, trace.rs, and smith-harness/src/inspection.rs. An extracted compaction module would be new, not an existing interface.

  • Apply the latest applicable edit after the compaction boundary; never resurrect discarded targets or regenerate a summary implicitly.
  • Traverse bounded durable metadata/chunks; decode retained context, not unrelated branches or a mandatory whole-history index.
  • Carry any projection checkpoint needed across compaction as recorded data; keep it reconstructible from authoritative entries.
  • Preserve unsupported records verbatim; fail explicitly if required edit semantics are unknown.

Proof: branch/fork/compaction fixtures exercised through resumed eval and raw inspection; navigation before the edit restores the earlier projection. Property tests cover ancestry isolation, latest-edit precedence, deterministic reopen, and rejected-batch invariance. Allocation/working-set tests vary irrelevant history while holding effective context fixed; account for retained effective context separately.

B3 · Inspect the exact request that crossed the provider boundary

Before every provider effect, durably capture its effective instructions, ordered messages, advertised definitions, accepted transformations, and generation provenance. Touch smith/src/provider.rs, smith-core/src/agent.rs, projection/codec/store/trace paths, smith-harness/src/runtime.rs, inspection.rs, and smith-cli/src/cli.rs.

  • Give the logical provider round and each attempt stable recorded identities; anchor projection to the selected leaf before provenance append.
  • Record accepted prompt/context outputs and runtime prompt/tool changes; never reconstruct historical requests by rerunning transforms or reading current configuration.
  • Take advertised definitions and generation identity from the B2 pinned-loadout handoff in the companion plan, not an independently sampled registry.
  • Persist a bounded self-contained request representation, with references only to immutable recorded inputs sufficient for offline reconstruction.
  • Redact registered secrets and retain secret references before persistence; credentials cannot appear in reconstruction, export, diagnostics, or replay.
  • Add bounded request selection to existing sessions get human/JSONL inspection; distinguish raw transcript from effective request context.
  • Keep current configuration unchanged; later runtime changes affect later requests only.

Proof: change prompt/tool configuration and reload between requests; inspect and ordinarily replay the earlier request with no credentials or network and obtain its original context and definitions. A stalled-provider signal lets a test submit a later change and prove the already-started request is unchanged. Intent/provenance append failure prevents dispatch; unknown required provenance fails explicitly. Ordinary context reconstruction does not execute prompt, context, or routing handlers; provider byte-decoder replay retains its existing pure-codec contract.

B4 · Record selection separately from every physical attempt

Resolve a selected alias/group offline, validate each candidate's physical capabilities, and record bounded retry/failover decisions before dispatch. Touch smith-ai/src/models.rs, mux.rs, smith/src/provider.rs, smith-harness/src/runtime.rs, and request/provenance paths.

  • Wire existing deterministic resolution and establishment-fault policy into the shared runtime; keep vendor transport/codec ownership in ordinary provider components.
  • Record selection, physical provider/model, account-bucket provenance, effective thinking level, routing source, catalog capabilities, and generation for every dispatched attempt.
  • Capture routing inputs/results before dispatch; external routing information must arrive through granted recorded effects.
  • Keep the initial round's generation, advertised definitions, and executable handlers through retries, failover, response, and complete tool-result batch.
  • Later attempts record their own physical decisions; a later round may acquire a replacement after preceding work settles.
  • Do not replay a partially consumed stream or completed/uncertain effects automatically.

Proof: two scripted loopback endpoints behind a real provider component; primary rate-limit/transient faults, backup success, physical capability mismatch, auth/invalid-request faults, and cancellation expose ordered attempt records. Reload during establishment retry and during tool execution cannot change that round's handlers; the next round can see the replacement. The selected label remains unchanged; unsupported physical-model options fail before its transport effect. Replay preserves recorded decisions without network resolution or current catalog/configuration.

B5 · Attribute physical and child usage once

Inspect totals for direct, retried, routed, compaction/summary, and nested work without charging parent reports twice or inventing zero for missing usage. Touch current accounting in smith-core/src/agent.rs, session/provenance codecs, smith/src/stream.rs, and existing eval/RPC result serialization. An extracted accounting module would be new, not an existing interface.

  • Associate each usage observation with its recorded operation/attempt, causal parent, physical model, and pricing inputs actually used.
  • Count observed work even when a later attempt or parent fails; do not infer usage for an attempt that reported none.
  • Consume companion B3 child-work attribution; parent summaries reference child observations rather than creating duplicate observations.
  • Preserve unknown usage and pricing explicitly through aggregates, inspection, replay, eval JSON, and RPC results.

Proof: scripted component provider and child effects report known-zero, missing, partial, and repeated referenced observations; arithmetic and order-independent deduplication properties hold. Binary eval/RPC/inspection agree with replayed totals after catalog price changes; physical backup pricing wins over alias pricing. Context edits and parent failure cannot erase recorded usage.

Behavior dependencies

Solid edges order local behavior; dashed edges identify companion interface handoffs, not whole-plan cycles.

flowchart LR
  B1["B1: durable edit in next request"] --> B2["B2: branch and compaction projection"]
  B2 --> B3["B3: recorded effective request"]
  B3 --> B4["B4: physical attempt provenance"]
  B4 --> B5["B5: once-only usage"]
  L["SLXZ2R9V B2: pinned loadout"] -.-> B3
  U["SLXZ2R9V B3: child attribution"] -.-> B5
  B1 -.-> E["SLXZ2R9V B4: context-edit actions"]

Interface ownership

Shared vocabulary carries immutable request/attempt identities and snapshots; core owns projection, append validation, reconstruction, and accounting. AI owns catalog resolution and physical capability validation; harness owns recorded plugin/effect execution and feeds ordered decisions into core. Frontends serialize the same core outputs; RPC never acquires a dependency on the harness crate. New projection/provenance modules are implementation candidates, not claims about existing APIs.

Library arrows point from consumer to dependency; core receives shared records, never harness types.

flowchart TB
  CLI["smith-cli: eval and request inspection"] --> H["smith-harness: ordinary plugin and effect runtime"]
  CLI --> R["smith-rpc: shared result serialization"]
  H --> C["smith-core: projection, records, accounting"]
  H --> A["smith-ai: resolution and validation"]
  R --> C
  C --> V["smith: request and attempt vocabulary"]
  A --> V
  R --> V

Verification and stop conditions

  • Every changed test file names SMH-SPEC-SPEC0001; companion policy tests name their owning spec.
  • Run scoped cargo x check, cargo x e2e, cargo x doc, and architecture gates after each integrated slice; run full relevant gates before completion.
  • Pin human/wire output with reviewed snapshots; typed codec roundtrips and projection/accounting laws use property tests.
  • Extend exact type/allocation bounds and run scoped mutation tests over projection, attempt classification, and deduplication.
  • New golden records require a deliberate frame-version decision; no silent migration or regeneration.
  • Keep existing instruction pins; profile regressions before optimizing; never silently re-pin or claim speed from implementation shape.
  • Stop if exact historical reconstruction needs current configuration, fresh transforms, secret plaintext, unrelated payload residency, or an undocumented semantic assumption.
  • Stop if a provider effect can start before provenance/intent publication, retries lose the round pin, or uncertain effects are replayed.
  • Stop if required plugin contracts are missing; do not broaden this plan into the harness/provider migration or introduce privileged fallback.

Draft creation proves no implementation state.

---
id: SMH-PLAN-99TPLN9I
type: plan
title: "Recorded Context and Dispatch Provenance"
spec: SMH-SPEC-SPEC0001
status: draft
depends_on: [SMH-PLAN-KFFL0_FT, SMH-PLAN-VOETK_DD, SMH-PLAN-PLUG0001]
---

# Recorded Context and Dispatch Provenance

## Outcome

An edited, resumed, or routed turn exposes exactly the secret-safe context and physical dispatch used by each provider attempt.
Raw history remains intact; replay uses recorded inputs; usage follows physical work once.

## Scope

Implement the amended [Smith Specification](../../specs/SMH-SPEC-SPEC0001-smith-specification/index.md): Sessions → Effective request context and Context edits; Providers → Dispatch identity; Compaction, cost, and secrets → physical and nested accounting.
Shared eval/RPC runtime integration belongs here; plugin authority and lifecycle arbitration belong to [Plugin Tool Authority and Run Settlement](../SMH-PLAN-SLXZ2R9V-plugin-tool-authority-and-run-settlement/index.md).

Exclude classifier routing, virtual-model APIs, image operations, cache warming, QuickJS, MCP clients, Brush/Nushell facilities, vendor/OAuth migration, and general TUI work.
Do not rewrite existing approved plans.

## Entry contracts

- [Paged storage](../SMH-PLAN-KFFL0_FT-borrowed-codecs/index.md) supplies complete-record publication, bounded readers, branch selection, observer enforcement, and inspection.
- [Auth and selection](../SMH-PLAN-VOETK_DD-auth-store-and-provider-selection/index.md) supplies explicit selection and secret-safe credential resolution.
- [Plugin harness](../SMH-PLAN-PLUG0001-plugin-harness/index.md) supplies ordinary component loading, granted recorded effects, and atomic generations.
- A real component provider supplied by the [provider-plugin plan](../SMH-PLAN-AIAGD002-provider-and-oauth-plugins/index.md), or its ordinary conformance fixture, is required for provider-dependent binary proof.
Compile and load that component through the ordinary host; a loopback fixture scripts transport bytes, never dispatch identities, generation pins, or provenance records.
These are prerequisite contracts, not claims that their plans are complete.
Missing contracts block dependent slices; do not replace them with native vendor dispatch or manufactured generation IDs.

Current seams: `smith-core/src/session.rs`, `content_codec.rs`, and `store.rs` supply typed records and bounded readers; `agent.rs` constructs requests from resident selected context.
`smith-ai/src/models.rs` and `mux.rs` resolve catalog plans and establishment failover, but `smith-harness/src/runtime.rs` selects native transports and does not wire that router into the binary runtime.
Current WIT is tool-only; provider, generation, and transform handoffs must be verified before consuming them.

Plan prerequisites and cross-spec interface handoffs are distinct.

```mermaid
flowchart TB
  S["KFFL0_FT: bounded storage"] --> C["99TPLN9I: recorded context and dispatch"]
  A["VOETK_DD: auth and selection"] --> C
  H["PLUG0001: ordinary plugin harness"] --> C
  P["AIAGD002: component provider contract"] -.-> C
  L["SLXZ2R9V B2: pinned loadout"] -.-> C
  U["SLXZ2R9V B3: causal child usage"] -.-> C
```

## Behaviors

Each slice includes its required codecs, runtime integration, public-interface proof, and fresh scoped gates.
Narrow pure tests establish rules; real files, components, sockets, and the built binary establish integration.
Fixture creation may use public session APIs; fixture injection is not a production command or tool.

### B1 · Edit future context without rewriting history

Append one validated omit/replace batch through the public session mutation API; the next resumed provider request sees the edited content while raw inspection retains the original message.
Touch `smith-core/src/session.rs`, `content_codec.rs`, `store.rs`, `agent.rs`, and a narrow new projection module.

- Encode one bounded edit batch as one typed semantic record; validate before append, publish only after durable acknowledgement.
- Resolve targets by durable identity on the selected ancestry; preserve target role, identity, lineage, raw bytes, effects, and usage.
- Validate the resulting complete context, including tool-call/result pairing and ordering; reject unknown or off-branch targets and invalid batches without changing the prior projection.
- Use the projection in actual request construction, not only an inspection helper.

Proof: public API appends to a real session; built `smith eval --session` dispatches the replacement through a component provider to a scripted loopback endpoint; `sessions get` still exposes original content.
Invalid paired-tool edits, oversized batches, read-only access, interrupted append, and persistence failure preserve the previous published projection.

### B2 · Preserve projection across branch, compaction, and reopen

Navigate before an edit, fork an unrelated branch, compact retained context, and reopen; each request follows only its selected history.
Touch the same projection path plus current compaction behavior in `smith-core/src/agent.rs`, `trace.rs`, and `smith-harness/src/inspection.rs`.
An extracted compaction module would be new, not an existing interface.

- Apply the latest applicable edit after the compaction boundary; never resurrect discarded targets or regenerate a summary implicitly.
- Traverse bounded durable metadata/chunks; decode retained context, not unrelated branches or a mandatory whole-history index.
- Carry any projection checkpoint needed across compaction as recorded data; keep it reconstructible from authoritative entries.
- Preserve unsupported records verbatim; fail explicitly if required edit semantics are unknown.

Proof: branch/fork/compaction fixtures exercised through resumed eval and raw inspection; navigation before the edit restores the earlier projection.
Property tests cover ancestry isolation, latest-edit precedence, deterministic reopen, and rejected-batch invariance.
Allocation/working-set tests vary irrelevant history while holding effective context fixed; account for retained effective context separately.

### B3 · Inspect the exact request that crossed the provider boundary

Before every provider effect, durably capture its effective instructions, ordered messages, advertised definitions, accepted transformations, and generation provenance.
Touch `smith/src/provider.rs`, `smith-core/src/agent.rs`, projection/codec/store/trace paths, `smith-harness/src/runtime.rs`, `inspection.rs`, and `smith-cli/src/cli.rs`.

- Give the logical provider round and each attempt stable recorded identities; anchor projection to the selected leaf before provenance append.
- Record accepted prompt/context outputs and runtime prompt/tool changes; never reconstruct historical requests by rerunning transforms or reading current configuration.
- Take advertised definitions and generation identity from the B2 pinned-loadout handoff in the companion plan, not an independently sampled registry.
- Persist a bounded self-contained request representation, with references only to immutable recorded inputs sufficient for offline reconstruction.
- Redact registered secrets and retain secret references before persistence; credentials cannot appear in reconstruction, export, diagnostics, or replay.
- Add bounded request selection to existing `sessions get` human/JSONL inspection; distinguish raw transcript from effective request context.
- Keep current configuration unchanged; later runtime changes affect later requests only.

Proof: change prompt/tool configuration and reload between requests; inspect and ordinarily replay the earlier request with no credentials or network and obtain its original context and definitions.
A stalled-provider signal lets a test submit a later change and prove the already-started request is unchanged.
Intent/provenance append failure prevents dispatch; unknown required provenance fails explicitly.
Ordinary context reconstruction does not execute prompt, context, or routing handlers; provider byte-decoder replay retains its existing pure-codec contract.

### B4 · Record selection separately from every physical attempt

Resolve a selected alias/group offline, validate each candidate's physical capabilities, and record bounded retry/failover decisions before dispatch.
Touch `smith-ai/src/models.rs`, `mux.rs`, `smith/src/provider.rs`, `smith-harness/src/runtime.rs`, and request/provenance paths.

- Wire existing deterministic resolution and establishment-fault policy into the shared runtime; keep vendor transport/codec ownership in ordinary provider components.
- Record selection, physical provider/model, account-bucket provenance, effective thinking level, routing source, catalog capabilities, and generation for every dispatched attempt.
- Capture routing inputs/results before dispatch; external routing information must arrive through granted recorded effects.
- Keep the initial round's generation, advertised definitions, and executable handlers through retries, failover, response, and complete tool-result batch.
- Later attempts record their own physical decisions; a later round may acquire a replacement after preceding work settles.
- Do not replay a partially consumed stream or completed/uncertain effects automatically.

Proof: two scripted loopback endpoints behind a real provider component; primary rate-limit/transient faults, backup success, physical capability mismatch, auth/invalid-request faults, and cancellation expose ordered attempt records.
Reload during establishment retry and during tool execution cannot change that round's handlers; the next round can see the replacement.
The selected label remains unchanged; unsupported physical-model options fail before its transport effect.
Replay preserves recorded decisions without network resolution or current catalog/configuration.

### B5 · Attribute physical and child usage once

Inspect totals for direct, retried, routed, compaction/summary, and nested work without charging parent reports twice or inventing zero for missing usage.
Touch current accounting in `smith-core/src/agent.rs`, session/provenance codecs, `smith/src/stream.rs`, and existing eval/RPC result serialization.
An extracted accounting module would be new, not an existing interface.

- Associate each usage observation with its recorded operation/attempt, causal parent, physical model, and pricing inputs actually used.
- Count observed work even when a later attempt or parent fails; do not infer usage for an attempt that reported none.
- Consume companion B3 child-work attribution; parent summaries reference child observations rather than creating duplicate observations.
- Preserve unknown usage and pricing explicitly through aggregates, inspection, replay, eval JSON, and RPC results.

Proof: scripted component provider and child effects report known-zero, missing, partial, and repeated referenced observations; arithmetic and order-independent deduplication properties hold.
Binary eval/RPC/inspection agree with replayed totals after catalog price changes; physical backup pricing wins over alias pricing.
Context edits and parent failure cannot erase recorded usage.

## Behavior dependencies

Solid edges order local behavior; dashed edges identify companion interface handoffs, not whole-plan cycles.

```mermaid
flowchart LR
  B1["B1: durable edit in next request"] --> B2["B2: branch and compaction projection"]
  B2 --> B3["B3: recorded effective request"]
  B3 --> B4["B4: physical attempt provenance"]
  B4 --> B5["B5: once-only usage"]
  L["SLXZ2R9V B2: pinned loadout"] -.-> B3
  U["SLXZ2R9V B3: child attribution"] -.-> B5
  B1 -.-> E["SLXZ2R9V B4: context-edit actions"]
```

## Interface ownership

Shared vocabulary carries immutable request/attempt identities and snapshots; core owns projection, append validation, reconstruction, and accounting.
AI owns catalog resolution and physical capability validation; harness owns recorded plugin/effect execution and feeds ordered decisions into core.
Frontends serialize the same core outputs; RPC never acquires a dependency on the harness crate.
New projection/provenance modules are implementation candidates, not claims about existing APIs.

Library arrows point from consumer to dependency; core receives shared records, never harness types.

```mermaid
flowchart TB
  CLI["smith-cli: eval and request inspection"] --> H["smith-harness: ordinary plugin and effect runtime"]
  CLI --> R["smith-rpc: shared result serialization"]
  H --> C["smith-core: projection, records, accounting"]
  H --> A["smith-ai: resolution and validation"]
  R --> C
  C --> V["smith: request and attempt vocabulary"]
  A --> V
  R --> V
```

## Verification and stop conditions

- Every changed test file names `SMH-SPEC-SPEC0001`; companion policy tests name their owning spec.
- Run scoped `cargo x check`, `cargo x e2e`, `cargo x doc`, and architecture gates after each integrated slice; run full relevant gates before completion.
- Pin human/wire output with reviewed snapshots; typed codec roundtrips and projection/accounting laws use property tests.
- Extend exact type/allocation bounds and run scoped mutation tests over projection, attempt classification, and deduplication.
- New golden records require a deliberate frame-version decision; no silent migration or regeneration.
- Keep existing instruction pins; profile regressions before optimizing; never silently re-pin or claim speed from implementation shape.
- Stop if exact historical reconstruction needs current configuration, fresh transforms, secret plaintext, unrelated payload residency, or an undocumented semantic assumption.
- Stop if a provider effect can start before provenance/intent publication, retries lose the round pin, or uncertain effects are replayed.
- Stop if required plugin contracts are missing; do not broaden this plan into the harness/provider migration or introduce privileged fallback.

Draft creation proves no implementation state.