id: SMH-RESEARCH-EVY30PHD
type: research
title: "Pi Durable as a Smith API Foundation"
Pi Durable as a Smith API Foundation
Answer
Candidate: durable kernel as Smith's shared foundation; compact agent facade above it; no second runtime beside it.
“Bottom” means below agent policy, not below binary framing, storage, sandbox, or effect execution.
Pi's layering transfers; its TypeScript/Chord implementation and exact recovery semantics do not.
Current Smith already contains the durable-history foundation, but not Pi's atomic multi-record/state/task kernel.
This is substantive architecture work, not a rename or free facade.
Scope
Checked 2026-10-11.
Pi target: 1.1.0, revision abe508e1b89912adde45528136c3221eb69acdd7.
Continuation of release-delta research.
Evidence: pinned README, exports, selected source, scoped normative sections; current Smith source and specs.
Upstream task/storage tests exist but were not executed; neither implementation received a full correctness audit.
The 246 KB upstream specification was inspected selectively, not completely.
Its “Pico5 is not implemented yet” sentence conflicts with concrete exported implementation; implementation evidence takes precedence for implemented behavior.1234
No code copied, performance claim, production edit, or spec amendment.
What is actually simpler?
Pi's happy path is compact: open harness → acquire conversation → submit → await receipt.5
The reduction is conceptual: one durable authority supports configuration, work, observations, and recovery.
It is not evidence of fewer total API declarations, less implementation code, lower memory, or higher speed than Smith.
Root exports include document families, transactions, task runtime, storage writes, registry, generation, compaction, tools, events, views, and errors.1
Broad machinery is hidden from ordinary callers, not eliminated.
Pi concept
Responsibility
Generality
Session / Tx / Storage
One serialized atomic mutation line and committed publication
Reusable nucleus, still carries conversation/entry/task vocabulary
Entry
Immutable history with optional model contribution
HarnessImpl extends SessionImpl and gives its scheduler the same Session.
Generation and tool execution are built-in durable tasks over that authority.367
Thus the generic kernel genuinely underlies the agent facade; it is not a sidecar database.
A bare createSession() supports transactions/state but does not itself provide the Harness scheduler.
Current scheduler implementation lives inside harness/; clean conceptual layers do not mean independent exported packages.
Pi's core design
One commit authority
Session holds its mutation line through callback, preparation, storage settlement, adoption, publication.
External effects run outside that line.24
A transaction can atomically change entries, documents, task checkpoint, ownership, and submission state.
Mutable drafts expire when the transaction callback settles; prepared state is immutable.
Storage succeeds before adopted state reaches commit observers.
Uncertain storage failure poisons the open Session; explicit StorageRejected alone proves no batch effect committed and permits continued use.2
Durability of a backend acknowledgement remains distinct from power-loss durability: SQLite NORMAL and optional JSONL fsync do not establish Smith's persistence boundary.5
Tasks are durable control, not persisted threads
Task phases checkpoint before/after effects; waiting tasks park without executing code.
States: pending, running, waiting, completing, terminal.
completing holds an outcome until foreground-owned work drains.
Terminal receipts remain queryable; task-scoped documents retire at terminal transition.64
History ancestry and work ownership are independent relationships.
Background work forms an explicit ownership boundary, rather than accidentally escaping a parent wait.
Cancelling a waiter stops that wait, not shared work; aborting work records intent and signals execution.
Closing stops invocations without declaring them completed; reopening reconciles running tasks to pending.
Missing/incompatible task code blocks execution rather than erasing durable work.
Effect safety is narrower than task durability
Submission requestId deduplicates admission within one conversation, not external side effects.54
Tool intent stores final arguments and replay policy before execution.
Recovery reruns only when both persisted and currently resolved tool policies say safe.
Otherwise ToolTask emits an interrupted error explaining that the tool may have partially run.7
Generation/custom task recovery has its own effect protocol; the scheduler cannot infer whether an external operation occurred.
Therefore “all interrupted effects retry” is false; “all effects are exactly once” is also false.
Task outcome and external mutation evidence are different dimensions.
Pi's task outcome variants are not a replacement for Smith's applied / not-applied / unknown effect record.
State and observation
Documents declare session/conversation/task scope; conversation documents also declare latest/rewindable history and fresh/current/as-of fork behavior.8
These are persisted semantic lifetimes, not allocator scopes.
Chord derives immutable values and operation batches from JSON drafts; docs preserve missing definitions and migrate on typed access.
Watch attachment captures baseline plus subscription atomically.
Slow observers retain at most 100 pending frames, then converge through a newest-state replacement.9
Conversation views mount active entries plus built-in documents; agent events are another derived view.10
A convergence watch can skip intermediate publications; durable history must answer exact replay/audit needs separately.
An item-count backlog bound does not bound snapshot bytes or total loaded state.
Important limits
Task records persist kind/version, not an immutable plugin generation.
Registry snapshots stabilize a task phase, then may refresh at later phases or restart.611
Tool implementation and cwd resolve for execution; changes can affect calls already emitted by a model.57
Ordinary extensions are native host code, not sandboxed guests.
Session document cache has no reviewed general byte/resident-item ceiling; idle context TTL is not an aggregate memory guarantee.25
Environment abstraction and conformance make execution replaceable, but do not themselves authorize effects.12
Current Smith: genuine substrate, incomplete general API
Existing symbol
Implemented seam
Missing relative to candidate
Session::append_with
Encode entry; persist callback before resident publication; rollback staged payload on failure
Atomic multi-record semantic commit, task/submission state
append_with is one entry, not a transaction over entry + checkpoint + queue + state.
Tool result and mutation metadata are separate appends; failure can leave partial semantic completion evidence.
Current Agent owns provider polling and tool execution; it is not yet the specified pure transition engine.
Public StreamFn exposes a futures stream; Agent consumes it through block_on_stream, contrary to the current no-futures core boundary law.
Runtime assembly still contains native vendor selection; ToolSession still contains built-in name dispatch.
Resident Session stores branch vectors and payload bytes; bounded reader APIs do not make every resident projection independently bounded.
These are source observations, not newly created issues or authorization to repair them.1314
“Use existing substrate” does not mean “existing implementation already satisfies all foundation laws.”
Natural Smith structure
Candidate logical layers preserve current crate roles; no new crate or new build system is implied.
Arrows below mean “uses”, not execution order.
Durable intent, causal parent, pins, completion/mutation state
Missing completion remains unknown; no automatic retry
Scoped state
Bounded typed updates and plugin payloads
Host-owned, branch/fork semantics explicit, no second log
Read view
Bounded immutable chunks and historical projection
Borrow in flight, validated payloads, explicit cursor limits
Subscription
Sequence-tagged updates and recovery position
Persisted transcript recoverable; transient progress may coalesce
“State” need not become a generic JSON database.
Known core state remains typed binary records; plugin-defined state can use bounded CBOR payloads with declared ownership/version.
State snapshots and deltas remain records in the same session authority; reconstructible caches are not a second source of truth.
Small bounded blobs may need only replacement/checkpoint records, not Chord-style arbitrary tree patches.
Session/branch/work state policy is distinct from process/session/turn/frame allocation accounting.
Frame scratch, VM-local scratch, and render caches are not made durable just because the API is general.
Agent policy above foundation
Agent becomes one consumer of recorded inputs and session state, not owner of storage/executor machinery.
It decides effective request, provider round, tool batch, queue delivery, compaction, continuation, settlement.
It emits typed changes and effect plans; host feeds recorded completions back in order.
Provider and tool work can share durable ownership/effect mechanisms without sharing domain outcome enums.
No persistent VM stack, Rust thread, closure, or Future is the checkpoint.
Checkpoint data describes the next valid step and pinned provenance; host reconstructs a fresh invocation.
A useful general core is still allowed to have typed agent-specific records above its neutral primitives.
Small public facade above foundation
Candidate host vocabulary: open/attach, submit, inspect, observe, abort, fork, close.
Submit returns a durable work/submission identity after admission; inspect/observe report settlement later.
Eval may offer a blocking “submit then wait” convenience while RPC/TUI use the same admitted operation.
RPC request IDs correlate transport messages; durable submission keys identify retried admission.
Process reopen reacquires receipts by durable ID; handles are not durable state.
Cancelling observation does not cancel work; abort is a separate authorized command.
This facade belongs naturally in smith-harness over shared smith-core state.
Shared IDs and message vocabulary belong in smith; schema/codec/reduction in smith-core.
Storage/executor threads remain host orchestration; abstract UI belongs in smith-ui; transports retain frontend roles.
Existing crate dependency constraints remain in force.
Conveniences should compose these operations, not implement another agent loop.
A typed Rust facade and WIT resources may express the same semantics without mirroring each other's syntax.
No async callbacks, Chord types, mutable transaction drafts, or ambient ExecutionEnv cross Smith's core/WIT boundary.
Plugin API is a constrained view, not raw database access
Guest sees per-call granted resources for state, work, and effects, with explicit bounds and provenance.
Host owns persistence, IDs, ordering, generation pin, budgets, and authoritative terminal outcomes.
Plugins request typed changes; they cannot rewrite arbitrary core task records or claim an effect succeeded.
Pure state changes can roll back before admission; external effects cannot roll back merely because later guest code fails.
Child composition shares caller authority/budget/generation and drains before handler return, as currently specified.
A durable background task would require an explicit independent ownership/admission contract; it cannot be hidden behind ordinary child work.
A general work model does not require exposing arbitrary workflow creation to every plugin.
Effect loop: one history, executor outside core
Candidate ordering retains Smith's effect evidence rather than importing Pi's replay-safe switch.
sequenceDiagram
participant K as Kernel
participant H as Host
participant S as Store
K->>H: Intent change
H->>S: Persist intent
S-->>H: Durable ack
H-->>K: Intent admitted
K->>H: Effect plan
H-->>K: Outcome observation
K->>H: Completion change
H->>S: Persist completion
S-->>H: Durable ack
H-->>K: Completion admitted
Host dispatch is eligible only after durable intent acknowledgement.
Core stages a pure completion transition from the observation; the bounded outcome input joins that completion commit before authoritative state adoption/publication.
Core never blocks render or performs network/filesystem effects inside a state transition.
A crash after dispatch but before completion leaves unknown evidence; reopening does not blindly re-execute the effect.
Explicit reconciliation may record an external receipt or user decision before workflow continuation.
Deterministic replay consumes prior inputs/effect evidence; recovery decides what uncompleted work may safely do.
These are different operations.
Foundation versus extra
Choice
Result
Assessment
Pi-like facade only over current Agent
Easier embedding, same lifecycle/storage gaps
Useful thin convenience, not durable foundation
Second JSON/doc/task runtime beside Smith sessions
One log/evidence authority; modes and plugins compose it
Preferred candidate
Full general workflow engine exposed immediately
New task DSL, migrations, document DB, scheduler policy
Unjustified breadth without consumers
Optional high-level workflow/agent package over same kernel
Extra convenience without extra persistence authority
Compatible if later consumers earn it
Base mechanisms can be foundation; high-level workflows may be extra; storage authority must not be extra.
No need to create Pi's enclosing multi-conversation Session table just to gain this shape.
Smith session identity can remain the durable file/home boundary; branches remain history choices, not parallel running conversations.
Independent concurrent agents can retain distinct Smith sessions.
Ownership across sessions does not grant cross-file atomic commits, writable fork aliasing, or distributed merge.
If a real workflow needs those semantics, they need separate explicit design rather than a renamed Conversation handle.
Compatibility with current Smith laws
Smith law
Pi transfer
Required adaptation
Binary self-contained append-only file
One transaction authority
New grouped-record semantics must survive torn/corrupt tails and unknown frames
Complete visibility differs from durable ack
Commit before publication
Expose exact acknowledgement level; do not equate flush with durability
Pure core over recorded ordered inputs
Durable checkpoints
Synchronous transitions, typed channels, injected recorded time/config
Effects recover unknown, never auto-retry
Intent/effect/outcome sandwich
Keep three-state evidence; safe replay declaration is not authority
Request generation/loadout pinned through tool batch
Registry snapshots
Record/retain immutable generation plus grants and request provenance
Host-owned state, metered wasm
Scoped documents/task control
Resources and declarative requests, not native closures or unrestricted drafts
Bounded working sets
Snapshot/delta observation
Bound bytes/items/depth; chunk/replay durable transcript, do not send whole-history snapshots
Remote sequence resume
Convergent watch
Recovery from durable sequence; snapshot cannot replace missing transcript history
Fleet home owns writes
Single mutation line
Commands route to home; generations content-hashed; reads remain replicas
Queue preserved on abort unless explicit action
Durable cancellation
Do not import Pi's queued-input withdrawal default
Current single-entry frames do not prove multi-record atomicity.
A bounded group frame or explicit commit envelope are candidate encodings, not chosen formats.
Readers, writer acknowledgements, recovery, unknown-record roundtrips, checkpoints, and observation must agree on the same commit boundary.
Appending several existing frames and delaying UI publication is insufficient for crash-recoverable semantic atomicity.
Generation pinning of a provider round is already required; arbitrary custom multi-phase workflows would need their own pin/rebind contract.
Do not silently promise permanent artifact availability across restart.
Conclusions
Pi demonstrates a real durable nucleus beneath an approachable agent API.
Smith can naturally use that pattern as foundation, with its existing binary session/effect system as substrate.
Host happy-path simplicity comes from coherent authority and receipts, not generalizing every feature into tasks or JSON documents.
A small facade can exist before a complete durable work runtime, but must not claim crash-resumable submissions it cannot reconstruct.
Agent/effect composition offers immediate consumers; a full workflow/document engine remains optional breadth.
Exact API names, record formats, generic task exposure, and crate restructuring remain candidates, not mandates.
Human approved later relevant spec revisions referencing this research on 2026-10-11.
Current specs and implementation unchanged.
Relevant existing contracts: SMH-SPEC-SPEC0001, SMH-SPEC-PLUG0001, SMH-SPEC-WASM0001, SMH-SPEC-RMTF0001, SMH-SPEC-FLET0001.
Unresolved questions
Is first public consumer embedding Rust, frontend protocol, or WIT composition; which operations are truly shared?
What bounded semantic group must commit atomically: effect result + evidence, queue admission + work, plugin state + checkpoint?
Does durable work need an extensible generic task definition now, or typed provider/tool/compaction checkpoints first?
Which plugin state scopes need historical rewind/fork behavior versus fresh state?
How are work checkpoints resumed when the original component/grant revision is unavailable, without silent rebinding?
Which unknown effects can be reconciled by recorded external receipts, and which require human action?
What sequence/cursor identifies a committed view across branching, maintenance, reconnect, and fleet replication?
Can bounded snapshot plus paged transcript recovery satisfy both frontend convergence and exact event resumption?
Which state/concurrency guarantees belong to one session file versus a host coordinating independent sessions?
Sources
Pinned upstream sources at abe508e1b89912adde45528136c3221eb69acdd7:
---
id: SMH-RESEARCH-EVY30PHD
type: research
title: "Pi Durable as a Smith API Foundation"
---
# Pi Durable as a Smith API Foundation
## Answer
Candidate: **durable kernel as Smith's shared foundation; compact agent facade above it; no second runtime beside it.**
“Bottom” means below agent policy, not below binary framing, storage, sandbox, or effect execution.
Pi's layering transfers; its TypeScript/Chord implementation and exact recovery semantics do not.
Current Smith already contains the durable-history foundation, but not Pi's atomic multi-record/state/task kernel.
This is substantive architecture work, not a rename or free facade.
## Scope
Checked 2026-10-11.
Pi target: 1.1.0, revision `abe508e1b89912adde45528136c3221eb69acdd7`.
Continuation of [release-delta research](../SMH-RESEARCH-6QMOC6X4-pi-1-0-0-to-1-1-0-features-and-architecture/index.md).
Evidence: pinned README, exports, selected source, scoped normative sections; current Smith source and specs.
Upstream task/storage tests exist but were not executed; neither implementation received a full correctness audit.
The 246 KB upstream specification was inspected selectively, not completely.
Its “Pico5 is not implemented yet” sentence conflicts with concrete exported implementation; implementation evidence takes precedence for implemented behavior.[^pi-index][^pi-session][^pi-harness][^pi-spec]
No code copied, performance claim, production edit, or spec amendment.
## What is actually simpler?
Pi's happy path is compact: open harness → acquire conversation → submit → await receipt.[^pi-readme]
The reduction is conceptual: one durable authority supports configuration, work, observations, and recovery.
It is not evidence of fewer total API declarations, less implementation code, lower memory, or higher speed than Smith.
Root exports include document families, transactions, task runtime, storage writes, registry, generation, compaction, tools, events, views, and errors.[^pi-index]
Broad machinery is hidden from ordinary callers, not eliminated.
| Pi concept | Responsibility | Generality |
| --- | --- | --- |
| Session / Tx / Storage | One serialized atomic mutation line and committed publication | Reusable nucleus, still carries conversation/entry/task vocabulary |
| Entry | Immutable history with optional model contribution | Transcript-aware, not completely model-neutral |
| Document | Typed definition, JSON state, lifetime, version, fork policy | General application state |
| Task | Checkpointed phase machine with owned work and terminal receipt | General durable work |
| Harness | Registry, scheduler, built-in agent tasks, conversation conveniences | Agent product runtime |
| Conversation | Input, config, context, fork, reset, compaction, abort, watch | Agent-facing scope |
| Submission | Durable admission and later settlement | Agent input/write workflow |
| Watch | Committed snapshot plus later frames | Convergent observation, not audit log |
`HarnessImpl` extends `SessionImpl` and gives its scheduler the same Session.
Generation and tool execution are built-in durable tasks over that authority.[^pi-harness][^pi-scheduler][^pi-tool]
Thus the generic kernel genuinely underlies the agent facade; it is not a sidecar database.
A bare `createSession()` supports transactions/state but does not itself provide the Harness scheduler.
Current scheduler implementation lives inside `harness/`; clean conceptual layers do not mean independent exported packages.
## Pi's core design
### One commit authority
Session holds its mutation line through callback, preparation, storage settlement, adoption, publication.
External effects run outside that line.[^pi-session][^pi-spec]
A transaction can atomically change entries, documents, task checkpoint, ownership, and submission state.
Mutable drafts expire when the transaction callback settles; prepared state is immutable.
Storage succeeds before adopted state reaches commit observers.
Uncertain storage failure poisons the open Session; explicit `StorageRejected` alone proves no batch effect committed and permits continued use.[^pi-session]
Durability of a backend acknowledgement remains distinct from power-loss durability: SQLite NORMAL and optional JSONL fsync do not establish Smith's persistence boundary.[^pi-readme]
### Tasks are durable control, not persisted threads
Task phases checkpoint before/after effects; waiting tasks park without executing code.
States: pending, running, waiting, completing, terminal.
`completing` holds an outcome until foreground-owned work drains.
Terminal receipts remain queryable; task-scoped documents retire at terminal transition.[^pi-scheduler][^pi-spec]
History ancestry and work ownership are independent relationships.
Background work forms an explicit ownership boundary, rather than accidentally escaping a parent wait.
Cancelling a waiter stops that wait, not shared work; aborting work records intent and signals execution.
Closing stops invocations without declaring them completed; reopening reconciles running tasks to pending.
Missing/incompatible task code blocks execution rather than erasing durable work.
### Effect safety is narrower than task durability
Submission `requestId` deduplicates admission within one conversation, not external side effects.[^pi-readme][^pi-spec]
Tool intent stores final arguments and replay policy before execution.
Recovery reruns only when both persisted and currently resolved tool policies say `safe`.
Otherwise ToolTask emits an interrupted error explaining that the tool may have partially run.[^pi-tool]
Generation/custom task recovery has its own effect protocol; the scheduler cannot infer whether an external operation occurred.
Therefore “all interrupted effects retry” is false; “all effects are exactly once” is also false.
Task outcome and external mutation evidence are different dimensions.
Pi's task outcome variants are not a replacement for Smith's `applied` / `not-applied` / `unknown` effect record.
### State and observation
Documents declare session/conversation/task scope; conversation documents also declare latest/rewindable history and fresh/current/as-of fork behavior.[^pi-documents]
These are persisted semantic lifetimes, not allocator scopes.
Chord derives immutable values and operation batches from JSON drafts; docs preserve missing definitions and migrate on typed access.
Watch attachment captures baseline plus subscription atomically.
Slow observers retain at most 100 pending frames, then converge through a newest-state replacement.[^pi-observation]
Conversation views mount active entries plus built-in documents; agent events are another derived view.[^pi-view]
A convergence watch can skip intermediate publications; durable history must answer exact replay/audit needs separately.
An item-count backlog bound does not bound snapshot bytes or total loaded state.
### Important limits
Task records persist kind/version, not an immutable plugin generation.
Registry snapshots stabilize a task phase, then may refresh at later phases or restart.[^pi-scheduler][^pi-registry]
Tool implementation and cwd resolve for execution; changes can affect calls already emitted by a model.[^pi-readme][^pi-tool]
Ordinary extensions are native host code, not sandboxed guests.
Session document cache has no reviewed general byte/resident-item ceiling; idle context TTL is not an aggregate memory guarantee.[^pi-session][^pi-readme]
Environment abstraction and conformance make execution replaceable, but do not themselves authorize effects.[^pi-env]
## Current Smith: genuine substrate, incomplete general API
| Existing symbol | Implemented seam | Missing relative to candidate |
| --- | --- | --- |
| `Session::append_with` | Encode entry; persist callback before resident publication; rollback staged payload on failure | Atomic multi-record semantic commit, task/submission state |
| `SessionWriter` / `SessionReader` | Typed binary frames, sync, recovery, branches, bounded traversal | General transaction envelope and committed sequence API |
| `StorageWorker` | Serialized channel-based disk worker; immutable results | Shared command/receipt/observation facade |
| `Agent::run_turn` / `drive_settled` | Input → provider rounds → tools → settlement | Durable admitted submission, resumable work checkpoint |
| `ToolSession::invoke_recorded` | Intent precedes effect; result and mutation metadata; stop after recording failure | Per-SDK-effect uniform recording and atomic completion grouping |
| `RpcState` | Worker owns Agent; bounded prompt/wait channels and abort signal | Durable receipt distinct from RPC correlation ID; reconnect observation |
| `smith-plugin.wit` | Tool declaration and JSON invoke exports | Granted effects, durable state/work resources, generation manager |
Sources: current [session](../../../smith-core/src/session.rs), [store](../../../smith-core/src/store.rs), [storage worker](../../../smith-harness/src/storage.rs), [agent](../../../smith-core/src/agent.rs), [tools](../../../smith-core/src/tools.rs), [RPC](../../../smith-rpc/src/lib.rs), [WIT](../../../wit/smith-plugin.wit).
Specific gaps matter:
- `append_with` is one entry, not a transaction over entry + checkpoint + queue + state.
- Tool result and mutation metadata are separate appends; failure can leave partial semantic completion evidence.
- Current Agent owns provider polling and tool execution; it is not yet the specified pure transition engine.
- Public `StreamFn` exposes a futures stream; Agent consumes it through `block_on_stream`, contrary to the current no-futures core boundary law.
- Runtime assembly still contains native vendor selection; ToolSession still contains built-in name dispatch.
- Resident Session stores branch vectors and payload bytes; bounded reader APIs do not make every resident projection independently bounded.
These are source observations, not newly created issues or authorization to repair them.[^smith-runtime][^smith-provider]
“Use existing substrate” does not mean “existing implementation already satisfies all foundation laws.”
## Natural Smith structure
Candidate logical layers preserve current crate roles; no new crate or new build system is implied.
Arrows below mean “uses”, not execution order.
```mermaid
flowchart TB
api["CLI, TUI, RPC, embedding"] --> host["Harness runtime facade"]
host --> agent["Agent policy"]
agent --> kernel["Durable session kernel"]
kernel --> storage["Binary session storage"]
host --> effects["Granted effect executors"]
```
### Foundation: Smith session kernel
Keep one authority for history, effective requests, state transitions, work receipts, effect evidence.
Candidate primitives:
| Primitive | Semantic job | Constraint |
| --- | --- | --- |
| Session access | Stable identity, writer/observer mode, selected history | One writer per file/home; mutation denied before effects |
| Committed change | Bounded validated record group plus revision receipt | All-or-nothing semantic publication and recoverable encoding |
| Recorded input | Ordered nondeterministic observation | No ambient clock, RNG, network, FS, registry/config input |
| Durable work | Identity, checkpoint, ownership, cancellation, terminal outcome | Scheduling policy separate from stored checkpoint |
| Effect evidence | Durable intent, causal parent, pins, completion/mutation state | Missing completion remains unknown; no automatic retry |
| Scoped state | Bounded typed updates and plugin payloads | Host-owned, branch/fork semantics explicit, no second log |
| Read view | Bounded immutable chunks and historical projection | Borrow in flight, validated payloads, explicit cursor limits |
| Subscription | Sequence-tagged updates and recovery position | Persisted transcript recoverable; transient progress may coalesce |
“State” need not become a generic JSON database.
Known core state remains typed binary records; plugin-defined state can use bounded CBOR payloads with declared ownership/version.
State snapshots and deltas remain records in the same session authority; reconstructible caches are not a second source of truth.
Small bounded blobs may need only replacement/checkpoint records, not Chord-style arbitrary tree patches.
Session/branch/work state policy is distinct from process/session/turn/frame allocation accounting.
Frame scratch, VM-local scratch, and render caches are not made durable just because the API is general.
### Agent policy above foundation
Agent becomes one consumer of recorded inputs and session state, not owner of storage/executor machinery.
It decides effective request, provider round, tool batch, queue delivery, compaction, continuation, settlement.
It emits typed changes and effect plans; host feeds recorded completions back in order.
Provider and tool work can share durable ownership/effect mechanisms without sharing domain outcome enums.
No persistent VM stack, Rust thread, closure, or Future is the checkpoint.
Checkpoint data describes the next valid step and pinned provenance; host reconstructs a fresh invocation.
A useful general core is still allowed to have typed agent-specific records above its neutral primitives.
### Small public facade above foundation
Candidate host vocabulary: open/attach, submit, inspect, observe, abort, fork, close.
Submit returns a durable work/submission identity after admission; inspect/observe report settlement later.
Eval may offer a blocking “submit then wait” convenience while RPC/TUI use the same admitted operation.
RPC request IDs correlate transport messages; durable submission keys identify retried admission.
Process reopen reacquires receipts by durable ID; handles are not durable state.
Cancelling observation does not cancel work; abort is a separate authorized command.
This facade belongs naturally in `smith-harness` over shared `smith-core` state.
Shared IDs and message vocabulary belong in `smith`; schema/codec/reduction in `smith-core`.
Storage/executor threads remain host orchestration; abstract UI belongs in `smith-ui`; transports retain frontend roles.
Existing crate dependency constraints remain in force.
Conveniences should compose these operations, not implement another agent loop.
A typed Rust facade and WIT resources may express the same semantics without mirroring each other's syntax.
No async callbacks, Chord types, mutable transaction drafts, or ambient ExecutionEnv cross Smith's core/WIT boundary.
### Plugin API is a constrained view, not raw database access
Guest sees per-call granted resources for state, work, and effects, with explicit bounds and provenance.
Host owns persistence, IDs, ordering, generation pin, budgets, and authoritative terminal outcomes.
Plugins request typed changes; they cannot rewrite arbitrary core task records or claim an effect succeeded.
Pure state changes can roll back before admission; external effects cannot roll back merely because later guest code fails.
Child composition shares caller authority/budget/generation and drains before handler return, as currently specified.
A durable background task would require an explicit independent ownership/admission contract; it cannot be hidden behind ordinary child work.
A general work model does not require exposing arbitrary workflow creation to every plugin.
## Effect loop: one history, executor outside core
Candidate ordering retains Smith's effect evidence rather than importing Pi's replay-safe switch.
```mermaid
sequenceDiagram
participant K as Kernel
participant H as Host
participant S as Store
K->>H: Intent change
H->>S: Persist intent
S-->>H: Durable ack
H-->>K: Intent admitted
K->>H: Effect plan
H-->>K: Outcome observation
K->>H: Completion change
H->>S: Persist completion
S-->>H: Durable ack
H-->>K: Completion admitted
```
Host dispatch is eligible only after durable intent acknowledgement.
Core stages a pure completion transition from the observation; the bounded outcome input joins that completion commit before authoritative state adoption/publication.
Core never blocks render or performs network/filesystem effects inside a state transition.
A crash after dispatch but before completion leaves unknown evidence; reopening does not blindly re-execute the effect.
Explicit reconciliation may record an external receipt or user decision before workflow continuation.
Deterministic replay consumes prior inputs/effect evidence; recovery decides what uncompleted work may safely do.
These are different operations.
## Foundation versus extra
| Choice | Result | Assessment |
| --- | --- | --- |
| Pi-like facade only over current Agent | Easier embedding, same lifecycle/storage gaps | Useful thin convenience, not durable foundation |
| Second JSON/doc/task runtime beside Smith sessions | Dual authority, bridge/replay/conflict machinery | Reject as candidate |
| Smith-native committed kernel beneath existing agent | One log/evidence authority; modes and plugins compose it | Preferred candidate |
| Full general workflow engine exposed immediately | New task DSL, migrations, document DB, scheduler policy | Unjustified breadth without consumers |
| Optional high-level workflow/agent package over same kernel | Extra convenience without extra persistence authority | Compatible if later consumers earn it |
**Base mechanisms can be foundation; high-level workflows may be extra; storage authority must not be extra.**
No need to create Pi's enclosing multi-conversation Session table just to gain this shape.
Smith session identity can remain the durable file/home boundary; branches remain history choices, not parallel running conversations.
Independent concurrent agents can retain distinct Smith sessions.
Ownership across sessions does not grant cross-file atomic commits, writable fork aliasing, or distributed merge.
If a real workflow needs those semantics, they need separate explicit design rather than a renamed Conversation handle.
## Compatibility with current Smith laws
| Smith law | Pi transfer | Required adaptation |
| --- | --- | --- |
| Binary self-contained append-only file | One transaction authority | New grouped-record semantics must survive torn/corrupt tails and unknown frames |
| Complete visibility differs from durable ack | Commit before publication | Expose exact acknowledgement level; do not equate flush with durability |
| Pure core over recorded ordered inputs | Durable checkpoints | Synchronous transitions, typed channels, injected recorded time/config |
| Effects recover unknown, never auto-retry | Intent/effect/outcome sandwich | Keep three-state evidence; safe replay declaration is not authority |
| Request generation/loadout pinned through tool batch | Registry snapshots | Record/retain immutable generation plus grants and request provenance |
| Host-owned state, metered wasm | Scoped documents/task control | Resources and declarative requests, not native closures or unrestricted drafts |
| Bounded working sets | Snapshot/delta observation | Bound bytes/items/depth; chunk/replay durable transcript, do not send whole-history snapshots |
| Remote sequence resume | Convergent watch | Recovery from durable sequence; snapshot cannot replace missing transcript history |
| Fleet home owns writes | Single mutation line | Commands route to home; generations content-hashed; reads remain replicas |
| Queue preserved on abort unless explicit action | Durable cancellation | Do not import Pi's queued-input withdrawal default |
Current single-entry frames do not prove multi-record atomicity.
A bounded group frame or explicit commit envelope are candidate encodings, not chosen formats.
Readers, writer acknowledgements, recovery, unknown-record roundtrips, checkpoints, and observation must agree on the same commit boundary.
Appending several existing frames and delaying UI publication is insufficient for crash-recoverable semantic atomicity.
Generation pinning of a provider round is already required; arbitrary custom multi-phase workflows would need their own pin/rebind contract.
Do not silently promise permanent artifact availability across restart.
## Conclusions
- Pi demonstrates a real durable nucleus beneath an approachable agent API.
- Smith can naturally use that pattern as foundation, with its existing binary session/effect system as substrate.
- Host happy-path simplicity comes from coherent authority and receipts, not generalizing every feature into tasks or JSON documents.
- A small facade can exist before a complete durable work runtime, but must not claim crash-resumable submissions it cannot reconstruct.
- Agent/effect composition offers immediate consumers; a full workflow/document engine remains optional breadth.
- Exact API names, record formats, generic task exposure, and crate restructuring remain candidates, not mandates.
Human approved later relevant spec revisions referencing this research on 2026-10-11.
Current specs and implementation unchanged.
Relevant existing contracts: `SMH-SPEC-SPEC0001`, `SMH-SPEC-PLUG0001`, `SMH-SPEC-WASM0001`, `SMH-SPEC-RMTF0001`, `SMH-SPEC-FLET0001`.
## Unresolved questions
1. Is first public consumer embedding Rust, frontend protocol, or WIT composition; which operations are truly shared?
2. What bounded semantic group must commit atomically: effect result + evidence, queue admission + work, plugin state + checkpoint?
3. Does durable work need an extensible generic task definition now, or typed provider/tool/compaction checkpoints first?
4. Which plugin state scopes need historical rewind/fork behavior versus fresh state?
5. How are work checkpoints resumed when the original component/grant revision is unavailable, without silent rebinding?
6. Which unknown effects can be reconciled by recorded external receipts, and which require human action?
7. What sequence/cursor identifies a committed view across branching, maintenance, reconnect, and fleet replication?
8. Can bounded snapshot plus paged transcript recovery satisfy both frontend convergence and exact event resumption?
9. Which state/concurrency guarantees belong to one session file versus a host coordinating independent sessions?
## Sources
Pinned upstream sources at `abe508e1b89912adde45528136c3221eb69acdd7`:
[^pi-readme]: [Durable README](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/README.md).[^pi-index]: [Public exports](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/index.ts).[^pi-spec]: [Normative Pico5 specification; selected sections](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/docs/spec.md).[^pi-session]: [Session commit, publication, poisoning, documents](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/session/session.ts).[^pi-harness]: [Harness over Session](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/harness/harness.ts).[^pi-scheduler]: [Task scheduling, recovery, ownership](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/harness/scheduler.ts).[^pi-tool]: [ToolTask intent, replay policy, settlement](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/harness/tool.ts).[^pi-documents]: [Document scope/history/fork/version semantics](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/documents.ts).[^pi-observation]: [Committed state/watch mechanics](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/session/observation.ts).[^pi-view]: [Conversation projection](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/harness/view.ts).[^pi-registry]: [Registry snapshots and definition lookup](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/harness/registry.ts).[^pi-env]: [Execution environment contracts](https://github.com/earendil-works/pi/blob/abe508e1b89912adde45528136c3221eb69acdd7/packages/durable/src/env/index.ts).[^smith-runtime]: [Current runtime assembly](../../../smith-harness/src/runtime.rs).[^smith-provider]: [Current public provider/futures boundary](../../../smith/src/provider.rs).