pi-ext bugabingas pi extensions
owned by admin
README Source History Refs Compare Notes Search fix(decay): recover from partial classification and salvage failed summaries fcbe0a672c6dabb63777fcd6060ae3ba8befa6f9
Oliver Krylow <o.krylow@isp-insoft.de > · 17 Sep 2026
SSH signature · not checked
Browse files Compare with HEAD
Gap-fill only the chunks a classifier omitted, tolerate exact duplicate chunk records, and render ranked atoms deterministically when the summarizer fails instead of discarding the whole run to default compaction.
Changed files (6) 6 + - Recover Decay from partial classification with up to two gap-fill rounds over only the missing chunks, tolerate exact duplicate chunk records, and salvage a deterministic summary from ranked atoms when the summarizer fails instead of discarding the whole run to default compaction
6 7 - Expose Klaus tools to Claude under their Pi names instead of the `mcp__klaus__` prefix, while still accepting prefixed names from resumed sessions and older runtimes
7 8 - Stop Klaus from replacing large Pi tool results with a `<persisted-output>` pointer into a deleted temp directory, and isolate the Klaus child from ambient Claude Code context, deferred tool loading, tool backgrounding, and non-streaming retries
8 9 - Capture Klaus tool identities from Claude's `PreToolUse` hook so tool calls keep working when a permission rule answers them without asking Klaus, and truncate tool results at Claude's result-size ceiling instead of losing them to a persisted file
extensions/decay/README.md Modified 18 18 Omit `model` or set it to `null` to use the current session model.
19 19 Omit `summarizerModel` or set it to `null` to reuse the classifier model.
21 - Failed Decay runs delegate to Pi's default compactor and append a bounded `decay-diagnostic` custom entry containing the stage, model, sanitized reason, and timestamp.
21 + Failed Decay runs delegate to Pi's default compactor and append a bounded `decay-diagnostic` custom entry containing the outcome, stage, model, sanitized reason, and timestamp.
22 22 The custom entry is persisted in session JSONL but excluded from model context.
24 + Failure handling differs per stage because only some stages can degrade without inventing memory.
25 + A transient classifier failure, or a stream that omits chunks, triggers up to two gap-fill rounds that reclassify only the missing chunks before falling back.
26 + Exact duplicate records for one chunk are tolerated, while conflicting reclassification, total-count drift, invalid indexes, authentication errors, and misconfigured models fall back immediately.
27 + Unclassified chunks are never given default records, since inventing or silently dropping memory would fail invisibly.
29 + A summarizer failure salvages instead of falling back: Decay renders the ranked atoms deterministically into the same section structure without a model call, marks the compaction `degraded: "summarizer"`, and records a `salvage` diagnostic.
30 + Salvaged summaries keep the classified facts, provenance, ranking, and recurrence identity, and lose only prose synthesis.
24 32 Repeated compactions provide the classifier a bounded prior-atom identity catalog so paraphrased facts can reuse exact recurrence keys without counting catalog entries as new occurrences.
26 34 Interactive progress uses a passive, borderless five-row widget above Pi's editor, preserving native steering submission and Escape cancellation during compaction.
38 46 The [original Decay plan](../../docs/super/plans/2026-08-22-decay-compaction.md) contains earlier visual artifacts.
39 47 The current widget uses five contiguous lines at every terminal height; aligned evidence-driven maps replace that plan's eight-line design.
48 + That plan's blanket "any non-abort failure uses default compaction" rule is superseded by the staged policy above.
extensions/decay/__tests__/core.test.ts Modified 8 8 findPreviousDecayDetails,
14 16 parseModelReference,
20 + renderFallbackSummary,
19 22 validateRecordChunk,
187 190 ).
toEqual ([
record (
1 ,
2 ,
"one" ),
record (
2 ,
2 ,
"two" )]);
190 - it (
"rejects duplicates, missing chunks, wrong totals, and invalid indexes" , ()
=> {
193 + it (
"tolerates exact duplicates and reports missing chunks" , ()
=> {
196 + [
record (
1 ,
2 ,
"one" ),
record (
1 ,
2 ,
"one" ),
record (
2 ,
2 ,
"two" )],
200 + expect (
missingChunks ([
record (
2 ,
3 ,
"two" )],
3 )).
toEqual ([
1 ,
3 ]);
201 + expect (
missingChunks ([
record (
1 ,
1 ,
"one" )],
1 )).
toEqual ([]);
204 + it (
"rejects conflicts, missing chunks, wrong totals, and invalid indexes" , ()
=> {
192 206 reconcileRecords ([
record (
1 ,
2 ,
"one" ),
record (
1 ,
2 ,
"again" )],
2 ),
193 - ).
toThrow (
"duplicated chunk 1" );
207 + ).
toThrow (
"reclassified chunk 1" );
194 208 expect (()
=> reconcileRecords ([
record (
1 ,
2 ,
"one" )],
2 )).
toThrow (
195 209 "omitted chunks: 2" ,
384 + describe (
"degraded summarizer" , ()
=> {
385 + function atom (overrides
: Partial <
MemoryAtom >)
: MemoryAtom {
399 + it (
"renders every heading and only classified facts" , ()
=> {
400 + const summary
= renderFallbackSummary (
403 + atom ({ key:
"goal" , text:
"Ship decay" , kind:
"goal" }),
404 + atom ({ key:
"blocked" , text:
"Waiting on auth" , kind:
"blocker" }),
405 + atom ({ key:
"path" , text:
"src/main.ts" , kind:
"exact" }),
406 + atom ({ key:
"dropped" , text:
"Old idea" , status:
"superseded" }),
411 + expect (summary).
toContain (
"Decay summarizer unavailable" );
412 + for (
const heading
of [
414 + "## Constraints & Preferences" ,
419 + "## Key Decisions" ,
421 + "## Critical Context" ,
423 + expect (summary).
toContain (heading);
424 + expect (summary).
toContain (
"- Ship decay" );
425 + expect (summary).
toContain (
"- Waiting on auth" );
426 + expect (summary).
toContain (
"- src/main.ts" );
427 + expect (summary).not.
toContain (
"Old idea" );
430 + it (
"classifies transient failures without treating aborts as retryable" , ()
=> {
431 + for (
const message
of [
432 + "429 rate limit exceeded" ,
433 + "upstream returned 503" ,
436 + "Decay classifier stream ended without completion" ,
438 + expect (
isTransientFailure (
new Error (message))).
toBe (
true );
439 + const abort
= new Error (
"The operation was aborted" );
440 + abort.
name = "AbortError" ;
441 + expect (
isTransientFailure (abort)).
toBe (
false );
443 + isTransientFailure (
new Error (
"Decay classifier called unknown tool: x" )),
446 + isTransientFailure (
new Error (
"Decay model authentication failed" )),
extensions/decay/__tests__/integration.test.ts Modified 362 362 expect (
JSON .
stringify (diagnostic)).not.
toContain (
"stack" );
365 - it (
"returns nothing when the summarizer emits a tool call" ,
async ()
=> {
365 + it (
"salvages a deterministic summary when the summarizer fails" ,
async ()
=> {
366 366 const { testSession, faux }
= await setup ();
367 367 faux.
setResponses ([
368 368 classifierMessage (),
374 + const result
= await testSession.session.extensionRunner.
emit (
377 + const compaction = (
379 + | { compaction: { summary: string; details: unknown } }
382 + expect (compaction?.
summary ).
toContain (
"Decay summarizer unavailable" );
383 + expect (compaction?.
summary ).
toContain (
"src/main.ts" );
384 + expect (compaction?.details).
toMatchObject ({
385 + decay: { degraded:
"summarizer" },
387 + const diagnostic
= testSession.session.sessionManager
391 + entry.
type === "custom" && entry.customType
=== "decay-diagnostic" ,
393 + expect (diagnostic).
toMatchObject ({
394 + data: { outcome:
"salvage" , stage:
"summarizer" },
398 + it (
"gap-fills chunks the classifier omitted after a transient failure" ,
async ()
=> {
399 + const { testSession, faux }
= await setup ();
400 + faux.
setResponses ([
401 + fauxAssistantMessage ([], {
402 + stopReason:
"error" ,
403 + errorMessage:
"429 rate limit exceeded" ,
405 + classifierMessage (),
406 + fauxAssistantMessage (
"## Goal\n\nRecovered." ),
409 + const result
= await testSession.session.extensionRunner.
emit (
375 - await testSession.session.extensionRunner.
emit (
compactionEvent ()),
413 + (result
as { compaction
: { summary
: string } }
| undefined )?.compaction
415 + ).
toContain (
"Recovered." );
379 418 it (
"reuses prior keys for paraphrased repeated compactions" ,
async ()
=> {
extensions/decay/core.ts Modified 34 34 export type AtomKind = (
typeof ATOM_KINDS)[
number ];
35 35 export type AtomStatus = "active" | "superseded" ;
36 36 export type ChunkSource = "legacy-summary" | "history" | "split-prefix" ;
37 + export type DegradedStage = "summarizer" ;
38 39 type Preparation = SessionBeforeCompactEvent [
"preparation" ];
39 40 type SourceMessage = Preparation [
"messagesToSummarize" ][
number ];
96 97 chunkCount : number;
97 98 keepRecentTokens : number;
98 99 estimatedKeptTokens
?: number;
100 + degraded
?: DegradedStage;
99 101 atoms : MemoryAtom[];
100 102 readFiles : string[];
101 103 modifiedFiles : string[];
261 - export function reconcileRecords (
263 + export function indexRecords (
262 264 records
: readonly RecordChunk [],
263 265 totalChunks
: number ,
266 + )
: Map <
number ,
RecordChunk > {
265 267 if (totalChunks
< 1 )
throw new Error (
"Decay has no source chunks" );
266 268 const byChunk
= new Map <
number ,
RecordChunk >();
267 269 for (
const record
of records) {
274 276 `Decay classifier reported invalid chunk ${record.chunk}` ,
276 - if (byChunk.
has (record.chunk))
277 - throw new Error (
`Decay classifier duplicated chunk ${record.chunk}` );
278 + const existing
= byChunk.
get (record.chunk);
280 + if (
canonicalRecord (existing)
!== canonicalRecord (record))
282 + `Decay classifier reclassified chunk ${record.chunk} with conflicting atoms` ,
278 286 byChunk.
set (record.chunk, record);
291 + export function missingChunks (
292 + records
: readonly RecordChunk [],
293 + totalChunks
: number ,
295 + const byChunk
= indexRecords (records, totalChunks);
296 + return Array .
from ({ length: totalChunks }, (_, index)
=> index
+ 1 ).
filter (
297 + (chunk)
=> ! byChunk.
has (chunk),
301 + export function reconcileRecords (
302 + records
: readonly RecordChunk [],
303 + totalChunks
: number ,
305 + const byChunk
= indexRecords (records, totalChunks);
280 306 if (byChunk.
size !== totalChunks) {
281 307 const missing
= Array .
from ({ length: totalChunks }, (_, index)
=> index
+ 1 )
282 308 .
filter ((chunk)
=> ! byChunk.
has (chunk))
286 312 return [
... byChunk.
values ()].
sort ((left, right)
=> left.chunk
- right.chunk);
315 + function canonicalRecord (record
: RecordChunk )
: string {
316 + return JSON .
stringify ({
317 + chunk: record.chunk,
319 + atoms: record.atoms.
map ((atom)
=> [
289 328 export function estimateKeptTokens (
290 329 event
: SessionBeforeCompactEvent ,
291 330 )
: number | undefined {
322 361 (
"estimatedKeptTokens" in decay
&& 323 362 (
! Number .
isInteger (decay.estimatedKeptTokens)
|| 324 363 (decay.estimatedKeptTokens
as number )
< 0 ))
|| 364 + (decay.degraded
!== undefined && decay.degraded
!== "summarizer" )
|| 325 365 ! Array .
isArray (decay.atoms)
|| 326 366 ! Array .
isArray (decay.readFiles)
|| 327 367 ! Array .
isArray (decay.modifiedFiles)
344 384 ... (decay.estimatedKeptTokens
=== undefined 346 386 : { estimatedKeptTokens: decay.estimatedKeptTokens
as number }),
387 + ... (decay.degraded
=== undefined 389 + : { degraded:
"summarizer" as const }),
348 391 readFiles : [
... (decay.readFiles
as string [])],
349 392 modifiedFiles : [
... (decay.modifiedFiles
as string [])],
575 + const FALLBACK_NOTE
= 576 + "> Decay summarizer unavailable. Memory below is rendered deterministically from classified atoms without prose synthesis." ;
578 + export function renderFallbackSummary (atoms
: readonly AllocatedAtom [])
: string {
579 + const byKind = (kinds
: readonly AtomKind [])
: string []
=> 581 + .
filter ((atom)
=> kinds.
includes (atom.kind))
582 + .
map ((atom)
=> `- ${atom. text . replace (/\s + / g , " "). trim ()}` );
583 + const section = (heading
: string , kinds
: readonly AtomKind [])
: string => {
584 + const lines
= byKind (kinds);
585 + return lines.
length > 0 ? `${heading}\n\n${lines. join ("\n")}` : heading;
589 + section (
"## Goal" , [
"goal" ]),
590 + section (
"## Constraints & Preferences" , [
"constraint" ]),
593 + section (
"### In Progress" , [
"state" ]),
594 + section (
"### Blocked" , [
"blocker" ]),
595 + section (
"## Key Decisions" , [
"decision" ]),
597 + section (
"## Critical Context" , [
"exact" ,
"context" ]),
601 + const TRANSIENT_PATTERN
= 602 + / \b (429 | 500 | 502 | 503 | 504) \b| rate. ? limit | too many requests | overloaded | temporarily unavailable | service unavailable | timeout | timed out | socket hang up | stream (?:ended without completion | closed | disconnected) | network | fetch failed | ECONNRESET | ECONNREFUSED | EPIPE | ETIMEDOUT | EAI_AGAIN | ENOTFOUND/ i ;
604 + export function isTransientFailure (error
: Error )
: boolean {
605 + if (error.
name === "AbortError" )
return false ;
606 + return TRANSIENT_PATTERN.
test (
`${error. name }: ${error.message}` );
532 609 export function formatFileAppendix (
533 610 readFiles
: readonly string [],
534 611 modifiedFiles
: readonly string [],
extensions/decay/index.ts Modified 18 18 createSourceChunks,
19 19 DECAY_DETAILS_VERSION,
21 22 estimateKeptTokens,
22 23 findPreviousDecayDetails,
23 24 formatFileAppendix,
28 31 parseModelReference,
35 + renderFallbackSummary,
32 37 validateRecordChunk,
33 38 } from
"./core.js" ;
51 56 details : DecayDetails;
52 57 estimatedTokensAfter
?: number;
58 + degraded
?: { stage: DegradedStage; error:
Error };
84 90 const DECAY_DIAGNOSTIC_ENTRY
= "decay-diagnostic" ;
91 + const CLASSIFIER_GAP_FILL_ROUNDS
= 2 ;
85 92 const DECAY_WIDGET
= "decay-progress" ;
87 94 export default function decayExtension (pi
: ExtensionAPI )
: void {
100 107 ? `Decay model not found: ${settings.model}; using default compaction` 101 108 : "Decay has no current model; using default compaction" ;
102 - persistFallbackDiagnostic (
105 113 settings.model
|| "current/unknown" ,
106 114 new Error (message),
118 126 settings.summarizerModel
=== "" 119 127 ? "Decay summarizerModel setting must be provider/model, null, or omitted; using default compaction" 120 128 : `Decay summarizer model not found: ${settings.summarizerModel}; using default compaction` ;
121 - persistFallbackDiagnostic (
124 133 settings.summarizerModel
|| "classifier" ,
125 134 new Error (message),
165 174 if (outcome.
type === "error" ) {
166 175 if (outcome.cancelled)
return { cancel:
true };
167 176 const stage
= failureStage (outcome.error);
168 - persistFallbackDiagnostic (
171 181 stage
=== "summarizer" ? summarizerModelReference
: modelReference,
191 + if (outcome.result.degraded) {
195 + outcome.result.degraded.stage,
196 + summarizerModelReference,
197 + outcome.result.degraded.error,
201 + `Decay summarizer failed: ${ boundedError (outcome.result.degraded.error)}; rendered classified memory without prose synthesis` ,
183 207 summary: outcome.result.
summary ,
195 219 const details
= parseDecayDetails (
event .compactionEntry.details)?.decay;
196 220 if (
! details)
return ;
198 - `Decay · ${ modelLabel (details.model)} · ${details.chunkCount}/${details.chunkCount} classified · ${details.estimatedKeptTokens === undefined ? "unknown" : `~${details.estimatedKeptTokens. toLocaleString ("en-US")}`} kept (target ${details.keepRecentTokens. toLocaleString ("en-US")}) · ${(details.elapsedMs / 1_000). toFixed (1)}s` ,
222 + `Decay · ${ modelLabel (details.model)} · ${details.chunkCount}/${details.chunkCount} classified · ${details.estimatedKeptTokens === undefined ? "unknown" : `~${details.estimatedKeptTokens. toLocaleString ("en-US")}`} kept (target ${details.keepRecentTokens. toLocaleString ("en-US")}) · ${(details.elapsedMs / 1_000). toFixed (1)}s${details.degraded ? " · salvaged" : ""}` ,
223 + details.degraded
? "warning" : "info" ,
305 329 if (chunks.
length === 0 )
throw new Error (
"Decay has no source chunks" );
306 330 stage
= "classifier" ;
307 331 log
= await CanonicalRecordLog.
create (callbacks.onRecord);
308 - const classifierUsage
= await classify (
332 + const classifierUsage
= await classifyAllChunks (
327 351 const allocated
= allocateAtoms (atoms, outputBudget);
328 352 callbacks.
onMerge (atoms, allocated);
329 353 stage
= "summarizer" ;
330 - const auth
= await ctx.modelRegistry.
getApiKeyAndHeaders (summarizerModel);
332 - throw new Error (auth.error
|| "Decay model authentication failed" );
333 - const provider
= ctx.modelRegistry.
getProvider (summarizerModel.provider);
335 - throw new Error (
`Decay provider not found: ${summarizerModel.provider}` );
336 - const stream
= provider.
streamSimple (
338 - buildSummarizerContext (allocated,
event .customInstructions),
340 - apiKey: auth.apiKey,
341 - headers: auth.
headers ,
343 - maxTokens: outputBudget,
355 + let summarizerUsage
: Usage = combineUsage ();
356 + let degraded
: DecayRunResult [
"degraded" ];
358 + const summarized
= await summarize (
363 + event .customInstructions,
345 - cacheRetention:
"none" ,
346 - sessionId:
uuidv7 (),
347 - reasoning: summarizerModel.reasoning
? "low" : undefined ,
350 - for await (
const item
of stream) {
351 - if (item.
type === "text_delta" ) {
352 - callbacks.
onSummary ?.(
353 - Math .
ceil (
contentText (item.partial.
content ).
length / 4 ),
357 - const response
= await stream.
result ();
358 - if (response.stopReason
!== "stop" )
360 - `Decay summarizer stopped with ${response.stopReason}: ${response.errorMessage ?? "no detail"}` ,
362 - if (response.
content .
some ((content)
=> content.
type === "toolCall" ))
363 - throw new Error (
"Decay summarizer returned a tool call" );
364 - const prose
= contentText (response.
content ).
trim ();
365 - if (
! prose)
throw new Error (
"Decay summarizer returned an empty summary" );
367 + prose
= summarized.prose;
368 + summarizerUsage
= summarized.usage;
370 + if (signal.aborted)
throw error;
371 + degraded
= { stage:
"summarizer" , error:
asError (error) };
372 + prose
= renderFallbackSummary (allocated);
373 + callbacks.
onSummary ?.(
Math .
ceil (prose.
length / 4 ));
366 375 const files
= mergeFileLists (previous,
event .preparation.fileOps);
368 377 prose
+ formatFileAppendix (files.readFiles, files.modifiedFiles);
370 379 const estimatedKeptTokens
= estimateKeptTokens (
event );
373 - usage:
combineUsage (classifierUsage, response.usage),
382 + usage:
combineUsage (classifierUsage, summarizerUsage),
383 + ... (degraded
? { degraded }
: {}),
374 384 estimatedTokensAfter:
375 385 estimatedKeptTokens
=== undefined 383 393 chunkCount : chunks.
length ,
384 394 keepRecentTokens :
event .preparation.settings.keepRecentTokens,
385 395 ... (estimatedKeptTokens
=== undefined ? {}
: { estimatedKeptTokens }),
396 + ... (degraded
? { degraded: degraded.stage }
: {}),
387 398 readFiles : files.readFiles,
388 399 modifiedFiles : files.modifiedFiles,
419 + async function summarize (
420 + ctx
: ExtensionContext ,
422 + atoms
: ReturnType <
typeof allocateAtoms>,
423 + outputBudget
: number ,
424 + manualInstructions
: string | undefined ,
425 + signal
: AbortSignal ,
426 + callbacks
: RunCallbacks ,
427 + )
: Promise <{ prose
: string ; usage
: Usage }> {
428 + const auth
= await ctx.modelRegistry.
getApiKeyAndHeaders (model);
430 + throw new Error (auth.error
|| "Decay model authentication failed" );
431 + const provider
= ctx.modelRegistry.
getProvider (model.provider);
432 + if (
! provider)
throw new Error (
`Decay provider not found: ${model.provider}` );
433 + const stream
= provider.
streamSimple (
435 + buildSummarizerContext (atoms, manualInstructions),
437 + apiKey: auth.apiKey,
438 + headers: auth.
headers ,
440 + maxTokens: outputBudget,
442 + cacheRetention:
"none" ,
443 + sessionId:
uuidv7 (),
444 + reasoning: model.reasoning
? "low" : undefined ,
447 + for await (
const item
of stream) {
448 + if (item.
type === "text_delta" ) {
449 + callbacks.
onSummary ?.(
450 + Math .
ceil (
contentText (item.partial.
content ).
length / 4 ),
454 + const response
= await stream.
result ();
455 + if (response.stopReason
!== "stop" )
457 + `Decay summarizer stopped with ${response.stopReason}: ${response.errorMessage ?? "no detail"}` ,
459 + if (response.
content .
some ((content)
=> content.
type === "toolCall" ))
460 + throw new Error (
"Decay summarizer returned a tool call" );
461 + const prose
= contentText (response.
content ).
trim ();
462 + if (
! prose)
throw new Error (
"Decay summarizer returned an empty summary" );
463 + return { prose, usage: response.usage };
466 + async function classifyAllChunks (
467 + ctx
: ExtensionContext ,
469 + chunks
: readonly SourceChunk [],
470 + priorAtoms
: readonly MemoryAtom [],
471 + reserveTokens
: number ,
472 + manualInstructions
: string | undefined ,
473 + signal
: AbortSignal ,
474 + log
: CanonicalRecordLog ,
476 + const usages
: Usage []
= [];
477 + let pending
= [
... chunks];
478 + for (
let round
= 0 ; ; round
++ ) {
479 + let failure
: Error | undefined ;
488 + manualInstructions,
494 + if (signal.aborted)
throw error;
495 + failure
= asError (error);
496 + if (
! isTransientFailure (failure))
throw failure;
499 + const missing
= missingChunks (log.records, chunks.
length );
500 + if (missing.
length === 0 )
return combineUsage (
... usages);
501 + if (round
>= CLASSIFIER_GAP_FILL_ROUNDS)
504 + new Error (
`Decay classifier omitted chunks: ${missing. join (", ")}` )
506 + pending
= chunks.
filter ((chunk)
=> missing.
includes (chunk.chunk));
408 510 async function classify (
409 511 ctx
: ExtensionContext ,
411 - chunks
: ReturnType <
typeof createSourceChunks>,
513 + chunks
: readonly SourceChunk [],
412 514 priorAtoms
: readonly MemoryAtom [],
413 515 reserveTokens
: number ,
414 516 manualInstructions
: string | undefined ,
489 591 return error
instanceof DecayRunError ? error.stage
: "prepare" ;
492 - function persistFallbackDiagnostic (
594 + function persistDiagnostic (
493 595 pi
: ExtensionAPI ,
596 + outcome
: "fallback" | "salvage" ,
494 597 stage
: DecayFailureStage ,
499 602 pi.
appendEntry (DECAY_DIAGNOSTIC_ENTRY, {
501 - outcome:
"fallback" ,
503 606 model:
sanitizeDiagnostic (model).
slice (
0 ,
160 ),
504 607 reason:
boundedError (error),
541 644 previewChunkCount,
545 650 maximumOutputTokens,