Luigit
repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by admin

extensions/tps/README.md

Raw
Rendered preview

tps

Tokens-per-second footer/status segment for Pi.

Install / load

Loaded through root pi-ext package.

Commands / tools / settings

  • Commands: none.
  • Tools: none.
  • Settings: none.
  • Hooks/events:
    • session_start
    • agent_start
    • message_start
    • message_update
    • message_end
    • agent_end
    • session_shutdown
    • emits footer:segment when the shared footer is loaded.

Behavior

  • session_start: shows 󰓅 —/s.
  • agent_start: resets run state, starts timing, shows 󰓅 0.0/s.
  • message_start: records assistant span start for non-streaming fallback.
  • message_update: reads string deltas from assistant events whose type ends with _delta.
  • The first non-empty streamed delta starts decode timing as a baseline and is not counted in the live TPS numerator.
  • Later streamed delta chars are counted as live output, estimated at 4 chars per token.
  • Live updates are throttled to every 250 ms.
  • message_end: closes the assistant span. Streamed spans with more than one delta add measured decode time; otherwise fallback timing uses the assistant span.
  • agent_end: sums usage.output from assistant messages only.
  • Final streamed TPS uses assistant output tokens scaled by measured streamed chars / total streamed chars, divided by measured decode seconds.
  • Fallback final TPS uses assistant output tokens divided by assistant span seconds.
  • Multiple assistant spans are summed; tool gaps are excluded from measured decode time.
  • No assistant output tokens or no measured duration returns to 󰓅 —/s.
  • Shared footer present: emits a footer:segment in the llm zone.
  • No shared footer: falls back to ctx.ui.setStatus("tps", ...).
  • session_shutdown: clears state and removes status/segment.

Debug

Opt-in metadata diagnostics: debug contract. Safe events: session.start, session.shutdown, rate.finish (measured | fallback | idle outcome).

# tps

Tokens-per-second footer/status segment for Pi.

## Install / load

Loaded through root [pi-ext](../../README.md) package.

## Commands / tools / settings

- Commands: none.
- Tools: none.
- Settings: none.
- Hooks/events:
  - `session_start`
  - `agent_start`
  - `message_start`
  - `message_update`
  - `message_end`
  - `agent_end`
  - `session_shutdown`
  - emits `footer:segment` when the shared footer is loaded.

## Behavior

- `session_start`: shows `󰓅 —/s`.
- `agent_start`: resets run state, starts timing, shows `󰓅 0.0/s`.
- `message_start`: records assistant span start for non-streaming fallback.
- `message_update`: reads string deltas from assistant events whose type ends
  with `_delta`.
- The first non-empty streamed delta starts decode timing as a baseline and is
  not counted in the live TPS numerator.
- Later streamed delta chars are counted as live output, estimated at 4 chars
  per token.
- Live updates are throttled to every 250 ms.
- `message_end`: closes the assistant span. Streamed spans with more than one
  delta add measured decode time; otherwise fallback timing uses the assistant
  span.
- `agent_end`: sums `usage.output` from assistant messages only.
- Final streamed TPS uses assistant output tokens scaled by measured streamed
  chars / total streamed chars, divided by measured decode seconds.
- Fallback final TPS uses assistant output tokens divided by assistant span
  seconds.
- Multiple assistant spans are summed; tool gaps are excluded from measured
  decode time.
- No assistant output tokens or no measured duration returns to `󰓅 —/s`.
- Shared footer present: emits a `footer:segment` in the `llm` zone.
- No shared footer: falls back to `ctx.ui.setStatus("tps", ...)`.
- `session_shutdown`: clears state and removes status/segment.

## Debug

Opt-in metadata diagnostics: [debug contract](../DEBUG.md).
Safe events: `session.start`, `session.shutdown`, `rate.finish` (`measured` | `fallback` | `idle` outcome).