repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by bugabinga

docs(ultra): require deliberate workflow compute

6b05191620de3a491b39c1f5b155f4fce957eeed

Oliver Jan Krylow <oliver@bugabinga.net> ·

unsigned

Browse filesCompare with HEAD

Changed files (3)

extensions/ultra/__tests__/skill.test.ts

Modified
54 54 expect(fs.existsSync(skill.filePath)).toBe(true);
57 + it("requires deliberate per-step model tiers and thinking levels", () => {
58 + const skill = fs.readFileSync(SKILL_PATH, "utf8");
59 + const reference = fs.readFileSync(SPEC_FORMAT_PATH, "utf8");
61 + expect(skill).toContain("Assign compute deliberately");
62 + expect(skill).toContain("Explicitly set both fields");
63 + for (const tier of ["`small`", "`medium`", "`big`"]) {
64 + expect(skill).toContain(tier);
65 + expect(reference).toContain(tier);
66 + }
67 + expect(reference).toContain("relative to that step's difficulty");
68 + expect(reference).toContain("re-evaluate each assignment");
69 + });
57 71 it("keeps the reference doc and the validator in lockstep: every complete JSON example parses", () => {
58 72 // Extract every ```json fenced block from the canonical reference and run
59 73 // each *complete* spec (one with both `name` and `phases`) through the

extensions/ultra/skills/authoring-workflows/SKILL.md

Modified
64 64 literal `{`/`}` out of prompts except these tokens. Set `return` to the
65 65 selector for the workflow's final value.
67 -6. **Self-check the spec (there is no validator tool).** You can't call
67 +6. **Assign compute deliberately.** For every step:
68 + - `big` — planning, specification, architecture, critique, synthesis.
69 + - `medium` — coding, testing, editing, routine implementation.
70 + - `small` — scouting, extraction, classification, easy text work.
71 + - Set `thinkingLevel` from that step's relative ambiguity, constraints, and cost of mistakes.
72 + - Explicitly set both fields. If every phase uses the same pair, challenge the assignment before saving.
74 +7. **Self-check the spec (there is no validator tool).** You can't call
68 75 `parseWorkflow` directly — verify by inspection against the reference: every
69 76 required field is present, every `fanout` has an `over`, every `when` is one selector, every string
70 77 `schema` names a key in `schemas`, and interpolation uses only the closed
71 78 grammar. The real validator is the load itself: saving and test-running
72 - (steps 7–8) parses the file through the runtime's `parseWorkflow`, which
79 + (steps 8–9) parses the file through the runtime's `parseWorkflow`, which
73 80 surfaces a TypeBox error naming the offending path if anything is malformed —
74 81 fix and re-run until it loads clean. One caveat: a JSON _syntax_ error (a
75 82 trailing comma, a missing quote) makes the file invisible to discovery
76 83 entirely, so `/ultra <name>` reports `unknown workflow` rather than a schema
77 84 error — if a saved workflow seems to vanish, confirm it is valid JSON first.
79 -7. **Save it.** Write the spec as `<name>.json` (filename matching the spec's
86 +8. **Save it.** Write the spec as `<name>.json` (filename matching the spec's
80 87 `name`):
81 88 - project-scoped (lives in the repo): `<cwd>/.pi/workflows/<name>.json`
82 89 - personal / global: `~/.pi/agent/workflows/<name>.json`
84 91 Ask which scope only if it isn't clear from context (a repo-specific review
85 92 workflow → project; a personal utility → global).
87 -8. **Offer a test run.** After saving, offer to run it: `/ultra <name>` (with
94 +9. **Offer a test run.** After saving, offer to run it: `/ultra <name>` (with
88 95 any `args` the spec's `hint` declares). Tighten prompts/schemas from what the
89 96 first run surfaces.

extensions/ultra/skills/authoring-workflows/references/spec-format.md

Modified
67 67 If a step sets `model` to `small`, `medium`, or `big`, ultra resolves it through that map; an unset tier falls back to the session model.
68 68 Tier values may include a thinking suffix like `openai/gpt-5-mini:low`, used only when the step omits `thinkingLevel`.
70 +### Model tier and thinking rubric
72 +Set both `model` and `thinkingLevel` on every authored step.
73 +Choose them from the work that step performs, not by copying another phase:
75 +| Work | Tier |
76 +| ---------------------------------------------------------- | -------- |
77 +| Planning, specification, architecture, critique, synthesis | `big` |
78 +| Coding, testing, editing, routine implementation | `medium` |
79 +| Scouting, extraction, classification, easy text work | `small` |
81 +Choose thinking independently from the tier and relative to that step's difficulty:
83 +- `off` / `minimal` — deterministic extraction, transcription, or formatting with negligible ambiguity.
84 +- `low` / `medium` — routine decisions with bounded context and cheap mistakes.
85 +- `high` / `xhigh` — ambiguous, multi-constraint reasoning where a wrong decision is costly downstream.
87 +If every phase has the same model/thinking pair, re-evaluate each assignment before saving.
88 +Uniform settings are valid only when the phases genuinely perform equally difficult work of the same kind.
90 +Examples:
92 +- Repository inventory: `small` + `low`.
93 +- Implement a well-specified edit: `medium` + `medium`.
94 +- Design architecture from conflicting requirements: `big` + `high`.
95 +- Adversarially critique a risky plan: `big` + `xhigh`.
70 97 ## Interpolation grammar
72 99 A closed mini-syntax — not an expression evaluator, no `eval`. Tokens