1 +#
Ultra Tool Guidance Implementation Plan 3 +> **For agentic workers:** REQUIRED SUB-SKILL: Use /skill:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
5 +**Goal:** Give `workflow` callers the semantic contract needed to produce valid inline specs on the first attempt.
7 +**Architecture:** Put invocation-critical guidance on the existing shared TypeBox schema so Pi delivers it with the tool contract. Add only two runtime constraints already required by interpolation—identifier-shaped, unique phase IDs—and synchronize the tool reminder, skill, canonical reference, README, and lockstep tests.
9 +**Tech Stack:** TypeScript, TypeBox, Vitest, Pi extension harness, Markdown
13 +**Phase:** Single-plan implementation
17 +##
Task 1: Validate phase identifiers 21 +- Modify: `extensions/ultra/__tests__/spec.test.ts`
22 +- Modify: `extensions/ultra/spec.ts`
24 +- [x] **Step 1: Write failing phase-ID tests**
26 +Add tests proving `parseWorkflow` rejects a hyphenated phase ID, reserved ID `args`, and duplicate phase IDs, while allowing `item`:
29 +it(
"requires interpolation-safe phase identifiers", ()
=> {
30 + const s
= validSpec();
31 + s.phases[
0].
id = "first-pass";
32 + expect(()
=> parseWorkflow(s)).
toThrow(
/id/i);
35 +it(
"rejects duplicate phase identifiers", ()
=> {
36 + const s
= validSpec();
37 + s.phases[
1].
id = s.phases[
0].
id;
38 + expect(()
=> parseWorkflow(s)).
toThrow(
/duplicate phase id.*review/i);
41 +it(
"reserves args but allows item as a phase identifier", ()
=> {
42 + const reserved
= validSpec();
43 + reserved.phases[
0].
id = "args";
44 + expect(()
=> parseWorkflow(reserved)).
toThrow(
/reserved phase id.*args/i);
46 + const allowed
= validSpec();
47 + allowed.phases[
0].
id = "item";
48 + expect(()
=> parseWorkflow(allowed)).not.
toThrow();
52 +- [x] **Step 2: Verify RED**
54 +Run: `mise run test -- extensions/ultra/__tests__/spec.test.ts`
55 +Expected: the hyphenated, reserved, and duplicate-ID assertions fail because all are currently accepted.
57 +- [x] **Step 3: Implement minimal validation**
59 +Define one phase identifier schema using `^[A-Za-z_][A-Za-z0-9_]*$` and use it for both phase variants.
60 +After structural validation, reject `args` with `Invalid workflow spec: reserved phase id "args".` and reject repeated phase IDs with `Invalid workflow spec: duplicate phase id "<id>".`
61 +Do not reserve `item`.
63 +- [x] **Step 4: Verify GREEN**
65 +Run: `mise run test -- extensions/ultra/__tests__/spec.test.ts`
68 +##
Task 2: Expose semantic guidance through the tool schema 72 +- Modify: `extensions/ultra/__tests__/skill.test.ts`
73 +- Modify: `extensions/ultra/__tests__/harness.test.ts`
74 +- Modify: `extensions/ultra/spec.ts`
75 +- Modify: `extensions/ultra/index.ts`
77 +- [x] **Step 1: Write failing lockstep tests**
79 +In `skill.test.ts`, serialize `WorkflowSpecSchema` and assert that its descriptions contain all critical contracts already present in the skill/reference:
82 +const toolGuidance
= JSON.
stringify(WorkflowSpecSchema);
85 + "exact key in top-level schemas",
86 + "{PHASE.results[].FIELD[]}",
87 + "single brace-wrapped selector",
88 + "read, grep, find, ls, bash",
92 + expect(toolGuidance).
toContain(rule);
96 +In `harness.test.ts`, assert the registered tool description includes `schema strings must exactly name top-level schemas` and `interpolation uses only the closed grammar`.
98 +- [x] **Step 2: Verify RED**
100 +Run: `mise run test -- extensions/ultra/__tests__/skill.test.ts extensions/ultra/__tests__/harness.test.ts`
101 +Expected: FAIL because schema descriptions, compact tool reminders, and `max` documentation are missing.
103 +- [x] **Step 3: Add shared TypeBox descriptions**
105 +Add concise descriptions to the existing schemas for:
107 +- workflow `name`, `description`, inputs, named schemas, ordered phases, and `return`;
108 +- phase `id`, `kind`, `when`, `over`, `step`, and concurrency;
109 +- step prompt, tools, model, thinking level, and output schema;
110 +- the string and object branches of `step.schema`;
111 +- the literal-array and selector branches of fan-out `over`.
113 +Descriptions must encode the approved selector matrix exactly:
116 +prompt: embedded grammar tokens; item only in fan-out; literal braces forbidden
117 +when: one selector over args/prior phases; false/null/undefined/empty string/empty array skips
118 +over: literal array or one selector over args/prior phases; result must be an array
119 +return: one selector over args or completed phase results/failures; any result type
122 +State the closed token forms verbatim and the read-only fan-out tool rule verbatim.
124 +- [x] **Step 4: Add compact top-level tool guidance**
126 +Extend the `workflow` description with two short sentences:
129 +For inline specs, schema strings must exactly name top-level schemas. Interpolation uses only the closed grammar described on the spec fields.
132 +Describe tool `args` as values exposed through `{args.NAME}` and keep `background` unchanged.
133 +The shared `WorkflowSpecSchema` itself supplies the inline `spec` description.
135 +- [x] **Step 5: Verify GREEN**
137 +Run: `mise run test -- extensions/ultra/__tests__/skill.test.ts extensions/ultra/__tests__/harness.test.ts`
140 +##
Task 3: Synchronize `max` thinking guidance 144 +- Modify: `extensions/ultra/skills/authoring-workflows/SKILL.md`
145 +- Modify: `extensions/ultra/skills/authoring-workflows/references/spec-format.md`
146 +- Modify: `extensions/ultra/README.md`
147 +- Modify: `extensions/ultra/runner.ts`
148 +- Modify: `extensions/ultra/__tests__/runner.test.ts`
149 +- Modify: `extensions/ultra/__tests__/spec.test.ts`
150 +- Modify: `extensions/ultra/__tests__/skill.test.ts`
152 +- [x] **Step 1: Update all guidance surfaces**
154 +Add `max` after `xhigh` everywhere the supported thinking-level set or model-tier suffix is documented.
155 +Describe `max` as the highest opt-in reasoning budget for models that support it.
156 +Keep existing role-based tier guidance unchanged.
158 +- [x] **Step 2: Write and verify the failing tier-suffix test**
160 +Add a runner test using `modelTiers: { small: "openai/gpt-5-mini:max" }` and assert the session receives `thinkingLevel: "max"`.
161 +Run: `mise run test -- extensions/ultra/__tests__/runner.test.ts`
162 +Expected: FAIL because the current suffix parser treats `:max` as part of the model ID.
164 +- [x] **Step 3: Update runtime and complete-example validation**
166 +Add `max` to `splitTierSpec` in `runner.ts`.
167 +Update `spec.test.ts`'s complete thinking-level loop to include `max`.
169 +Change the complete-spec thinking-level assertion to accept:
172 +/^(off|minimal|low|medium|high|xhigh|max)$/;
175 +Add a lockstep assertion that runtime schema, skill, reference, and README all mention `max`.
177 +- [x] **Step 4: Run focused tests**
179 +Run: `mise run test -- extensions/ultra/__tests__/skill.test.ts extensions/ultra/__tests__/spec.test.ts extensions/ultra/__tests__/harness.test.ts extensions/ultra/__tests__/runner.test.ts`
182 +##
Task 4: Verify and commit 186 +- Modify: `docs/super/plans/2026-07-17-ultra-tool-guidance.md`
188 +- [x] **Step 1: Mark completed plan steps**
190 +Change every completed checkbox in this plan from `[ ]` to `[x]`.
192 +- [x] **Step 2: Run repository verification sequentially**
199 +mise run test -- extensions/ultra
203 +Expected: formatting makes no unexpected semantic changes; lint reports zero errors; all Ultra tests pass; typecheck and static checks pass.
205 +- [x] **Step 3: Inspect isolation**
207 +Run: `git status --short --branch`
208 +Expected: only Ultra guidance implementation files and this plan are changed in `.worktrees/ultra-tool-guidance`; the original `fix/leaks-reliability` worktree remains untouched.
210 +- [x] **Step 4: Commit**
213 +git add docs/super/plans/2026-07-17-ultra-tool-guidance.md docs/super/specs/2026-07-17-ultra-tool-guidance-design.md extensions/ultra/AGENTS.md extensions/ultra/spec.ts extensions/ultra/index.ts extensions/ultra/runner.ts extensions/ultra/README.md extensions/ultra/skills/authoring-workflows/SKILL.md extensions/ultra/skills/authoring-workflows/references/spec-format.md extensions/ultra/__tests__/spec.test.ts extensions/ultra/__tests__/skill.test.ts extensions/ultra/__tests__/harness.test.ts extensions/ultra/__tests__/runner.test.ts
214 +git commit -m "fix(ultra): guide valid inline workflows"