1 +#
Tune Cross-Platform Runner Implementation Plan 3 +> **For agentic workers:** REQUIRED SUB-SKILL: Use /skill:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
5 +**Goal:** Remove Tune's Bash/WSL dependency by running its fixed MJS benchmark with Pi's Node executable and invoking Git directly without regressing experiment behavior.
7 +**Architecture:** Keep production orchestration in `extensions/tune/index.ts`. Replace only process launch, termination, tool schema, and Git discard; use `process.execPath` for every Tune MJS file and preserve existing state, output, metrics, checks, VCS, and UI behavior.
9 +**Tech Stack:** TypeScript, Node standard library, Pi extension API, TypeBox, Vitest, Pi test harness, Mise
13 +**Phase:** Single-plan implementation
17 +###
Task 1: Establish Regression Harness and No-Bash RED 21 +- Modify: `extensions/tune/__tests__/harness.test.ts`
22 +- Modify: `scripts/test_extensions_static_check.mjs`
24 +- [ ] **Step 1: Add a deterministic active-session helper**
26 +Activate Tune without invoking `/tune`, because `/tune` sends an asynchronous user message.
27 +Create the persisted config, acquire the real session lock, and reconstruct through `session_tree`:
30 +async function activeTune(maxIterations
: number | null = null) {
31 + const extensionPath
= path.
resolve(
import.
meta.dirname,
"../index.ts");
32 + t
= await createTestSession({ extensions: [extensionPath] });
33 + fs.
mkdirSync(path.
join(t.cwd,
".pi",
"tune"), { recursive:
true });
35 + path.
join(t.cwd,
".pi",
"tune",
"session.jsonl"),
36 + "{\"type\":\"config\",\"name\":\"runtime\",\"metricName\":\"score\",\"metricUnit\":\"\",\"bestDirection\":\"lower\"}\n",
38 + if (maxIterations
!== null) {
40 + path.
join(t.cwd,
".pi",
"settings.json"),
41 + JSON.
stringify({ tune: { maxIterations } }),
44 + __test.
acquireTuneLock(t.cwd, t.session.sessionManager.
getSessionId());
45 + await t.session.extensionRunner.
emit({
46 + type:
"session_tree",
50 + return t.session.extensionRunner.extensions[
0];
54 +Import `calls`, `says`, and `when` from the harness.
55 +All public benchmark and VCS assertions must call the registered tools through `t.run`.
56 +Direct definition execution is allowed only where the harness DSL cannot expose `onUpdate` or an external `AbortSignal`.
58 +- [ ] **Step 2: Add the static no-Bash contract**
60 +Add before production changes:
63 +const tuneIndex
= readFileSync(join(ROOT,
"extensions/tune/index.ts"),
"utf8");
64 +assert(
!tuneIndex.includes(
"spawn(\"bash\""));
65 +assert(
!tuneIndex.includes(
"exec(\"bash\""));
68 +- [ ] **Step 3: Verify RED**
76 +Expected: static checking fails because Tune still contains both Bash invocations.
78 +###
Task 2: Replace Bash-Based Git Discard 82 +- Modify: `extensions/tune/__tests__/unit.test.ts`
83 +- Modify: `extensions/tune/__tests__/harness.test.ts`
84 +- Modify: `extensions/tune/index.ts:715-747`
86 +- [ ] **Step 1: Add exact-call and failure-order tests**
88 +Add mocked unit cases for direct arguments and failure ordering:
91 +it(
"uses direct Git calls on discard",
async ()
=> {
92 + const calls
: Array<[
string,
string[]]>
= [];
93 + const exec = async (cmd
: string, args
: string[])
=> {
94 + calls.
push([cmd, args]);
95 + return { code: cmd
=== "jj" ? 1 : 0, stdout:
"", stderr:
"" };
97 + await __test.
discardVcsChanges(exec,
"/repo",
"discard");
98 + expect(calls).
toEqual([
104 + ":(exclude,glob).pi/tune/**",
105 + ":(exclude,glob).pi/tune/*",
107 + [
"git", [
"clean",
"-fd",
"-e",
".pi/tune",
"-e",
".pi/tune/**"]],
111 +it(
"does not clean after checkout failure",
async ()
=> {
112 + const calls
: string[]
= [];
113 + const exec = async (cmd
: string, args
: string[])
=> {
114 + calls.
push(
`${cmd} ${args[0]}`);
115 + if (cmd
=== "jj")
return { code:
1, stdout:
"", stderr:
"" };
116 + return { code:
1, stdout:
"", stderr:
"checkout failed" };
118 + await expect(__test.
discardVcsChanges(exec,
"/repo",
"discard"))
119 + .rejects.
toThrow(
"git checkout failed: checkout failed");
120 + expect(calls).
toEqual([
"jj root",
"git checkout"]);
124 +Add the equivalent nonzero `git clean` case and require `git clean failed:` with bounded stderr.
126 +- [ ] **Step 2: Add a real Git repository test through `log_experiment`**
128 +In `harness.test.ts`, initialize a real temporary Git repository, configure local author identity, commit `tracked.txt`, then modify it and add `untracked.txt` plus `.pi/tune/ideas.md`.
129 +Activate Tune with Task 1's helper and call:
133 + when(
"discard regression", [
134 + calls(
"log_experiment", {
138 + description:
"regression",
139 + asi: { hypothesis:
"worse", next_action_hint:
"revert" },
146 +Assert the tracked file is restored, the untracked file is deleted, and both `ideas.md` and the appended discard entry in `session.jsonl` survive.
147 +This test detects incorrect Git pathspec semantics that mocked argument tests cannot.
149 +- [ ] **Step 3: Verify RED**
154 +mise run test -- extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/harness.test.ts
157 +Expected: mocked calls observe `bash -c`; the real repository test does not exercise direct Git calls.
159 +- [ ] **Step 4: Implement direct Git discard**
161 +Execute and check these calls sequentially:
164 +const checkout
= await exec(
"git", [
168 + ":(exclude,glob).pi/tune/**",
169 + ":(exclude,glob).pi/tune/*",
170 +], { cwd, timeout:
10000 });
171 +if (checkout.
code !== 0) {
173 + `git checkout failed: ${ 174 + (checkout.stdout + checkout.stderr).trim().slice(0, 200) 178 +const clean
= await exec(
"git", [
185 +], { cwd, timeout:
10000 });
186 +if (clean.
code !== 0) {
188 + `git clean failed: ${(clean.stdout + clean.stderr).trim().slice(0, 200)}`,
193 +Keep jj and the existing success text unchanged.
195 +- [ ] **Step 5: Verify GREEN and commit**
198 +mise run test -- extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/harness.test.ts
199 +git add extensions/tune/index.ts extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/harness.test.ts
200 +git commit -m "fix(tune): discard through direct git calls"
203 +Expected: all focused tests pass.
205 +###
Task 3: Characterize and Replace the Benchmark Runner 209 +- Modify: `extensions/tune/__tests__/harness.test.ts`
210 +- Modify: `extensions/tune/__tests__/unit.test.ts`
211 +- Modify: `extensions/tune/__tests__/remaining.test.ts`
212 +- Modify: `extensions/tune/index.ts:145-157,289-299,596-606,1980-2320`
214 +- [ ] **Step 1: Add fixed-schema and real-tool behavior tests**
216 +Assert `run_experiment` has no `command` property.
217 +Through `t.run(when(...calls("run_experiment", params)))`, cover:
219 +- success with stdout and stderr both captured
220 +- primary and secondary `METRIC` parsing
221 +- nonzero benchmark exit
222 +- missing benchmark, called twice with `maxIterations: 1`, never reporting the iteration limit
224 +- checks nonzero failure
226 +- output beyond line and byte limits
227 +- full-output spill file exists and contains early plus late output
229 +Use tiny per-test `benchmark.mjs` and `checks.mjs` fixtures.
235 + "console.log(`NODE=${process.execPath}`); console.error(\"stderr\"); console.log(\"METRIC score=42\"); console.log(\"METRIC bytes_kb=7\");\n",
237 +await t
!.
run(
when(
"run", [
calls(
"run_experiment", {}),
says(
"measured")]));
238 +const result
= t
!.events.
toolResultsFor(
"run_experiment")[
0];
239 +expect(result.details).
toMatchObject({
243 + parsedMetrics: { score:
42, bytes_kb:
7 },
245 +expect(result.
text).
toContain(
`NODE=${process.execPath}`);
246 +expect(result.
text).
toContain(
"stderr");
249 +- [ ] **Step 2: Add partial-update and abort tests on the registered definition**
251 +Call the same registered definition with `createContext()` only because harness records do not expose `onUpdate` or external cancellation.
252 +A benchmark prints immediately and exits after 1.2 seconds; assert at least one update includes its output.
253 +For abort, use an `AbortController`, abort after spawn, and assert rejection with `aborted`.
255 +- [ ] **Step 3: Add descendant termination tests through the real tool**
257 +Write a benchmark that spawns a child scheduled to create a marker after 300 ms, then remains alive.
258 +Run once with `timeout_seconds: 0.05` through `t.run`, and once through registered-definition execution with an abort signal.
259 +Wait 500 ms after each run and assert the marker never exists.
260 +This proves termination kills descendants, not only the benchmark parent.
262 +- [ ] **Step 4: Verify RED**
267 +mise run test -- extensions/tune/__tests__/*.test.ts
270 +Expected: schema and `{}` tool calls fail against the command-string runner.
272 +- [ ] **Step 5: Remove command strings and launch the fixed MJS file**
274 +Delete `isBenchmarkCommand`, its `__test` export, and its unit/remaining tests.
275 +Remove `command` from `RunParams`.
276 +Require `.pi/tune/benchmark.mjs` before changing runtime state.
277 +Use the fixed display value `node .pi/tune/benchmark.mjs` in runtime and result details.
281 +const child
= spawn(
process.
execPath, [benchmarkPath], {
283 + detached:
process.
platform !== "win32",
285 + stdio: [
"ignore",
"pipe",
"pipe"],
289 +Keep output accumulation, updates, spill files, timing, parsing, checks, and result construction otherwise unchanged.
290 +Update tool guidance and rendering to show the fixed benchmark and `run_experiment({})`.
292 +- [ ] **Step 6: Implement deterministic single-flight termination**
294 +Change `killTree` to return `Promise<void>` and accept optional platform/taskkill/direct-signal dependencies for controlled tests while production uses real defaults.
295 +On POSIX, retain negative-PID signalling with direct-PID fallback.
296 +On Windows, await `taskkill /F /T /PID <pid>`; call direct signalling only after taskkill spawn error or nonzero close.
298 +Inside the runner, define exactly one termination promise:
301 +let termination
: Promise<
void>
| null = null;
302 +const terminateOnce = ()
: Promise<
void>
=> {
303 + if (termination)
return termination;
304 + termination
= child.pid
305 + ? killTree(child.pid)
306 + : new Promise((resolve)
=> {
307 + child.
once(
"spawn", ()
=> void killTree(child.pid
!).
then(resolve));
308 + child.
once(
"error", ()
=> resolve());
310 + return termination;
314 +Timeout marks `processTimedOut` then calls `void terminateOnce()`.
315 +Abort calls `void terminateOnce()` before or after spawn.
316 +Use one settlement guard for `error` and `close`.
317 +Both paths clear timers/listeners, await termination, close the spill stream, then settle once.
318 +The close path rejects on abort and otherwise resolves the benchmark result.
320 +- [ ] **Step 7: Add Windows taskkill availability and controlled tests**
322 +Resolve `taskkill.exe` once from `SystemRoot\\System32`.
323 +On `session_start`, report one info notification when it is absent; direct child signalling remains fallback.
325 +Unit-test successful taskkill arguments and awaited nonzero fallback with a controlled executable and injected direct-signal callback.
326 +On native Windows, run the real Tune timeout path with a controlled fake taskkill executable on `PATH` or injected resolved path and assert its log is written before the tool returns.
327 +Emit two `session_start` events with an injected missing path and assert only one info notification.
328 +Skip only the native-Windows real-tool fixture on other platforms; helper ordering tests run everywhere.
330 +- [ ] **Step 8: Verify GREEN and commit**
333 +mise run test -- extensions/tune/__tests__/*.test.ts
334 +git add extensions/tune/index.ts extensions/tune/__tests__/harness.test.ts extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/remaining.test.ts
335 +git commit -m "refactor(tune): run fixed mjs benchmark"
338 +Expected: all Tune tests pass, including timeout and abort with no descendant marker.
340 +###
Task 4: Use Pi's Node Runtime for Checks and Hooks 344 +- Modify: `extensions/tune/__tests__/unit.test.ts`
345 +- Modify: `extensions/tune/__tests__/harness.test.ts`
346 +- Modify: `extensions/tune/hooks.ts:100-110`
347 +- Modify: `extensions/tune/index.ts:2220-2230`
349 +- [ ] **Step 1: Add PATH-shadow regression tests**
351 +Prepend a controlled fake `node` executable that writes a marker and exits nonzero.
352 +Run a real hook MJS and a passing benchmark plus checks MJS.
353 +Both scripts print `process.execPath`.
354 +Assert hook/check output contains Pi's executable and the fake marker is absent.
355 +Restore `PATH` in `finally`.
356 +This fails against current `spawn("node")` and `pi.exec("node", ...)` calls.
358 +- [ ] **Step 2: Verify RED**
361 +mise run test -- extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/harness.test.ts
364 +Expected: hook or checks reaches the fake PATH executable.
366 +- [ ] **Step 3: Replace PATH-based Node calls**
371 +spawn(
process.
execPath, [script], { cwd: payload.cwd, timeout: TIMEOUT_MS });
377 +pi.
exec(
process.
execPath, [checksPath], {
379 + timeout: checksTimeout,
384 +- [ ] **Step 4: Verify GREEN and commit**
387 +mise run test -- extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/harness.test.ts
388 +git add extensions/tune/index.ts extensions/tune/hooks.ts extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/harness.test.ts
389 +git commit -m "fix(tune): use pi node for mjs scripts"
392 +Expected: focused tests pass and the fake PATH node is unused.
394 +###
Task 5: Documentation and Repository Verification 398 +- Modify: `extensions/tune/README.md`
399 +- Modify: `extensions/tune/CHANGELOG.md`
400 +- Modify: `CHANGELOG.md`
402 +- [ ] **Step 1: Update documentation**
404 +Document `run_experiment` without a command argument.
405 +State that benchmark, checks, and hooks run with Pi's Node executable.
406 +State that experiments and VCS handling require no Bash, Git Bash, PowerShell, or WSL.
407 +Remove accepted-command and wrapper examples.
408 +Record the breaking tool parameter removal and direct Git discard.
410 +- [ ] **Step 2: Run repository checks**
419 +Expected: every command exits zero, including the no-Bash static contract added before implementation.
421 +- [ ] **Step 3: Commit documentation and static guard**
424 +git add scripts/test_extensions_static_check.mjs extensions/tune/README.md extensions/tune/CHANGELOG.md CHANGELOG.md
425 +git commit -m "docs(tune): document shell-free runner"
428 +###
Task 6: Release Gate 434 +- [ ] **Step 1: Run Tune browser E2E**
437 +mise //extensions/tune:e2e
440 +Expected: Chromium and Firefox dashboard tests pass.
442 +- [ ] **Step 2: Run full CI**
448 +Expected: check, lint, test, and E2E exit zero.
450 +- [ ] **Step 3: Inspect final state**
453 +git status --short --branch
454 +git log --oneline -10
457 +Expected: clean implementation worktree and focused commits for the spec, plan, Git discard, MJS runner, runtime consistency, and documentation.