repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by bugabinga

docs: plan tune e2e and ci promotion

7f1e0bf4727c8d79c83e98c739fd528db1af3805

Oliver Jan Krylow <oliver@bugabinga.net> ·

unsigned

Browse filesCompare with HEAD

Changed files (1)

docs/super/plans/2026-07-12-mise-task-architecture-phase-4-tune-e2e-ci.md

Added
new file mode 100644
1 +# Mise Task Architecture Phase 4 Implementation Plan
3 +> **For agentic workers:** REQUIRED SUB-SKILL: Use /skill:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
5 +**Goal:** Cover tune’s live dashboard in Chromium and Firefox, make all four browser suites mandatory in CI, and finalize the root task surface.
7 +**Architecture:** Extract tune’s embedded dashboard HTTP/SSE server into one focused module used by production and Playwright. Tune owns its local suite and task, while root mise installs browser system dependencies only under CI and runs the release gate in `check → lint → test → e2e` order.
9 +**Tech Stack:** Playwright 1.61, Chromium, Firefox, Node HTTP/SSE, mise monorepo tasks, TypeScript, GitHub Actions.
11 +**Roadmap:** [`docs/super/roadmaps/2026-07-12-mise-task-architecture-roadmap.md`](../roadmaps/2026-07-12-mise-task-architecture-roadmap.md)
13 +**Phase:** Phase 4: Tune Browser E2E And CI Promotion
15 +---
17 +## Design Note
19 +Tune’s dashboard is read-only.
20 +It has no browser-to-extension mutation or result path.
21 +The E2E contract therefore covers the real server-to-browser JSONL load and SSE refresh path plus the dashboard’s theme interaction instead of inventing a write API.
23 +## File Structure
25 +**Create:**
27 +- `extensions/tune/dashboard.ts`: focused dashboard HTTP/SSE lifecycle.
28 +- `extensions/tune/playwright.config.ts`: local overlay on shared browser policy.
29 +- `extensions/tune/__e2e__/dashboard.spec.ts`: real dashboard load, interaction, and live-update smoke test.
30 +- `extensions/tune/mise.toml`: local E2E task ownership.
32 +**Modify:**
34 +- `extensions/tune/index.ts`: delegate dashboard serving, updates, and shutdown to `dashboard.ts`.
35 +- `extensions/tune/package.json`: declare Playwright.
36 +- `package-lock.json`: record the workspace dependency.
37 +- `mise.toml`: install Linux browser dependencies under CI and order the final release gate.
38 +- `AGENTS.md`: document E2E as part of CI.
39 +- `extensions/tune/README.md`: document dashboard E2E usage.
41 +**Unchanged:**
43 +- `extensions/tune/assets/template.html`: browser behavior remains unchanged.
44 +- Existing tune Vitest suites remain detailed logic coverage.
45 +- `.github/workflows/check.yml`: continues to call only `mise install` and `mise run ci`.
46 +- Benchmark tasks remain outside CI.
48 +### Task 1: Extract The Dashboard Server Behind A Tested API
50 +**Files:**
52 +- Create: `extensions/tune/dashboard.ts`
53 +- Create: `extensions/tune/__e2e__/dashboard.spec.ts`
54 +- Create: `extensions/tune/playwright.config.ts`
55 +- Modify: `extensions/tune/package.json`
56 +- Modify: `package-lock.json`
58 +- [ ] **Step 1: Record the baseline**
60 +Run:
62 +```bash
63 +git status --short
64 +mise tasks --all --name-only
65 +```
67 +Expected: unrelated dirty work remains visible and preserved.
68 +`//extensions/tune:e2e` does not exist.
70 +- [ ] **Step 2: Create tune’s local Playwright overlay**
72 +Create `extensions/tune/playwright.config.ts`:
74 +```ts
75 +import { defineConfig } from "@playwright/test";
76 +import base from "../../test/playwright.base.ts";
78 +export default defineConfig(base, {
79 + testDir: "./__e2e__",
80 + testMatch: "*.spec.ts",
81 + timeout: 30_000,
82 + expect: { timeout: 5_000 },
83 +});
84 +```
86 +- [ ] **Step 3: Declare Playwright locally**
88 +Add to tune’s `devDependencies`:
90 +```json
91 +"@playwright/test": "^1.61.0"
92 +```
94 +Keep existing scripts and dependencies unchanged.
95 +Run `mise run lock`.
96 +Expected: only tune’s workspace dependency metadata changes.
98 +- [ ] **Step 4: Write the failing dashboard E2E contract**
100 +Create `extensions/tune/__e2e__/dashboard.spec.ts`:
102 +```ts
103 +import { expect, test } from "@playwright/test";
104 +import { appendFile, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
105 +import { tmpdir } from "node:os";
106 +import { join } from "node:path";
107 +import { startTuneDashboard, type TuneDashboard } from "../dashboard.ts";
109 +let dashboard: TuneDashboard | undefined;
110 +let dir = "";
112 +test.afterEach(async () => {
113 + dashboard?.close();
114 + dashboard = undefined;
115 + if (dir) await rm(dir, { recursive: true, force: true });
116 + dir = "";
117 +});
119 +test("loads runs, switches theme, and receives live updates", async ({ page }) => {
120 + const pageErrors: string[] = [];
121 + page.on("pageerror", (error) => pageErrors.push(error.message));
122 + dir = await mkdtemp(join(tmpdir(), "tune-e2e-"));
123 + const jsonlPath = join(dir, "session.jsonl");
124 + const template = await readFile(
125 + join(import.meta.dirname, "..", "assets", "template.html"),
126 + "utf8",
127 + );
128 + const html = template.replace("__TUNE_TITLE__", "Latency");
129 + await writeFile(
130 + jsonlPath,
132 + JSON.stringify({
133 + type: "config",
134 + name: "Latency",
135 + metricName: "latency",
136 + metricUnit: "ms",
137 + bestDirection: "lower",
138 + }),
139 + JSON.stringify({
140 + run: 1,
141 + metric: 100,
142 + status: "keep",
143 + description: "baseline",
144 + }),
145 + ].join("\n") + "\n",
146 + );
148 + dashboard = await startTuneDashboard(html, jsonlPath);
149 + const eventsReady = page.waitForResponse(
150 + (response) =>
151 + response.url().endsWith("/events") && response.status() === 200,
152 + );
153 + await page.goto(dashboard.url);
154 + await eventsReady;
155 + await expect(page.locator("#title")).toHaveText("Latency");
156 + await expect(page.locator("#summary")).toContainText("latency (ms)");
157 + await expect(page.locator("#runs")).toHaveText("1");
158 + await expect(page.locator("#rows")).toContainText("baseline");
160 + await page.locator("#theme-mode").selectOption("dark");
161 + await expect(page.locator("html")).toHaveAttribute("data-theme", "dark");
163 + await appendFile(
164 + jsonlPath,
165 + JSON.stringify({
166 + run: 2,
167 + metric: 80,
168 + status: "keep",
169 + description: "faster",
170 + }) + "\n",
171 + );
172 + dashboard.broadcast();
174 + await expect(page.locator("#runs")).toHaveText("2");
175 + await expect(page.locator("#rows")).toContainText("faster");
176 + await expect(page.locator("#baseline-best")).toHaveText("100ms → 80ms");
177 + expect(pageErrors).toEqual([]);
178 +});
179 +```
181 +Run:
183 +```bash
184 +npm exec --workspace extensions/tune -- playwright test --config playwright.config.ts
185 +```
187 +Expected: FAIL because `dashboard.ts` does not exist.
189 +- [ ] **Step 5: Implement the minimal dashboard lifecycle module**
191 +Create `extensions/tune/dashboard.ts`:
193 +```ts
194 +import { readFile } from "node:fs";
195 +import { createServer, type Server, type ServerResponse } from "node:http";
197 +export type TuneDashboard = {
198 + url: string;
199 + update(html: string, jsonlPath: string): void;
200 + broadcast(): void;
201 + close(): void;
204 +export async function startTuneDashboard(
205 + html: string,
206 + jsonlPath: string,
207 +): Promise<TuneDashboard> {
208 + let currentHtml = html;
209 + let currentJsonlPath = jsonlPath;
210 + let closed = false;
211 + const clients = new Set<ServerResponse>();
212 + const server = createServer((req, res) => {
213 + const url = new URL(req.url ?? "/", "http://127.0.0.1");
214 + if (url.pathname === "/events") {
215 + res.writeHead(200, {
216 + "Content-Type": "text/event-stream",
217 + "Cache-Control": "no-cache",
218 + Connection: "keep-alive",
219 + });
220 + res.write("retry: 1000\n\n");
221 + clients.add(res);
222 + res.on("close", () => clients.delete(res));
223 + return;
225 + if (url.pathname === "/") {
226 + res.writeHead(200, {
227 + "Content-Type": "text/html; charset=utf-8",
228 + "Cache-Control": "no-store",
229 + });
230 + res.end(currentHtml);
231 + return;
233 + if (url.pathname === "/session.jsonl") {
234 + readFile(currentJsonlPath, (error, data) => {
235 + if (error) {
236 + res.writeHead(404);
237 + res.end();
238 + return;
240 + res.writeHead(200, {
241 + "Content-Type": "application/x-ndjson; charset=utf-8",
242 + "Cache-Control": "no-store",
243 + });
244 + res.end(data);
245 + });
246 + return;
248 + res.writeHead(404);
249 + res.end();
250 + });
251 + const port = await listen(server);
252 + return {
253 + url: `http://127.0.0.1:${port}`,
254 + update(nextHtml, nextJsonlPath) {
255 + currentHtml = nextHtml;
256 + currentJsonlPath = nextJsonlPath;
257 + },
258 + broadcast() {
259 + for (const client of clients) {
260 + try {
261 + client.write("event: jsonl-updated\n");
262 + client.write(`data: ${Date.now()}\n\n`);
263 + } catch {
264 + clients.delete(client);
267 + },
268 + close() {
269 + if (closed) return;
270 + closed = true;
271 + for (const client of clients) client.end();
272 + clients.clear();
273 + server.close();
274 + },
275 + };
278 +function listen(server: Server): Promise<number> {
279 + return new Promise((resolve, reject) => {
280 + server.once("error", reject);
281 + server.listen(0, "127.0.0.1", () => {
282 + server.off("error", reject);
283 + const address = server.address();
284 + if (!address || typeof address === "string") {
285 + reject(new Error("Failed to bind tune dashboard server"));
286 + return;
288 + resolve(address.port);
289 + });
290 + });
292 +```
294 +No authentication is added because the existing dashboard is loopback-only and read-only.
295 +No filesystem path is accepted from HTTP requests.
297 +- [ ] **Step 6: Verify the E2E contract turns green**
299 +Run:
301 +```bash
302 +npm exec --workspace extensions/tune -- playwright test --config playwright.config.ts
303 +mise run check
304 +```
306 +Expected: one dashboard test passes in Chromium and Firefox, and static checks pass with Playwright declared locally.
308 +- [ ] **Step 7: Verify static checks and commit the tested dashboard module**
310 +Run:
312 +```bash
313 +git add extensions/tune/dashboard.ts extensions/tune/playwright.config.ts extensions/tune/__e2e__/dashboard.spec.ts extensions/tune/package.json package-lock.json
314 +git diff --cached --check
315 +git commit -m "test(tune): cover live dashboard in browsers"
316 +```
318 +Expected: the dashboard module, Playwright overlay, E2E test, and declared Playwright dependency are committed.
320 +### Task 2: Route Production Dashboard Behavior Through The Module
322 +**Files:**
324 +- Modify: `extensions/tune/index.ts:10-17,3024-3180`
326 +- [ ] **Step 1: Replace dashboard server imports**
328 +Remove this import from `extensions/tune/index.ts`:
330 +```ts
331 +import { createServer, type Server, type ServerResponse } from "node:http";
332 +```
334 +Add beside other local imports:
336 +```ts
337 +import { startTuneDashboard, type TuneDashboard } from "./dashboard.ts";
338 +```
340 +- [ ] **Step 2: Collapse dashboard state to one owner**
342 +Replace:
344 +```ts
345 +let dashboardServer: Server | null = null;
346 +let dashboardServerPort: number | null = null;
347 +let dashboardIndexPath: string | null = null;
348 +let dashboardJsonlPath: string | null = null;
349 +const dashboardSseClients = new Set<ServerResponse>();
350 +```
352 +with:
354 +```ts
355 +let dashboardServer: TuneDashboard | null = null;
356 +```
358 +- [ ] **Step 3: Delegate broadcast and shutdown**
360 +Replace `broadcastDashboardUpdate()` with:
362 +```ts
363 +function broadcastDashboardUpdate(_cwd: string): void {
364 + dashboardServer?.broadcast();
366 +```
368 +Replace `stopDashboardServer()` with:
370 +```ts
371 +function stopDashboardServer(): void {
372 + dashboardServer?.close();
373 + dashboardServer = null;
375 +```
377 +- [ ] **Step 4: Remove temporary HTML export and inline server creation**
379 +Inside `exportDashboard()`, keep template lookup, JSONL reading, title extraction, and HTML injection.
380 +Replace everything from creation of `exportDir` through assignment of `dashboardServerPort` with:
382 +```ts
383 +if (dashboardServer) dashboardServer.update(html, jsonlPath);
384 +else dashboardServer = await startTuneDashboard(html, jsonlPath);
385 +```
387 +Replace:
389 +```ts
390 +const url = `http://127.0.0.1:${dashboardServerPort}`;
391 +openInBrowser(url);
392 +ctx.ui.notify(`Dashboard at ${url} (live updates)`, "info");
393 +```
395 +with:
397 +```ts
398 +openInBrowser(dashboardServer.url);
399 +ctx.ui.notify(`Dashboard at ${dashboardServer.url} (live updates)`, "info");
400 +```
402 +Do not change command behavior, template rendering, or browser opener policy.
404 +- [ ] **Step 5: Run focused deterministic tests**
406 +Run:
408 +```bash
409 +mise run test -- extensions/tune/__tests__/unit.test.ts extensions/tune/__tests__/remaining.test.ts
410 +npm exec --workspace extensions/tune -- playwright test --config playwright.config.ts
411 +```
413 +Expected: existing dashboard/template tests and both browser projects pass.
415 +- [ ] **Step 6: Commit production integration**
417 +Run:
419 +```bash
420 +git add extensions/tune/index.ts
421 +git diff --cached --check
422 +git commit -m "refactor(tune): isolate dashboard server"
423 +```
425 +Expected: only the production delegation is committed.
427 +### Task 3: Add Tune E2E Ownership
429 +**Files:**
431 +- Create: `extensions/tune/mise.toml`
433 +- [ ] **Step 1: Create tune’s local task**
435 +Create `extensions/tune/mise.toml`:
437 +```toml
438 +[tasks.e2e]
439 +description = "Run tune dashboard browser E2E tests"
440 +depends = ["//:install"]
441 +run = "npm exec -- playwright test --config playwright.config.ts"
442 +```
444 +- [ ] **Step 2: Verify qualified ownership**
446 +Run:
448 +```bash
449 +mise tasks validate
450 +mise tasks --all --name-only
451 +mise run //extensions/tune:e2e
452 +```
454 +Expected: tune’s qualified task is discovered and passes in Chromium and Firefox.
456 +- [ ] **Step 3: Commit local ownership**
458 +Run:
460 +```bash
461 +git add extensions/tune/mise.toml
462 +git diff --cached --check
463 +git commit -m "build(tune): add dashboard e2e task"
464 +```
466 +Expected: only tune’s local task is committed.
468 +### Task 4: Promote E2E Into The Release Gate
470 +**Files:**
472 +- Modify: `mise.toml:54-86`
473 +- Verify: `.github/workflows/check.yml`
475 +- [ ] **Step 1: Verify the current CI order excludes E2E**
477 +Run:
479 +```bash
480 +mise run --dry-run ci
481 +```
483 +Expected: lint, test, and check appear; E2E does not.
485 +- [ ] **Step 2: Install browser system dependencies only under CI**
487 +In root `tasks.e2e`, replace the browser install command with this templated value:
489 +```toml
490 +"npm exec -- playwright install{% if env.CI %} --with-deps{% endif %} chromium firefox",
491 +```
493 +Local `mise run e2e` downloads browser engines without mutating host system packages.
494 +GitHub Actions sets `CI`, so the same root task uses Playwright’s supported dependency installation path.
496 +- [ ] **Step 3: Set the final release-gate order**
498 +Replace root `tasks.ci.run` with:
500 +```toml
501 +run = [
502 + "mise run check",
503 + "mise run lint",
504 + "mise run test",
505 + "mise run e2e",
507 +```
509 +Check runs first and fails fastest on type or package-structure errors.
510 +E2E runs last because it is the most expensive gate.
512 +- [ ] **Step 4: Verify task resolution locally**
514 +Run:
516 +```bash
517 +mise tasks validate
518 +mise run --dry-run e2e
519 +mise run --dry-run ci
520 +```
522 +Expected:
524 +- Local E2E install omits `--with-deps`.
525 +- CI order is check, lint, test, e2e.
526 +- Root E2E discovers diff-review, UI, the-system, and tune.
528 +Run:
530 +```bash
531 +CI=true mise run --dry-run e2e
532 +```
534 +Expected: browser install includes `--with-deps`.
536 +- [ ] **Step 5: Verify GitHub Actions remains generic**
538 +Read `.github/workflows/check.yml`.
539 +Expected: it installs mise/tools and invokes only `mise run ci`; no extension-specific test command is added.
541 +- [ ] **Step 6: Commit release-gate promotion**
543 +Run:
545 +```bash
546 +git add mise.toml
547 +git diff --cached --check
548 +git commit -m "ci: require extension browser e2e"
549 +```
551 +Expected: only browser installation policy and CI ordering change.
553 +### Task 5: Document And Verify The Final Browser Gate
555 +**Files:**
557 +- Modify: `AGENTS.md`
558 +- Modify: `extensions/tune/README.md`
560 +- [ ] **Step 1: Update root CI documentation**
562 +Change the `CI` task description in `AGENTS.md` to:
564 +```markdown
565 +Release gate (check + lint + test + e2e)
566 +```
568 +The existing root E2E row remains unchanged.
570 +- [ ] **Step 2: Document tune E2E usage**
572 +Add near tune’s dashboard behavior documentation:
574 +```markdown
575 +## E2E
577 +Run root `mise run e2e` first to install browsers and execute every extension suite.
578 +After that bootstrap, run `mise run e2e` here, or `mise //extensions/tune:e2e` from the repository root, to test dashboard loading, theme interaction, and live updates in Chromium and Firefox.
579 +```
581 +- [ ] **Step 3: Format only changed documentation**
583 +Run:
585 +```bash
586 +mise exec -- dprint fmt AGENTS.md extensions/tune/README.md
587 +mise exec -- dprint check AGENTS.md extensions/tune/README.md
588 +```
590 +Expected: both files are formatted without touching unrelated dirty work.
592 +- [ ] **Step 4: Commit documentation**
594 +Run:
596 +```bash
597 +git add AGENTS.md extensions/tune/README.md
598 +git diff --cached --check
599 +git commit -m "docs: explain mandatory browser gate"
600 +```
602 +Expected: only final task documentation is committed.
604 +- [ ] **Step 5: Run all browser suites**
606 +Run:
608 +```bash
609 +mise run e2e
610 +```
612 +Expected:
614 +- Diff-review passes in Chromium and Firefox.
615 +- UI fake-RPC passes in Chromium and Firefox; real-RPC remains skipped.
616 +- The-system mockup passes in Chromium and Firefox.
617 +- Tune dashboard passes in Chromium and Firefox.
619 +- [ ] **Step 6: Run the complete release gate**
621 +Run:
623 +```bash
624 +mise run ci
625 +```
627 +Expected: check, lint, all Vitest tests, and all browser E2E suites pass in documented order.
629 +- [ ] **Step 7: Inspect final phase scope**
631 +Run:
633 +```bash
634 +git status --short
635 +git log -8 --oneline
636 +```
638 +Expected: unrelated dirty work remains untouched.
639 +No bundling or emitted JavaScript exists.
640 +Phase 5 TypeScript standardization remains unimplemented.
642 +- [ ] **Step 8: Report completion evidence**
644 +Report:
646 +- Tune browser results.
647 +- Aggregate four-extension E2E results.
648 +- Final `mise run ci` order and result.
649 +- Commits created.
650 +- Confirmation that GitHub Actions remains extension-agnostic.
651 +- Confirmation that Phase 5 remains unimplemented.