1 +#
Leaks Detector Reliability Phase 1 Implementation Plan 3 +> **For agentic workers:** REQUIRED SUB-SKILL: Use /skill:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
5 +**Goal:** Replace broad detector guesses with precision-first parsing and lock every detector family behind deterministic and generated positive/negative tests.
7 +**Architecture:** Strict provider formats keep regexes. Structured formats use bounded candidate scans followed by independent validation. Ambiguous assignments and metadata remain visible. Dedicated fast-check tests are excluded from unit tests.
9 +**Tech Stack:** TypeScript ESM, Node standard library, Vitest 4, fast-check 4.9, @fast-check/vitest 0.4, Mise.
11 +**Roadmap:** `docs/super/roadmaps/2026-07-17-leaks-detector-reliability-roadmap.md`
13 +**Phase:** Phase 1: Detector Precision
17 +###
Task 1: Wire Dedicated Property Tests 19 +**Files:** `package.json`, `package-lock.json`, `mise.toml`, `extensions/leaks/vitest.fuzz.config.ts`, `extensions/leaks/__fuzz__/detectors.fuzz.test.ts`
21 +- [ ] Add these root dev dependencies:
24 +"@fast-check/vitest":
"^0.4.1",
25 +"fast-check":
"^4.9.0" 28 +- [ ] Create `extensions/leaks/vitest.fuzz.config.ts`:
31 +import { defineConfig }
from "vitest/config";
32 +export default defineConfig({
34 + include: [
"extensions/leaks/__fuzz__/**/*.test.ts"],
35 + reporters: [
"default"],
38 + testTimeout:
110_000,
43 +- [ ] Add a smoke property using `test.prop({ text: fc.string() })` that calls `redactText` twice on fresh engines and expects equal changed/count/labels output.
45 +- [ ] Add a Mise `fuzz` task that runs Vitest through Node and enforces a hard 120-second process deadline with Node `spawnSync(..., { timeout: 120_000 })`; exit 124 on timeout.
47 +- [ ] Define replay inputs `FC_SEED` and `FC_PATH` in the fuzz setup and pass them as `seed` and `path` to fast-check when set.
49 +- [ ] Run `mise run lock && mise run fuzz`; expect PASS.
51 +- [ ] Commit `test(leaks): add dedicated property suite`.
53 +###
Task 2: Establish The Precision Contract First 55 +**Files:** `extensions/leaks/SPEC.md`
57 +- [ ] Replace “JWT decoding is out of scope” with exact JWT requirements: three strict Base64URL segments, decoded non-array JSON objects, nonempty string `alg`, nonempty strict signature.
59 +- [ ] Replace substring-sensitive names with separator-aware complete tokens.
61 +- [ ] State that ambiguous assignments and context-only env names remain visible unless the value has a provider-specific shape or parsed secret component.
63 +- [ ] Replace PuTTY-to-EOF behavior with exact `Private-Lines` consumption.
65 +- [ ] State Phase 1 still uses volatile HMAC12 mappings; durable V1 mappings remain Phase 2.
67 +- [ ] Run `git diff --check -- extensions/leaks/SPEC.md`; expect no errors.
69 +- [ ] Commit `docs(leaks): specify precision-first detectors`.
71 +###
Task 3: Parse Structured Candidates With Paired TDD 73 +**Files:** `extensions/leaks/src/structures.ts`, `extensions/leaks/src/safety.ts`, `extensions/leaks/__tests__/structures.test.ts`, `extensions/leaks/__tests__/false-positives.test.ts`
75 +- [ ] Add JWT red/green cases:
78 +expect(
findJwtSpans(
"agendaFlick.visibleArea.yPosition")).
toEqual([]);
79 +expect(
findJwtSpans(
"application/vnd.openxmlformats-officedocument")).
toEqual([]);
80 +expect(
findJwtSpans(validJwt)).
toMatchObject([{ value: validJwt, labels: [{ tag:
"JWT" }] }]);
81 +expect(engine.
restoreText(engine.
redactText(
`x=${validJwt}`).
text)).
toBe(
`x=${validJwt}`);
84 +- [ ] Implement strict Base64URL validation before `Buffer.from`:
87 +function decodeJsonObject(segment
: string) {
88 + if (
!/^[A-Za-z0-9_-]+$/.
test(segment))
return;
89 + const bytes
= Buffer.
from(segment,
"base64url");
90 + if (bytes.
toString(
"base64url").
replace(
/=+$/,
"")
!== segment)
return;
91 + const value
= JSON.
parse(
new TextDecoder(
"utf-8", { fatal:
true }).
decode(bytes));
92 + if (
!value
|| typeof value
!== "object" || Array.
isArray(value))
return;
93 + return value
as Record<
string,
unknown>;
97 +Catch parse/decode errors and return no span.
98 +Require `typeof header.alg === "string" && header.alg.length > 0`.
100 +- [ ] Add query/cookie red/green cases:
103 +expect(
findSensitiveQuerySpans(
"?monkey=ordinary-value")).
toEqual([]);
104 +expect(
findSensitiveQuerySpans(
"?token=ordinary-secret-value")).
toHaveLength(
1);
105 +expect(
findCookieSpans(
"Cookie: keyboard=mechanical-layout")).
toEqual([]);
106 +expect(
findCookieSpans(
"Cookie: session=ordinary-secret-value")).
toHaveLength(
1);
109 +- [ ] Export one `isSensitiveName` helper from `safety.ts` that normalizes `-` to `_`, splits complete tokens, accepts explicit compound names such as `API_KEY` and `CLIENT_SECRET`, and never uses `includes`.
111 +- [ ] Scan URL candidates with a bounded regex, parse each with `new URL(candidate)`, and locate raw component spans by searching inside the unchanged candidate string; reject a candidate when a decoded value cannot be mapped uniquely to one raw source span.
113 +- [ ] Require strict Basic credentials with `/^[A-Za-z0-9+/]+={0,2}$/`, canonical Base64 re-encoding, and decoded `:`.
115 +- [ ] Parse cookies only from a complete `Cookie:` line and classify complete pair names with `isSensitiveName`.
117 +- [ ] Add assignment red/green cases for source constants, lower-camel references, type syntax, XML prose, provider-shaped config values, exact env-style values, transformed text, exact labels, and restoration.
119 +- [ ] Replace broad assignment authority with line-anchored recognizers. Reject lines beginning with `const`, `let`, `var`, `function`, `class`, `type`, `interface`, or containing template/interpolation syntax. For JSON/YAML/TOML/XML-like forms, redact only `highConfidenceSecretShape(value)`; dotenv/env exact learning handles arbitrary opaque values.
121 +- [ ] Run `mise run test -- extensions/leaks/__tests__/structures.test.ts extensions/leaks/__tests__/false-positives.test.ts`; expect PASS.
123 +- [ ] Commit `fix(leaks): parse structured secret candidates`.
125 +###
Task 4: Bound Provider, Private-Key, And Environment Detection With Paired TDD 127 +**Files:** `extensions/leaks/src/detectors.ts`, `extensions/leaks/src/dotenv.ts`, `extensions/leaks/__tests__/detectors.test.ts`, `extensions/leaks/__tests__/dotenv.test.ts`
129 +- [ ] For every provider rule, generate one deterministic valid control and three exact near misses: changed prefix, one-short length, and invalid alphabet. Assert exact zero or one span and exact label.
131 +- [ ] Replace the PuTTY regex with a line parser that finds `Private-Lines: N`, consumes exactly N subsequent Base64 lines, and ends before `Private-MAC`. Add a trailing sentinel assertion proving unrelated text remains unchanged after redaction/restoration.
133 +- [ ] Add env red/green cases:
136 +expect(
learnEnvironmentValues({ GPG_KEY_ID:
"0123456789abcdef" }).values).
toEqual([]);
137 +expect(
learnEnvironmentValues({ SESSION_MANAGER:
"local/session-manager" }).values).
toEqual([]);
138 +expect(
learnEnvironmentValues({ PROD_ACCESS_TOKEN: opaque }).values.
map(v
=> v.
value)).
toEqual([opaque]);
141 +Also cover `KEY`, `AUTH`, `SESSION`, `COOKIE`, `JWT`, `SEED`, public prefixes, and credentialed/uncredentialed URLs.
143 +- [ ] Split normalized env names into explicit high-precision, context-required, metadata-suffix, and public-prefix classes matching the spec table.
145 +- [ ] Parse env URL values with `URL`; learn raw password and exact sensitive query values only. Do not learn the complete URL.
147 +- [ ] Run `mise run test -- extensions/leaks/__tests__/detectors.test.ts extensions/leaks/__tests__/dotenv.test.ts`; expect PASS.
149 +- [ ] Commit `fix(leaks): tighten provider and environment detection`.
151 +###
Task 5: Fuzz Every Detector Family 153 +**Files:** `extensions/leaks/__fuzz__/arbitraries.ts`, `extensions/leaks/__fuzz__/detectors.fuzz.test.ts`
155 +- [ ] Build semantic arbitraries without importing production regexes for provider tokens, valid/invalid JWTs, URLs, query names, headers, cookies, assignments, private keys, dotenv/env names, and custom regex rules.
157 +- [ ] Every family must generate empty, boundary-length, Unicode-surrounded, punctuation, quoted, multiline, and oversized cases.
159 +- [ ] Positive properties assert exact span/value/label, transformed text, unchanged surrounding bytes, and restore identity.
161 +- [ ] Negative properties mutate one semantic requirement and assert zero spans and unchanged text.
163 +- [ ] Add custom-regex properties for zero/one/multiple captures, empty-match progress, unsafe-pattern rejection, span boundaries, and adversarial runtime.
165 +- [ ] Add engine properties for idempotence, no source mutation, selected non-overlap, and preservation outside selected spans.
167 +- [ ] Prove shrinking/replay by temporarily forcing a property failure, capture reported seed/path, rerun with `FC_SEED` and `FC_PATH`, then restore the assertion. No failing code is committed.
169 +- [ ] Run `mise run fuzz && mise run test -- extensions/leaks/__tests__`; expect PASS.
171 +- [ ] Commit `test(leaks): fuzz every detector boundary`.
173 +###
Task 6: Phase Verification 175 +**Files:** all Phase 1 files
177 +- [ ] Run `mise run fmt`.
179 +- [ ] Run `mise run lint`.
181 +- [ ] Run `mise run test`.
183 +- [ ] Run `mise run fuzz`; verify wall time is below two minutes.
185 +- [ ] Run `mise run check`.
187 +- [ ] Inspect `git status --short` and preserve unrelated dirty files.
189 +- [ ] Commit only any formatter changes belonging to Phase 1 as `style(leaks): format detector precision work`.
191 +README and changelog remain Phase 3 scope.