repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by bugabinga

test(leaks): gate durable behavior in CI

3134c7522e119daf0fd79a164501142a5f5d7dc5

Oliver Jan Krylow <oliver@bugabinga.net> ·

unsigned

Browse filesCompare with HEAD

Changed files (18)

extensions/leaks/CHANGELOG.md

Modified
1 1 # Changelog
3 +## Unreleased
5 +- Parse structured detectors conservatively and keep ambiguous values visible.
6 +- Persist HMAC32 placeholder mappings as authenticated AES-GCM session entries.
7 +- Rebuild mappings from the active branch on start, reload, and tree navigation.
8 +- Block every tool on unknown placeholders or incomplete input traversal.
9 +- Add deterministic regressions, fast-check suites, tracked-source scanning, and the CI fuzz gate.
3 11 ## 0.0.3
5 13 - Replace Bun test/build wiring with npm and Vitest

extensions/leaks/README.md

Modified
2 2 # leaks
4 4 Provider-bound secret redaction for Pi.
6 -`leaks` redacts secrets before provider requests.
7 -Local Pi behavior stays transparent when placeholders round-trip exactly.
9 -## Install / load
11 -Loaded through the root pi-ext package. See [../../README.md](../../README.md).
6 +`leaks` replaces only high-confidence secret values before provider requests.
7 +Local Pi restores authenticated placeholders before tools and user-visible output.
13 9 ## Threat model
15 -Provider sees placeholders, not classified secret values.
16 -Pi keeps restore mappings in memory only.
17 -Mappings are lost on process exit, reload, new session, or `/leaks clear`.
11 +Provider payloads receive placeholders instead of classified values.
12 +Pi persists encrypted mappings in custom session entries so reload, resume, fork, and tree navigation can restore active-branch placeholders.
13 +The root key and ciphertext share the session file, so this is context isolation and obfuscation, not encryption at rest.
19 -`leaks` does not stop tool exfiltration, malicious model behavior, prompt
20 -injection, or unsafe commands.
21 -Use Pi permissions, sandboxing, and tool policy for that.
15 +`leaks` is not DLP, a sandbox, or a permission system.
23 17 ## Placeholder model
25 -Provider text uses this shape:
27 19 ```text
28 -__LEAK_<TAG>[_<TAG>...]_<HMAC12>[_N]__
20 +__LEAK_V1_<TAG>[_<TAG>...]_<HMAC32>__
31 -Examples use fake metadata only:
23 +HMAC32 is derived from a random generation root key.
24 +Mappings use AES-256-GCM with a fresh nonce and authenticated generation metadata.
25 +A mapping is appended before its placeholder can reach the provider.
26 +Persistence, encryption, collision, or capacity failure leaves the original value visible and warns.
33 -```text
34 -__LEAK_STRIPE_LIVE_SECRET_KEY_STRIPE_TOKEN_8f31aa77c901__
35 -__LEAK_DATABASE_URL_DB_PASSWORD_POSTGRES_URL_ba29d4ff8810__
36 -```
28 +Only authenticated mappings on `sessionManager.getBranch()` hydrate.
29 +Unknown current or legacy placeholders block every built-in and custom tool.
30 +Cycles, unsupported objects, and traversal beyond depth 64 also block tools because inspection is incomplete.
38 -The digest is HMAC12 with a random in-memory key.
39 -Tags are safe metadata.
40 -Secret length, prefix, suffix, alphabet, and entropy are not encoded.
42 -## Commands / tools / settings
44 -Commands:
32 +## Commands
46 34 - `/leaks status` reports safe counters and labels.
47 -- `/leaks reload` reloads settings and dotenv/env learning.
48 -- `/leaks test <text>` redacts local diagnostic text.
49 -- `/leaks clear` clears volatile placeholder mappings after confirmation.
35 +- `/leaks reload` rebuilds active-branch mappings and reloads settings and dotenv/environment learning.
36 +- `/leaks test <text>` reports a local match count.
37 +- `/leaks clear` appends a fresh generation after confirmation.
51 -Tools: none.
39 +The extension registers no tools.
40 +Non-UI diagnostics go to stderr with a `[leaks]` prefix.
53 -Settings: only custom regex rules are accepted.
42 +## Settings
44 +Only custom regex rules are accepted.
55 46 ```json
62 53 Labels must match `[A-Z0-9_]{1,64}`.
63 -Invalid or missing labels fall back to `CUSTOM_REGEX` and warn.
64 -Flags and unsupported fields are ignored with warnings.
65 -Unsafe patterns are rejected.
54 +Invalid rules warn and are ignored or use the documented label fallback.
55 +Unsupported compatibility settings are rejected.
67 -Rejected settings: `enabled`, `keywords`, `exclude`, `secretFiles`.
68 -They are intentionally not compatibility features.
57 +## Detector policy
70 -## Behavior
59 +Precision outranks provider secrecy.
60 +Ambiguous values remain visible.
72 -Built-ins are always active.
73 -They cover high-confidence AI, VCS, package, cloud, payment, messaging, webhook,
74 -mail, auth, OAuth, JWT, cookie, session, database, cache, private-key, and
75 -sensitive-assignment patterns.
62 +Structured formats require parsing or complete bounded context:
77 -Dotenv learning reads only cwd files:
64 +- JWTs require three canonical Base64URL segments, object header/payload JSON, and nonempty `alg`.
65 +- URLs use `URL` and redact only parsed passwords and exact sensitive query values.
66 +- Basic authorization requires canonical Base64 that decodes with a `:` separator.
67 +- Cookie and assignment names use complete separator-aware tokens.
68 +- PuTTY private payloads obey declared line counts and canonical Base64.
69 +- Environment URLs learn only parsed password and sensitive query components.
79 -- `.env`
80 -- `.env.local`
81 -- `.env.*.local`
71 +Dotenv discovery reads `.env`, `.env.local`, and `.env.*.local` under cwd.
72 +Public and metadata names remain visible unless the value has a provider-specific shape.
83 -Process environment values use the same classifier.
84 -Status collapses process env names to `ENV_KEY`.
74 +## Property tests
86 -URLs, query params, headers, cookies, and assignments preserve safe structure.
87 -Only secret components are redacted.
88 -JWTs and private-key material redact as whole logical values.
89 -Public keys and certificates stay visible.
76 +Run the dedicated suite separately from unit tests:
91 -Provider-bound context and provider request payloads are redacted on copied
92 -values.
93 -Original local objects are not mutated.
94 -Provider authentication headers are preserved.
78 +```console
79 +mise run fuzz
80 +```
96 -Tool-call args and finalized assistant messages are restored locally when they
97 -contain exact placeholders.
98 -Restore only matches complete placeholders.
82 +Replay one detector property exactly:
100 -Traversal is bounded.
101 -Cycles are preserved.
102 -Primitive and callable values pass through.
103 -Detector failures fail open and warn once.
84 +```console
85 +FC_TEST=jwt-positive FC_SEED=123 FC_PATH='0:1' mise run fuzz
86 +```
105 -Mappings and learned values are capped at 8192 entries.
106 -Overflow placeholders may not be restorable.
88 +The runner force-kills the suite after two minutes and exits 124 on timeout.
89 +`mise run ci` includes this fuzz gate.
108 -## False-positive policy
91 +## Limits
110 -Metadata stays visible aggressively.
111 -No entropy-only redaction.
112 -Names, hosts, domains, ordinary URLs, refs, branch names, package names, IDs,
113 -codenames, env references, public keys, and certificates stay visible unless a
114 -custom regex explicitly matches them.
116 -## Limitations
118 -Out of scope:
120 -- image, audio, binary, OCR redaction
121 -- semantic secret-byte editing
122 -- homomorphic encryption
123 -- symbolic secret transforms
124 -- arbitrary DLP/privacy redaction
125 -- tool sandboxing
126 -- permission enforcement
127 -- repository secret scanning
128 -- git history cleanup
93 +Mappings and learned values are capped at 8192 entries each.
94 +Secret-byte editing, images, audio, binary data, OCR, homomorphic transforms, repository cleanup, sandboxing, and permission enforcement are out of scope.

extensions/leaks/__fuzz__/lifecycle.fuzz.test.ts

Modified
10 10 kind: fc.constant("add" as const),
11 11 id: fc.integer({ min: 0, max: 40 }),
13 + fc.record({
14 + kind: fc.constant("fork" as const),
15 + branch: fc.integer({ min: 0, max: 4 }),
16 + }),
17 + fc.record({
18 + kind: fc.constant("switch" as const),
19 + branch: fc.integer({ min: 0, max: 4 }),
20 + }),
15 23 test.prop(
16 24 { operations: fc.array(operation, { minLength: 1, maxLength: 100 }) },
17 25 base,
18 26 )("reconstructs only the modeled active generation", ({ operations }) => {
19 - const entries: PersistedEntry[] = [];
20 - const append = (customType: string, data: unknown) =>
21 - entries.push({ type: "custom" as const, customType, data });
22 - const store = new __test.PlaceholderStore();
23 - store.configure(entries, append);
24 - const active = new Map<string, string>();
25 - const inactive = new Map<string, string>();
27 + const branches = new Map<number, PersistedEntry[]>([[0, []]]);
28 + const models = new Map<number, Map<string, string>>([[0, new Map()]]);
29 + const known = new Map<string, string>();
30 + let activeBranch = 0;
31 + let store = new __test.PlaceholderStore();
32 + const configure = () => {
33 + const entries = branches.get(activeBranch) as PersistedEntry[];
34 + store = new __test.PlaceholderStore();
35 + store.configure(entries, (customType, data) =>
36 + entries.push({ type: "custom", customType, data }),
37 + );
38 + };
39 + configure();
26 40 for (const item of operations) {
27 41 if (item.kind === "clear") {
28 - for (const entry of active) inactive.set(...entry);
29 - active.clear();
42 + models.set(activeBranch, new Map());
30 43 store.clear();
31 44 continue;
46 + if (item.kind === "fork") {
47 + branches.set(
48 + item.branch,
49 + structuredClone(branches.get(activeBranch) as PersistedEntry[]),
50 + );
51 + models.set(
52 + item.branch,
53 + new Map(models.get(activeBranch) as Map<string, string>),
54 + );
55 + activeBranch = item.branch;
56 + configure();
57 + continue;
58 + }
59 + if (item.kind === "switch") {
60 + if (!branches.has(item.branch)) continue;
61 + activeBranch = item.branch;
62 + configure();
63 + continue;
64 + }
33 65 const value = `lifecycle-fixture-${item.id}-ordinary`;
34 66 const placeholder = store.create(value, [
35 67 { tag: "TOKEN", priority: 10, source: "custom" },
37 - if (placeholder) active.set(placeholder, value);
69 + if (placeholder) {
70 + models.get(activeBranch)?.set(placeholder, value);
71 + known.set(placeholder, value);
72 + }
74 + configure();
75 + const active = models.get(activeBranch) as Map<string, string>;
76 + for (const [placeholder, value] of known) {
77 + expect(store.restore(placeholder)).toBe(
78 + active.get(placeholder) ?? placeholder,
79 + );
80 + if (active.has(placeholder)) expect(active.get(placeholder)).toBe(value);
81 + }
82 +});
84 +test.prop(
85 + {
86 + field: fc.constantFrom("nonce", "authTag", "ciphertext", "placeholder"),
87 + },
88 + base,
89 +)("disables creation after generated record corruption", ({ field }) => {
90 + const entries: PersistedEntry[] = [];
91 + const store = new __test.PlaceholderStore();
92 + store.configure(entries, (customType, data) =>
93 + entries.push({ type: "custom", customType, data }),
94 + );
95 + store.create("ordinary-lifecycle-fixture", [
96 + { tag: "TOKEN", priority: 10, source: "custom" },
97 + ]);
98 + const mapping = entries[1].data as Record<string, string>;
99 + mapping[field] = `${mapping[field]}!`;
39 100 const resumed = new __test.PlaceholderStore();
40 - resumed.configure(entries, append);
41 - for (const [placeholder, value] of active)
42 - expect(resumed.restore(placeholder)).toBe(value);
43 - for (const placeholder of inactive.keys())
44 - if (!active.has(placeholder))
45 - expect(resumed.restore(placeholder)).toBe(placeholder);
101 + resumed.configure(entries, () => {});
102 + expect(resumed.stats().persistenceEnabled).toBe(false);
103 + expect(
104 + resumed.create("another-ordinary-fixture", [
105 + { tag: "TOKEN", priority: 10, source: "custom" },
106 + ]),
107 + ).toBeUndefined();
48 110 test.prop(

extensions/leaks/__tests__/__snapshots__/demo-corpus.test.ts.snap

Added
new file mode 100644
1 +// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
3 +exports[`leaks demo corpus > matches the exact demo corpus labels and spans 1`] = `
4 +[
5 + {
6 + "fixtureClass": "README.md",
7 + "labels": [],
8 + "redactionCount": 0,
9 + "relativePath": "README.md",
10 + "restoredExactly": true,
11 + },
12 + {
13 + "fixtureClass": "ai",
14 + "labels": [],
15 + "redactionCount": 0,
16 + "relativePath": "ai/langchain.env",
17 + "restoredExactly": true,
18 + },
19 + {
20 + "fixtureClass": "ai",
21 + "labels": [
22 + "ANTHROPIC_API_KEY",
23 + "ANTHROPIC_TOKEN",
24 + "GOOGLE_API_KEY",
25 + "HUGGINGFACE_TOKEN",
26 + "OPENAI_API_KEY",
27 + "OPENAI_TOKEN",
28 + ],
29 + "redactionCount": 5,
30 + "relativePath": "ai/llm-providers.env",
31 + "restoredExactly": true,
32 + },
33 + {
34 + "fixtureClass": "analytics",
35 + "labels": [],
36 + "redactionCount": 0,
37 + "relativePath": "analytics/analytics.env",
38 + "restoredExactly": true,
39 + },
40 + {
41 + "fixtureClass": "api-tokens",
42 + "labels": [
43 + "AUTHORIZATION_BEARER",
44 + "X_API_KEY",
45 + ],
46 + "redactionCount": 2,
47 + "relativePath": "api-tokens/generic.txt",
48 + "restoredExactly": true,
49 + },
50 + {
51 + "fixtureClass": "api-tokens",
52 + "labels": [
53 + "JWT",
54 + ],
55 + "redactionCount": 1,
56 + "relativePath": "api-tokens/jwt.txt",
57 + "restoredExactly": true,
58 + },
59 + {
60 + "fixtureClass": "api-tokens",
61 + "labels": [],
62 + "redactionCount": 0,
63 + "relativePath": "api-tokens/oauth-client.json",
64 + "restoredExactly": true,
65 + },
66 + {
67 + "fixtureClass": "auth",
68 + "labels": [
69 + "JWT",
70 + ],
71 + "redactionCount": 1,
72 + "relativePath": "auth/auth-providers.env",
73 + "restoredExactly": true,
74 + },
75 + {
76 + "fixtureClass": "auth",
77 + "labels": [],
78 + "redactionCount": 0,
79 + "relativePath": "auth/passport-session.env",
80 + "restoredExactly": true,
81 + },
82 + {
83 + "fixtureClass": "backup",
84 + "labels": [],
85 + "redactionCount": 0,
86 + "relativePath": "backup/backup.env",
87 + "restoredExactly": true,
88 + },
89 + {
90 + "fixtureClass": "backup",
91 + "labels": [
92 + "OAUTH_TOKEN",
93 + ],
94 + "redactionCount": 1,
95 + "relativePath": "backup/rclone.conf",
96 + "restoredExactly": true,
97 + },
98 + {
99 + "fixtureClass": "cdn-dns",
100 + "labels": [],
101 + "redactionCount": 0,
102 + "relativePath": "cdn-dns/cdn-dns.env",
103 + "restoredExactly": true,
104 + },
106 + "fixtureClass": "chat",
107 + "labels": [],
108 + "redactionCount": 0,
109 + "relativePath": "chat/messaging-extra.env",
110 + "restoredExactly": true,
111 + },
113 + "fixtureClass": "chat",
114 + "labels": [
115 + "SLACK_TOKEN",
116 + ],
117 + "redactionCount": 1,
118 + "relativePath": "chat/slack-discord.env",
119 + "restoredExactly": true,
120 + },
122 + "fixtureClass": "ci",
123 + "labels": [],
124 + "redactionCount": 0,
125 + "relativePath": "ci/.terraformrc",
126 + "restoredExactly": true,
127 + },
129 + "fixtureClass": "ci",
130 + "labels": [],
131 + "redactionCount": 0,
132 + "relativePath": "ci/ansible-vault.yml",
133 + "restoredExactly": true,
134 + },
136 + "fixtureClass": "ci",
137 + "labels": [],
138 + "redactionCount": 0,
139 + "relativePath": "ci/circleci.env",
140 + "restoredExactly": true,
141 + },
143 + "fixtureClass": "ci",
144 + "labels": [
145 + "GITHUB_TOKEN",
146 + "JWT",
147 + ],
148 + "redactionCount": 2,
149 + "relativePath": "ci/github-actions.env",
150 + "restoredExactly": true,
151 + },
153 + "fixtureClass": "ci",
154 + "labels": [
155 + "OPENSSH_PRIVATE_KEY",
156 + ],
157 + "redactionCount": 1,
158 + "relativePath": "ci/gitlab-ci.yml",
159 + "restoredExactly": true,
160 + },
162 + "fixtureClass": "ci",
163 + "labels": [],
164 + "redactionCount": 0,
165 + "relativePath": "ci/jenkins-credentials.xml",
166 + "restoredExactly": true,
167 + },
169 + "fixtureClass": "ci",
170 + "labels": [],
171 + "redactionCount": 0,
172 + "relativePath": "ci/more-ci.env",
173 + "restoredExactly": true,
174 + },
176 + "fixtureClass": "ci",
177 + "labels": [],
178 + "redactionCount": 0,
179 + "relativePath": "ci/pulumi.env",
180 + "restoredExactly": true,
181 + },
183 + "fixtureClass": "ci",
184 + "labels": [
185 + "PRIVATE_KEY",
186 + ],
187 + "redactionCount": 1,
188 + "relativePath": "ci/terraform.tfvars",
189 + "restoredExactly": true,
190 + },
192 + "fixtureClass": "cloud",
193 + "labels": [],
194 + "redactionCount": 0,
195 + "relativePath": "cloud/alibaba.env",
196 + "restoredExactly": true,
197 + },
199 + "fixtureClass": "cloud",
200 + "labels": [
201 + "AWS_ACCESS_KEY_ID",
202 + ],
203 + "redactionCount": 2,
204 + "relativePath": "cloud/aws-credentials",
205 + "restoredExactly": true,
206 + },
208 + "fixtureClass": "cloud",
209 + "labels": [
210 + "AWS_ACCESS_KEY_ID",
211 + ],
212 + "redactionCount": 1,
213 + "relativePath": "cloud/aws-env.sh",
214 + "restoredExactly": true,
215 + },
217 + "fixtureClass": "cloud",
218 + "labels": [],
219 + "redactionCount": 0,
220 + "relativePath": "cloud/azure.env",
221 + "restoredExactly": true,
222 + },
224 + "fixtureClass": "cloud",
225 + "labels": [],
226 + "redactionCount": 0,
227 + "relativePath": "cloud/cloudflare.env",
228 + "restoredExactly": true,
229 + },
231 + "fixtureClass": "cloud",
232 + "labels": [],
233 + "redactionCount": 0,
234 + "relativePath": "cloud/digitalocean.env",
235 + "restoredExactly": true,
236 + },
238 + "fixtureClass": "cloud",
239 + "labels": [
240 + "PRIVATE_KEY",
241 + ],
242 + "redactionCount": 1,
243 + "relativePath": "cloud/gcp-service-account.json",
244 + "restoredExactly": true,
245 + },
247 + "fixtureClass": "cloud",
248 + "labels": [],
249 + "redactionCount": 0,
250 + "relativePath": "cloud/ibm.env",
251 + "restoredExactly": true,
252 + },
254 + "fixtureClass": "cloud",
255 + "labels": [],
256 + "redactionCount": 0,
257 + "relativePath": "cloud/linode-hcloud-scaleway.env",
258 + "restoredExactly": true,
259 + },
261 + "fixtureClass": "cloud",
262 + "labels": [],
263 + "redactionCount": 0,
264 + "relativePath": "cloud/oci-config",
265 + "restoredExactly": true,
266 + },
268 + "fixtureClass": "cloud",
269 + "labels": [
270 + "PRIVATE_KEY",
271 + ],
272 + "redactionCount": 1,
273 + "relativePath": "cloud/oci_api_key.pem",
274 + "restoredExactly": true,
275 + },
277 + "fixtureClass": "cloud",
278 + "labels": [],
279 + "redactionCount": 0,
280 + "relativePath": "cloud/openstack.env",
281 + "restoredExactly": true,
282 + },
284 + "fixtureClass": "cloud",
285 + "labels": [],
286 + "redactionCount": 0,
287 + "relativePath": "cloud/tencent.env",
288 + "restoredExactly": true,
289 + },
291 + "fixtureClass": "cm-tools",
292 + "labels": [
293 + "AWS_ACCESS_KEY_ID",
294 + ],
295 + "redactionCount": 1,
296 + "relativePath": "cm-tools/chef-knife.rb",
297 + "restoredExactly": true,
298 + },
300 + "fixtureClass": "cm-tools",
301 + "labels": [
302 + "OPENSSH_PRIVATE_KEY",
303 + ],
304 + "redactionCount": 1,
305 + "relativePath": "cm-tools/client.pem",
306 + "restoredExactly": true,
307 + },
309 + "fixtureClass": "cm-tools",
310 + "labels": [
311 + "AWS_ACCESS_KEY_ID",
312 + ],
313 + "redactionCount": 1,
314 + "relativePath": "cm-tools/packer-vars.pkrvars.hcl",
315 + "restoredExactly": true,
316 + },
318 + "fixtureClass": "cm-tools",
319 + "labels": [
320 + "OPENSSH_PRIVATE_KEY",
321 + ],
322 + "redactionCount": 1,
323 + "relativePath": "cm-tools/puppet-hiera.yaml",
324 + "restoredExactly": true,
325 + },
327 + "fixtureClass": "cm-tools",
328 + "labels": [],
329 + "redactionCount": 0,
330 + "relativePath": "cm-tools/salt-pillar.sls",
331 + "restoredExactly": true,
332 + },
334 + "fixtureClass": "code-signing",
335 + "labels": [],
336 + "redactionCount": 0,
337 + "relativePath": "code-signing/codesign.crt",
338 + "restoredExactly": true,
339 + },
341 + "fixtureClass": "code-signing",
342 + "labels": [
343 + "PRIVATE_KEY",
344 + ],
345 + "redactionCount": 1,
346 + "relativePath": "code-signing/codesign.key",
347 + "restoredExactly": true,
348 + },
350 + "fixtureClass": "code-signing",
351 + "labels": [
352 + "COSIGN_PRIVATE_KEY",
353 + "ENCRYPTED_COSIGN_PRIVATE_KEY",
354 + ],
355 + "redactionCount": 1,
356 + "relativePath": "code-signing/cosign.key",
357 + "restoredExactly": true,
358 + },
360 + "fixtureClass": "code-signing",
361 + "labels": [],
362 + "redactionCount": 0,
363 + "relativePath": "code-signing/signing.env",
364 + "restoredExactly": true,
365 + },
367 + "fixtureClass": "commerce",
368 + "labels": [],
369 + "redactionCount": 0,
370 + "relativePath": "commerce/commerce.env",
371 + "restoredExactly": true,
372 + },
374 + "fixtureClass": "config",
375 + "labels": [],
376 + "redactionCount": 0,
377 + "relativePath": "config/.env",
378 + "restoredExactly": true,
379 + },
381 + "fixtureClass": "config",
382 + "labels": [
383 + "DB_PASSWORD",
384 + ],
385 + "redactionCount": 1,
386 + "relativePath": "config/.envrc",
387 + "restoredExactly": true,
388 + },
390 + "fixtureClass": "config",
391 + "labels": [],
392 + "redactionCount": 0,
393 + "relativePath": "config/application.properties",
394 + "restoredExactly": true,
395 + },
397 + "fixtureClass": "config",
398 + "labels": [],
399 + "redactionCount": 0,
400 + "relativePath": "config/appsettings.json",
401 + "restoredExactly": true,
402 + },
404 + "fixtureClass": "config",
405 + "labels": [
406 + "AUTHORIZATION_BEARER",
407 + "DB_PASSWORD",
408 + ],
409 + "redactionCount": 2,
410 + "relativePath": "config/bash_history.txt",
411 + "restoredExactly": true,
412 + },
414 + "fixtureClass": "config",
415 + "labels": [],
416 + "redactionCount": 0,
417 + "relativePath": "config/config.ini",
418 + "restoredExactly": true,
419 + },
421 + "fixtureClass": "config",
422 + "labels": [],
423 + "redactionCount": 0,
424 + "relativePath": "config/cookies.txt",
425 + "restoredExactly": true,
426 + },
428 + "fixtureClass": "config",
429 + "labels": [],
430 + "redactionCount": 0,
431 + "relativePath": "config/htpasswd",
432 + "restoredExactly": true,
433 + },
435 + "fixtureClass": "config",
436 + "labels": [],
437 + "redactionCount": 0,
438 + "relativePath": "config/password-manager-export.csv",
439 + "restoredExactly": true,
440 + },
442 + "fixtureClass": "config",
443 + "labels": [],
444 + "redactionCount": 0,
445 + "relativePath": "config/secrets.toml",
446 + "restoredExactly": true,
447 + },
449 + "fixtureClass": "config",
450 + "labels": [],
451 + "redactionCount": 0,
452 + "relativePath": "config/secrets.yaml",
453 + "restoredExactly": true,
454 + },
456 + "fixtureClass": "containers",
457 + "labels": [],
458 + "redactionCount": 0,
459 + "relativePath": "containers/.dockercfg",
460 + "restoredExactly": true,
461 + },
463 + "fixtureClass": "containers",
464 + "labels": [
465 + "GHCR_PAT",
466 + "GITHUB_TOKEN",
467 + ],
468 + "redactionCount": 1,
469 + "relativePath": "containers/registry-auths.env",
470 + "restoredExactly": true,
471 + },
473 + "fixtureClass": "crypto",
474 + "labels": [],
475 + "redactionCount": 0,
476 + "relativePath": "crypto/keys.env",
477 + "restoredExactly": true,
478 + },
480 + "fixtureClass": "crypto",
481 + "labels": [],
482 + "redactionCount": 0,
483 + "relativePath": "crypto/minisign.key",
484 + "restoredExactly": true,
485 + },
487 + "fixtureClass": "databases",
488 + "labels": [],
489 + "redactionCount": 0,
490 + "relativePath": "databases/.my.cnf",
491 + "restoredExactly": true,
492 + },
494 + "fixtureClass": "databases",
495 + "labels": [],
496 + "redactionCount": 0,
497 + "relativePath": "databases/.pgpass",
498 + "restoredExactly": true,
499 + },
501 + "fixtureClass": "databases",
502 + "labels": [
503 + "DB_PASSWORD",
504 + ],
505 + "redactionCount": 6,
506 + "relativePath": "databases/database-urls.txt",
507 + "restoredExactly": true,
508 + },
510 + "fixtureClass": "databases",
511 + "labels": [],
512 + "redactionCount": 0,
513 + "relativePath": "databases/kafka.env",
514 + "restoredExactly": true,
515 + },
517 + "fixtureClass": "databases",
518 + "labels": [],
519 + "redactionCount": 0,
520 + "relativePath": "databases/redis.conf",
521 + "restoredExactly": true,
522 + },
524 + "fixtureClass": "databases",
525 + "labels": [
526 + "DB_PASSWORD",
527 + ],
528 + "redactionCount": 1,
529 + "relativePath": "databases/search-and-cache.env",
530 + "restoredExactly": true,
531 + },
533 + "fixtureClass": "databases",
534 + "labels": [
535 + "PRIVATE_KEY",
536 + ],
537 + "redactionCount": 1,
538 + "relativePath": "databases/warehouses.env",
539 + "restoredExactly": true,
540 + },
542 + "fixtureClass": "ftp",
543 + "labels": [],
544 + "redactionCount": 0,
545 + "relativePath": "ftp/.netrc",
546 + "restoredExactly": true,
547 + },
549 + "fixtureClass": "ftp",
550 + "labels": [],
551 + "redactionCount": 0,
552 + "relativePath": "ftp/lftp.conf",
553 + "restoredExactly": true,
554 + },
556 + "fixtureClass": "gpg",
557 + "labels": [
558 + "PGP_PRIVATE_KEY",
559 + ],
560 + "redactionCount": 1,
561 + "relativePath": "gpg/private.asc",
562 + "restoredExactly": true,
563 + },
565 + "fixtureClass": "hosting",
566 + "labels": [
567 + "GOOGLE_API_KEY",
568 + ],
569 + "redactionCount": 1,
570 + "relativePath": "hosting/firebase.json",
571 + "restoredExactly": true,
572 + },
574 + "fixtureClass": "hosting",
575 + "labels": [
576 + "JWT",
577 + ],
578 + "redactionCount": 1,
579 + "relativePath": "hosting/paas.env",
580 + "restoredExactly": true,
581 + },
583 + "fixtureClass": "hosting",
584 + "labels": [
585 + "JWT",
586 + ],
587 + "redactionCount": 2,
588 + "relativePath": "hosting/supabase.env",
589 + "restoredExactly": true,
590 + },
592 + "fixtureClass": "identity",
593 + "labels": [
594 + "PRIVATE_KEY",
595 + ],
596 + "redactionCount": 1,
597 + "relativePath": "identity/oidc.env",
598 + "restoredExactly": true,
599 + },
601 + "fixtureClass": "identity",
602 + "labels": [],
603 + "redactionCount": 0,
604 + "relativePath": "identity/saml-idp.xml",
605 + "restoredExactly": true,
606 + },
608 + "fixtureClass": "infra",
609 + "labels": [],
610 + "redactionCount": 0,
611 + "relativePath": "infra/networking.env",
612 + "restoredExactly": true,
613 + },
615 + "fixtureClass": "infra",
616 + "labels": [
617 + "OPENSSH_PRIVATE_KEY",
618 + ],
619 + "redactionCount": 1,
620 + "relativePath": "infra/openvpn-client.ovpn",
621 + "restoredExactly": true,
622 + },
624 + "fixtureClass": "infra",
625 + "labels": [],
626 + "redactionCount": 0,
627 + "relativePath": "infra/wg0.conf",
628 + "restoredExactly": true,
629 + },
631 + "fixtureClass": "iot",
632 + "labels": [
633 + "PRIVATE_KEY",
634 + ],
635 + "redactionCount": 1,
636 + "relativePath": "iot/iot.env",
637 + "restoredExactly": true,
638 + },
640 + "fixtureClass": "kubernetes",
641 + "labels": [],
642 + "redactionCount": 0,
643 + "relativePath": "kubernetes/kubeconfig.yaml",
644 + "restoredExactly": true,
645 + },
647 + "fixtureClass": "kubernetes",
648 + "labels": [
649 + "X_API_KEY",
650 + ],
651 + "redactionCount": 1,
652 + "relativePath": "kubernetes/secret.yaml",
653 + "restoredExactly": true,
654 + },
656 + "fixtureClass": "licenses",
657 + "labels": [
658 + "JWT",
659 + ],
660 + "redactionCount": 1,
661 + "relativePath": "licenses/license-and-activation.env",
662 + "restoredExactly": true,
663 + },
665 + "fixtureClass": "mail",
666 + "labels": [
667 + "DB_PASSWORD",
668 + "SENDGRID_TOKEN",
669 + ],
670 + "redactionCount": 2,
671 + "relativePath": "mail/mail.env",
672 + "restoredExactly": true,
673 + },
675 + "fixtureClass": "mail",
676 + "labels": [],
677 + "redactionCount": 0,
678 + "relativePath": "mail/sms-voice.env",
679 + "restoredExactly": true,
680 + },
682 + "fixtureClass": "manifest-files.txt",
683 + "labels": [],
684 + "redactionCount": 0,
685 + "relativePath": "manifest-files.txt",
686 + "restoredExactly": true,
687 + },
689 + "fixtureClass": "maps",
690 + "labels": [
691 + "GOOGLE_API_KEY",
692 + ],
693 + "redactionCount": 1,
694 + "relativePath": "maps/maps.env",
695 + "restoredExactly": true,
696 + },
698 + "fixtureClass": "mlops",
699 + "labels": [
700 + "JWT",
701 + ],
702 + "redactionCount": 1,
703 + "relativePath": "mlops/mlops.env",
704 + "restoredExactly": true,
705 + },
707 + "fixtureClass": "mobile",
708 + "labels": [
709 + "PRIVATE_KEY",
710 + ],
711 + "redactionCount": 1,
712 + "relativePath": "mobile/AuthKey_DEMO123456.p8",
713 + "restoredExactly": true,
714 + },
716 + "fixtureClass": "mobile",
717 + "labels": [
718 + "PRIVATE_KEY",
719 + ],
720 + "redactionCount": 1,
721 + "relativePath": "mobile/mobile.env",
722 + "restoredExactly": true,
723 + },
725 + "fixtureClass": "monitoring",
726 + "labels": [],
727 + "redactionCount": 0,
728 + "relativePath": "monitoring/monitoring-extra.env",
729 + "restoredExactly": true,
730 + },
732 + "fixtureClass": "monitoring",
733 + "labels": [],
734 + "redactionCount": 0,
735 + "relativePath": "monitoring/observability.env",
736 + "restoredExactly": true,
737 + },
739 + "fixtureClass": "oauth",
740 + "labels": [],
741 + "redactionCount": 0,
742 + "relativePath": "oauth/oauth-providers.env",
743 + "restoredExactly": true,
744 + },
746 + "fixtureClass": "orchestration",
747 + "labels": [
748 + "GITHUB_TOKEN",
749 + "password",
750 + ],
751 + "redactionCount": 1,
752 + "relativePath": "orchestration/argocd-repository-secret.yaml",
753 + "restoredExactly": true,
754 + },
756 + "fixtureClass": "orchestration",
757 + "labels": [
758 + "DB_PASSWORD",
759 + ],
760 + "redactionCount": 1,
761 + "relativePath": "orchestration/docker-compose.yml",
762 + "restoredExactly": true,
763 + },
765 + "fixtureClass": "orchestration",
766 + "labels": [],
767 + "redactionCount": 0,
768 + "relativePath": "orchestration/dockerconfig-secret.yaml",
769 + "restoredExactly": true,
770 + },
772 + "fixtureClass": "orchestration",
773 + "labels": [],
774 + "redactionCount": 0,
775 + "relativePath": "orchestration/helm-values.yaml",
776 + "restoredExactly": true,
777 + },
779 + "fixtureClass": "orchestration",
780 + "labels": [],
781 + "redactionCount": 0,
782 + "relativePath": "orchestration/kustomization.yaml",
783 + "restoredExactly": true,
784 + },
786 + "fixtureClass": "orchestration",
787 + "labels": [
788 + "DB_PASSWORD",
789 + "STRIPE_LIVE_SECRET_KEY",
790 + ],
791 + "redactionCount": 2,
792 + "relativePath": "orchestration/serverless.yml",
793 + "restoredExactly": true,
794 + },
796 + "fixtureClass": "orchestration",
797 + "labels": [],
798 + "redactionCount": 0,
799 + "relativePath": "orchestration/terraform.tfstate",
800 + "restoredExactly": true,
801 + },
803 + "fixtureClass": "package-managers",
804 + "labels": [
805 + "NPM_TOKEN",
806 + ],
807 + "redactionCount": 1,
808 + "relativePath": "package-managers/.npmrc",
809 + "restoredExactly": true,
810 + },
812 + "fixtureClass": "package-managers",
813 + "labels": [],
814 + "redactionCount": 0,
815 + "relativePath": "package-managers/.pypirc",
816 + "restoredExactly": true,
817 + },
819 + "fixtureClass": "package-managers",
820 + "labels": [],
821 + "redactionCount": 0,
822 + "relativePath": "package-managers/cargo-credentials.toml",
823 + "restoredExactly": true,
824 + },
826 + "fixtureClass": "package-managers",
827 + "labels": [],
828 + "redactionCount": 0,
829 + "relativePath": "package-managers/docker-config.json",
830 + "restoredExactly": true,
831 + },
833 + "fixtureClass": "package-managers",
834 + "labels": [],
835 + "redactionCount": 0,
836 + "relativePath": "package-managers/gradle.properties",
837 + "restoredExactly": true,
838 + },
840 + "fixtureClass": "package-managers",
841 + "labels": [
842 + "GITHUB_TOKEN",
843 + ],
844 + "redactionCount": 1,
845 + "relativePath": "package-managers/more-package-managers.env",
846 + "restoredExactly": true,
847 + },
849 + "fixtureClass": "package-managers",
850 + "labels": [
851 + "DB_PASSWORD",
852 + ],
853 + "redactionCount": 1,
854 + "relativePath": "package-managers/pip.conf",
855 + "restoredExactly": true,
856 + },
858 + "fixtureClass": "package-managers",
859 + "labels": [],
860 + "redactionCount": 0,
861 + "relativePath": "package-managers/settings.xml",
862 + "restoredExactly": true,
863 + },
865 + "fixtureClass": "payment",
866 + "labels": [],
867 + "redactionCount": 0,
868 + "relativePath": "payment/payment.env",
869 + "restoredExactly": true,
870 + },
872 + "fixtureClass": "product",
873 + "labels": [],
874 + "redactionCount": 0,
875 + "relativePath": "product/cms.env",
876 + "restoredExactly": true,
877 + },
879 + "fixtureClass": "product",
880 + "labels": [],
881 + "redactionCount": 0,
882 + "relativePath": "product/feature-flags.env",
883 + "restoredExactly": true,
884 + },
886 + "fixtureClass": "product",
887 + "labels": [],
888 + "redactionCount": 0,
889 + "relativePath": "product/saas-tools.env",
890 + "restoredExactly": true,
891 + },
893 + "fixtureClass": "quality",
894 + "labels": [],
895 + "redactionCount": 0,
896 + "relativePath": "quality/dev-quality.env",
897 + "restoredExactly": true,
898 + },
900 + "fixtureClass": "search",
901 + "labels": [],
902 + "redactionCount": 0,
903 + "relativePath": "search/search.env",
904 + "restoredExactly": true,
905 + },
907 + "fixtureClass": "secret-management",
908 + "labels": [],
909 + "redactionCount": 0,
910 + "relativePath": "secret-management/consul-nomad.env",
911 + "restoredExactly": true,
912 + },
914 + "fixtureClass": "secret-management",
915 + "labels": [],
916 + "redactionCount": 0,
917 + "relativePath": "secret-management/sops-age.txt",
918 + "restoredExactly": true,
919 + },
921 + "fixtureClass": "secret-management",
922 + "labels": [],
923 + "redactionCount": 0,
924 + "relativePath": "secret-management/vault.env",
925 + "restoredExactly": true,
926 + },
928 + "fixtureClass": "security",
929 + "labels": [],
930 + "redactionCount": 0,
931 + "relativePath": "security/security-tools.env",
932 + "restoredExactly": true,
933 + },
935 + "fixtureClass": "social",
936 + "labels": [],
937 + "redactionCount": 0,
938 + "relativePath": "social/social.env",
939 + "restoredExactly": true,
940 + },
942 + "fixtureClass": "ssh",
943 + "labels": [
944 + "OPENSSH_PRIVATE_KEY",
945 + ],
946 + "redactionCount": 1,
947 + "relativePath": "ssh/id_ed25519",
948 + "restoredExactly": true,
949 + },
951 + "fixtureClass": "ssh",
952 + "labels": [],
953 + "redactionCount": 0,
954 + "relativePath": "ssh/id_ed25519.pub",
955 + "restoredExactly": true,
956 + },
958 + "fixtureClass": "ssh",
959 + "labels": [
960 + "OPENSSH_PRIVATE_KEY",
961 + ],
962 + "redactionCount": 1,
963 + "relativePath": "ssh/id_rsa",
964 + "restoredExactly": true,
965 + },
967 + "fixtureClass": "ssh",
968 + "labels": [],
969 + "redactionCount": 0,
970 + "relativePath": "ssh/id_rsa.pub",
971 + "restoredExactly": true,
972 + },
974 + "fixtureClass": "ssh",
975 + "labels": [
976 + "PUTTY_PRIVATE_KEY",
977 + ],
978 + "redactionCount": 1,
979 + "relativePath": "ssh/id_rsa_putty.ppk",
980 + "restoredExactly": true,
981 + },
983 + "fixtureClass": "testing",
984 + "labels": [],
985 + "redactionCount": 0,
986 + "relativePath": "testing/testing-platforms.env",
987 + "restoredExactly": true,
988 + },
990 + "fixtureClass": "tls",
991 + "labels": [],
992 + "redactionCount": 0,
993 + "relativePath": "tls/server.crt",
994 + "restoredExactly": true,
995 + },
997 + "fixtureClass": "tls",
998 + "labels": [
999 + "EC_PRIVATE_KEY",
1000 + ],
1001 + "redactionCount": 1,
1002 + "relativePath": "tls/server.ec.key",
1003 + "restoredExactly": true,
1004 + },
1006 + "fixtureClass": "tls",
1007 + "labels": [
1008 + "PRIVATE_KEY",
1009 + ],
1010 + "redactionCount": 1,
1011 + "relativePath": "tls/server.key",
1012 + "restoredExactly": true,
1013 + },
1015 + "fixtureClass": "tls",
1016 + "labels": [
1017 + "PRIVATE_KEY",
1018 + ],
1019 + "redactionCount": 1,
1020 + "relativePath": "tls/server.pem",
1021 + "restoredExactly": true,
1022 + },
1024 + "fixtureClass": "vcs",
1025 + "labels": [],
1026 + "redactionCount": 0,
1027 + "relativePath": "vcs/bitbucket.env",
1028 + "restoredExactly": true,
1029 + },
1031 + "fixtureClass": "vcs",
1032 + "labels": [
1033 + "DB_PASSWORD",
1034 + ],
1035 + "redactionCount": 3,
1036 + "relativePath": "vcs/git-credentials",
1037 + "restoredExactly": true,
1038 + },
1040 + "fixtureClass": "vcs",
1041 + "labels": [
1042 + "PRIVATE_KEY",
1043 + ],
1044 + "redactionCount": 1,
1045 + "relativePath": "vcs/github-app-private-key.pem",
1046 + "restoredExactly": true,
1047 + },
1049 + "fixtureClass": "vcs",
1050 + "labels": [
1051 + "GITHUB_CLASSIC_PAT",
1052 + "GITHUB_TOKEN",
1053 + "JWT",
1054 + "PRIVATE_KEY",
1055 + ],
1056 + "redactionCount": 3,
1057 + "relativePath": "vcs/github.env",
1058 + "restoredExactly": true,
1059 + },
1061 + "fixtureClass": "vcs",
1062 + "labels": [],
1063 + "redactionCount": 0,
1064 + "relativePath": "vcs/sourcehut.env",
1065 + "restoredExactly": true,
1066 + },
1068 + "fixtureClass": "vector",
1069 + "labels": [],
1070 + "redactionCount": 0,
1071 + "relativePath": "vector/vector-dbs.env",
1072 + "restoredExactly": true,
1073 + },
1075 + "fixtureClass": "web3",
1076 + "labels": [
1077 + "JWT",
1078 + ],
1079 + "redactionCount": 1,
1080 + "relativePath": "web3/web3.env",
1081 + "restoredExactly": true,
1082 + },
1084 + "fixtureClass": "webhooks",
1085 + "labels": [],
1086 + "redactionCount": 0,
1087 + "relativePath": "webhooks/webhook-secrets.env",
1088 + "restoredExactly": true,
1089 + },

extensions/leaks/__tests__/corpus.test.ts

Added
new file mode 100644
1 +import { spawnSync } from "node:child_process";
2 +import { readFileSync } from "node:fs";
3 +import { describe, expect, it } from "vitest";
4 +import { __test } from "../index.ts";
6 +function productionFiles() {
7 + const result = spawnSync("git", ["ls-files", "-z"], {
8 + cwd: process.cwd(),
9 + encoding: "utf8",
10 + });
11 + if (result.status !== 0) throw new Error("git ls-files failed");
12 + return result.stdout
13 + .split("\0")
14 + .filter(Boolean)
15 + .filter(
16 + (file) =>
17 + /^(?:extensions\/[^/]+\/(?:index\.ts|src\/.*\.(?:ts|js|mjs))|scripts\/.*\.(?:ts|js|mjs))$/.test(
18 + file,
19 + ) && !file.includes("/__fixtures__/"),
20 + );
23 +describe("leaks tracked production corpus", () => {
24 + it("has no unexpected built-in or structured matches", () => {
25 + const failures: string[] = [];
26 + for (const file of productionFiles()) {
27 + const text = readFileSync(file, "utf8");
28 + for (const span of [
29 + ...__test.findStructureSpans(text),
30 + ...__test.findBuiltinSpans(text),
31 + ]) {
32 + const line = text.slice(0, span.start).split("\n").length;
33 + const labels = span.labels.map((label) => label.tag).join(",");
34 + failures.push(
35 + `${file}:${line} labels=${labels} chars=${span.end - span.start}`,
36 + );
37 + }
38 + }
39 + expect(failures).toEqual([]);
40 + });
41 +});

extensions/leaks/__tests__/demo-corpus.test.ts

Modified
12 12 relativePath: string;
13 13 fixtureClass: string;
14 14 redactionCount: number;
15 + labels: string[];
16 + restoredExactly: boolean;
17 19 describe("leaks demo corpus", () => {
18 - it("redacts demo corpus values without exposing fixture contents", () => {
20 + it("matches the exact demo corpus labels and spans", () => {
19 21 const rows = scanCorpus(corpusRoot);
20 - const redacted = rows.filter((row) => row.redactionCount > 0);
21 - const classes = new Set(rows.map((row) => row.fixtureClass));
22 - const redactedClasses = new Set(redacted.map((row) => row.fixtureClass));
24 - expect(rows.length).toBeGreaterThanOrEqual(120);
25 - expect(redacted.length).toBeGreaterThanOrEqual(50);
26 - expect(classes.size).toBeGreaterThanOrEqual(40);
27 - expect(redactedClasses.size).toBeGreaterThanOrEqual(20);
22 + expect(rows).toMatchSnapshot();
28 23 expect(
29 - redacted.reduce((sum, row) => sum + row.redactionCount, 0),
30 - ).toBeGreaterThanOrEqual(70);
24 + rows
25 + .filter((row) => row.redactionCount > 0)
26 + .every((row) => row.restoredExactly),
27 + ).toBe(true);
33 30 it("covers provider-shaped demo fixture classes", () => {
56 53 .filter((file) => !readFileSync(file).includes(0))
57 54 .map((file) => {
58 55 const relativePath = path.relative(root, file);
56 + const source = readFileSync(file, "utf8");
57 + const result = engine.redactText(source, "provider");
59 58 return {
60 59 relativePath,
61 60 fixtureClass: relativePath.split(path.sep)[0] ?? relativePath,
62 - redactionCount: engine.redactText(
63 - readFileSync(file, "utf8"),
64 - "provider",
65 - ).count,
61 + redactionCount: result.count,
62 + labels: [...new Set(result.labels)].toSorted(),
63 + restoredExactly: engine.restoreText(result.text) === source,

extensions/leaks/__tests__/fuzz-runner.test.ts

Added
new file mode 100644
1 +import { spawnSync } from "node:child_process";
2 +import process from "node:process";
3 +import { describe, expect, it } from "vitest";
5 +describe("leaks fuzz runner", () => {
6 + it("force-kills timed out children with exit 124", () => {
7 + const result = spawnSync(process.execPath, ["scripts/run_leaks_fuzz.mjs"], {
8 + cwd: process.cwd(),
9 + env: {
10 + ...process.env,
11 + LEAKS_FUZZ_TEST_HANG: "1",
12 + LEAKS_FUZZ_TIMEOUT_MS: "20",
13 + },
14 + });
15 + expect(result.status).toBe(124);
16 + });
17 +});

extensions/leaks/__tests__/harness.test.ts

Modified
2 2 import { mkdirSync, writeFileSync } from "node:fs";
3 3 import os from "node:os";
4 4 import path from "node:path";
5 5 import { afterEach, describe, expect, it } from "vitest";
6 -import { createTestSession, type TestSession } from "../../../test/harness";
6 +import {
7 + calls,
8 + createTestSession,
9 + says,
10 + type TestSession,
11 + when,
12 +} from "../../../test/harness";
8 14 function extPath() {
9 15 return path.resolve(import.meta.dirname, "../index.ts");
18 +function captureExtension(pi: {
19 + registerTool(tool: Record<string, unknown>): void;
20 +}) {
21 + pi.registerTool({
22 + name: "capture_value",
23 + label: "capture value",
24 + description: "Return the provided value for integration testing.",
25 + parameters: {
26 + type: "object",
27 + properties: { value: { type: "string" } },
28 + required: ["value"],
29 + additionalProperties: false,
30 + },
31 + async execute(_toolCallId: string, params: { value: string }) {
32 + return { content: [{ type: "text", text: params.value }], details: {} };
33 + },
34 + });
37 +function latestPlaceholder(session: TestSession) {
38 + const entry = session.session.sessionManager
39 + .getBranch()
40 + .toReversed()
41 + .find(
42 + (item) =>
43 + item.type === "custom" && item.customType === "leaks-mapping-v1",
44 + );
45 + if (!entry || entry.type !== "custom") throw new Error("missing mapping");
46 + return (entry.data as { placeholder: string }).placeholder;
12 49 const secret = "sk-or-v1-" + "d".repeat(64);
13 50 const dotenvSecret = ["dotenv", "fixture", "secret", "123456789"].join("-");
133 170 expect(JSON.stringify(message)).toContain(secret);
173 + it("rebuilds mappings from only the active tree branch", async () => {
174 + t = await createTestSession({ extensions: [extPath()] });
175 + const manager = t.session.sessionManager;
176 + const runner = t.session.extensionRunner;
177 + const firstResult = await runner.emitBeforeProviderRequest({
178 + text: secret,
179 + });
180 + const firstPlaceholder = JSON.stringify(firstResult).match(
181 + /__LEAK_V1_[A-Z0-9_]+_[a-f0-9]{32}__/,
182 + )?.[0] as string;
183 + const firstMapping = manager
184 + .getBranch()
185 + .find(
186 + (entry) =>
187 + entry.type === "custom" && entry.customType === "leaks-mapping-v1",
188 + );
189 + if (!firstMapping) throw new Error("missing first mapping");
190 + const generation = manager
191 + .getBranch()
192 + .find(
193 + (entry) =>
194 + entry.type === "custom" && entry.customType === "leaks-generation-v1",
195 + );
196 + if (!generation) throw new Error("missing generation");
198 + manager.branch(generation.id);
199 + await runner.emit({
200 + type: "session_tree",
201 + oldLeafId: firstMapping.id,
202 + newLeafId: generation.id,
203 + });
204 + const siblingFixture = "sk-or-v1-" + "e".repeat(64);
205 + const secondResult = await runner.emitBeforeProviderRequest({
206 + text: siblingFixture,
207 + });
208 + const secondPlaceholder = JSON.stringify(secondResult).match(
209 + /__LEAK_V1_[A-Z0-9_]+_[a-f0-9]{32}__/,
210 + )?.[0] as string;
211 + const firstCall = {
212 + type: "tool_call",
213 + toolName: "custom",
214 + toolCallId: "first-branch",
215 + input: { value: firstPlaceholder },
216 + };
217 + expect(await runner.emitToolCall(firstCall as never)).toMatchObject({
218 + block: true,
219 + });
220 + const secondCall = {
221 + type: "tool_call",
222 + toolName: "custom",
223 + toolCallId: "second-branch",
224 + input: { value: secondPlaceholder },
225 + };
226 + expect(await runner.emitToolCall(secondCall as never)).toBeUndefined();
227 + expect(secondCall.input.value).toBe(siblingFixture);
229 + const siblingLeaf = manager.getLeafId();
230 + manager.branch(firstMapping.id);
231 + await runner.emit({
232 + type: "session_tree",
233 + oldLeafId: siblingLeaf,
234 + newLeafId: firstMapping.id,
235 + });
236 + const restoredFirst = {
237 + type: "tool_call",
238 + toolName: "custom",
239 + toolCallId: "restored-first",
240 + input: { value: firstPlaceholder },
241 + };
242 + expect(await runner.emitToolCall(restoredFirst as never)).toBeUndefined();
243 + expect(restoredFirst.input.value).toBe(secret);
244 + });
246 + it("hydrates built-in tool input through the real agent loop", async () => {
247 + let received = "";
248 + t = await createTestSession({
249 + extensions: [extPath()],
250 + mockTools: {
251 + write: (params: { content: string }) => {
252 + received = params.content;
253 + return "written";
254 + },
255 + },
256 + });
257 + await t.run(
258 + when(`store ${secret}`, [
259 + calls("write", () => ({
260 + path: "result.txt",
261 + content: latestPlaceholder(t as TestSession),
262 + })),
263 + says("done"),
264 + ]),
265 + );
266 + expect(received).toBe(secret);
267 + });
269 + it("hydrates custom tool input through the real agent loop", async () => {
270 + t = await createTestSession({
271 + extensions: [extPath()],
272 + extensionFactories: [captureExtension as never],
273 + });
274 + await t.run(
275 + when(`capture ${secret}`, [
276 + calls("capture_value", () => ({
277 + value: latestPlaceholder(t as TestSession),
278 + })),
279 + says("done"),
280 + ]),
281 + );
282 + expect(t.events.toolResultsFor("capture_value")[0].text).toBe(secret);
283 + });
136 285 it("persists encrypted dotenv mappings without plaintext", async () => {
137 286 const cwd = path.join(os.tmpdir(), `leaks-harness-${Date.now()}`);
138 287 mkdirSync(cwd, { recursive: true });

extensions/leaks/__tests__/pi-direct.test.ts

Modified
1 1 import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
2 2 import { tmpdir } from "node:os";
3 3 import path from "node:path";
4 -import { describe, expect, it } from "vitest";
4 +import { describe, expect, it, vi } from "vitest";
5 5 import leaks from "../src/pi.ts";
7 7 type Handler = (event: any, ctx: any) => unknown | Promise<unknown>;
197 197 ).toBe(true);
200 + it("writes sanitized non-UI diagnostics only to stderr", async () => {
201 + const runtime = fakePi();
202 + leaks(runtime.pi as never);
203 + const view = ui();
204 + const ctx = { ...view.ctx, hasUI: false };
205 + const stderr = vi.spyOn(console, "error").mockImplementation(() => {});
206 + const stdout = vi.spyOn(console, "log").mockImplementation(() => {});
207 + try {
208 + await runtime.handlers.get("session_start")?.({}, ctx);
209 + await runtime.handlers.get("before_provider_request")?.(
211 + payload: {
212 + text: ["agendaFlick", "visibleArea", "yPosition"].join("."),
213 + },
214 + },
215 + ctx,
216 + );
217 + await runtime.commands.get("leaks")?.handler("status", ctx);
218 + expect(stderr.mock.calls.length).toBeGreaterThan(0);
219 + expect(
220 + stderr.mock.calls.every(([line]) =>
221 + String(line).startsWith("[leaks] "),
222 + ),
223 + ).toBe(true);
224 + expect(
225 + stderr.mock.calls.some(([line]) =>
226 + String(line).includes("agendaFlick"),
227 + ),
228 + ).toBe(false);
229 + expect(stdout).not.toHaveBeenCalled();
230 + } finally {
231 + stderr.mockRestore();
232 + stdout.mockRestore();
234 + });
200 236 it("keeps provider redaction when UI ctx is stale", async () => {
201 237 const runtime = fakePi();
202 238 leaks(runtime.pi as never);

extensions/leaks/__tests__/structures.test.ts

Modified
108 108 expect(__test.findJwtSpans(jwt).map((span) => span.value)).toEqual([jwt]);
111 + it("reports malformed structured candidates without values", () => {
112 + const dotted = ["agendaFlick", "visibleArea", "yPosition"].join(".");
113 + const malformedUrl = ["https://[bad/?tok", "en=ordinary-value-123456"].join(
114 + "",
115 + );
116 + const malformedBasic = [
117 + "Authorization",
118 + " Basic ordinary-value-123456",
119 + ].join(":");
120 + expect(
121 + __test
122 + .findStructureAmbiguities(
123 + [dotted, malformedUrl, malformedBasic].join("\n"),
125 + .map((item) => item.family)
126 + .toSorted(),
127 + ).toEqual(["JWT", "URL_QUERY", "BASIC_AUTH"].toSorted());
128 + });
111 130 it("rejects malformed structured contexts", () => {
112 131 const ordinary = ["ordinary", "metadata", "value", "123456"].join("-");
113 132 expect(

extensions/leaks/__tests__/unit.test.ts

Modified
22 22 changed: false,
23 23 count: 0,
24 24 redactions: [],
25 + ambiguities: [],
26 27 expect(engine.redactDeep(null, "provider")).toEqual({
27 28 value: null,
28 29 changed: false,
29 30 count: 0,
30 31 redactions: [],
32 + ambiguities: [],
32 34 const image = { type: "image", data: fixtureSecret };
33 35 expect(engine.redactDeep(image, "provider").value).toEqual(image);

extensions/leaks/index.ts

Modified
38 38 findJwtSpans,
39 39 findPrivateKeySpans,
40 40 findSensitiveQuerySpans,
41 + findStructureAmbiguities,
41 42 findStructureSpans,
42 43 structureRegexFuzzTargets,
43 44 } from "./src/structures.ts";
68 69 countCaptures,
69 70 isUnsafeRegex,
70 71 findStructureSpans,
72 + findStructureAmbiguities,
71 73 findCredentialedUrlSpans,
72 74 findSensitiveQuerySpans,
73 75 findHeaderSpans,

extensions/leaks/src/engine.ts

Modified
13 13 compileCustomRegex,
14 14 customRegexSpans,
15 15 } from "./settings.ts";
16 -import { findStructureSpans } from "./structures.ts";
16 +import { findStructureAmbiguities, findStructureSpans } from "./structures.ts";
17 17 import type {
18 18 DeepRedactionResult,
19 19 Detector,
46 46 private learned: LearnedValue[] = [];
47 47 private lastProviderRedactionCount = 0;
48 48 private totalProviderRedactionCount = 0;
49 + private lastProviderAmbiguityCount = 0;
49 50 private recentLabels: string[] = [];
50 51 private latestTags: string[] = [];
51 52 readonly notifiedPlaceholders = new Set<string>();
71 72 text: string,
72 73 origin: RedactionOrigin = "provider",
73 74 ): RedactionResult {
75 + const ambiguities = findStructureAmbiguities(text).map((item) => ({
76 + path: "",
77 + family: item.family,
78 + }));
74 79 let spans: SecretSpan[] = [
75 80 ...findStructureSpans(text),
76 81 ...findBuiltinSpans(text),
92 97 labels: [],
93 98 placeholders: [],
94 99 redactions: [],
100 + ambiguities,
118 124 const count = placeholders.length;
119 125 if (origin === "provider") {
120 126 this.lastProviderRedactionCount = count;
127 + this.lastProviderAmbiguityCount = ambiguities.length;
121 128 this.totalProviderRedactionCount += count;
123 130 if (labels.length > 0) this.recentLabels = labels.slice(-12);
126 133 placeholder.replace(/^__LEAK_V1_/, "").replace(/_[a-f0-9]{32}__$/, ""),
129 - return { text: redacted, count, labels, placeholders, redactions };
136 + return {
137 + text: redacted,
138 + count,
139 + labels,
140 + placeholders,
141 + redactions,
142 + ambiguities,
143 + };
132 146 redactDeep<T>(
204 218 clear() {
205 219 this.placeholders.clear();
206 220 this.lastProviderRedactionCount = 0;
221 + this.lastProviderAmbiguityCount = 0;
207 222 this.recentLabels = [];
208 223 this.latestTags = [];
209 224 this.notifiedPlaceholders.clear();
223 238 mappingCount: this.placeholders.stats().mappingCount,
224 239 lastProviderRedactionCount: this.lastProviderRedactionCount,
225 240 totalProviderRedactionCount: this.totalProviderRedactionCount,
241 + lastProviderAmbiguityCount: this.lastProviderAmbiguityCount,
226 242 recentLabels: [...this.recentLabels],
227 243 latestTags: [...this.latestTags],
228 244 warnings: [...this.warnings],
251 267 ): DeepRedactionResult<T> {
252 268 const seen = new WeakMap<object, unknown>();
253 269 const redactions: RedactionRecord[] = [];
270 + const ambiguities: Array<{ path: string; family: string }> = [];
254 271 let count = 0;
255 272 let changed = false;
256 273 const visit = (value: unknown, depth: number, trace: string): unknown => {
268 285 count += result.count;
269 286 for (const item of result.redactions)
270 287 redactions.push({ ...item, path: trace || "$" });
288 + for (const item of result.ambiguities)
289 + ambiguities.push({ ...item, path: trace || "$" });
271 290 return result.text;
273 292 if (value === null || typeof value !== "object") return value;
311 330 return copy;
313 332 const value = visit(input, 0, "") as T;
314 - if (origin === "provider" && !restore)
333 + if (origin === "provider" && !restore) {
315 334 this.lastProviderRedactionCount = count;
316 - return { value, changed, count, redactions };
335 + this.lastProviderAmbiguityCount = ambiguities.length;
337 + return { value, changed, count, redactions, ambiguities };
319 340 private warnOnce(code: string, message: string) {

extensions/leaks/src/pi.ts

Modified
139 139 result: RedactionResult,
142 + if (result.ambiguities.length > 0)
143 + notify(ctx, formatAmbiguities(result.ambiguities), "warning");
142 144 if (!ctx.hasUI) return;
143 145 ctx.ui.setStatus("leaks", `leaks on | last ${result.count}`);
144 146 if (result.count === 0) return;
218 220 if (value === undefined) return;
219 221 const issue = engine.hydrateToolInput(value).issue;
220 - if (!issue || issue.label.startsWith("TRAVERSAL_")) return;
222 + if (
223 + !issue ||
224 + issue.label.startsWith("TRAVERSAL_") ||
225 + issue.label === "UNSUPPORTED_OBJECT"
227 + return;
221 228 notify(
223 230 `unresolved leaks placeholder in ${surface} at ${issue.path} (${issue.label})`,
288 295 }).settings;
298 +function formatAmbiguities(
299 + ambiguities: Array<{ path: string; family: string }>,
300 +) {
301 + const counts = new Map<string, number>();
302 + for (const item of ambiguities) {
303 + const key = `${item.path || "$"} — ${item.family}`;
304 + counts.set(key, (counts.get(key) ?? 0) + 1);
306 + return [
307 + `leaks kept ${ambiguities.length} ambiguous candidate(s) visible`,
308 + ...[...counts].map(
309 + ([key, count]) => `• ${key}${count > 1 ? ` ×${count}` : ""}`,
310 + ),
311 + ].join("\n");
291 314 function formatNotice(result: {
292 315 count: number;
293 316 redactions: Array<{ path: string; labels: string[] }>;
335 358 `mappings: ${stats.mappingCount}`,
336 359 `last provider redactions: ${stats.lastProviderRedactionCount}`,
337 360 `total provider redactions: ${stats.totalProviderRedactionCount}`,
361 + `last provider ambiguities: ${stats.lastProviderAmbiguityCount}`,
338 362 `recent labels: ${stats.recentLabels.map(noticeLabel).join(", ")}`,
339 363 `latest tags: ${stats.latestTags.join(", ")}`,
340 364 ].join("\n");

extensions/leaks/src/structures.ts

Modified
67 67 regex: regex as RegExp,
70 +export function findStructureAmbiguities(text: string) {
71 + const ambiguities: Array<{ family: string; start: number }> = [];
72 + for (const match of text.matchAll(JWT_RE))
73 + if (findJwtSpans(match[0]).length === 0)
74 + ambiguities.push({ family: "JWT", start: match.index as number });
75 + for (const match of text.matchAll(AUTH_RE))
76 + if (
77 + match[1]?.toLowerCase() === "basic" &&
78 + findHeaderSpans(match[0]).length === 0
79 + )
80 + ambiguities.push({ family: "BASIC_AUTH", start: match.index as number });
81 + for (const match of text.matchAll(URL_CANDIDATE_RE)) {
82 + if (!/[?&](?:token|secret|password|api[_-]?key)=/i.test(match[0])) continue;
83 + if (findSensitiveQuerySpans(match[0]).length === 0)
84 + ambiguities.push({ family: "URL_QUERY", start: match.index as number });
85 + }
86 + return ambiguities;
70 89 export function findStructureSpans(text: string): SecretSpan[] {
71 90 return [
72 91 ...findPrivateKeySpans(text),

extensions/leaks/src/types.ts

Modified
45 45 placeholders: string[];
48 +export type AmbiguityRecord = {
49 + path: string;
50 + family: string;
51 +};
48 53 export type RedactionResult = {
49 54 text: string;
50 55 count: number;
51 56 labels: string[];
52 57 placeholders: string[];
53 58 redactions: RedactionRecord[];
59 + ambiguities: AmbiguityRecord[];
56 62 export type DeepRedactionResult<T = unknown> = {
58 64 changed: boolean;
59 65 count: number;
60 66 redactions: RedactionRecord[];
67 + ambiguities: AmbiguityRecord[];
63 70 export type EngineStats = {
71 78 mappingCount: number;
72 79 lastProviderRedactionCount: number;
73 80 totalProviderRedactionCount: number;
81 + lastProviderAmbiguityCount: number;
74 82 recentLabels: string[];
75 83 latestTags: string[];
76 84 warnings: WarningRecord[];

mise.toml

Modified
54 54 [tasks.fuzz]
55 55 description = "Run property tests"
56 56 depends = ["install"]
57 -run = '''
58 -node -e "const {spawnSync}=require('node:child_process');const r=spawnSync(process.execPath,['node_modules/vitest/vitest.mjs','run','--config','extensions/leaks/vitest.fuzz.config.ts'],{stdio:'inherit',timeout:120000,killSignal:'SIGKILL'});process.exit(r.error?.code==='ETIMEDOUT'?124:r.status??1)"
59 -'''
57 +run = "node scripts/run_leaks_fuzz.mjs"
61 59 [tasks.bench]
62 60 description = "Run all extension benchmarks sequentially"
87 85 "mise run check",
88 86 "mise run lint",
89 87 "mise run test",
88 + "mise run fuzz",
90 89 "mise run e2e",

scripts/run_leaks_fuzz.mjs

Added
new file mode 100644
1 +import { spawnSync } from "node:child_process";
2 +import process from "node:process";
3 +import { fileURLToPath } from "node:url";
5 +export function runProcess(command, args, timeout) {
6 + const result = spawnSync(command, args, {
7 + stdio: "inherit",
8 + timeout,
9 + killSignal: "SIGKILL",
10 + });
11 + return result.error?.code === "ETIMEDOUT" ? 124 : (result.status ?? 1);
14 +if (process.argv[1] === fileURLToPath(import.meta.url)) {
15 + const timeout = Number(process.env.LEAKS_FUZZ_TIMEOUT_MS ?? 120_000);
16 + const args = process.env.LEAKS_FUZZ_TEST_HANG
17 + ? ["-e", "setInterval(() => {}, 1000)"]
18 + : [
19 + "node_modules/vitest/vitest.mjs",
20 + "run",
21 + "--config",
22 + "extensions/leaks/vitest.fuzz.config.ts",
23 + ];
24 + process.exit(runProcess(process.execPath, args, timeout));