Luigit
repositories / smith

smith

There are many coding harnesses - but this one is fast

owned by admin

.system/issues/SMH-ISSUE-MUTTMPDR-inspection-memory-test-fails-under-mutation-scratch/index.md

Raw
Rendered preview

id: SMH-ISSUE-MUTTMPDR type: issue title: "Inspection memory test fails under mutation scratch paths" specs: [SMH-SPEC-SPEC0001]

Inspection memory test fails under mutation scratch paths

Symptom before fix

cargo x mutants with the repository configuration reports every mutant as caught. Each mutant fails on the same unrelated test, inspection_large_history_peak_is_independent_of_payload_bytes in smith-harness. Nextest fail-fast then stops before the mutated crate's own tests run. The same test passes under cargo x check.

Impact

Mutation results from full-configuration runs are invalid: "caught" proves nothing. Missed mutants stay hidden until someone narrows the test scope by hand.

Cause

cargo x mutants sets TMPDIR to the absolute scratch directory target/mutants-tmp, a 43-byte path in this checkout. The test pads its fixture directory to a pinned 41-byte canonical path so that path-allocation sizes stay constant. A temporary base longer than that budget fails the padding precondition, so the test panics. The test therefore depends on the ambient temporary directory length, not only on the code under test.

sequenceDiagram
  participant X as cargo x mutants
  participant N as nextest
  participant T as inspection memory test
  X->>N: run tests with TMPDIR = target/mutants-tmp
  N->>T: run test
  T->>T: canonical temp base 43 bytes, budget 41
  T-->>N: panic: temporary path exceeds pinned fixture length
  N-->>X: fail-fast stop, mutant recorded as caught

Evidence

  • Unsafe-law work: full-config mutation run with 96 caught mutants, all failing on this test; a valid rerun with tests limited to xtask found 22 misses.
  • Fixture budget and padding: inspection tests.
  • Scratch directory selection: xtask.

Acceptance

  • The inspection memory test holds its allocation ceilings for any temporary directory length the harness can produce.
  • A full-configuration mutation run reaches the mutated crate's tests, and its caught and missed counts reflect them.

Resolution

Removed the 41-byte temporary-path precondition. The regression exercises the harness temporary root plus 64-byte and 128-byte prefixes. Required path capacity is computed through std outside the measured implementation. Peak accounting replaces exactly one list path or two grep paths with the original 164-byte reference allowance per path. The 65,776-byte list ceiling, 131,676-byte grep ceiling, history-growth assertions, and 256 KiB absolute grep guard remain unchanged. Extra retained paths or payload storage still consume the fixed inspection budget. No production behavior changes.

Verification

  • Regression first: the long-prefix case failed with canonical temporary path exceeds pinned fixture length; the fixed test passed.
  • cargo x check: 440 tests passed; formatting, lint, architecture, docs, unused-dependency gates passed.
  • cargo x e2e: 32 tests passed.
  • Full-config mutation smoke: cargo x mutants -f smith/src/config.rs --re 'config.rs:76:33: replace < with <=' -- --no-fail-fast.
  • All 463 configured tests ran under mutation scratch; inspection regression passed.
  • The mutation was caught by smith config tests; three failures all reject the mutated 1024-byte inclusive lower bound.
  • Result: 1 caught, 0 missed, 0 timeouts, 0 unviable.
  • Evidence: mutation outcomes and test log, run started 2026-10-01T23:50:36Z.

Earlier full-config mutation counts remain invalid. This smoke proves runner health, not workspace-wide mutation completeness.

---
id: SMH-ISSUE-MUTTMPDR
type: issue
title: "Inspection memory test fails under mutation scratch paths"
specs: [SMH-SPEC-SPEC0001]
---

# Inspection memory test fails under mutation scratch paths

## Symptom before fix

`cargo x mutants` with the repository configuration reports every mutant as caught.
Each mutant fails on the same unrelated test, `inspection_large_history_peak_is_independent_of_payload_bytes` in smith-harness.
Nextest fail-fast then stops before the mutated crate's own tests run.
The same test passes under `cargo x check`.

## Impact

Mutation results from full-configuration runs are invalid: "caught" proves nothing.
Missed mutants stay hidden until someone narrows the test scope by hand.

## Cause

`cargo x mutants` sets TMPDIR to the absolute scratch directory `target/mutants-tmp`, a 43-byte path in this checkout.
The test pads its fixture directory to a pinned 41-byte canonical path so that path-allocation sizes stay constant.
A temporary base longer than that budget fails the padding precondition, so the test panics.
The test therefore depends on the ambient temporary directory length, not only on the code under test.

```mermaid
sequenceDiagram
  participant X as cargo x mutants
  participant N as nextest
  participant T as inspection memory test
  X->>N: run tests with TMPDIR = target/mutants-tmp
  N->>T: run test
  T->>T: canonical temp base 43 bytes, budget 41
  T-->>N: panic: temporary path exceeds pinned fixture length
  N-->>X: fail-fast stop, mutant recorded as caught
```

## Evidence

- Unsafe-law work: full-config mutation run with 96 caught mutants, all failing on this test; a valid rerun with tests limited to xtask found 22 misses.
- Fixture budget and padding: [inspection tests](../../../smith-harness/src/inspection.rs).
- Scratch directory selection: [xtask](../../../xtask/src/main.rs).

## Acceptance

- [x] The inspection memory test holds its allocation ceilings for any temporary directory length the harness can produce.
- [x] A full-configuration mutation run reaches the mutated crate's tests, and its caught and missed counts reflect them.

## Resolution

Removed the 41-byte temporary-path precondition.
The regression exercises the harness temporary root plus 64-byte and 128-byte prefixes.
Required path capacity is computed through std outside the measured implementation.
Peak accounting replaces exactly one list path or two grep paths with the original 164-byte reference allowance per path.
The 65,776-byte list ceiling, 131,676-byte grep ceiling, history-growth assertions, and 256 KiB absolute grep guard remain unchanged.
Extra retained paths or payload storage still consume the fixed inspection budget.
No production behavior changes.

## Verification

- Regression first: the long-prefix case failed with `canonical temporary path exceeds pinned fixture length`; the fixed test passed.
- `cargo x check`: 440 tests passed; formatting, lint, architecture, docs, unused-dependency gates passed.
- `cargo x e2e`: 32 tests passed.
- Full-config mutation smoke: `cargo x mutants -f smith/src/config.rs --re 'config.rs:76:33: replace < with <=' -- --no-fail-fast`.
- All 463 configured tests ran under mutation scratch; inspection regression passed.
- The mutation was caught by `smith` config tests; three failures all reject the mutated 1024-byte inclusive lower bound.
- Result: 1 caught, 0 missed, 0 timeouts, 0 unviable.
- Evidence: [mutation outcomes](../../../target/mutants.out/outcomes.json) and [test log](../../../target/mutants.out/log/smith__src__config.rs_line_76_col_33.log), run started `2026-10-01T23:50:36Z`.

Earlier full-config mutation counts remain invalid.
This smoke proves runner health, not workspace-wide mutation completeness.