Luigit
repositories / termux-janitor

termux-janitor

Interactive cleanup assistant for Termux: transparent, safe, confirmed disk reclamation.

owned by admin

spec/experiments/testing_strategy_core.md

Raw
Rendered preview

Zig testing-strategy core experiment

Question

Can the core approach in spec/TESTING.md produce fast, deterministic, legible safety tests with Zig 0.16.0?

Method

An isolated throwaway Zig program implemented production-style state transitions, a typed effect ledger, fixed-capacity storage, allocator sealing, and a 40-column cell grid. Four tests covered protected selection, confirmation-gated mutation, capacity exhaustion, and screen output. The program was also changed deliberately to emit a mutation during review.

The valid suite was run 20 times through zig test; complete output was hashed for determinism. A separate width check measured rendered rows independently from the golden screen text.

Observations

  • All four valid tests passed without allocation after initialization.
  • The deliberate authorization defect failed both no-mutation assertions and exposed an unchecked write at zero effect capacity.
  • Twenty compile-and-test invocations took 19.27 seconds in total and produced byte-identical 500-byte outputs.
  • The first golden screen passed despite containing 39 visible columns for a nominal 40-column layout. An independent width check found the defect. After separating cell storage from newline formatting, repeated outputs remained identical and all rows measured 40 columns.

Conclusion

The core strategy is supported: fixed-capacity state-machine tests, typed forbidden-effect assertions, allocator sealing, and cell-grid evidence are practical, fast, and deterministic in Zig 0.16.0.

Golden screens are not sufficient structural oracles. Layout tests must assert dimensions and other structural invariants independently from expected screen text.

This experiment does not verify PTY behavior, filesystem races, process control, image artifacts, or suite performance at production scale. Those require implementation-specific experiments.

# Zig testing-strategy core experiment

## Question

Can the core approach in [`spec/TESTING.md`](../TESTING.md) produce fast, deterministic, legible
safety tests with Zig 0.16.0?

## Method

An isolated throwaway Zig program implemented production-style state transitions, a typed effect
ledger, fixed-capacity storage, allocator sealing, and a 40-column cell grid.
Four tests covered protected selection, confirmation-gated mutation, capacity exhaustion, and
screen output.
The program was also changed deliberately to emit a mutation during review.

The valid suite was run 20 times through `zig test`; complete output was hashed for determinism.
A separate width check measured rendered rows independently from the golden screen text.

## Observations

- All four valid tests passed without allocation after initialization.
- The deliberate authorization defect failed both no-mutation assertions and exposed an unchecked
  write at zero effect capacity.
- Twenty compile-and-test invocations took 19.27 seconds in total and produced byte-identical
  500-byte outputs.
- The first golden screen passed despite containing 39 visible columns for a nominal 40-column
  layout. An independent width check found the defect. After separating cell storage from newline
  formatting, repeated outputs remained identical and all rows measured 40 columns.

## Conclusion

**The core strategy is supported:** fixed-capacity state-machine tests, typed forbidden-effect
assertions, allocator sealing, and cell-grid evidence are practical, fast, and deterministic in
Zig 0.16.0.

**Golden screens are not sufficient structural oracles.** Layout tests must assert dimensions and
other structural invariants independently from expected screen text.

This experiment does not verify PTY behavior, filesystem races, process control, image artifacts,
or suite performance at production scale. Those require implementation-specific experiments.