Can the core approach in spec/TESTING.md produce fast, deterministic, legible
safety tests with Zig 0.16.0?
Method
An isolated throwaway Zig program implemented production-style state transitions, a typed effect
ledger, fixed-capacity storage, allocator sealing, and a 40-column cell grid.
Four tests covered protected selection, confirmation-gated mutation, capacity exhaustion, and
screen output.
The program was also changed deliberately to emit a mutation during review.
The valid suite was run 20 times through zig test; complete output was hashed for determinism.
A separate width check measured rendered rows independently from the golden screen text.
Observations
All four valid tests passed without allocation after initialization.
The deliberate authorization defect failed both no-mutation assertions and exposed an unchecked
write at zero effect capacity.
Twenty compile-and-test invocations took 19.27 seconds in total and produced byte-identical
500-byte outputs.
The first golden screen passed despite containing 39 visible columns for a nominal 40-column
layout. An independent width check found the defect. After separating cell storage from newline
formatting, repeated outputs remained identical and all rows measured 40 columns.
Conclusion
The core strategy is supported: fixed-capacity state-machine tests, typed forbidden-effect
assertions, allocator sealing, and cell-grid evidence are practical, fast, and deterministic in
Zig 0.16.0.
Golden screens are not sufficient structural oracles. Layout tests must assert dimensions and
other structural invariants independently from expected screen text.
This experiment does not verify PTY behavior, filesystem races, process control, image artifacts,
or suite performance at production scale. Those require implementation-specific experiments.
# Zig testing-strategy core experiment
## Question
Can the core approach in [`spec/TESTING.md`](../TESTING.md) produce fast, deterministic, legible
safety tests with Zig 0.16.0?
## Method
An isolated throwaway Zig program implemented production-style state transitions, a typed effect
ledger, fixed-capacity storage, allocator sealing, and a 40-column cell grid.
Four tests covered protected selection, confirmation-gated mutation, capacity exhaustion, and
screen output.
The program was also changed deliberately to emit a mutation during review.
The valid suite was run 20 times through `zig test`; complete output was hashed for determinism.
A separate width check measured rendered rows independently from the golden screen text.
## Observations
- All four valid tests passed without allocation after initialization.
- The deliberate authorization defect failed both no-mutation assertions and exposed an unchecked
write at zero effect capacity.
- Twenty compile-and-test invocations took 19.27 seconds in total and produced byte-identical
500-byte outputs.
- The first golden screen passed despite containing 39 visible columns for a nominal 40-column
layout. An independent width check found the defect. After separating cell storage from newline
formatting, repeated outputs remained identical and all rows measured 40 columns.
## Conclusion
**The core strategy is supported:** fixed-capacity state-machine tests, typed forbidden-effect
assertions, allocator sealing, and cell-grid evidence are practical, fast, and deterministic in
Zig 0.16.0.
**Golden screens are not sufficient structural oracles.** Layout tests must assert dimensions and
other structural invariants independently from expected screen text.
This experiment does not verify PTY behavior, filesystem races, process control, image artifacts,
or suite performance at production scale. Those require implementation-specific experiments.