Luigit
repositories / smith

smith

There are many coding harnesses - but this one is fast

owned by admin

.pi/skills/deflake/SKILL.md

Raw
Rendered preview

name: deflake description: "Use when a test sleeps, polls, retries, or is intermittently red: replace waiting on wall time with injected time or an explicit signal plus a deadline."

Deflake

A flaky test is a bug with a random error message.

Find the wait

  • sleep, poll until, retries = N, recv() without timeout, "give it a moment" comments.
  • Timing assertions with tight margins on shared CI machines.

Replace it

  • Time-dependent logic → inject a clock or an instant; the test sets it.
  • "Wait for the system to settle" → the system emits a signal (channel item, atomic flag, end event); the test waits on that with a deadline (recv_timeout), never on a duration.
  • Ordering between threads → a channel handoff or a barrier, not a sleep.
  • Real socket or process fixtures → keep, but every read carries a deadline and the assertion is on order and content, not on elapsed time.
  • Elapsed-time assertions → assert a bound with a wide margin only where time is the contract (timeouts); otherwise drop them.

Rules

  • Retries are never the fix.
  • Run the test in a loop (--test-threads high, 20 iterations) before calling it fixed.
  • If the flake is in the code under test, that is the real finding; fix it there.

Sources

---
name: deflake
description: "Use when a test sleeps, polls, retries, or is intermittently red: replace waiting on wall time with injected time or an explicit signal plus a deadline."
---

# Deflake

A flaky test is a bug with a random error message.

## Find the wait

- `sleep`, `poll until`, `retries = N`, `recv()` without timeout, "give it a moment" comments.
- Timing assertions with tight margins on shared CI machines.

## Replace it

- Time-dependent logic → inject a clock or an instant; the test sets it.
- "Wait for the system to settle" → the system emits a signal (channel item, atomic flag, end event); the test waits on that with a deadline (`recv_timeout`), never on a duration.
- Ordering between threads → a channel handoff or a barrier, not a sleep.
- Real socket or process fixtures → keep, but every read carries a deadline and the assertion is on order and content, not on elapsed time.
- Elapsed-time assertions → assert a bound with a wide margin only where time is the contract (timeouts); otherwise drop them.

## Rules

- Retries are never the fix.
- Run the test in a loop (`--test-threads` high, 20 iterations) before calling it fixed.
- If the flake is in the code under test, that is the real finding; fix it there.

## Sources

- [nextest: flaky tests and retries](https://nexte.st/docs/features/retries/)