repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by bugabinga

docs(system): add gratis synthetic provider spec and research bundles

55017f549ec0b062218f50e8d8977ad6aa1ae9bf

Oliver Krylow <o.krylow@isp-insoft.de> ·

SSH signature · not checked

Browse filesCompare with HEAD

Changed files (6)

.system/research/PX-RESEARCH-8WQ4KZ2M-opencode-free-tier-lock/index.md

Added
new file mode 100644
1 +---
2 +id: PX-RESEARCH-8WQ4KZ2M
3 +type: research
4 +title: OpenCode Free Tier Lock
5 +---
7 +## OpenCode Free Tier Lock
9 +Why `opencode`/`zen` cannot be a gratis backend. Complements [Kilo Gateway Free Access](../PX-RESEARCH-KZ2M6WQ4-kilo-gateway-free-access/index.md).
10 +All probes 2026, live HTTP and public source; no secrets involved.
12 +### Facts
14 +- Two surfaces exist and must not be conflated: **opencode free** (zero-setup models inside the OpenCode TUI) and **Zen** (the `opencode.ai/zen/v1` gateway with console API keys). In OpenCode's own source and models.dev, provider `opencode` = "OpenCode Zen" (env `OPENCODE_API_KEY`); Zen catalog on models.dev lists ~15 free models (`big-pickle`, `grok-code`, `*-free`, `muse-spark-*-contributor-free`, …).
15 +- Live probe: `GET /zen/v1/models` anonymous → 200, but **zero free IDs** in the public response; free roster is not publicly listed.
16 +- Live probe: `POST /zen/v1/chat/completions` (`big-pickle`) anonymous → **403 `FreeTierError`** "OpenCode's free tier can only be used from within OpenCode".
17 +- Independent confirmation: freellmapi server catalog (issues #1249, #1204) — free models return the same 403 **on a valid Zen key**, and sending OpenCode client headers does not change it; they tried, reverted, and disabled the free rows. The provider remains registered there only for paid models.
18 +- OpenCode client source (`packages/core/src/plugin/provider/opencode.ts`): the "opencode" integration authenticates via OAuth device flow against `https://opencode.ai/console` (client `opencode-cli`) or a service-account key; free access inside the TUI rides this client identity, enforced server-side beyond simple headers.
19 +- A Zen console key is free to create, but it unlocks **paid models only** — it grants no free inference from non-OpenCode clients.
21 +### Conclusion
23 +The TUI's "open opencode and it just works" experience is real but is a client-locked free tier: server-attested OpenCode client, not a key, and not reproducible with plain HTTP.
24 +Consequences for gratis: no `opencode` backend, no `GRATIS_OPENCODE_TOKEN`; any workaround would mean mimicking client identity — fragile and against the provider's intent.
25 +This mirrors the post-mortem lesson: do not build on unlabeled, uncontractual access.

.system/research/PX-RESEARCH-KMBSJUZR-free-llm-inference-providers/index.md

Added
new file mode 100644
1 +---
2 +id: PX-RESEARCH-KMBSJUZR
3 +type: research
4 +title: Free LLM Inference Providers
5 +---
7 +## Free LLM Inference Providers
9 +Free-tier LLM inference providers that are established (multi-year track record), their access requirements, and estimated free usage.
10 +Numbers change frequently; each row cites its source and all sources were checked at research time.
12 +### Landscape
14 +| Provider | Since | Access requirements | Free usage estimate | Caveats |
15 +| --- | --- | --- | --- | --- |
16 +| [OpenRouter](https://openrouter.ai/docs/faq) | 2023 | Account + API key, no card | `:free` models: 50 req/day, 20 req/min; with ≥\$10 lifetime credit purchase: 1000 req/day | Free endpoints may log prompts; 429s common; lowest tier caps prompt tokens per request |
17 +| [OpenCode Zen](https://opencode.ai/docs/zen) | 2025 | OpenCode account + API key, no card | Rotating free stealth/trial models (Big Pickle, MiMo-V2.5 Free, Nemotron Free, Ling Free); no published rate limit | Free = "limited time"; data may be collected to improve model; NVIDIA-served ones are trial-only |
18 +| [Google AI Studio / Gemini API](https://ai.google.dev/gemini-api/docs/rate-limits) | 2023 | Google account, no card | Per model, viewed in AI Studio; Flash class ≈ 10-15 RPM, 250K-1M TPM, 250-1500 RPD; RPD resets midnight PT | No published universal table anymore; free tier data may be used for product improvement |
19 +| [Groq](https://console.groq.com/docs/rate-limits) | API free tier 2024 | Account, no card | Per model; e.g. gpt-oss / qwen3-27b: 30 RPM, 1K RPD, 8K TPM, 200K TPD; audio + guard models higher RPD | Curated open models only; limits are org-level; exact values on limits page |
20 +| [Cerebras](https://inference-docs.cerebras.ai/) | 2024 | Account, no card | Free trial: 5 RPM, 30K TPM, 1M tokens/day (gpt-oss-120b, GLM-4.7) | 5 RPM breaks parallel agent tool calls; eval-only |
21 +| [Mistral La Plateforme](https://help.mistral.ai/en/articles/698531-why-am-i-hitting-api-rate-limits-and-how-do-i-increase-them) | Free tier 2024 | Account + phone (SMS) verification, no card | "Experiment" plan: ≈1 RPS, 500K TPM, ≈1B tokens/month, all models incl. Codestral | 1 RPS slow for concurrency; free data may train unless opted out |
22 +| [GitHub Models](https://docs.github.com/en/copilot/concepts/billing/copilot-requests) | 2024 | GitHub account + PAT | ≈10 RPM, ≈50 RPD on free Copilot tier; higher with paid Copilot plans | Hard 8K input / 4K output token cap per request; prototyping only |
23 +| [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/platform/pricing/) | GA 2024/25 | Cloudflare account | 10,000 Neurons/day ≈ 100-300 LLM requests/day; text generation ≈300 RPM | Frontier models (kimi, glm, deepseek) need paid plan or prepaid AI Gateway credits |
24 +| [NVIDIA NIM](https://build.nvidia.com/models) | 2024 | NVIDIA account | Free prototype endpoints, ≈40 RPM most models, no per-token billing | Prototype/trial use; older credit system (1000 credits signup) replaced by RPM limits |
25 +| [Cohere](https://docs.cohere.com/docs/rate-limits) | Trial keys since ~2023 | Account, no card | Trial key: 20 req/min chat, 1000 API calls/month total | Embed 2000 inputs/min; production key requires billing |
26 +| [Z.ai / Zhipu](https://docs.z.ai/guides/llm/glm-4.5) | GLM-4.5 free tier 2025 | Account, no card | GLM-4.5-Flash and GLM-4.7-Flash: \$0/token for registered users, 128K-200K ctx | Rate limits unpublished; concurrency caps by account tier |
27 +| [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers/pricing) | 2024/25 | HF account, no card | \$0.10 credits/month free users; \$2/month with PRO (\$9/mo) | After credits: must purchase credits; smallest allowance in this list |
28 +| [Vercel AI Gateway](https://vercel.com/docs/ai-gateway/pricing) | 2025 | Vercel team account | \$5 credits per 30 days for non-paying teams; free-tier model subset, lower per-model rate limits | Buying credits ends monthly free credit; BYOK needs paid tier |
30 +Discontinued free access worth knowing: [Chutes](https://chutes.ai/pricing) removed its \$5-deposit 200 req/day free tier in Aug 2025 → paid subscriptions only.
31 +[NanoGPT](https://nano-gpt.com/get-started) offers one anonymous free model only; paid access from \$0.10 crypto / \$1 card deposit.
33 +### Reading the estimates
35 +Free request budgets per day are the headline number, but token budgets decide usability for coding agents:
37 +- OpenRouter 50 req/day ≈ 1-3M tokens/day at agent-typical 20-50K tokens/request → one short coding session.
38 +- Gemini free Flash ≈ 250-1500 RPD with 250K-1M TPM → the largest generous allowance; viable for real daily agent use.
39 +- Groq 200K-500K TPD on main models → small contexts only.
40 +- Mistral 1B tokens/month ≈ 33M/day → largest total volume, throttled to 1 RPS.
41 +- Cerebras 1M tokens/day but 5 RPM → single-threaded experiments.
42 +- GitHub Models 50 RPD × 8K in/4K out → smoke tests, not agents.
44 +### Conclusions
46 +1. Best sustainable free coding-agent budget: Google AI Studio (Flash class), then Mistral (volume, 1 RPS) and OpenRouter with the one-time \$10 top-up (1000 free req/day, permanent).
47 +2. Cheapest "free forever" unlock: OpenRouter \$10 one-time purchase multiplies free quota 20x; no subscription.
48 +3. All free tiers can change or vanish (Chutes precedent); none is contractual capacity.
49 +4. Privacy: free tiers routinely log or train on prompts (Google, Mistral, OpenRouter free endpoints, Zen free models); avoid proprietary code on them.
51 +### Unresolved questions
53 +- OpenCode Zen free-model rate limits: undocumented; only observable by hitting 429s.
54 +- Google per-model free limits: no longer published as a static table; must be read per project in AI Studio.
55 +- Z.ai free-model concurrency caps: unpublished, vary by account tier.

.system/research/PX-RESEARCH-KZ2M6WQ4-kilo-gateway-free-access/index.md

Added
new file mode 100644
1 +---
2 +id: PX-RESEARCH-KZ2M6WQ4
3 +type: research
4 +title: Kilo Gateway Free Access
5 +---
7 +## Kilo Gateway Free Access
9 +Re-evaluation of Kilo as a gratis backend, requested after the [Free Extension Post-Mortem](../PX-RESEARCH-LXU8HWNJ-free-extension-post-mortem/index.md) flagged Kilo as unresolved.
10 +Sources: [Authentication](https://kilo.ai/docs/gateway/authentication), [Models & Providers](https://kilo.ai/docs/gateway/models-and-providers), [Using Kilo for Free](https://kilo.ai/docs/getting-started/using-kilo-for-free), [Gateway overview](https://kilo.ai/docs/gateway).
12 +### Facts
14 +- OpenAI-compatible gateway at `https://api.kilo.ai/api/gateway` (`/chat/completions`, SSE streaming, tool calling).
15 +- **Anonymous free access: `:free`-suffixed models need no API key at all.** Unauthenticated requests are IP-identified and rate limited to 200 requests/hour/IP. This is the only known free tier with zero account requirement.
16 +- Model catalog endpoint `GET /api/gateway/models` requires no authentication; returns pricing, context window, modalities → live discovery is possible and necessary.
17 +- Free pool (rotates; sample at research time): `minimax/minimax-m2.1:free`, `z-ai/glm-5:free`, `stepfun/step-3.7-flash:free`, `poolside/laguna-s-2.1:free`, `poolside/laguna-xs-2.1:free`, `nvidia/nemotron-3-ultra-550b-a55b:free`, `tencent/hy3:free`, plus virtual `openrouter/free` (best available free model) and `kilo-auto/free` (server-side routing over the free pool, per-session affinity).
18 +- Optional `KILO_API_KEY` (JWT from a Kilo account) raises limits beyond anonymous; paid credits only for non-free models.
19 +- Data caveat: `kilo-auto/free` may route to providers that log prompts/outputs for service improvement; free-pool providers vary.
21 +### Churn evidence
23 +The old extension hardcoded Kilo IDs (`inclusionai/ling-2.6-1t:free`, `tencent/hy3-preview:free`, `baidu/qianfan-ocr-fast:free`, …) in April 2026 — none appear in the current pool.
24 +Confirms the post-mortem: Kilo's failure mode was catalog rot, not auth. Discovery at load + tolerance for empty pools is mandatory.
26 +### Verdict
28 +Kilo belongs in gratis v1 — as the keyless tier:
30 +- Only backend delivering the "zero config" goal literally: no env var, no account, visible out of the box.
31 +- Requires a spec amendment: current spec ties visibility to key resolution; Kilo is visible without a key (network permitting), `KILO_API_KEY` optional as enhancement.
32 +- Anonymity is IP-scoped → shared egress (office/VPN) shares the 200 req/h budget; keyed mode avoids that.
34 +### Unresolved questions
36 +- Whether a `KILO_API_KEY` actually raises the free-model rate limit above 200 req/h/IP (docs state anonymous limiting; keyed free-model limit unpublished).
37 +- `openrouter/free` and `kilo-auto/free` are virtual models whose underlying mapping changes server-side; their effective context window is unknown → catalog must treat virtual models conservatively.

.system/research/PX-RESEARCH-LXU8HWNJ-free-extension-post-mortem/index.md

Added
new file mode 100644
1 +---
2 +id: PX-RESEARCH-LXU8HWNJ
3 +type: research
4 +title: Free Extension Post-Mortem
5 +---
7 +## Free Extension Post-Mortem
9 +Prior attempt at the same idea as [Gratis Synthetic Free Provider](../../specs/PX-SPEC-CNDFHNO2-gratis-synthetic-free-provider/index.md): a `free` extension in the dotfiles Pi setup, April-May 2026.
10 +Sources: dotfiles git history commits `5fbf178c` (added), `b87a61ad` (packaged), `5ca1296c` (reworked), `a6c52214` (deleted).
12 +### What it was
14 +Single extension registering three synthetic providers over keyless free endpoints:
16 +| Synthetic provider | Upstream | API | Catalog |
17 +| --- | --- | --- | --- |
18 +| `free` | Kilo gateway (`api.kilo.ai/api/gateway/v1`) | openai-completions | hardcoded, manual `/free-refresh` re-fetch |
19 +| `zen` | opencode.ai/zen (big-pickle, minimax-m2.5-free) | anthropic-messages | startup fetch of `zen/v1/models` |
20 +| `zen-free` | opencode.ai/zen (hy3, ling, nemotron `-free`) | openai-completions | same startup fetch |
22 +Model IDs leaked upstream paths (`kilo-auto/free`, `openrouter/free`, `nvidia/nemotron-3-super-120b-a12b:free`).
23 +Auth was `apiKey: "-"` with `authHeader: false` — both endpoints served free models without accounts in April 2026.
25 +### Timeline
27 +```mermaid
28 +flowchart LR
29 + add["2026-05-04 added<br/>5fbf178c"] --> pkg["2026-05-04 packaged<br/>b87a61ad"] --> rw["2026-05-09 reworked<br/>5ca1296c"] --> del["2026-05-21 deleted<br/>a6c52214"]
30 +```
32 +Deleted in a commit otherwise titled "config: add kimi coding model" (same change migrated all extensions to a new package scope and rewrote `models.json`) — no failure note, dropped as dead weight after ~2.5 weeks.
34 +### Failure analysis
36 +1. **Keyless assumption expired.** Free access without any account was the load-bearing design. Zen now requires login + API key; Kilo free models moved behind its own gateway account. When endpoints started rejecting unauthenticated calls, every registered model broke at request time.
37 +2. **Visible but broken, the worst failure mode.** Fake key `-` meant providers always registered. Stale catalogs and dead endpoints surfaced as per-request errors, never as absence.
38 +3. **Provider sprawl.** Three synthetic providers named after API shapes (`zen` vs `zen-free`) forced users to know transport details to pick a model.
39 +4. **Catalog churn.** Hardcoded Kilo list rotted between manual `/free-refresh` runs; a session-start notify reminded every session, which is noise.
40 +5. **No resilience.** No retry, no failover, no cooldown; one 429 ended the request.
41 +6. **Static model params blocked the auto router.** Pi model attributes (context window, max tokens) were fixed per registered model entry at the time; there was no way to give one synthetic router model per-backend parameters. (Human-reported; consistent with the single-entry-per-model design of the old code.)
43 +Current Pi resolves 6: `ProviderModelConfig` supports per-model `api` and `baseUrl` overrides, `registerProvider` calls after load apply immediately (model sets replaceable live), custom `streamSimple` gives full per-request control, and `message_end` error normalization makes context-overflow recovery work for custom providers ([custom-provider docs](https://github.com/earendil-works/pi/blob/main/packages/ai/test)). Viable auto strategy: advertise the minimum context window across keyed backends, normalize upstream overflow errors to `context_length_exceeded`, and treat overflow like exhaustion in the cascade.
45 +Points 1-4 are evidenced by the code and current provider docs; point 5's removal trigger (commit pairing) is inferred, not documented.
47 +### Lessons vs current spec
49 +| Old failure | Gratis spec countermeasure |
50 +| --- | --- |
51 +| Keyless endpoints died | GRATIS_*/native key chain; unkeyed backend invisible |
52 +| Visible but broken models | Register only keyed backends; no key → no provider |
53 +| Three API-shaped providers | One `grts` provider, backend-nested model IDs |
54 +| Stale hardcoded catalogs | Static curated + Zen live discovery of `-free` models |
55 +| No failover | `grts/auto` cascade with per-backend cooldown |
57 +### New constraint discovered
59 +Zen free models are split across two API types: `big-pickle`/`minimax`-class via anthropic-messages, others via openai-completions — and the split can change between catalog refreshes.
60 +The spec's backends table pins opencode to "OpenAI completions (Zen v1)"; correct model is per-model API type carried by each catalog entry, not per-backend API choice.
61 +Same applies in principle to every backend: catalog entries, not backends, own the API type.
63 +### Unresolved questions
65 +- Why exactly deletion happened on 2026-05-21: no note in commits; if session logs hold the actual breakage story, it would confirm inference 1 vs an unrelated cleanup motive.
66 +- Kilo re-evaluated separately: [Kilo Gateway Free Access](../PX-RESEARCH-KZ2M6WQ4-kilo-gateway-free-access/index.md) — verdict: include as the keyless v1 tier; the old Kilo failure was catalog rot, not auth.

.system/research/PX-RESEARCH-Q7N4M2K9-pi-model-sdk-and-failover-prior-art/index.md

Added
new file mode 100644
1 +---
2 +id: PX-RESEARCH-Q7N4M2K9
3 +type: research
4 +title: Pi Model SDK and Failover Prior Art
5 +---
7 +## Pi Model SDK and Failover Prior Art
9 +SDK surfaces around models/providers, and extensions that already attempted free-model catalogs, model switching, or failover.
10 +Sources: installed Pi docs (`extensions.md`, `custom-provider.md`), pi-ext sources, GitHub community extensions (probed 2026).
12 +### Pi SDK surfaces relevant to gratis
14 +- `pi.registerProvider()`: full `createProvider` or legacy config form; async extension factories finish before startup, so discovered models exist for `/model` and `pi --list-models`; registrations after load apply immediately, so model sets can be replaced at runtime without `/reload`.
15 +- `pi.setModel(model)`: programmatic model switch, `await`-able, returns success; used by `masks` (with restore) and every community failover extension.
16 +- `ctx.modelRegistry`: `find(provider, id)`, `getProvider(provider)` (incl. `.auth.oauth`), `getApiKeyForProvider(provider)` — key resolution without touching env directly (pi-ext `quota`, `model-info` rely on it).
17 +- Events: `model_select`, `before_provider_headers`, `before_provider_request` (payload rewrite per request), `after_provider_response` (status + headers, pre-stream-consume), `message_end` (error-message rewrite → feeds Pi's overflow recovery), `agent_end`.
18 +- Pi's own retry: regex-classified retryable errors (429/5xx/network) get automatic retry with backoff; failover extensions classify with a copy of Pi's `_isRetryableError` regex to decide "swap now vs wait for Pi's retry".
19 +- Per-model `api` and `baseUrl` overrides exist in `ProviderModelConfig`; custom `streamSimple` gives full request control.
21 +### Prior art
23 +| Extension | Approach | Mechanism |
24 +| --- | --- | --- |
25 +| dotfiles `free` (deleted) | keyless free catalog (Zen, Kilo) | 3 synthetic providers; died with endpoint lock (see post-mortem) |
26 +| [shyim/pi-orcarouter](https://github.com/shyim/pi-orcarouter) | one gateway as provider | dynamic catalog: fetch → disk cache → baked-in generated fallback; stale cache triggers visible refresh; only chat-capable models registered |
27 +| [Columpio/pi-extensions](https://github.com/Columpio/pi-extensions) `pi-failover` | error-triggered model swap | `message_end` classification (copy of Pi's retryable regex) + consecutive-error threshold → `pi.setModel(fallback)` + prompt re-run; per-model cooldown with `degradedUntil`; own synthetic failover provider |
28 +| [continua-ai/pi-lab](https://github.com/continua-ai/pi-lab) `subscription-fallback` | auth-route rotation | preference stack of vendor routes (oauth + api_key per vendor), model policy `follow_current`, command UX + tool bridge |
29 +| season179/pi-ecosystem `pi-model-fallback` | persisted fallback chain | `setModel` with `internalSwitch` guard against own `model_select` recursion; persisted active fallback + restore cooldown to avoid churn |
30 +| `opencode-rate-limit` (npm, opencode ecosystem) | provider pool fallback | virtual model pool: on 429/concurrency error, transparently switch to next configured model, session context preserved, cooldown memory |
32 +### Conclusions
34 +1. The swap-on-error pattern (setModel + re-run + cooldown) is established and proven in at least four independent extensions; gratis's in-provider cascade is the less common but cleaner pattern — it keeps `grts/auto` a single selectable model instead of mutating the user's active model.
35 +2. gratis can reuse, not reinvent: Pi's retryable-regex classification, `after_provider_response` for status/header observation, `message_end` for overflow normalization, `modelRegistry.getApiKeyForProvider` for native-key resolution (stronger than raw env: also finds OAuth/auth-store keys).
36 +3. Catalog pattern proven in the wild: fetch → cache → baked-in fallback (pi-orcarouter); gratis static catalogs can adopt the same three-layer scheme.
37 +4. The `internalSwitch` guard (season179) is a required detail for any gratis code that touches `setModel` or `model_select`; gratis avoids it entirely by not switching models.
38 +5. Key-resolution delta worth a spec check: `getApiKeyForProvider` surfaces keys stored in Pi's auth store — gratis's spec says "process env only"; using the registry would honor user keys from `auth.json` too (e.g. an existing OpenRouter entry), broadening zero-config reach at the cost of the env-only simplicity rule.
40 +### Unresolved questions
42 +- Whether `getApiKeyForProvider` works for a *custom* provider id (gratis backends are not Pi-native providers); if not, env-only stands as the only uniform mechanism.
43 +- Whether `after_provider_response` fires for wrapped errors (Kilo's 429-with-body) before Pi's own retry consumes them — needs implementation-phase confirmation.

.system/specs/PX-SPEC-CNDFHNO2-gratis-synthetic-free-provider/index.md

Added
new file mode 100644
1 +---
2 +id: PX-SPEC-CNDFHNO2
3 +type: spec
4 +title: Gratis Synthetic Free Provider
5 +research:
6 + - PX-RESEARCH-KMBSJUZR
7 + - PX-RESEARCH-LXU8HWNJ
8 + - PX-RESEARCH-KZ2M6WQ4
9 + - PX-RESEARCH-8WQ4KZ2M
10 + - PX-RESEARCH-Q7N4M2K9
11 +---
13 +## Gratis Synthetic Free Provider
15 +Extension `gratis` registers one synthetic Pi provider `grts` that exposes curated free-tier LLM backends as normal models.
16 +Intent: free inference with zero gratis-specific configuration; invisible when unusable.
18 +### Scope
20 +In: synthetic provider, model addressing, key resolution, backend visibility, failover cascade, free-model catalog for five backends.
21 +Out: paid models, key acquisition or signup flows, non-LLM free tiers (image, audio), usage accounting dashboards.
23 +### Addressing
25 +Nested per-backend model IDs plus one synthetic router model.
26 +`<backend>` is one of `kilo`, `google`, `openrouter`, `groq`, `mistral`.
28 +```text
29 +grts/<backend>/<model> pinned: request routes to that backend only
30 +grts/auto router: request cascades across available backends
31 +```
33 +`grts/auto` follows backend priority: kilo → google → openrouter → groq → mistral.
34 +Kilo leads because it is keyless and IP-budgeted: spend its anonymous quota before touching keyed quotas.
35 +Remaining order is by sustainable free budget (research: Gemini largest RPD/TPM; OpenRouter largest unlockable RPD; Groq, Mistral after).
36 +Within the kilo backend, virtual router models (`openrouter/free`, `kilo-auto/free`) rank first: they absorb upstream per-model saturation server-side (verified: pinned models 429 while virtual models answered in the same minute).
38 +#### Auto model parameters
40 +Pi holds model attributes per registered model entry, not per request (post-mortem failure 6).
41 +`grts/auto` therefore advertises the minimum context window and max output tokens across its candidate set, and the transport layer may tighten parameters per hop.
42 +An upstream context-overflow error on `grts/auto` normalizes to a recognized overflow error and advances the cascade exactly like exhaustion; pinned models surface it honestly.
44 +### Key resolution
46 +Per backend, evaluated independently at extension load:
48 +1. `GRATIS_<BACKEND>_TOKEN` (for example `GRATIS_OPENROUTER_TOKEN`)
49 +2. Native env of the backend (`OPENROUTER_API_KEY`, `GEMINI_API_KEY`, `GROQ_API_KEY`, `MISTRAL_API_KEY`)
50 +3. Pi's credential resolution for the backend's native provider id (`modelRegistry` auth lookup), which covers keys stored in the auth store and OAuth — env-only access is not required
51 +4. none
53 +A backend that resolves a key by any step → its `grts/<backend>/*` models register.
54 +No backend resolves a key → `grts` never registers → provider invisible in `/model`.
55 +Partial keys → only resolved backends' models appear.
56 +**Kilo exception:** kilo is keyless — its free models register whenever the catalog is reachable, no key required.
57 +`GRATIS_KILO_TOKEN` or `KILO_API_KEY` optionally raise it above anonymous 200 req/h/IP and decouple it from shared egress IPs.
58 +Keys are read from process env only; gratis never prompts, stores, or logs key material.
60 +### Transport and backends
62 +API type is a per-catalog-entry property, never a backend property; a single backend can serve several transports and the split can shift between catalog refreshes (post-mortem).
63 +All API types Pi implements are in scope from the start — the google backend alone needs its own API shape — and gratis does not assume openai-completions suffices.
64 +A catalog entry carries: model id, API type, endpoint base, context window, max output tokens, input modalities, reasoning flag.
66 +| Backend | Auth | Free models | Catalog source |
67 +| --- | --- | --- | --- |
68 +| kilo | none (key optional) | `:free` suffix + virtual `kilo-auto/free`, `openrouter/free` | live discovery via unauthenticated models endpoint |
69 +| google | Google account key | Flash class free tier | static curated |
70 +| openrouter | OpenRouter key | `:free` variants | live discovery via public models endpoint; static list as fallback |
71 +| groq | Groq key | free-tier models | static curated |
72 +| mistral | Mistral key | all models on Experiment plan | static curated |
74 +Free-set identification never relies on a suffix alone: discovery intersects live lists with a gratis-maintained free-set of names and patterns, updated with the catalog.
75 +Virtual router models advertise their published catalog parameters when present (`kilo-auto/free`: 256K ctx, 10K out); conservative fallback values apply only when nothing is published.
76 +Static catalogs are extension data; stale entries are corrected by catalog update, never by user config.
78 +### Failover
80 +Cascade applies to `grts/auto` only; pinned models surface errors honestly.
81 +Cascade operates at two granularities: models within a backend, then backends.
82 +Within an available backend, the router tries candidate models in catalog rank — virtual router models first — before cooling the backend; a 429 or quota response on one candidate advances to the next candidate.
83 +Only when every candidate fails does the backend cool until its window resets.
84 +An upstream context-overflow error on `grts/auto` advances the cascade without cooling; the router retries the same prompt on the next candidate or backend.
85 +The request fails only when every backend is cooling; the error then lists exhausted backends.
87 +```mermaid
88 +stateDiagram-v2
89 + [*] --> Available: backend available at load
90 + Available --> Cooling: 429 or quota exhausted
91 + Cooling --> Available: rate window resets
92 + Available --> [*]: unavailable at next load
93 + Cooling --> [*]: unavailable at next load
94 +```
96 +```mermaid
97 +flowchart LR
98 + req["grts/auto request"] --> pick{"backend available<br/>and not cooling?"}
99 + pick -->|yes| model["pick candidate model:<br/>virtual first"]
100 + pick -->|no| next["next backend"]
101 + next --> pick
102 + model --> send["send"]
103 + send -->|success| done["stream response"]
104 + send -->|429 or quota,<br/>candidates remain| model
105 + send -->|429 or quota,<br/>candidates exhausted| cool["cool backend"] --> next
106 + send -->|overflow| next
107 + send -->|other error| err["surface error"]
108 +```
110 +### Constraints
112 +- Every routed model costs 0; gratis never emits paid-model fallbacks.
113 +- Free tiers commonly log or train on prompts; `grts` models are labeled in their description so users can avoid them for proprietary code.
114 +- Kilo's free pool rotates hard (post-mortem: every April 2026 ID is gone); discovery must tolerate arbitrary churn and empty pools.
115 +- Rate-limit and quota responses arrive status-wrapped with provider text in the body; failure classification inspects status and body text, never status alone.
116 +- Kilo exposes no rate-limit response headers; budget awareness is observation-based (cool on observed 429), never header-based.
117 +- opencode and Zen are rejected backends: the free tier is locked to the OpenCode client (403 `FreeTierError` even on a valid key); a Zen key grants paid models only. Neither returns for gratis unless OpenCode changes the lock (research: OpenCode Free Tier Lock).
118 +- Rate-limit metadata in catalogs is advisory, never user-facing config.
119 +- No new Pi config surface: `GRATIS_*` env vars are the only configuration gratis adds; native env and Pi's credential store are read, never written.
121 +### Acceptance criteria
123 +- [ ] With no GRATIS-style and native keys set and network up, `grts/kilo/*` models appear and `grts/auto` includes kilo; all other `grts` backends stay invisible.
124 +- [ ] With no keys set and the Kilo catalog unreachable, `grts` does not appear in provider or model lists.
125 +- [ ] With exactly one native key set, only that backend's and kilo's models appear under `grts/`.
126 +- [ ] `GRATIS_OPENROUTER_TOKEN` overrides a simultaneously set native OpenRouter key.
127 +- [ ] A key stored in Pi's credential store for a native provider, with no env var set, registers the matching `grts` backend.
128 +- [ ] `grts/auto` on an exhausted first-priority backend completes on the next available backend without user action.
129 +- [ ] A context-overflow response during an `grts/auto` request advances the cascade; it never triggers cooling or fails while a next backend remains.
130 +- [ ] A pinned model that 429s returns the upstream error, not a cascade.
131 +- [ ] A pinned Kilo model returning upstream-saturation 429 while a virtual router model is available completes on that model without cooling the kilo backend.
132 +- [ ] Virtual models advertise published catalog parameters when present.
133 +- [ ] Catalog entries with non-openai-completions API types (anthropic-messages, google-shaped) stream correctly without conversion assumptions.
134 +- [ ] OpenRouter discovery with zero `:free` variants and Kilo discovery with zero `:free` models each yield no models for that backend and no error.
135 +- [ ] All registered models report input/output cost 0 and a free-tier data caveat in their description.