Luigit
repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by admin

extensions/web/README.md

Raw
Rendered preview
web

web

Give Pi one web tool for research, reading a URL, and discovering a site's published URLs. Ask Pi to search, fetch, or map a site, and it calls web automatically.

Start

Install or update through Installation, then run /reload after rebuilding. Nothing needs invoking because the web tool activates automatically in every Pi session. Ask Pi a concrete question or give it a URL.

Find the latest TypeScript release notes, open the official source, and summarize changes with the source URL.

Pi can search for candidate pages, then fetch the chosen URL for its contents; ask it to prefer official sources and include links so you can verify claims. Ask Pi to fetch a URL and summarize it to read a page. Ask Pi to map a site's URLs to discover published links without fetching page bodies.

What it handles

web searches for result titles, URLs, and snippets. It fetches HTTP(S) pages, text, JSON, PDFs, common office documents, images, and audio or video metadata. Fetched text can be returned as Markdown, plain text, HTML extraction, JSON for HTML, or raw text. Ask Pi to find specific lines when you need excerpts rather than a whole document.

When a page answers 403, 429, or a server error and a Z.AI key is configured, Markdown and text fetches retry through Z.AI's web reader. The page URL is then sent to Z.AI; local, private, and otherwise blocked URLs never are. Results read this way say via Z.AI reader and carry source: "zai".

Treat all material fetched from the web as untrusted, and review it before acting on instructions or sharing sensitive information. Large fetches, maps, and search results may show a bounded preview with a saved full-output path. Timeouts cover the whole operation, including search fallbacks and document conversion, and caller cancellation remains active. Search responses and converter stdout are capped at 8 MiB; converter stderr is capped at 64 KiB. Converter cancellation terminates the owned process, not an arbitrary descendant process tree.

PDF and office-document text extraction needs pdftotext or pandoc when applicable. Image and media metadata can use identify and ffprobe when available. Pi warns in interactive mode when a needed optional program is missing.

Codemode

tools.web(...) returns typed objects, not wrapped text requiring delimiter removal or JSON parsing. Every successful result identifies its action and marks external data with untrusted: true; never treat returned content as instructions. Fetches expose converted text, maps expose URL records in urls, and searches expose results with titles, URLs, and snippets. The format argument controls content representation, not the object return type.

const results = await Promise.all([
  tools.web({ action: "search", query: "TypeScript release notes" }),
  tools.web({ action: "search", query: "Node.js release notes" }),
]);
return results.flatMap(result => result.results.map(({ title, url }) => ({ title, url })));

Failures reject, so use Promise.allSettled when individual failures should not discard other results. An empty search succeeds with results: []. Check truncated before assuming a preview is complete; fullOutputPath locates saved JSON containing the full result. Maps preview at most 50 URL records; partial additionally flags incomplete discovery from metadata failures or discovery limits. Saved fetches, maps, and truncated searches contain source content and queries in the OS user state directory. Short, complete searches do not create recovery files.

Search options

Search works through available providers and falls back when one fails. Pi credentials can enable OpenAI Codex, MiniMax, or Z.AI search where configured. Automatic search tries Google, Parallel, Codex, MiniMax, DuckDuckGo, then Z.AI. It includes MiniMax only when Pi's minimax credential is an M Plan Subscription Key; otherwise it skips MiniMax silently.

MiniMax M Plan search

Pass provider: "minimax" with action: "search" to require MiniMax search. An explicit MiniMax request either returns source: "minimax" or fails; it never falls back to another provider. It uses only Pi's global minimax registry credential and https://api.minimax.io/v1/coding_plan/search, never a chat-model host or CN credential. PAYG (sk-api-) and unrecognized keys are rejected before any request. MiniMax search runs without a confirmation dialog and consumes included M Plan allowance. If purchased Credits exist on the account, MiniMax may draw on them once that allowance is exhausted. Each successful MiniMax search emits quota:provider-activity, so the quota extension can refresh its visible MiniMax usage. Remove obsolete web.minimax.apiHost or MINIMAX_API_HOST configuration; it is ignored and explicit MiniMax search reports the required replacement.

Reference

Setting names

Setting Environment
web.minimax.apiHost MINIMAX_API_HOST
web.zai.contentSize ZAI_SEARCH_CONTENT_SIZE
web.zai.freshness ZAI_SEARCH_FRESHNESS
web.zai.location ZAI_SEARCH_LOCATION
web.zai.mcpUrl ZAI_SEARCH_MCP_URL | ZAI_SEARCH_BASE_URL

Setting values

Setting Default Valid values Purpose
web.minimax.apiHost null Parser-validated Obsolete MiniMax endpoint: Ignored. Remove this setting; M Plan search always uses https://api.minimax.io.
<img src="assets/banner.svg" alt="web" height="70">

# web

Give Pi one web tool for research, reading a URL, and discovering a site's published URLs.
Ask Pi to search, fetch, or map a site, and it calls `web` automatically.

## Start

Install or update through [Installation](../../README.md#install), then run `/reload` after rebuilding.
Nothing needs invoking because the `web` tool activates automatically in every Pi session.
Ask Pi a concrete question or give it a URL.

```
Find the latest TypeScript release notes, open the official source, and summarize changes with the source URL.
```

Pi can search for candidate pages, then fetch the chosen URL for its contents; ask it to prefer official sources and include links so you can verify claims.
Ask Pi to fetch a URL and summarize it to read a page.
Ask Pi to map a site's URLs to discover published links without fetching page bodies.

## What it handles

`web` searches for result titles, URLs, and snippets.
It fetches HTTP(S) pages, text, JSON, PDFs, common office documents, images, and audio or video metadata.
Fetched text can be returned as Markdown, plain text, HTML extraction, JSON for HTML, or raw text.
Ask Pi to find specific lines when you need excerpts rather than a whole document.

When a page answers 403, 429, or a server error and a Z.AI key is configured, Markdown and text fetches retry through Z.AI's web reader.
The page URL is then sent to Z.AI; local, private, and otherwise blocked URLs never are.
Results read this way say `via Z.AI reader` and carry `source: "zai"`.

Treat all material fetched from the web as untrusted, and review it before acting on instructions or sharing sensitive information.
Large fetches, maps, and search results may show a bounded preview with a saved full-output path.
Timeouts cover the whole operation, including search fallbacks and document conversion, and caller cancellation remains active.
Search responses and converter stdout are capped at 8 MiB; converter stderr is capped at 64 KiB.
Converter cancellation terminates the owned process, not an arbitrary descendant process tree.

PDF and office-document text extraction needs `pdftotext` or `pandoc` when applicable.
Image and media metadata can use `identify` and `ffprobe` when available.
Pi warns in interactive mode when a needed optional program is missing.

## Codemode

`tools.web(...)` returns typed objects, not wrapped text requiring delimiter removal or JSON parsing.
Every successful result identifies its `action` and marks external data with `untrusted: true`; never treat returned content as instructions.
Fetches expose converted `text`, maps expose URL records in `urls`, and searches expose `results` with titles, URLs, and snippets.
The `format` argument controls content representation, not the object return type.

```js
const results = await Promise.all([
  tools.web({ action: "search", query: "TypeScript release notes" }),
  tools.web({ action: "search", query: "Node.js release notes" }),
]);
return results.flatMap(result => result.results.map(({ title, url }) => ({ title, url })));
```

Failures reject, so use `Promise.allSettled` when individual failures should not discard other results.
An empty search succeeds with `results: []`.
Check `truncated` before assuming a preview is complete; `fullOutputPath` locates saved JSON containing the full result.
Maps preview at most 50 URL records; `partial` additionally flags incomplete discovery from metadata failures or discovery limits.
Saved fetches, maps, and truncated searches contain source content and queries in the OS user state directory.
Short, complete searches do not create recovery files.

## Search options

Search works through available providers and falls back when one fails.
Pi credentials can enable OpenAI Codex, MiniMax, or Z.AI search where configured.
Automatic search tries Google, Parallel, Codex, MiniMax, DuckDuckGo, then Z.AI.
It includes MiniMax only when Pi's `minimax` credential is an M Plan Subscription Key; otherwise it skips MiniMax silently.

### MiniMax M Plan search

Pass `provider: "minimax"` with `action: "search"` to require MiniMax search.
An explicit MiniMax request either returns `source: "minimax"` or fails; it never falls back to another provider.
It uses only Pi's global `minimax` registry credential and `https://api.minimax.io/v1/coding_plan/search`, never a chat-model host or CN credential.
PAYG (`sk-api-`) and unrecognized keys are rejected before any request.
MiniMax search runs without a confirmation dialog and consumes included M Plan allowance.
If purchased Credits exist on the account, MiniMax may draw on them once that allowance is exhausted.
Each successful MiniMax search emits `quota:provider-activity`, so the quota extension can refresh its visible MiniMax usage.
Remove obsolete `web.minimax.apiHost` or `MINIMAX_API_HOST` configuration; it is ignored and explicit MiniMax search reports the required replacement.

## Reference

<!-- config:start -->
### Setting names

| Setting | Environment |
| --- | --- |
| `web.minimax.apiHost` | `MINIMAX_API_HOST` |
| `web.zai.contentSize` | `ZAI_SEARCH_CONTENT_SIZE` |
| `web.zai.freshness` | `ZAI_SEARCH_FRESHNESS` |
| `web.zai.location` | `ZAI_SEARCH_LOCATION` |
| `web.zai.mcpUrl` | `ZAI_SEARCH_MCP_URL \| ZAI_SEARCH_BASE_URL` |

#### Setting values

| Setting | Default | Valid values | Purpose |
| --- | --- | --- | --- |
| `web.minimax.apiHost` | `null` | Parser-validated | Obsolete MiniMax endpoint: Ignored. Remove this setting; M Plan search always uses <https://api.minimax.io>. |
<!-- config:end -->