repositories / pi-ext

pi-ext

bugabingas pi extensions

owned by bugabinga

Iteration 19: Improved pandoc-md-to-pdf.lua with real PDF engine auto-detection. If PANDOC_PDF_ENGINE is unset, it smoke-tests xelatex/lualatex/pdflatex/typst/wkhtmltopdf/weasyprint and selects the first working engine; if none work, prints actionable diagnostics instead of blindly failing on xelatex. Preserves env override behavior. Updated SKILL wording. Verified no-engine diagnostic, forced-engine path, and 9/9 filters pass.

6e53ac99f449bcdc3afe142116bf42e3cd50b24f

Oliver Jan Krylow <oliver@bugabinga.net> ·

unsigned

Browse filesCompare with HEAD

Result: {"status":"keep","skill_quality":38}

Changed files (4)

.pi/autoresearch/session.jsonl

Modified
6 6 {"run":15,"commit":"44eb971","metric":34,"metrics":{},"status":"keep","description":"Iteration 15: Improved autolink.lua URL boundary handling. Previous filter included trailing prose punctuation in hrefs (e.g. https://example.com. linked with the period). Added trailing punctuation splitting for . , ; : ! ? ) ]. Verified period/comma/paren/question punctuation stay outside links, multiple URLs still work, 9/9 filters pass.","timestamp":1780158167127,"segment":0,"confidence":null,"asi":{"hypothesis":"Fix subtle but common URL autolinking bug; generated hrefs must not include sentence punctuation","bug_fixed":"trailing punctuation included in link target","validation":"period/comma/paren/question stripped; multiple URLs OK; 9/9 filters pass","changed_file":"references/filters/autolink.lua"}}
7 7 {"run":16,"commit":"bf8645a","metric":35,"metrics":{},"status":"keep","description":"Iteration 16: Fixed extract-headings.lua to emit a real nested BulletList AST instead of escaped literal Markdown paragraphs. The filter now preserves heading hierarchy, creates actual internal links, handles skipped heading levels, and returns empty output for documents without headings. Verified Markdown list output, HTML nested ul/li output, link hrefs, and 9/9 filters pass.","timestamp":1780158253729,"segment":0,"confidence":null,"asi":{"hypothesis":"Fix existing filter correctness: escaped Markdown output is misleading; agents need real outline links/lists","bug_fixed":"extract-headings produced literal escaped '- [Heading](#id)' paragraphs instead of list/link AST","validation":"nested markdown list, HTML ul/li, internal hrefs, skipped levels, no-heading input, 9/9 filters","changed_file":"references/filters/extract-headings.lua"}}
8 8 {"run":17,"commit":"1d7d3c1","metric":36,"metrics":{},"status":"keep","description":"Iteration 17: Fixed center-images.lua LaTeX corruption bug. The prior Image/Plain approach double-wrapped images inside Figure, producing malformed minipage/center LaTeX. Rewrote filter to return explicit filter tables, use topdown traversal, skip Figure children for LaTeX/Beamer, center only image-only paragraphs, and leave inline images unchanged. Verified clean LaTeX figures, centered no-alt image paragraphs, unchanged inline images, HTML styling, and 9/9 filters pass.","timestamp":1780158416994,"segment":0,"confidence":null,"asi":{"hypothesis":"Continue fixing existing filters for real correctness; center-images had subtle LaTeX corruption for captioned figures","bug_fixed":"LaTeX Figure children were being centered, corrupting figure output","validation":"captioned figure clean, image-only paragraph centered, inline image unchanged, HTML style present, 9/9 filters","changed_file":"references/filters/center-images.lua"}}
9 +{"run":18,"commit":"5694044","metric":37,"metrics":{},"status":"keep","description":"Iteration 18: Improved pandoc-doctor.lua accuracy for PDF diagnostics. Doctor no longer misleadingly reports pdf as normal reader/writer; it now labels PDF as writer via engine and runs real smoke tests for available PDF engines. This catches cases where engine binaries exist but PDF generation fails (verified: typst/wkhtmltopdf version OK but smoke FAIL in current env). Updated SKILL.md helper script note. 9/9 filters pass.","timestamp":1780158545607,"segment":0,"confidence":null,"asi":{"hypothesis":"Doctor version checks alone were misleading; real PDF smoke tests better diagnose agent PDF failures","bug_fixed":"pdf shown as -- format despite extension output support; engine binary OK did not mean pandoc PDF works","validation":"doctor prints PDF smoke tests, catches real failures, markdown HTML smoke OK, 9/9 filters","changed_file":"scripts/pandoc-doctor.lua"}}

skills/pandoc/SKILL.md

Modified
966 966 --highlight-style=kate
968 -# Or use the helper script
968 +# Or use the helper script (auto-detects a working PDF engine)
969 969 pandoc lua scripts/pandoc-md-to-pdf.lua report.md
1106 1106 # Batch convert all .md files in current directory to .html
1107 1107 pandoc lua scripts/pandoc-batch-convert.lua md html -s --toc
1109 -# Markdown to polished PDF
1109 +# Markdown to polished PDF (auto-detects a working PDF engine)
1110 1110 pandoc lua scripts/pandoc-md-to-pdf.lua report.md
1111 1111 pandoc lua scripts/pandoc-md-to-pdf.lua report.md draft.pdf --draft

skills/pandoc/ideas.md

Modified
11 11 - Extract headings filter emits real nested BulletList links, not escaped Markdown text
12 12 - Center images filter skips LaTeX Figure children to avoid corrupting generated figure output
13 13 - pdf-report.yaml and html-report.yaml defaults
14 -- Cross-platform Lua scripts (batch-convert, md-to-pdf, docx-to-md, web-to-md, wordcount, doctor with real PDF smoke tests)
14 +- Cross-platform Lua scripts (batch-convert, md-to-pdf with PDF engine auto-detection, docx-to-md, web-to-md, wordcount, doctor with real PDF smoke tests)
15 15 - pandoc.system.list_directory for portable dir listing
16 16 - Multi-file documents & Jupyter notebook section
17 17 - Decision Guide with 18 agent task entries

skills/pandoc/scripts/pandoc-md-to-pdf.lua

Modified
1 1 -- pandoc-md-to-pdf.lua — Markdown to polished PDF with sensible defaults
2 2 -- Usage: pandoc lua scripts/pandoc-md-to-pdf.lua <input.md> [output.pdf] [--draft|--final]
3 3 -- Cross-platform: works on Linux, macOS, Windows (requires pandoc 3.x)
4 --- Requires: xelatex (or set PANDOC_PDF_ENGINE)
4 +--
5 +-- Engine selection:
6 +-- - Set PANDOC_PDF_ENGINE to force an engine.
7 +-- - Otherwise, the script smoke-tests common engines and picks the first that works.
9 +local system = require 'pandoc.system'
10 +local path = require 'pandoc.path'
6 12 local args = arg
7 13 if #args < 1 then
30 36 if output == input then output = input .. ".pdf" end
33 --- PDF engine: env var override or default to xelatex
34 -local engine = os.getenv("PANDOC_PDF_ENGINE") or "xelatex"
39 +local candidates = {
40 + {name = "xelatex", opts = {}},
41 + {name = "lualatex", opts = {}},
42 + {name = "pdflatex", opts = {}},
43 + {name = "typst", opts = {}},
44 + -- wkhtmltopdf often needs local-file access to read pandoc's temp HTML file.
45 + {name = "wkhtmltopdf", opts = {"--pdf-engine-opt=--enable-local-file-access"}},
46 + {name = "weasyprint", opts = {}},
49 +local function one_line(x)
50 + return tostring(x):match("([^\r\n]+)") or tostring(x)
51 +end
53 +local function smoke_engine(engine, opts)
54 + local ok, result = false, "not run"
55 + system.with_temporary_directory('pandoc-md-to-pdf', function(tmpdir)
56 + local out = path.join({tmpdir, 'smoke.pdf'})
57 + local pandoc_args = {
58 + "-f", "markdown",
59 + "--pdf-engine=" .. engine,
60 + "-o", out,
61 + }
62 + for _, opt in ipairs(opts or {}) do
63 + pandoc_args[#pandoc_args + 1] = opt
64 + end
65 + ok, result = pcall(pandoc.pipe, "pandoc", pandoc_args,
66 + "---\ntitle: Smoke\n---\n\n# Smoke\n\nHello.\n")
67 + end)
68 + return ok, one_line(result)
69 +end
71 +local function choose_engine()
72 + local forced = os.getenv("PANDOC_PDF_ENGINE")
73 + if forced and forced ~= "" then
74 + return forced, {}, "forced by PANDOC_PDF_ENGINE"
75 + end
77 + local failures = {}
78 + for _, candidate in ipairs(candidates) do
79 + local ok, msg = smoke_engine(candidate.name, candidate.opts)
80 + if ok then
81 + return candidate.name, candidate.opts, "auto-detected"
82 + end
83 + failures[#failures + 1] = string.format(" %-12s %s", candidate.name, msg)
84 + end
86 + io.stderr:write("No working PDF engine found. Tried:\n")
87 + io.stderr:write(table.concat(failures, "\n") .. "\n")
88 + io.stderr:write("Install TeX/Typst/WeasyPrint/wkhtmltopdf, or generate HTML/docx instead.\n")
89 + os.exit(47)
90 +end
92 +local engine, engine_opts, engine_reason = choose_engine()
36 94 local pandoc_args = {
37 95 input,
47 105 "--highlight-style=kate",
108 +for _, opt in ipairs(engine_opts) do
109 + pandoc_args[#pandoc_args + 1] = opt
110 +end
50 112 if mode == "draft" then
51 113 pandoc_args[#pandoc_args + 1] = "-V"
52 114 pandoc_args[#pandoc_args + 1] = "fontsize=12pt"
57 119 pandoc_args[#pandoc_args + 1] = "fontsize=11pt"
60 -io.stdout:write(string.format("Converting: %s -> %s (engine=%s, mode=%s)\n",
61 - input, output, engine, mode))
122 +io.stdout:write(string.format("Converting: %s -> %s (engine=%s, mode=%s, %s)\n",
123 + input, output, engine, mode, engine_reason))
63 125 local ok, result = pcall(pandoc.pipe, "pandoc", pandoc_args, "")
64 126 if ok then