Skip to content

Build tooling work, commits C65d-C75: pdf-lib shim gate, impexp parity, one command-line convention - #213

Merged
KubaO merged 21 commits into
twinbasic:mainfrom
KubaO:staging
Sep 30, 2026
Merged

KubaO merged 21 commits into
twinbasic:mainfrom
KubaO:staging

Conversation

@KubaO

@KubaO KubaO commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Puts a test around the book's pdf-lib shims and the two impexp editions, then gives every Node tool in the repository the same command-line behaviour: the same --help, the same refusal of a bad command line, and the same exit codes.

  • book: the twelve pdf-lib shims are now checked against stock pdf-lib. The gate is new, and it found three defects on the way, which are fixed first.
    • fast-dict-onebuf.mjs replaced six of pdf-lib's eight factories for PDFDict, PDFCatalog, PDFPageTree and PDFPageLeaf. The other two, PDFPageTree.withContext and PDFPageLeaf.withContextAndParent, still built on a Map, so insertPage, addPage and PDFDocument.create failed under the shim. Both are replaced now. Stock and shimmed pdf-lib give the same bytes for create, addPage, insertPage, drawText and save. The book never adds a page, so it was not affected.
    • Fixes.md said there were thirteen shims; there are twelve. It names them without a count now.
    • check_pdf_shims_equiv (new, in test.bat, the shared CI action and Tools.md) loads, changes and saves one generated document with stock pdf-lib and with the shims, in child processes, and compares the two files object by object with streams inflated. A second pair builds a document with PDFDocument.create. It also lists the 72 members of pdf-lib the shims patch, and fails when a listed member is not patched, a patched one is not listed, an unmarked one never runs, or a marked one runs. Only the two context setters stay marked as unreached. Today it reports the same 25 objects for the loaded document and the same 11 for the created one. It runs in about 0.5 s.
    • The two computeBufferSize overrides in fast-sync-load.mjs are deleted. No storage change forced them, the measurements found no reliable gain, and ParallelStreamWriter overrides the method on the book's only path.
    • book/lib/pdf-lib-internals.mjs holds the 33 pdf-lib internals the shims import (44 before the unused ones went), replacing a createRequire block repeated in nine shims. book/lib/onebuf-range.mjs holds the range machinery the two onebuf shims had copied. book/lib/shim-targets.mjs gives each shim a load-time check of the members it overwrites (arity and a source fingerprint, 78 targets), so a changed pdf-lib throws naming the shim instead of being patched silently.
    • Three renders of book.html before and after the onebuf change give 2,299 pages and 2,466 outline entries each, in the same time within noise. The files differ only in /CreationDate and /ModDate, and in a byte of deflate size in some renders.
  • test.bat: check_impexp_parity (new) runs both --self-test suites (19 tests each) and then 21 commands through impexp.mjs and impexp.py on the repository's fixtures, comparing exit code, output and written files byte for byte. Without Python it reports itself skipped and passes; with CI=true it fails. About 4 s. test.bat is now fifteen steps, and CI runs the gates the wrappers run.
  • cli: every Node tool (45) answers --help and -h by printing usage to stdout and exiting 0, with no side effect. Before, gen_attribute_probes took --help as its output folder and created --help/Sources, render-book rejected it, and twelve tools ignored it.
  • cli: a refused command line exits 2 in every tool. An unknown flag, a value flag with no value, an empty value, and a positional the tool does not take are all errors, where eleven tools ignored an unknown flag. check_links's tolerance for stray flags and its unused --threads are gone. A term that starts with a dash goes after --. Only tbdocs's --baseurl may be empty. Two hazards close with it: build_corpus --dest "" removed the current folder, and convert_em_dash_separators --chek rewrote docs/.
  • cli: values are checked straight after the parse, before an IDE, browser, request or file removal. lib/cli.mjs gains choiceOption, regexOption, urlOption, dateOption and refuseTogether, and every number goes through numberOption. This closes cases such as crawl_check --concurrency abc and addin_test --jobs abc, which looped for ever, addin_test's and run_case's timeout of 0 or text, which killed every child at once, and tbdocs --url foo, which crashed partway through the build. Conflicting options are refused, for example --show with --hide, and two of check_examples's modes. build_corpus refuses a --dest that is or contains the repository root, the current folder or --src.
  • cli: tbdocs and check_links exit 0 clean, 1 when the build or check found a problem of any kind, and 2 on a refused command line or a crash, like every other tool. They gave a bitmask (1, 2, 3 and 4) that nothing read: the wrappers test errorlevel 1, and CI and compare_trees test non-zero. tbdocs's stall watchdog and a failed --serve start now exit 2 as well.
  • cli: --json now always means "print to stdout" and --src always means a documentation root. check_a11y_fingerprint's --json FILE is --out FILE. census_attributes and build_package_api take --exported <dir> for the package tree. build_corpus and nav_hops take --repo, since their --src named a repository root that holds docs/. The three perf/ rigs follow. The old spellings are refused as unknown options.
  • cli: each tool's usage text ends with one Exit codes: block, and its Tools.md section (or eval/README.md, or Wisdom.md) ends with the same codes. The convention is 0 clean, 1 a finding, 2 the tool could not do its job, and a tool's own codes above 2. Three codes that meant two things are split: addin_test exits 3 when the registry or a work folder was not put back, wisdom exits 3 when it reaches its request cap, and tbrun exits 4 when the compiler crashed, as tbbuild does. Several tools that exited 1 when they could not run now exit 2, and a crash exits 2 in every tool; exitOnCrash moved into lib/cli.mjs for that.
  • scripts: addin_test puts the registry and the add-ins' settings back when it crashes, not only at the end of a normal run. The crash path ends the lanes, waits up to 10 s for them, and runs the same put-back. It exits 2, or 3 if the put-back found a problem. Before, a crash left the user's registry changed.
  • serve.bat: returns tbdocs's exit code, as the other wrappers do. With its port held, it now exits 2 where it exited 0.
  • docs: Tools.md, Building.md, Builder.md, Extending.md, PDF-Generation.md, Pipeline-Stages.md, Wisdom.md, Fixes.md, Fixes-PDFLib.md, eval/README.md and the WIP notes are updated to match. Tools.md now names --exported for the two package tools.
  • plan: the Landed entries for Phase 2 and Phase 3 in builder/PLAN-TOOLING-REVIEW.md are cut to what later work needs. Their full text stays in history.

check_cli grows from 258 to 860 probes: a --help case and an empty-folder probe for every tool, an unknown-flag case for all 45 tools, empty-value and bad-value cases, and a probe that each tool's --help ends with exactly one exit-code table. Faults put into lib/cli.mjs failed 69 of 568 probes (unknown-option refusal) and 88 of 810 (numberOption). test.bat also gained the two new gates named above.

@KubaO
KubaO merged commit b6ee575 into twinbasic:main Sep 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant