ci: pin TLS-verified patch hosts on macOS compat legs (fix hosted DNS flake)

[Mikola Lysenko (mikolalysenko)]

Problem

The macOS native legs of Bun patch compatibility (and, more rarely, vlt and Poetry) fail intermittently, and a re-run of the same code passes.

Root cause (from the logs)

GitHub's hosted macOS runner intermittently cannot resolve patch.socket.dev, sometimes for minutes, at job start or partway through a job. Meanwhile the service is up.

• The package managers fail, not the CLI. In each failing cell, the CLI's own calls to patches-api.socket.dev succeed: it finds the patch and rewrites the lock. What fails is the package manager downloading the tarball from patch.socket.dev:
  • bun: error: FailedToOpenSocket downloading tarball minimist@https://patch.socket.dev/...
  • python (harness): <urlopen error [Errno 8] nodename nor servname provided, or not known>
  • vlt harness probe: curl: (6) Could not resolve host: patch.socket.dev
  • Poetry: Failed to resolve 'patch.socket.dev' ([Errno 8] ...)
  • Bun 1.3.x workspace installs don't print the error; they never exit and hit the 180 s timeout.
• It comes in time windows. Sometimes every hosted cell in a job fails within about 30 s. Other times the job loses DNS partway through and gets it back. For example, 1.3.10 in 36415924573 passed its first 6 hosted cells, lost DNS from about 11:37 to 11:43, then passed the rest. 1.3.0 in 36420943072 lost DNS from 12:24 to about 12:31. The vlt macOS failure (36420943301, 12:22 to 12:24) falls in the same window as four Bun macOS failures in 36420943072.
• Vendored cells pass. They never contact patch.socket.dev. The only vendored failures are the two conversion shapes (hosted-then-vendored, vendored-then-hosted), which run hosted mode.
• The harness's own retry doesn't help. Bun's classifier only matches CLI transport errors, so these cells were never retried. vlt's probe-based retry fires, but 3 attempts over about 35 s can't outlast a multi-minute outage.

Frequency (Bun native, last 60 runs / 69 attempts, all attempts counted)

Four runs failed on every leg of every OS (code regressions in those PRs); they are excluded here.

OS	failed jobs	cause
macos-latest	29 (in 22 of 69 run attempts)	patch.socket.dev DNS failure, as above
ubuntu-latest	3	different issue: patches-api.socket.dev returned Connection reset by peer / 504 to the CLI's own requests (see below)
windows-latest	0	n/a

vlt native over the last 62 runs: 1 macOS failure (same DNS cause), 1 Windows failure (not this issue), 0 on ubuntu. The vlt workflow's red status on recent PRs comes from install-proof on older vlt eras. That fails on every run, on all three OSes. It's deterministic and unrelated to this PR.

Fix

A new composite action, .github/actions/pin-socket-hosts, runs on macOS only, before the backtest, in the Bun, vlt and Poetry native jobs. It calls scripts/pin-socket-hosts.py, which:

1. resolves patch.socket.dev and patches-api.socket.dev, first with the system resolver, then with DNS-over-HTTPS to IP-literal endpoints (1.1.1.1, 8.8.8.8), which need no DNS. It keeps retrying with backoff for up to 5 minutes.
2. keeps only addresses where a TLS handshake with that hostname verifies the certificate. An unreachable IPv6 address or a wrong IP is never pinned.
3. pins them in /etc/hosts and flushes the resolver cache. If a host can't be pinned, the job fails at this step and names the host, instead of failing a hundred cells later.

Every cell still talks to the production service over TLS verified for the hostname. Only the runner's DNS resolver, which isn't what we're testing, is out of the path.

Why not the other options

• Local mock server: these workflows are deliberately the "production-service twin" of the hermetic wiremock real-bun and real-vlt suites. Those already run on every PR in ci.yml's e2e matrix, including macOS. backtest-vlt.py says it runs "with no service doubles". Both harnesses' rows are imported by depscan as production captures, and the existing loopback IdentityMirror is documented as "never an import source". Moving these legs to a mock would duplicate ci.yml and lose the only production signal.
• Retrying in the CLI: the CLI's transport isn't what fails on macOS. bun, vlt, curl and python do the failing lookups.
• A wait-for-DNS step before the job: it can't cover outages that start partway through a job, which the 1.3.x jobs above show.
• Longer or broader cell retries: the outage windows last minutes. That would mean blind retries costing minutes per cell, which is what this PR is trying to avoid.

Separate issue, not fixed here: ubuntu (and Poetry ubuntu-latest, 1.1.15 on #293/#294)

The ubuntu failures are not DNS. The CLI's own requests to patches-api.socket.dev got client error (Connect): Connection reset by peer (os error 104) or API request failed with status 504, for example Bun jobs 108648998493 and 108733768066. On Poetry 1.1.15 the reset hit vex's /patch/view call. ApiClient::send_json_request deliberately never retries transport errors ("Transport errors are never retried"), and only retries 429/503. Retrying idempotent GETs on connect or reset errors, and treating 502/504 like 503, would be a real product improvement. It changes a documented policy, though, so I've left it for a separate PR.

Validation

• New scripts/tests/test_pin_socket_hosts.py (6 hermetic tests): system resolver path, fallback to DoH, unverified addresses rejected, IPv6-only as a last resort, retry until the resolver recovers, and fail-closed with no output.
• python3 -m unittest discover -s scripts/tests: 111 tests pass.
• Ran locally on macOS: pins the real addresses. With the system resolver forced to fail with EAI_NONAME, DoH takes over and the unreachable IPv6 addresses are rejected. A wrong IP fails with Hostname mismatch.
• actionlint: no new findings. The existing SC2086/SC2206 notes are in steps this PR doesn't touch. shellcheck is clean on the action script.
• cargo clippy --locked --workspace --all-targets -- -D warnings: clean (no Rust changes).
• CI re-runs: see comments below.

🤖 Generated with Claude Code

---

Note

Low Risk
CI-only change affecting macOS job /etc/hosts via sudo; no application or auth logic changes, with TLS verification before pinning.

Overview
Adds a macOS-only CI workaround for GitHub hosted runners intermittently failing to resolve patch.socket.dev / patches-api.socket.dev (EAI_NONAME) while production is healthy, which was flaking Bun, vlt, and Poetry production backtest jobs.

A new composite action .github/actions/pin-socket-hosts runs scripts/pin-socket-hosts.py before native backtests in bun-compatibility, poetry-compatibility, and vlt-compatibility. The script resolves each host via the system resolver, then DNS-over-HTTPS to IP literals (1.1.1.1 / 8.8.8.8), keeps only TLS-verified addresses, appends them to /etc/hosts, and flushes the macOS resolver cache. Jobs still hit the real production endpoints with normal hostname TLS; only runner DNS is bypassed.

Path filters on those workflows now include the action and script. docs/testing/bun-compatibility.md and docs/testing/vlt-compatibility.md document the step. scripts/tests/test_pin_socket_hosts.py adds hermetic tests for resolver fallback, verification gating, IPv6 last-resort, retries, and fail-closed behavior.

^{Reviewed by Cursor Bugbot for commit 6f46cb2. Configure here.}

[Mikola Lysenko (mikolalysenko)]

#279 landed on release/v5-prerelease as f6bdad5; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#282 landed on release/v5-prerelease as 14a9cb0; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#291 landed on release/v5-prerelease as f9cb7e1; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#292 landed on release/v5-prerelease as a7b0d00; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#296 landed on release/v5-prerelease as 1e3ace6; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#293 landed on release/v5-prerelease as b9e106d; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#294 landed on release/v5-prerelease as 180f10f; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code