[ralph-bitgo]

Description

Rebuilds the BitGo Express image's TLS trust configuration so outbound
TLS can no longer validate against the eMudhra roots compromised in the
October 2026 CA incident.

Why: bitgo/express:latest (17.6.0) validates TLS only against
Node's embedded root store and ships no system CA bundle, so
TLS-sensitive workloads behind Express accept eMudhra-rooted chains
(the WCN-3730 reproduction returned ACCEPTED HTTP=200 for a chain
rooted at emSign ECC Root CA - G3). Until this is released, the
current image should not be used for TLS-sensitive workloads.

What changed (4 files):

• Dockerfile (runtime stage): installs ca-certificates so a usable
  system CA bundle exists, and strips the emSign/eMudhra anchors
  Debian 12 (ca-certificates 20250419~deb12u2) still ships —
  installing the package alone would swap one eMudhra trust path for
  another. Sets NODE_OPTIONS=--use-openssl-ca and
  SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt so outbound TLS
  validates against that scrubbed bundle instead of Node's embedded
  roots. The Node base pins are untouched: master already carries
  node:22.23.4 (merged fix(express): bump Dockerfile Node base image to 22.23.4 #10022), the ticket's corrected-trust-data
  floor.
• scripts/verify-express-image-trust.sh (new): asserts the bundle
  exists and carries no eMudhra/emSign trust, and that TLS
  verification follows the configured store — a throwaway probe CA is
  ACCEPTED when bundled and REJECTED UNABLE_TO_VERIFY_LEAF_SIGNATURE
  when not. Both directions are asserted, so removing the scrub or
  dropping --use-openssl-ca breaks CI.
• .github/workflows/ci.yml: runs the script in docker-build, the
  only gate that builds the real image — it re-verifies every future
  base bump too.
• modules/express/README.md: documents the TLS trust model and how
  to add an extra CA (NODE_EXTRA_CA_CERTS extends the trusted set;
  pointing SSL_CERT_FILE at another bundle replaces it).

Supersedes #10021 (closed): the same remediation minus its Node
24.15.0 base bump, which conflicts with the base-pin work in #10022
(merged) and #10023 (open, edits only the three FROM pins).

Issue Number

WCN-3730 — https://linear.app/bitgo/issue/WCN-3730/rebuild-bitgo-express-image-to-remove-compromised-emudhra-trust

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Reduced-image smoke in this sandbox (the full docker build OOM-kills
in the pre-existing lerna run build step, unchanged by this diff; CI
docker-build is the full-build gate and ran green for these hunks on
#10021):

• Built the runtime-stage recipe (exact Dockerfile lines) on
  node:22.23.4-bookworm-slim and ran
  scripts/verify-express-image-trust.sh against it — passed:
  system bundle certs: 145, 0 eMudhra/emSign in the bundle; probe CA
  ACCEPTED when in the configured bundle,
  REJECTED UNABLE_TO_VERIFY_LEAF_SIGNATURE when not.
• Ticket reproduction routes dv-g3b-emudhra.we-have-a-problem.com
  and dv-g3b-aaa-emudhra.we-have-a-problem.com are both REJECTED
  in the rebuilt image. Note: both routes also reject on the stock
  node:22.23.4 base now (CERT_REVOKED), so they no longer reproduce
  the ticket's ACCEPTED without this change either — the
  discriminating evidence for the trust policy is the probe pair above.
• Node 22.23.4's embedded store still lists one eMudhra anchor
  (emSign Root CA - G1); --use-openssl-ca keeps embedded trust out
  of verification entirely (asserted by the probes).
• bash -n on the new script. No TS/JS source changed; the
  dockerfile-check gate's managed COPY/LINK blocks are untouched.

Outstanding after merge (cannot be done from a PR): run the release
workflow to publish the rebuilt image under a new immutable digest and
re-point latest — ticket remediation item 4.

Pre-PR ACI gate skipped: no .github/ai-checks/pr-risk on master.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• My code compiles correctly for both Node and Browser environments (n/a — no TS/JS source changed)
• I have commented my code, particularly in hard-to-understand areas
• My commits follow Conventional Commits and I have properly described any BREAKING CHANGES
• The ticket or github issue was included in the commit message as a reference
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective (scripts/verify-express-image-trust.sh, run in the CI docker-build job)
• New and existing unit tests pass locally with my changes (no unit-test-covered source changed; CI matrix pending on this branch)

Ticket: WCN-3730

[linear-code]

WCN-3730

[ralph-bitgo]

Debian 12's ca-certificates (20250419~deb12u2) still ships eMudhra/emSign roots, so installing the package alone would swap one eMudhra trust path for another. This scrub is temporary and idempotent: once the distro drops the roots the loop and sed match nothing, and the verify script asserts the end state either way.

[ralph-bitgo]

NODE_EXTRA_CA_CERTS can only add anchors to the trusted set — it cannot remove compromised roots. So verification moves off Node's embedded store entirely: in Node's NewRootCertStore() --use-openssl-ca selects X509_STORE_set_default_paths as a mutually exclusive alternative to the bundled roots, and SSL_CERT_FILE pins those paths to the scrubbed bundle. That also means future OS trust updates take effect without a Node bump.

[ralph-bitgo]

The accept/reject pair is the real assertion: accept proves verification follows the configured bundle (dropping --use-openssl-ca breaks it, since the bundled roots then reject the probe CA), and reject proves the default bundle grants no trust by accident (removing the scrub or replacing the config with NODE_EXTRA_CA_CERTS alone breaks it). The embedded-store line is informational because that store is never consulted under the configured flags.

[ralph-bitgo]

docker-build is the only gate that builds the real image from the Dockerfile, so the trust check lives here — it re-verifies every future base-image bump too, which is exactly how stale embedded trust data in a new base would get caught.

[ralph-bitgo]

Wording matters here and I verified it empirically against the built image: NODE_EXTRA_CA_CERTS extends the configured store (it does not replace it), while pointing SSL_CERT_FILE at another bundle replaces the default one.