Repository navigation
fix(express): rebuild image to drop eMudhra TLS trust #10021
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,7 +2,7 @@ | |
| # An elaborated scheme to build all the dependencies of all packages first in a cached layer | ||
| # https://stackoverflow.com/a/63142468/134409 | ||
| # https://medium.com/@emilefugulin/building-a-sane-docker-image-for-typescript-lerna-and-prisma-2-76d8ff9926e4 | ||
| FROM node:22.22.0-bookworm-slim@sha256:f86be15afa9a8277608e141ce2a8aa55d3d9c40845921b8511f4fb7897be2554 AS filter-packages-json | ||
| FROM node:24.15.0-bookworm-slim@sha256:4e6b70dd6cbfc88c8157ba19aa3d9f9cce6ba4703576d55459e45efcbc9c5f5d AS filter-packages-json | ||
|
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The digest is the upstream Docker Hub RepoDigest for |
||
| LABEL maintainer="Developer Relations <developer-relations-team@bitgo.com>" | ||
|
|
||
| COPY package.json yarn.lock lerna.json ./ | ||
|
|
@@ -12,7 +12,7 @@ COPY modules ./modules | |
| # delete all the non package.json files under `./modules/` | ||
| RUN find modules \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf | ||
|
|
||
| FROM node:22.22.0-bookworm-slim@sha256:f86be15afa9a8277608e141ce2a8aa55d3d9c40845921b8511f4fb7897be2554 AS builder | ||
| FROM node:24.15.0-bookworm-slim@sha256:4e6b70dd6cbfc88c8157ba19aa3d9f9cce6ba4703576d55459e45efcbc9c5f5d AS builder | ||
| ARG SOCKET_SECURITY_MODE=monitor | ||
| ENV SOCKET_SECURITY_MODE=${SOCKET_SECURITY_MODE} | ||
| RUN apt-get update && apt-get install -y git python3 make g++ libtool autoconf automake | ||
|
|
@@ -34,8 +34,20 @@ RUN \ | |
| rm -r modules/*/src | ||
|
|
||
|
|
||
| FROM node:22.22.0-bookworm-slim@sha256:f86be15afa9a8277608e141ce2a8aa55d3d9c40845921b8511f4fb7897be2554 | ||
| RUN apt-get update && apt-get install -y tini | ||
| FROM node:24.15.0-bookworm-slim@sha256:4e6b70dd6cbfc88c8157ba19aa3d9f9cce6ba4703576d55459e45efcbc9c5f5d | ||
| # The runtime validates TLS against the system CA bundle (NODE_OPTIONS at the | ||
| # bottom), so ca-certificates must be installed. Debian's package still trusts | ||
| # eMudhra roots after the October 2026 CA compromise, so they are stripped here | ||
| # until the distro removes them; scripts/verify-express-image-trust.sh asserts | ||
| # that none remain. | ||
| RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates tini && rm -rf /var/lib/apt/lists/* && \ | ||
| for f in $(find /usr/share/ca-certificates -name '*.crt'); do \ | ||
| if openssl x509 -in "$f" -noout -subject 2>/dev/null | grep -qi 'emudhra\|emsign'; then \ | ||
| echo "removing compromised trust anchor: $f" && rm "$f"; \ | ||
| fi; \ | ||
| done && \ | ||
| sed -i -e '/emSign/d' -e '/emudhra/Id' /etc/ca-certificates.conf && \ | ||
| update-ca-certificates --fresh | ||
|
Comment on lines
+43
to
+50
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Debian 12's |
||
| # copy the root node_modules to the bitgo-express parent node_modules | ||
| COPY --from=builder /tmp/bitgo/node_modules /var/node_modules/ | ||
| COPY --from=builder /tmp/bitgo/modules/express /var/bitgo-express/ | ||
|
|
@@ -374,5 +386,9 @@ LABEL git_hash=${GIT_HASH} | |
| USER node | ||
| ENV NODE_ENV=production | ||
| ENV BITGO_BIND=0.0.0.0 | ||
| # Validate outbound TLS against the system CA bundle instead of Node's embedded | ||
| # root store, so OS trust updates (e.g. revoked roots) take effect. | ||
| ENV NODE_OPTIONS=--use-openssl-ca | ||
| ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt | ||
|
Comment on lines
+389
to
+392
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| EXPOSE 3080 | ||
| ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/node", "/var/bitgo-express/bin/bitgo-express"] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Verify a BitGo Express image cannot validate TLS chains rooted in eMudhra | ||
| # (October 2026 CA compromise): the system CA bundle carries no eMudhra/emSign | ||
| # trust, and TLS verification follows that bundle rather than Node's embedded | ||
| # root store. | ||
| # | ||
| # Usage: scripts/verify-express-image-trust.sh <image> | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| IMAGE="${1:?usage: verify-express-image-trust.sh <image>}" | ||
|
|
||
| WORK_DIR=$(mktemp -d) | ||
| trap 'rm -rf "$WORK_DIR"' EXIT | ||
|
|
||
| echo "Verifying CA trust configuration of $IMAGE" | ||
|
|
||
| # The system CA bundle must exist and contain certificates. | ||
| if ! docker run --rm --entrypoint /bin/sh "$IMAGE" -c \ | ||
| 'test -s /etc/ssl/certs/ca-certificates.crt && grep -q "BEGIN CERTIFICATE" /etc/ssl/certs/ca-certificates.crt'; then | ||
| echo "::error::missing or empty /etc/ssl/certs/ca-certificates.crt in $IMAGE" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # The system bundle must carry no eMudhra/emSign trust. Node's embedded root | ||
| # store is reported but only matters when the runtime consults it, which the | ||
| # probes below assert it does not. | ||
| docker run --rm --entrypoint /usr/local/bin/node "$IMAGE" -e ' | ||
| const fs = require("fs"); | ||
| const tls = require("tls"); | ||
| const { X509Certificate } = require("crypto"); | ||
| const pem = fs.readFileSync("/etc/ssl/certs/ca-certificates.crt", "utf8"); | ||
| const blocks = pem.match(/-----BEGIN CERTIFICATE-----[\s\S]*?-----END CERTIFICATE-----/g) || []; | ||
| const compromised = (x509) => /emudhra|emsign/i.test(`${x509.subject}\n${x509.issuer}`); | ||
| const embedded = tls.rootCertificates.map((block) => new X509Certificate(block)); | ||
| console.log(`system bundle certs: ${blocks.length}, embedded roots: ${embedded.length}`); | ||
| const hits = blocks.map((block) => new X509Certificate(block)).filter(compromised); | ||
| if (hits.length > 0) { | ||
| console.error(`eMudhra/emSign trust found in the system bundle:\n${hits.map((x509) => x509.subject).join("\n")}`); | ||
| process.exit(1); | ||
| } | ||
| const embeddedHits = embedded.filter(compromised); | ||
| if (embeddedHits.length > 0) { | ||
| console.log(`note: Node embedded root store lists ${embeddedHits.length} eMudhra/emSign root(s); NODE_OPTIONS=--use-openssl-ca keeps them out of TLS verification (asserted by the probes below)`); | ||
| } | ||
| ' | ||
|
|
||
| # TLS verification must follow the configured system bundle. A server whose | ||
|
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The accept/reject pair is the real assertion: accept proves verification follows the configured bundle (dropping |
||
| # chain is anchored in a throwaway probe CA is accepted only when that CA is in | ||
| # the bundle Node validates against; Node's embedded roots alone reject it. | ||
| openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 -nodes \ | ||
| -keyout "$WORK_DIR/probe-ca.key" -out "$WORK_DIR/probe-ca.crt" -days 1 \ | ||
| -subj "/CN=Express Trust Probe CA" | ||
| openssl req -newkey ec -pkeyopt ec_paramgen_curve:P-256 -nodes \ | ||
| -keyout "$WORK_DIR/probe-leaf.key" -out "$WORK_DIR/probe-leaf.csr" \ | ||
| -subj "/CN=localhost" -addext "subjectAltName=DNS:localhost,IP:127.0.0.1" | ||
| openssl x509 -req -in "$WORK_DIR/probe-leaf.csr" \ | ||
| -CA "$WORK_DIR/probe-ca.crt" -CAkey "$WORK_DIR/probe-ca.key" -CAcreateserial \ | ||
| -out "$WORK_DIR/probe-leaf.crt" -days 1 -copy_extensions copy | ||
|
|
||
| PROBE_JS=' | ||
| const fs = require("fs"); | ||
| const tls = require("tls"); | ||
| if (process.env.PROBE_CA) { | ||
| fs.writeFileSync("/tmp/probe-bundle.pem", process.env.PROBE_CA); | ||
| } | ||
| const server = tls.createServer( | ||
| { key: process.env.PROBE_KEY, cert: process.env.PROBE_LEAF }, | ||
| (socket) => socket.end("ok") | ||
| ); | ||
| server.listen(0, "127.0.0.1", () => { | ||
| const socket = tls.connect( | ||
| { host: "127.0.0.1", port: server.address().port, servername: "localhost", rejectUnauthorized: true }, | ||
| () => { | ||
| console.log("ACCEPTED"); | ||
| socket.end(); | ||
| server.close(); | ||
| } | ||
| ); | ||
| socket.on("error", (err) => { | ||
| console.log(`REJECTED ${err.code || err.message}`); | ||
| server.close(); | ||
| process.exitCode = 2; | ||
| }); | ||
| }); | ||
| ' | ||
|
|
||
| LEAF_KEY=$(cat "$WORK_DIR/probe-leaf.key") | ||
| LEAF_CERT=$(cat "$WORK_DIR/probe-leaf.crt") | ||
| PROBE_CA=$(cat "$WORK_DIR/probe-ca.crt") | ||
|
|
||
| echo "Probe CA present in configured bundle -> chain must verify:" | ||
| printf '%s' "$PROBE_JS" | docker run -i --rm \ | ||
| -e PROBE_KEY="$LEAF_KEY" -e PROBE_LEAF="$LEAF_CERT" -e PROBE_CA="$PROBE_CA" \ | ||
| -e SSL_CERT_FILE=/tmp/probe-bundle.pem \ | ||
| --entrypoint /usr/local/bin/node "$IMAGE" - > "$WORK_DIR/accept.log" 2>&1 || true | ||
| cat "$WORK_DIR/accept.log" | ||
| if ! grep -q '^ACCEPTED' "$WORK_DIR/accept.log"; then | ||
| echo "::error::TLS verification does not follow the system CA store in $IMAGE" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Probe CA absent from configured bundle -> chain must be rejected:" | ||
| printf '%s' "$PROBE_JS" | docker run -i --rm \ | ||
| -e PROBE_KEY="$LEAF_KEY" -e PROBE_LEAF="$LEAF_CERT" \ | ||
| --entrypoint /usr/local/bin/node "$IMAGE" - > "$WORK_DIR/reject.log" 2>&1 || true | ||
| cat "$WORK_DIR/reject.log" | ||
| if ! grep -q '^REJECTED' "$WORK_DIR/reject.log"; then | ||
| echo "::error::untrusted probe CA was accepted via the default bundle in $IMAGE" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "✅ CA trust verification passed for $IMAGE" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
docker-build is the only gate that builds the real image from the Dockerfile, so the trust check lives here — it re-verifies every future base-image bump too, which is exactly how stale embedded trust data in a new base would get caught.