RTG-5132 Bump BoringSSL

[bwesterb]

Most of our patches required work:

1. pq: (a) Upstream removed Kyber. We keep X25519Kyber768 and P256Kyber768 by implementing Kyber768 on top of ML-KEM-768.

   (b) Improve documentation of what's currently supported.

2. allow-crl-extensions-bad-version: Properly document what the patch actually does. Also improve its tests.

3. relax-cert-validation: Upstream removed enforce_rsa_key_usage; inline the behaviour instead.

I tested the Kyber key exchanges of the new BoringSSL against the old both as client and server. I didn't add a Go test for it to keep the patch small (not wanting to bring in a Kyber implementation.)

[Lekensteyn]

LGTM. I was worried about the impact of the RSA Key Usage not relaxing TLS 1.3, but that was already the status quo before.

What wasn't immediately obvious to me, and explains why the reported kind of breakage is not more widespread: X.509 certificates with RSA keys pass the key usage check either (1) when the requested key usage bit is set, or (2) when the Key Usage extension is missing.

The last case should cover the majority of RSA keys, like the one generated with openssl req -new -x509 -keyout server.pem -out server.pem -sha256 -nodes -subj /

[Lekensteyn]

While trying to apply the boring-pq patch on boringssl, I noticed some more minor issues. See comments.

[Lekensteyn]

Unit tests (part of ninja run_tests) fail unless this is applied:

--- a/include/openssl/prefix_symbols.h
+++ b/include/openssl/prefix_symbols.h
@@ -2102,2 +2102,3 @@
 #pragma redefine_extname SSL_CTX_use_psk_identity_hint BORINGSSL_ADD_USER_LABEL_AND_PREFIX(SSL_CTX_use_psk_identity_hint)
+#pragma redefine_extname SSL_CTX_use_second_keyshare BORINGSSL_ADD_USER_LABEL_AND_PREFIX(SSL_CTX_use_second_keyshare)
 #pragma redefine_extname SSL_ECH_KEYS_add BORINGSSL_ADD_USER_LABEL_AND_PREFIX(SSL_ECH_KEYS_add)
@@ -2451,2 +2452,3 @@
 #pragma redefine_extname SSL_use_psk_identity_hint BORINGSSL_ADD_USER_LABEL_AND_PREFIX(SSL_use_psk_identity_hint)
+#pragma redefine_extname SSL_use_second_keyshare BORINGSSL_ADD_USER_LABEL_AND_PREFIX(SSL_use_second_keyshare)
 #pragma redefine_extname SSL_used_hello_retry_request BORINGSSL_ADD_USER_LABEL_AND_PREFIX(SSL_used_hello_retry_request)
@@ -5239,2 +5241,3 @@
 #define SSL_CTX_use_psk_identity_hint BORINGSSL_ADD_PREFIX(SSL_CTX_use_psk_identity_hint)
+#define SSL_CTX_use_second_keyshare BORINGSSL_ADD_PREFIX(SSL_CTX_use_second_keyshare)
 #define SSL_ECH_KEYS_add BORINGSSL_ADD_PREFIX(SSL_ECH_KEYS_add)
@@ -5588,2 +5591,3 @@
 #define SSL_use_psk_identity_hint BORINGSSL_ADD_PREFIX(SSL_use_psk_identity_hint)
+#define SSL_use_second_keyshare BORINGSSL_ADD_PREFIX(SSL_use_second_keyshare)
 #define SSL_used_hello_retry_request BORINGSSL_ADD_PREFIX(SSL_used_hello_retry_request)

This possibly changed in commit google/boringssl@d32c35e (committed 2026-01-27, part of 0.20260204.0).

[Lekensteyn]

To prevent this From line from changing after patch changes, I used --zero-commit:

git format-patch --zero-commit --no-numbered --no-signature

[Lekensteyn]

Let's prepend SSL_GROUP_X25519_MLKEM768 here to ensure it is enabled when using SSL_set_compliance_policy(ssl, ssl_compliance_policy_fips_202205).

Alternatively, we stop patching this and require our users to move away from this API.

[bwesterb]

We have users of the API

[Lekensteyn]

This note is obsolete and needs to be removed, X25519MLKEM768 is already enabled by default since 0.20260713.0 (commit date 2026-07-09) via google/boringssl@e748fac

[Lekensteyn]

This diff below seems incorrect and should be removed:

--- a/ssl/test/runner/common.go
+++ b/ssl/test/runner/common.go
@@ -2186,7 +2186,7 @@ type ProtocolBugs struct {
        FailIfHelloRetryRequested bool

        // FailIfPostQuantumOffered will cause a server to reject a ClientHello if
-       // post-quantum curves are supported.
+       // post-quantum curves are not supported.
        FailIfPostQuantumOffered bool

        // ExpectKeyShares, if not nil, lists (in order) the curves that a ClientHello

The previous comment was correct. If this flag is set, the peer will reject a Client Hello if PQ curves are included (supported).

[cjpatton]

This comment is likely to become stale. Perhaps add an AGENTS.md with a note about this.

[Lekensteyn]

We have no AGENTS.md, but thanks for the reminder. I updated it. We also need to bump the crate major version according the the comment.

[Lekensteyn]

So, one of the tests is failing:

thread 'x509::tests::trusted_first::test_verify_cert' (9925) panicked at boring/src/x509/tests/trusted_first.rs:50:5:
assertion `left == right` failed
  left: Err(X509VerifyError { code: 10, error: "certificate has expired" })
 right: Ok(())

Presumably caused by google/boringssl@defe581 ("Always enable X509_V_FLAG_TRUSTED_FIRST", part of 0.20260903.0).

As a result, this test always fails -- permitting alternative trust paths that validate:

    assert_eq!(
        Err(X509VerifyError::CERT_HAS_EXPIRED),
        verify(
            &leaf,
            &[&root1, &root2],
            &[&intermediate, &root1_cross],
            |param| param.clear_flags(X509VerifyFlags::TRUSTED_FIRST), // <-- no-op
         )
    );

@nox this test was introduced in commit dfd49f4. Is anything relying on the ability to disable the effects of this flag?

[nox]

@nox this test was introduced in commit dfd49f4. Is anything relying on the ability to disable the effects of this flag?

I added this to make the older FIPS build match the newer BoringSSL default. Clearing the flag was just a regression-test control; our application code enables it. Making the behavior unconditional is consistent with the original fix.