Preserve Next.js and Google Tag Manager rewrites in streamed pages

CHANGELOG.md

@@ -28,12 +28,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Security
 
+- Publisher and page-bids processing now reject malformed effective request hosts with 400 before inserting request-derived bytes into rewrite output. `/first-party/sign` also returns 400 for a malformed effective host when the request URI has no scheme and request metadata is needed to determine it. Host selection remains `Forwarded` (`host=`), then `X-Forwarded-Host`, then `Host`; a malformed or explicitly empty selected host is rejected without fallback. Valid local hostnames, underscore labels, ports, and bracketed IPv6 remain supported. Forwarded-header trust policy is unchanged (#1206, #1207, #1208).
 - `/first-party/sign` now rejects valid targets outside `proxy.allowed_domains` before minting a proxy token. The creative runtime keeps image and iframe assignments blocked after this `403` policy response instead of loading the rejected URL directly; fetch-time checks still cover the initial target and every redirect.
 - Reserved the complete admin namespace at the publisher-fallback boundary. Percent-encoded separators (`/_ts/admin%2Fec`, `%2f`, and double-encoded forms) matched the `^/_ts/admin` Basic-auth handler but escaped the literal-slash namespace check, so an authenticated request fell through to publisher fallback and forwarded its `Authorization` header and body to the publisher origin. The reservation now spans the whole `/_ts/admin` prefix plus the retired `/admin/keys` aliases — including trailing, descendant, and encoded-separator forms — evaluated on the raw path and on each of its bounded percent-decodings, so multi-encoded separators such as `/admin%252Fkeys/rotate` cannot survive to fallback for a proxy or origin to decode again, and applies to every adapter.
 - Validate synthetic ID format on inbound values from the `x-synthetic-id` header and `synthetic_id` cookie; values that do not match the expected format (`64-hex-hmac.6-alphanumeric-suffix`) are discarded and a fresh ID is generated rather than forwarded to response headers, cookies, or third-party APIs
 
 ### Fixed
 
+- Next.js and Google Tag Manager script rewrites now compose in streamed HTML instead of overwriting each other, preserving Pages Router JSON and App Router Flight payloads with recomputed decoded `T`-record lengths. GTM/GA Flight rewrites require both integrations to be enabled. Buffering is bounded and per document; an uncaptured data head (including inert text quoting a Flight push), an unsupported or binary push, or a size limit keeps the remaining Flight stream raw and restores queued originals, but cannot undo already-streamed rewrites. Complete serialized form state does not disable later rewriting, foreign property references no longer hide a later qualified Flight push, and placeholder-stage document fallback emits a reason-bearing warning without source data (#1206, #1207, #1208).
 - Publisher responses now map origin URLs in `Location`, `Content-Location`, `Refresh`, and `Link` headers to the serving host, and each CSP source naming the origin gains a serving-host source beside it (#1128). A redirect to the current URL under a different scheme, or a `GET`/`HEAD` redirect to the current URL that sets no cookie (or any such redirect under a `Host` override), is left unchanged to avoid a loop. `<meta>` refresh and CSP tags in the page body are not rewritten. See "Origin URLs in proxied response headers" in the configuration guide.
 - Image Optimizer configuration now rejects profile and profile-set names within the same map that collide after trimming, even when their values match, instead of silently overwriting an entry. Empty names continue to be discarded.
 - Browser integration bundles now share context providers and the configured log level with TSJS core (#1196). With Permutive enabled and cohorts available, `tsjs.requestAds()` now sends `config.permutive_segments` to `/auction`; the server retains that key only when `auction.allowed_context_keys` includes `"permutive_segments"`. This restores a previously inactive data flow, not a consent gate: review publisher consent handling before relying on it. Integration logs now follow `tsjs.setConfig` and `tsjs.log.setLevel`, and creative no longer raises the default `warn` level to `info`. The `?tsdebug=1` query parameter or `localStorage['tsdebug'] = '1'` intentionally enables `debug` logging across all bundles when creative installs, overriding any log level set earlier through `tsjs.setConfig` or `tsjs.log.setLevel`. Later calls to either API can change the level again.

crates/trusted-server-adapter-axum/tests/routes.rs

@@ -7,8 +7,10 @@
 use axum::body::Body as AxumBody;
 use axum::http::Request;
 use edgezero_adapter_axum::service::EdgeZeroAxumService;
+use std::sync::Arc;
 use tower::{Service as _, ServiceExt as _};
 use trusted_server_adapter_axum::app::TrustedServerApp;
+use trusted_server_core::test_support::nextjs_auction;
 
 const LEGACY_ADMIN_DENY_METHODS: &[&str] =
     &["GET", "POST", "HEAD", "OPTIONS", "PUT", "PATCH", "DELETE"];
@@ -878,10 +880,6 @@ async fn first_party_proxy_rebuild_is_routed() {
 /// Fastly adapter alone and not by this test.
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn nextjs_auction_output_holds_until_the_structural_body_close() {
-    use std::sync::Arc;
-
-    use trusted_server_core::test_support::nextjs_auction;
-
     let client = Arc::new(nextjs_auction::NextJsAuctionOrigin::default());
     let router = TrustedServerApp::routes_with_settings_and_services(
         nextjs_auction::settings(),
@@ -939,6 +937,161 @@ async fn nextjs_auction_output_holds_until_the_structural_body_close() {
     );
 }
 
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn nextjs_gtm_rejects_malformed_request_hosts() {
+    for header in ["host", "x-forwarded-host", "forwarded"] {
+        let origin = Arc::new(nextjs_auction::NextJsAuctionOrigin::with_html_response(
+            &nextjs_auction::script_composition_html(),
+            false,
+            true,
+        ));
+        let router = TrustedServerApp::routes_with_settings_and_services(
+            nextjs_auction::script_composition_settings(),
+            nextjs_auction::services(origin),
+        )
+        .expect("should build script composition router");
+        let mut service = EdgeZeroAxumService::new(router);
+        let hostile_host = "example.com</script><script>alert(1)</script>";
+        let value = if header == "forwarded" {
+            format!("host=\"{hostile_host}\"")
+        } else {
+            hostile_host.to_owned()
+        };
+        let mut request = Request::builder()
+            .uri("/script-composition")
+            .header("host", nextjs_auction::PUBLISHER_HOST)
+            .body(AxumBody::empty())
+            .expect("should build hostile host request");
+        request.headers_mut().insert(
+            header,
+            value.parse().expect("should encode hostile host header"),
+        );
+        let response = service
+            .ready()
+            .await
+            .expect("should be ready")
+            .call(request)
+            .await
+            .expect("should respond to hostile host request");
+        assert_eq!(
+            response.status().as_u16(),
+            400,
+            "should reject malformed {header} rather than rewrite script source"
+        );
+        let body = axum::body::to_bytes(response.into_body(), 1024)
+            .await
+            .expect("should consume host rejection response");
+        assert!(
+            !String::from_utf8_lossy(&body).contains(hostile_host),
+            "should not reflect the rejected host in the response"
+        );
+    }
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn nextjs_gtm_accepts_local_request_hosts_and_ports() {
+    for host in [
+        "localhost:3000",
+        "example.com:8443",
+        "[::1]:3000",
+        "my_service:8080",
+        "_service_.example.com:8443",
+    ] {
+        let origin = Arc::new(nextjs_auction::NextJsAuctionOrigin::with_html_response(
+            &nextjs_auction::script_composition_html(),
+            false,
+            true,
+        ));
+        let router = TrustedServerApp::routes_with_settings_and_services(
+            nextjs_auction::script_composition_settings(),
+            nextjs_auction::services(origin),
+        )
+        .expect("should build script composition router");
+        let mut service = EdgeZeroAxumService::new(router);
+        let request = Request::builder()
+            .uri("/script-composition")
+            .header("host", host)
+            .body(AxumBody::empty())
+            .expect("should build local host request");
+        let response = service
+            .ready()
+            .await
+            .expect("should be ready")
+            .call(request)
+            .await
+            .expect("should serve local host request");
+        assert_eq!(
+            response.status().as_u16(),
+            200,
+            "should accept host: {host}"
+        );
+        let body = axum::body::to_bytes(response.into_body(), usize::MAX)
+            .await
+            .expect("should consume local host response");
+        let html = String::from_utf8(body.to_vec()).expect("should emit UTF-8 HTML");
+        assert!(
+            html.contains(&format!("http://{host}/app")),
+            "should preserve host and port in rewritten Flight URL: {host}"
+        );
+        assert!(
+            !html.contains("__ts_rsc_"),
+            "should resolve every local host payload"
+        );
+        assert!(
+            html.ends_with("</body></html>"),
+            "should complete local response"
+        );
+    }
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn nextjs_gtm_html_response_is_complete() {
+    for gzip in [false, true] {
+        let origin = Arc::new(nextjs_auction::NextJsAuctionOrigin::with_html_response(
+            &nextjs_auction::script_composition_html(),
+            gzip,
+            false,
+        ));
+        let router = TrustedServerApp::routes_with_settings_and_services(
+            nextjs_auction::script_composition_settings(),
+            nextjs_auction::services(origin),
+        )
+        .expect("should build mixed integration router");
+        let mut service = EdgeZeroAxumService::new(router);
+        let request = Request::builder()
+            .method("GET")
+            .uri("/article")
+            .header("host", nextjs_auction::PUBLISHER_HOST)
+            .header("accept", "text/html")
+            .body(AxumBody::empty())
+            .expect("should build navigation");
+        let response = service
+            .ready()
+            .await
+            .expect("should be ready")
+            .call(request)
+            .await
+            .expect("should serve HTML");
+        assert_eq!(
+            response.status().as_u16(),
+            200,
+            "should return a successful composed publisher response"
+        );
+        assert_eq!(
+            response
+                .headers()
+                .get("content-encoding")
+                .map(|value| value.to_str().expect("should have ASCII coding")),
+            gzip.then_some("gzip"),
+            "should retain the requested response compression"
+        );
+        let body = axum::body::to_bytes(response.into_body(), usize::MAX)
+            .await
+            .expect("should consume entire adapter body");
+        nextjs_auction::assert_script_composition_response(&body, gzip);
+    }
+}
+
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn health_reports_git_version() {
     let mut service = make_service();

crates/trusted-server-adapter-fastly/src/app.rs

@@ -1390,6 +1390,7 @@ mod tests {
     use std::net::{IpAddr, Ipv4Addr};
 
     use error_stack::Report;
+    use futures::StreamExt as _;
     use futures::executor::block_on;
     use trusted_server_core::constants::HEADER_X_GEO_INFO_AVAILABLE;
     use trusted_server_core::ec::device::DeviceSignals;
@@ -1406,6 +1407,7 @@ mod tests {
         TemplateCacheMiss, TemplateCacheReservation, TemplateEntry, TemplateMetadata,
     };
     use trusted_server_core::settings::Settings;
+    use trusted_server_core::test_support::nextjs_auction;
 
     #[test]
     fn hooks_store_metadata_matches_edgezero_manifest() {
@@ -3147,6 +3149,58 @@ mod tests {
         );
     }
 
+    #[test]
+    fn dispatch_fallback_nextjs_gtm_response_is_complete() {
+        for gzip in [false, true] {
+            let state = build_state_from_settings(nextjs_auction::script_composition_settings())
+                .expect("should build mixed state");
+            let origin = Arc::new(nextjs_auction::NextJsAuctionOrigin::with_html_response(
+                &nextjs_auction::script_composition_html(),
+                gzip,
+                true,
+            ));
+            let services = nextjs_auction::services(origin);
+            let request = request_builder()
+                .method(Method::GET)
+                .uri(format!(
+                    "https://{}/article",
+                    nextjs_auction::PUBLISHER_HOST
+                ))
+                .header("host", nextjs_auction::PUBLISHER_HOST)
+                .header("accept", "text/html")
+                .body(Body::empty())
+                .expect("should build navigation");
+            let response = block_on(super::dispatch_fallback(&state, &services, request));
+            assert_eq!(
+                response.status(),
+                StatusCode::OK,
+                "should return a successful composed publisher response"
+            );
+            assert!(
+                !response.headers().contains_key(header::CONTENT_LENGTH),
+                "should drop the stale content length after streaming rewrites"
+            );
+            assert_eq!(
+                response
+                    .headers()
+                    .get(header::CONTENT_ENCODING)
+                    .map(|value| value.to_str().expect("should have ASCII coding")),
+                gzip.then_some("gzip"),
+                "should retain the requested response compression"
+            );
+            let Body::Stream(mut stream) = response.into_body() else {
+                panic!("should retain lazy Fastly body");
+            };
+            let mut bytes = Vec::new();
+            block_on(async {
+                while let Some(chunk) = stream.next().await {
+                    bytes.extend_from_slice(&chunk.expect("should process every stream item"));
+                }
+            });
+            nextjs_auction::assert_script_composition_response(&bytes, gzip);
+        }
+    }
+
     #[test]
     fn dispatch_fallback_streams_publisher_body_without_buffering() {
         // Regression guard for the publisher streaming cutover (#849): a