review: whole-repo round — keep-alive stream stall (TCP_NODELAY), wrong comments, dead items, json! copies

[CMGS]

Whole-repo review round over main@55abd059: every production file read in full (82 files, 29.5k lines), README and docs checked against the code, a LOC-justify pass, then a full Linux regression on a test host. The run turned up one real defect, a ~44 ms stall at the end of every streamed response on a keep-alive connection. Everything else is comment corrections, small cuts and ownership fixes.

Commits

• fix: set TCP_NODELAY on accepted connections: on a keep-alive connection every streamed response ended about 44 ms late. Nagle held the last small SSE write until the client's delayed ACK. SDK clients and reverse proxies pool connections, so every stream paid it. The listener now sets TCP_NODELAY on each accepted connection (axum::serve::ListenerExt::tap_io).
• fix: comments the code contradicts, and one fact stated once: TenantConf.fallback_model still said a request over quota with no fallback passes through. Since model quota: refuse an over-quota request with no usable fallback #150 it is refused. The in-memory ledger_usage note said usage is never capped, but billing_record clamps every token count. The config generation's cache-key role was stated at three places. The README now names both triggers for account failover.
• cut: a write-only response field, an unreachable break, a role nothing sends:
  • GatewayResponse.step: written on video replies and read nowhere.
  • pump_sse: a trailing if out.aborted { break } that every abort path had already left.
  • role::STORAGE: came with the initial import and no code creates such a turn. A client-sent one reaches the vendor, so the estimate counts it now.
• cut: one rollup-bucket fold for the in-process store.
• perf: move owned values into JSON envelopes instead of json! copies: four json! calls that deep-copied an owned Value.
• review: one line per comment: six two-line inline comments and four multi-line private docs.
• docs: docs/performance.md records the stall and the numbers below.

Net: production −33 lines, comments −20, tests −8.

Evidence

Gates

• macOS: fmt and clippy (-D warnings) clean. cargo test --workspace against Postgres 17 + Redis 7: 763 passed / 0 failed / 4 ignored.
• Every commit: clippy clean and tests green.
• Linux test host (Rust 1.99.0, Postgres 17, Redis): fmt and clippy clean, 763 / 0 / 4, with the gwtest_* databases and Redis keys created by the env-gated suites.

Stream stall (Linux, oha against the mock upstream, main vs this branch):

Shape	main	this branch
/v1/chat/completions stream, 1 connection	22 req/s, p50 48 ms	7,255 req/s, p50 0.12 ms
/v1/messages stream, 1 connection	22 req/s, p50 44 ms	5,941 req/s, p50 0.16 ms
chat stream, 64 connections	1,445 req/s	52,820 req/s
messages stream, 64 connections	1,444 req/s	54,697 req/s
non-stream chat, 64 connections	50,656 req/s	57,248 req/s

Hot path: instructions per request (perf stat, 200k requests per arm, arms interleaved with order swapped).

• Small and 52 KB chat in the plain, day-budget and Redis-budget configs: all within ±0.2% of main.
• Streamed /v1/messages throughput: from ~1.4k to 40k–90k req/s.

Live vendor matrix (real keys, through the branch build): 191/194.

• The 3 failures are the Bedrock cases in the grok group; no Bedrock key is currently valid.
• An earlier pass on this branch (before the TCP_NODELAY commit) had 2 transient xAI reasoning-omission failures; the xai group then reran 23/23 on both main and the branch.