Skip to content

Disable gateway proxy buffering for model services - #4326

Merged
r4victor merged 1 commit into
masterfrom
issue_4269_gateway_buffering
Sep 30, 2026
Merged

r4victor merged 1 commit into
masterfrom
issue_4269_gateway_buffering

Conversation

@r4victor

@r4victor r4victor commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #4269

The fix applies only to new services. Existing services are not affected.

@r4victor
r4victor merged commit deac309 into master Sep 30, 2026
26 checks passed
@r4victor
r4victor deleted the issue_4269_gateway_buffering branch September 30, 2026 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gateway buffers streamed LLM responses, inflating TTFT to 16 KB worth of tokens

1 participant