Repository navigation
fix(agent-core): only send prompt_cache_key to official OpenAI endpoints - #2240
creatiVision wants to merge 17 commits into
Conversation
🦋 Changeset detectedLatest commit: 4c248f5 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
c313025 to
6c9fbe6
Compare
9cba0aa to
d18b4de
Compare
8f1a867 to
2024ea0
Compare
|
I'd love to see this merged soon. I can't use Kimi + Kimi Code on Azure as it is. |
|
Rebased and synced with latest The PR diff is now clean and focused strictly on gating |
475f680 to
30e00d4
Compare
|
Rebased and synced with latest
All merge conflicts are resolved and the PR is clean, mergeable, and ready for review. |
|
Superseded by #3660 — the same v2 gating fix, reconciled onto a single clean branch based on the latest |
30e00d4 to
0c0ae57
Compare
1 similar comment
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0c0ae57e1e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "@moonshot-ai/kimi-code": patch | ||
| --- | ||
|
|
||
| Only send the OpenAI prompt cache key to the official OpenAI API; custom OpenAI-compatible endpoints no longer receive the unknown parameter and stop rejecting requests with HTTP 400. No newline at end of file |
There was a problem hiding this comment.
The clause after the semicolon explains the mechanism and benefit through the unknown parameter and HTTP 400 response, rather than keeping the release note to one short user-facing statement of what changed. Trim the entry to the behavior change alone before it is emitted in the CLI changelog.
AGENTS.md reference: AGENTS.md:L86-L88
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 0620582. The changeset entry is now trimmed to the behavior change alone: "Only send the OpenAI prompt cache key to the official OpenAI API." The clause about the unknown parameter and HTTP 400 rejection was removed from the release-note text.
| }); | ||
|
|
||
| describe('openai-responses requester responseFormat', () => { | ||
| it('omits prompt_cache_key for third-party endpoints', async () => { |
There was a problem hiding this comment.
This case tests prompt_cache_key behavior rather than response formatting, and an existing cache-key.test.ts already contains the corresponding requester coverage. Keeping the Responses variant here scatters tests for the same component across unrelated suites instead of extending the existing test file.
AGENTS.md reference: AGENTS.md:L61-L62
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 0620582. The Responses cache-key test was moved out of response-format.test.ts into the existing cache-key.test.ts (new "openai-responses requester cacheKey" describe block with a responses client stub). The duplicated test was removed from response-format.test.ts, which now only covers response formatting again. cache-key.test.ts: 6 tests, response-format.test.ts: 7 tests, both green.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a1d7ac0e2a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -0,0 +1,6 @@ | |||
| --- | |||
| "@moonshot-ai/agent-core-v2": patch | |||
There was a problem hiding this comment.
Because this user-visible fix reaches users through the CLI, the required gen-changesets package-selection rule says to list only @moonshot-ai/kimi-code; @moonshot-ai/agent-core-v2 is a private internal package and should not receive its own release entry here. Remove this line and retain the CLI patch entry.
AGENTS.md reference: AGENTS.md:L87-L87
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 0620582. Removed the "@moonshot-ai/agent-core-v2": patch line from the changeset frontmatter; only "@moonshot-ai/kimi-code": patch remains, since the fix reaches users through the CLI. Verified with pnpm changeset status.
…t-cache # Conflicts: # packages/agent-core-v2/src/human/llm/requester/bases/openai-responses/requester.ts # packages/agent-core-v2/src/human/llm/requester/bases/openai/requester.ts
…hangeset and tests
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0620582c69
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| kwargs = | ||
| trait?.encodeCacheKey?.(input.cacheKey, ctx) ?? encodeOpenAIResponsesCacheKey(input.cacheKey); | ||
| trait?.encodeCacheKey?.(input.cacheKey, ctx) ?? | ||
| encodeOpenAIResponsesCacheKey(input.cacheKey, ctx.model.baseUrl); |
There was a problem hiding this comment.
When a Kimi model uses the openai_responses protocol, its provider registration supplies kimiConnection but no Responses trait; the connection resolves the base URL to https://api.moonshot.ai/v1, so this fallback now returns {} and silently removes the session prompt_cache_key that these requests previously sent. Kimi's chat-completions route avoids the regression through kimiOpenAITrait.encodeCacheKey, so the Responses route needs an equivalent vendor-specific hook before applying this host gate.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 04511c6. Registered kimiOpenAIResponsesTrait with encodeCacheKey: (key) => ({ prompt_cache_key: key }) for Kimi's openai_responses protocol in both human/llm-kimi and llm-adapter, preserving the session cache key on Kimi Responses requests. Added unit tests in cache-key.test.ts verifying that traits supply the cache key on Responses requests.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7152a7f5da
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } | ||
| try { | ||
| const hostname = new URL(baseUrl).hostname; | ||
| return hostname === 'api.openai.com' || hostname.endsWith('.api.openai.com'); |
There was a problem hiding this comment.
When the llm-adapter custom-provider path routes an OpenAI/Responses model through a forwarding gateway that accepts or passes through prompt_cache_key, this hostname allowlist makes both default encoders return {} although the same configuration previously sent the session key. The only bypass is a registered internal encodeCacheKey trait (the Kimi special case); the model/provider configuration has no way to select this for an arbitrary gateway, so users silently lose cache affinity or hits. Keep the omission default for strict endpoints, but add a config or env opt-in for compatible gateways.
AGENTS.md reference: AGENTS.md:L102-L104
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 6c990ea. Added KIMI_CODE_OPENAI_PROMPT_CACHE_KEY environment opt-in (1 / true) to isOfficialOpenAIBaseUrl() so custom forwarding gateways can opt into sending prompt_cache_key on both the OpenAI and OpenAI-Responses protocols. Added unit tests in cache-key.test.ts verifying that setting the environment variable sends prompt_cache_key to third-party endpoints for both protocols. Updated the changeset to name the flipped default and state the escape hatch.
|
Rebased and synced with latest upstream Resolved merge conflicts in
The PR is conflict-free, mergeable, and ready for review. |
…r custom gateways
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6c990ea9f2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -0,0 +1,15 @@ | |||
| export function isOfficialOpenAIBaseUrl(baseUrl: string | undefined): boolean { | |||
| const optIn = process.env['KIMI_CODE_OPENAI_PROMPT_CACHE_KEY']; | |||
There was a problem hiding this comment.
When a compatible custom OpenAI gateway needs to retain session cache affinity, this new process variable is now the only supported escape hatch, but a repo-wide rg finds it only in this helper, the tests, and the changeset—not in either docs/en/configuration/env-vars.md or its Chinese mirror. After the release note is no longer in front of users, the configuration reference therefore says nothing about how to restore prompt_cache_key or which values are accepted; add the variable to both environment-variable pages.
AGENTS.md reference: AGENTS.md:L94-L96
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in commit 877d7caf7. Added KIMI_CODE_OPENAI_PROMPT_CACHE_KEY documentation to both docs/en/configuration/env-vars.md and docs/zh/configuration/env-vars.md.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 877d7caf72
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -0,0 +1,15 @@ | |||
| export function isOfficialOpenAIBaseUrl(baseUrl: string | undefined): boolean { | |||
| const optIn = process.env['KIMI_CODE_OPENAI_PROMPT_CACHE_KEY']; | |||
| if (optIn === '1' || optIn === 'true') { | |||
There was a problem hiding this comment.
When a user sets KIMI_CODE_OPENAI_PROMPT_CACHE_KEY=TRUE (explicitly allowed by docs/en/configuration/env-vars.md:63 and its Chinese mirror), this exact comparison remains false, so a compatible third-party gateway still silently loses prompt_cache_key; normalize the value before checking it or narrow the documented contract and tests.
AGENTS.md reference: AGENTS.md:L118-L120
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in commit 4229b95d4. isOfficialOpenAIBaseUrl now normalizes KIMI_CODE_OPENAI_PROMPT_CACHE_KEY case-insensitively and trims whitespace (accepting 1, true, yes, on).
| @@ -0,0 +1,15 @@ | |||
| export function isOfficialOpenAIBaseUrl(baseUrl: string | undefined): boolean { | |||
| const optIn = process.env['KIMI_CODE_OPENAI_PROMPT_CACHE_KEY']; | |||
There was a problem hiding this comment.
Because both cache-key formatters invoke this helper, formatting a request now reads mutable process.env and classifies provider endpoints. The required llm module guide defines format stages as pure and assigns endpoint/env resolution to the provider connection; move this gate into requester/connection composition and keep the format encoders deterministic.
AGENTS.md reference: packages/agent-core-v2/src/human/llm/AGENTS.md:L1-L1
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in commit 4229b95d4. Moved the isOfficialOpenAIBaseUrl endpoint gate out of encodeOpenAICacheKey / encodeOpenAIResponsesCacheKey into requester composition (prepareOpenAIRequest / prepareOpenAIResponsesRequest), keeping protocol format encoders pure and deterministic.
| ### `KIMI_CODE_OPENAI_PROMPT_CACHE_KEY` | ||
|
|
||
| ::: info Added | ||
| Added in 0.4.3. |
There was a problem hiding this comment.
This page documents Kimi Code CLI, but apps/kimi-code/package.json:3 is version 2.1.1 and this changeset targets that public package; 0.4.3 is instead the already-current version of the private package at packages/agent-core-v2/package.json:3-4. The callout therefore tells users this new variable existed in an unrelated internal release, so remove it until the public release number is known or use the CLI release version, and update the Chinese mirror as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in commit 4229b95d4. Removed the internal package version callout block from both docs/en/configuration/env-vars.md and docs/zh/configuration/env-vars.md.
…ase-insensitively, drop doc callout
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4229b95d43
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| ### `KIMI_CODE_OPENAI_PROMPT_CACHE_KEY` | ||
|
|
||
| Controls whether `prompt_cache_key` is attached to requests sent to non-official OpenAI endpoints. By default, prompt cache keys are only sent to official OpenAI endpoints (`api.openai.com`) to prevent incompatibilities with third-party proxies and gateways that reject unknown fields. |
There was a problem hiding this comment.
For type = "kimi" providers (including managed Kimi), kimiResponsesTrait and kimiOpenAITrait always return prompt_cache_key (packages/agent-core-v2/src/human/llm-kimi/trait.ts:78-90), and both requesters use that trait result before the hostname gate. The key therefore remains enabled with this environment variable unset for Kimi's non-OpenAI hosts and for any Kimi base_url override, so saying it is sent only to api.openai.com makes this variable look like a global policy and can mislead users diagnosing a strict Kimi-compatible gateway; qualify the statement as applying only when no provider trait encodes the key.
AGENTS.md reference: AGENTS.md:L98-L100
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in commit 4c248f5ee. Updated both docs/en/configuration/env-vars.md and docs/zh/configuration/env-vars.md to qualify that KIMI_CODE_OPENAI_PROMPT_CACHE_KEY applies when no provider trait explicitly encodes the cache key (for openai and openai_responses providers), noting that dedicated kimi providers always attach prompt_cache_key via their provider trait regardless of endpoint.
This branch has not been deployed
2 participants
Related Issue
Resolves #2166 (NVIDIA NIM:
400 Validation: Unsupported parameter(s): prompt_cache_key). Related: #2611 (Azure Foundry:400 Unrecognized request argument supplied: prompt_cache_key).Problem
Since v0.29.0 (#1970), every OpenAI-compatible provider receives the session
prompt_cache_keyin the request body. Strictly-validating endpoints (e.g. NVIDIA NIM, Azure Foundry) reject the unknown parameter with HTTP 400, making customopenai/openai_responsesproviders unusable for agent sessions.What changed
The field is now only sent when the effective base URL targets the official OpenAI API (
api.openai.comor a regional*.api.openai.comvariant, or unset → client default):isOfficialOpenAIBaseUrl()helper inpackages/agent-core-v2/src/human/llm/requester/bases/openai-base-url.ts.prompt_cache_keyfallback behind it in both the OpenAI chat-completions and Responses bases (openai/format.ts,openai-responses/format.ts). A vendor that supports the field on other hosts keeps encoding it via thecacheKeytrait hook (e.g. Kimi), which is unaffected.cache-key.test.ts(official endpoint keeps the key; third-party endpoint omits it) andresponse-format.test.ts(Responses base omits it for third-party endpoints).Checklist
/approve).gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.