Skip to content

fix(chat): tolerate missing fields in prompt_tokens_details - #483

Open
pcbeingused333 wants to merge 3 commits into
MacPaw:mainfrom
pcbeingused333:c0570-optional-token-details
Open

pcbeingused333 wants to merge 3 commits into
MacPaw:mainfrom
pcbeingused333:c0570-optional-token-details

Conversation

@pcbeingused333

Copy link
Copy Markdown

What

ChatResult.CompletionUsage.PromptTokensDetails now decodes with decodeIfPresent, defaulting a missing audio_tokens or cached_tokens to 0. A regression test decodes a response whose prompt_tokens_details contains only cached_tokens.

Why

audioTokens and cachedTokens were decoded as required Ints. OpenAI-compatible providers (DeepSeek, Gemini's compat endpoint) often send only cached_tokens. Decoding prompt_tokens_details then threw, and the try? that guards usage in ChatResult and ChatStreamResult swallowed the error, so the entire usage (including prompt_tokens, completion_tokens, total_tokens) came back nil instead of just lacking the missing detail.

Affected Areas

ChatResult.CompletionUsage.PromptTokensDetails decoding, which is used by both ChatResult and ChatStreamResult. The property types stay Int, so there is no public API change. (Making them Int?, like CompletionTokensDetails, would be an alternative but source-breaking.)

Tests: full swift test in a swift:6.2 container (223 tests) passes; the new test fails without the change (XCTUnwrap gets a nil usage). No linked issue.


This change was prepared with the help of an AI assistant (Claude Code), and the author is responsible for the change.

🤖 Generated with Claude Code

OpenAI-compatible providers (e.g. DeepSeek, Gemini) send only cached_tokens
or only a subset of the fields. Decoding then failed and the whole usage was
silently dropped by the try? in ChatResult/ChatStreamResult. Default the
missing counters to 0.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@nezhyborets nezhyborets left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, please see the comment

Comment on lines +272 to +277
public init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
// OpenAI-compatible providers (e.g. DeepSeek, Gemini) often send only one of these fields.
self.audioTokens = try container.decodeIfPresent(Int.self, forKey: .audioTokens) ?? 0
self.cachedTokens = try container.decodeIfPresent(Int.self, forKey: .cachedTokens) ?? 0
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks! Please see other uses of decodeIfPresent, the usage of parsingOptions. Looks something like this

public init(from decoder: any Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    let parsingOptions = decoder.userInfo[.parsingOptions] as? ParsingOptions ?? []
    self.id = try container.decodeString(forKey: .id, parsingOptions: parsingOptions)
    ...
}

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@pcbeingused333

Copy link
Copy Markdown
Author

Done, now via decodeInt(forKey:parsingOptions:) like the rest of ChatResult; the test uses .relaxed.

This comment was drafted with Claude Code.

@nezhyborets nezhyborets left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just one more change before merging. Let me also see if Copilot (balanced) wants something else

Edit: Copilot didn't provide anything meaningful so let's just add the public init

case cachedTokens = "cached_tokens"
}

public init(from decoder: any Decoder) throws {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declaring init(from:) in the struct body stops Swift from synthesizing the memberwise init(audioTokens:cachedTokens:), so PromptTokensDetails can now only be created by decoding, even from @testable tests. Could you either move init(from:) into an extension, or add a public init(audioTokens:cachedTokens:) like CompletionTokensDetails has? A public one would also make the promptTokensDetails: parameter of CompletionUsage.init usable for anyone mocking usage.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Missing detail fields still cause default-configured clients to discard the entire usage object, and the regression test’s relaxed parsing setting does not cover this default behavior.

1 open issue
What changed in this PR

This PR aims to preserve chat usage data when OpenAI-compatible providers omit prompt-token detail fields, without changing the public Int property types.

Changes:

  • Adds custom decoding for prompt-token details using parsing options, shared by chat and streaming responses.
  • Adds a regression test for a response containing only cached_tokens, with relaxed parsing enabled.
File Description
Tests/​OpenAITests/​ChatResultTests.swift Tests usage decoding when audio_tokens is omitted.
Sources/​OpenAI/​Public/​Models/​ChatResult.swift Adds parsing-option-aware decoding for prompt-token details.

Review effort: Balanced

@nezhyborets

Copy link
Copy Markdown
Collaborator

Also just ran the checks to see if they find anything that needs fixing

Declaring init(from:) in the struct body suppressed the synthesized
memberwise initializer; restore it as a public init, matching
CompletionTokensDetails.
@pcbeingused333

Copy link
Copy Markdown
Author

Added public init(audioTokens:cachedTokens:) to PromptTokensDetails (same shape as CompletionTokensDetails), plus a small test that builds CompletionUsage with it. ChatResultTests pass.

This comment was drafted with Claude Code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants