Conversation
FallbackModel's default fallback_on=(ModelAPIError,) does not cover UnexpectedModelBehavior, which pydantic-ai raises for a 200 response whose body doesn't match the expected schema (seen against OpenRouter: a ChatCompletion with choices/model/object all None — likely a transient provider hiccup, not something tied to the specific input). Because UnexpectedModelBehavior is a sibling of ModelAPIError under AgentRunError (not a subclass), it silently bypasses the configured fallback-model entirely and kills the module outright, even though a fallback-model was explicitly configured. Observed once in 376 generated pages on a real run (voll-intelligence) — low frequency, but the failure mode is a genuine gap: the fallback exists specifically to catch this kind of thing, and doesn't. Fix: pass fallback_on=(ModelAPIError, UnexpectedModelBehavior) explicitly.
…tic-ai 1.0.6) requirements.txt pins pydantic-ai==1.0.6, and that version's pydantic_ai.exceptions has no ModelAPIError (it was added in a later release as a broader base class over ModelHTTPError) — only ModelHTTPError and UnexpectedModelBehavior exist. Importing ModelAPIError broke collection for every test module that imports llm_services, transitively or directly. FallbackModel's own default in 1.0.6 is fallback_on=(ModelHTTPError,), so using ModelHTTPError here instead of ModelAPIError keeps the exact same fix intent (widen the default to also cover UnexpectedModelBehavior) while working with the pinned dependency version. CI: ImportError: cannot import name 'ModelAPIError' from 'pydantic_ai.exceptions' (9 collection errors).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
FallbackModel's defaultfallback_on=(ModelAPIError,)does not coverUnexpectedModelBehavior, which pydantic-ai raises for a 200 response whose body doesn't match the expected schema. Seen against OpenRouter on a real run (voll-intelligence):Likely a transient provider hiccup, not something tied to the specific input.
Because
UnexpectedModelBehavioris a sibling ofModelAPIErrorunderAgentRunError(not a subclass), it silently bypasses the configuredfallback-modelentirely and kills the module outright — even though a fallback-model was explicitly configured for exactly this kind of situation.Scope
Observed once in 376 generated pages on the run in question — low frequency, but the failure mode is a genuine gap: the fallback exists specifically to catch this kind of thing, and doesn't.
Fix
Pass
fallback_on=(ModelAPIError, UnexpectedModelBehavior)explicitly increate_fallback_models.🤖 Generated with Claude Code