Repository navigation
Catch up on the community PR backlog (35 PRs) - #1040
Merged
Merged
Conversation
… tool loop ## Context When Anthropic's server-side tools (like `web_search`) are used alongside regular user-defined tools, the model can do both in a single response: perform a web search, write text with citations referencing the search results, and call a regular tool. Because a regular tool was called, Prism enters its multi-step tool loop. It executes the tool, then replays the entire conversation back to the API for the next turn. The problem is that when Prism builds the replayed assistant message, it includes the text with citations but drops the `server_tool_use` and `web_search_tool_result` content blocks that the citations reference. The API validates that every citation points to an existing search result, finds none, and rejects the request with: `invalid_request_error - Could not find search result for citation index.` This only triggers when the model performs a server-side tool call AND a regular tool call in the same response. If either happens alone, everything works fine. ## Changes Both the Text and Stream handlers had the same gap in their tool loop replay logic. **Text handler (`Text.php`):** Added `extractProviderToolContent()` that pulls `server_tool_use` and `*_tool_result` content blocks from the API response and stores them in `additionalContent` as `provider_tool_calls` and `provider_tool_results`, the same keys that `MessageMap::mapAssistantMessage()` already reads and serializes back to the API. This follows the existing pattern of `extractText()`, `extractCitations()`, and `extractThinking()`. **Stream handler (`Stream.php`):** The stream state already tracked provider tool calls, provider tool results, and citations during streaming, but `handleToolCalls()` only included `thinking` and `thinking_signature` in the replayed `AssistantMessage`'s `additionalContent`. Now it also includes `citations`, `provider_tool_calls`, and `provider_tool_results`.
Add topK as a first-class generation parameter alongside temperature and topP. Wire it into all three Gemini handlers (Text, Stream, Structured) via the generationConfig object. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allow fine-tuned models (e.g. ft:gpt-4.1-mini-2025-04-14:company:model-name:hash) to use structured output by extracting the base model and using prefix matching for version-dated variants. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Two changes to improve compatibility with OpenAI-compatible providers (e.g. Cloudflare Workers AI) that use the xAI driver: 1. MessageMap: Use plain string content for text-only user messages, array content format only when images are present. Some /chat/completions endpoints only accept string content and reject the array format with errors like "Type mismatch of '/messages/0/content', 'array' not in 'string'". 2. Structured handler: Handle content returned as a parsed object/array instead of a JSON string. Some providers return structured output directly as an object in the content field rather than a JSON-encoded string with a separate parsed field. The handler now normalizes this by json_encoding array content and using it as the parsed data when no parsed field is present. Both changes are backward-compatible with xAI's native API.
Mistral does not allow `response_format` and `tools` in the same request. This adds a two-phase approach to the structured handler: Phase 1: When tools are present, send the request with tools but without response_format. Handle tool calls in a loop until the model stops calling tools or max steps is exhausted. Phase 2: Send a final request without tools but with json_schema response_format to get the structured JSON output. Also upgrades from json_object mode (relies on system prompt instruction) to json_schema mode (strict server-side schema enforcement by Mistral). The implementation follows the same patterns as the Mistral text handler: reusable addStep() method, ExtractsText/ExtractsThinking traits, rate limit processing, and shouldContinue() for max steps.
…g items are sent in the correct order. Update tests to confirm mapping is as expected.
When a provider returns a tool call with no arguments, some models send
the literal JSON string "null" (valid JSON representing a null value).
json_decode('null', true) correctly parses it but returns PHP null,
which violates the declared `: array` return type and throws a TypeError
at the call site.
Fix: guard the json_decode result with is_array() and fall back to []
so callers always receive an array regardless of the model's output.
Reproducer: any tool with no required parameters called by a model that
emits "arguments": "null" instead of "arguments": "{}".
Llama models running on Groq (e.g. llama-3.3-70b-versatile) occasionally
return tool calls where the entire argument payload is concatenated into
the function.name field:
"function": { "name": "get_transactions,{\"from_date\":\"2026-03-06\"}", "arguments": "" }
Prism passes this mangled string as the ToolCall name. On the follow-up
turn Groq validates the assistant message's tool_calls against the
original request.tools list and rejects the request because
"get_transactions,{...}" was never registered as a tool:
Groq Error [400]: tool call validation failed: attempted to call tool
'get_transactions,{"from_date":"..."}' which was not in request.tools
Fix: in mapToolCalls(), detect the "name,{...}" pattern, validate the
suffix is valid JSON, and split into a clean name + arguments string.
Llama models on Groq (e.g. llama-3.3-70b-versatile) serialize all tool
call arguments as JSON strings regardless of the declared schema type.
Groq validates the model output against the schema we send and rejects
any mismatch before returning it to the caller:
Groq Error [400]: tool call validation failed: parameters for tool
get_transactions did not match schema: errors: [
`/is_cash_in`: expected boolean, but got string,
`/limit`: expected number, but got string
]
Fix: wrap boolean, number, and integer property types in anyOf so that
Groq accepts either the declared type or a plain string from the model.
The caller is responsible for coercing string values to the expected
PHP type before invoking the tool handler.
StructuredModeResolver::supportsStructuredMode() uses in_array with exact string matching. Versioned model names like gpt-5-mini-2025-08-07 don't match gpt-5-mini, causing them to silently fall into JSON mode (json_object) instead of Structured mode (json_schema + strict). In JSON mode, the schema is embedded as plain text in a system message with no constrained decoding, resulting in significantly more verbose model output. Use a hybrid approach: - Keep exact matching for gpt-4o and o3-mini families where only specific versions support structured output - Use prefix matching for gpt-4.1+, gpt-4.5+, and gpt-5+ families where structured output is a baseline feature across all versions
…erties Gemini's JSON Schema subset does not support the standard draft-4 array-type notation for nullability (e.g. `"type": ["integer", "null"]`). It expects `"type": "integer", "nullable": true` instead. When schemas are passed as raw arrays (e.g. via Laravel AI SDK's ObjectSchema which wraps Illuminate's JsonSchema serializer), the per-property SchemaMap normalization is never applied — those schemas bypass the `instanceof ObjectSchema` branch and their properties are spread directly into the payload with array-type notation intact. Add `normalizeProperties()` to intercept properties in `$schemaArray` before they reach Gemini, converting `["type", "null"]` arrays into the `nullable: true` format Gemini understands.
Normalizes legacy formats to audio/wav
Follow-up to #975 (closes upstream #974 fully): the response_json_schema field takes standard JSON Schema, which Prism schema objects already emit natively — type arrays for nullability, null entries in nullable enums, additionalProperties. Running them through the OpenAPI-style SchemaMap lost nullable semantics (Gemini ignores the OpenAPI `nullable` flag in JSON Schema mode). SchemaMap remains in use for tool function declarations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # tests/Providers/Gemini/GeminiStreamTest.php # tests/Providers/Gemini/GeminiStructuredTest.php # tests/Providers/Gemini/GeminiTextTest.php
…function tools
…fter information
…oning_tokens from usage
…ode for tool calls # Conflicts: # src/Providers/DeepSeek/Handlers/Text.php
…tails and upstream provider error messages # Conflicts: # src/Providers/OpenRouter/Handlers/Structured.php # src/Providers/OpenRouter/Handlers/Text.php
# Conflicts: # src/Providers/Mistral/Handlers/Structured.php
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sons # Conflicts: # src/Providers/Anthropic/Handlers/Text.php
Preview deployments for prism ⚡️
Commit: Deployment ID: Static site name: |
… types Models routinely serialize every tool argument as a JSON string (notably Llama models on Groq) even when the schema declares boolean or number, and under strict types the handler crashed with a raw TypeError. Tool::handle now reflects the handler signature and coerces string values into declared int/float/bool types and converts string/int values into BackedEnum instances. Arguments that do not match a declared parameter pass through untouched so the existing validation-error reporting still fires. Closes upstream #1016; addresses the coercion half of #1007. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Gemini API specifies `tools` as `Tool[]`, but four handlers built it in ways
that break that contract once more than one kind of tool is in play.
**Mixed keys → JSON object.** Gemini/Text, Gemini/Stream and Vertex/Text map
provider tools into a numerically keyed list and then assign
`$tools['function_declarations'] = …` on top. The result is a mixed-key array,
which `json_encode` emits as an object:
"tools": {"0": {"google_search": {}}, "function_declarations": [...]}
Gemini and Vertex both reject that, so combining Google Search grounding with
any custom tool fails outright.
**Silent overwrite.** Vertex/Structured reassigns `$tools` instead of appending,
so provider tools vanish whenever custom tools are present.
**Mutual exclusion.** Vertex/Stream chained `elseif`, so custom tools were only
ever sent when there were neither provider tools nor `searchGrounding` — the
combination was impossible rather than merely malformed.
All four now append a separate Tool entry, matching Gemini/Structured, which
already had it right and serves as the reference:
"tools": [{"google_search": {}}, {"function_declarations": [...]}]
Provider-tool precedence in Vertex/Stream (explicit provider tools over the
legacy `searchGrounding` option) is preserved; only the custom-tool branch
becomes additive.
Adds a regression test asserting `array_is_list($data['tools'])` plus both
entries. Verified it fails on the current code and passes with the fix; the 169
Gemini + Vertex tests and Pint stay green.
Restores the package name, README, install docs and links that the fork changed when it renamed the package. The Laravel and testbench constraints from the fork are kept.
A streamed error response is not seekable, so reading body() and then json() decoded an empty string and every 4xx/5xx surfaced as "Unknown error". Read the body once and decode from that.
The handler skipped the rest of the first data chunk after emitting StreamStart, so any content or tool call in that chunk was lost.
sixlive
force-pushed
the
adopt-fork-v0.103.0
branch
from
October 13, 2026 22:00
c19393a to
54b5f91
Compare
This was referenced Oct 13, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This brings in 35 community PRs that have been waiting since v0.100.1, all in one go.
@wishborn already merged and tested these in a fork (#1028), so I'm building on that work. The branch is the fork at its
v0.103.0tag, plus:pause_turnand refusal stop reasons), using the fork's own merge of iteffort), merged on toptoolsas an array when provider tools and function tools are mixedI kept the original commits so everyone's authorship stays intact. That means this has to go in as a merge commit. A squash would throw all of that away. The PRs in here should flip to merged on their own once it lands.
The package name, README, and install docs stay as
prism-php/prism.This also fixes #936 and #1010.
What I checked
I ran the full suite locally on PHP 8.3 with Laravel 12 (lowest and stable), PHP 8.4 with Laravel 13 (lowest), and PHP 8.5 with Laravel 13. 1,608 tests pass. PHPStan, Pint, and Rector are clean.
I also ran the test suite from
mainagainst this branch to see what changes for existing users. Everything that broke traces back to one of the changes listed below.Breaking Changes
^12.61.1or^13.12.0. Composer won't install any Laravel 11 release anymore because of open security advisories, which is why CI onmaincan't pass right now.$response->finishReason. (fix: handle unknown finish reasons gracefully #996)refusalandpause_turnmap to the newFinishReason::RefusalandFinishReason::Pause. (feat(anthropic): handle pause_turn and refusal stop reasons #1003)json_schemanow. It used to sendjson_objectplus a system message. Tools passed to a Mistral structured request get called now too. (Add tool call support to Mistral structured handler #965)response_json_schemain place ofresponse_schema. (fix(gemini): switch structured output to response_json_schema #975)promptTokensno longer includes cache hits. (fix(deepseek): extract prompt_cache_hit_tokens and reasoning_tokens from usage #1021)nullin theenumlist. (Fix: Nullable Enum Schema #1024)Text\RequestandStructured\Requesttake a newtopKconstructor argument. (feat(gemini): add topK generation parameter support #961)executeToolCall()andresolveTool()inCallsToolsare static. (Fix: concurrent tool execution fails with ProcessDriver due to $this capture and circular references #1009)What's left out
The rest of #1028. Batch and Files, the new providers, human-in-the-loop, and telemetry all need a closer look, so I'm going through those separately.