Skip to content

fix(anthropic): preserve server tool content blocks during multi-step tool loop#952

Open
lorisleiva wants to merge 1 commit intoprism-php:mainfrom
lorisleiva:fix/text-handler-server-tool-replay
Open

fix(anthropic): preserve server tool content blocks during multi-step tool loop#952
lorisleiva wants to merge 1 commit intoprism-php:mainfrom
lorisleiva:fix/text-handler-server-tool-replay

Conversation

@lorisleiva
Copy link
Contributor

Context

When Anthropic's server-side tools (like web_search) are used alongside regular user-defined tools, the model can do both in a single response: perform a web search, write text with citations referencing the search results, and call a regular tool. Because a regular tool was called, Prism enters its multi-step tool loop. It executes the tool, then replays the entire conversation back to the API for the next turn.

The problem is that when Prism builds the replayed assistant message, it includes the text with citations but drops the server_tool_use and web_search_tool_result content blocks that the citations reference. The API validates that every citation points to an existing search result, finds none, and rejects the request with: invalid_request_error - Could not find search result for citation index.

This only triggers when the model performs a server-side tool call AND a regular tool call in the same response. If either happens alone, everything works fine.

Changes

Both the Text and Stream handlers had the same gap in their tool loop replay logic.

Text handler (Text.php): Added extractProviderToolContent() that pulls server_tool_use and *_tool_result content blocks from the API response and stores them in additionalContent as provider_tool_calls and provider_tool_results, the same keys that MessageMap::mapAssistantMessage() already reads and serializes back to the API. This follows the existing pattern of extractText(), extractCitations(), and extractThinking().

Stream handler (Stream.php): The stream state already tracked provider tool calls, provider tool results, and citations during streaming, but handleToolCalls() only included thinking and thinking_signature in the replayed AssistantMessage's additionalContent. Now it also includes citations, provider_tool_calls, and provider_tool_results.

… tool loop

## Context

When Anthropic's server-side tools (like `web_search`) are used alongside regular user-defined tools, the model can do both in a single response: perform a web search, write text with citations referencing the search results, and call a regular tool. Because a regular tool was called, Prism enters its multi-step tool loop. It executes the tool, then replays the entire conversation back to the API for the next turn.

The problem is that when Prism builds the replayed assistant message, it includes the text with citations but drops the `server_tool_use` and `web_search_tool_result` content blocks that the citations reference. The API validates that every citation points to an existing search result, finds none, and rejects the request with: `invalid_request_error - Could not find search result for citation index.`

This only triggers when the model performs a server-side tool call AND a regular tool call in the same response. If either happens alone, everything works fine.

## Changes

Both the Text and Stream handlers had the same gap in their tool loop replay logic.

**Text handler (`Text.php`):** Added `extractProviderToolContent()` that pulls `server_tool_use` and `*_tool_result` content blocks from the API response and stores them in `additionalContent` as `provider_tool_calls` and `provider_tool_results`, the same keys that `MessageMap::mapAssistantMessage()` already reads and serializes back to the API. This follows the existing pattern of `extractText()`, `extractCitations()`, and `extractThinking()`.

**Stream handler (`Stream.php`):** The stream state already tracked provider tool calls, provider tool results, and citations during streaming, but `handleToolCalls()` only included `thinking` and `thinking_signature` in the replayed `AssistantMessage`'s `additionalContent`. Now it also includes `citations`, `provider_tool_calls`, and `provider_tool_results`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant