🦌 Fix incomplete tool call recovery on token limit truncation#50
🦌 Fix incomplete tool call recovery on token limit truncation#50masudahiroto merged 13 commits intomainfrom
Conversation
| // When the stream is cut mid-tool-input, tool-input-start fires (adding to activeToolCalls) | ||
| // but tool-call never fires (not added to completedToolCalls). | ||
| // We inject error tool_results so the model can retry with shorter arguments. | ||
| const incompleteToolCallIds = [...activeToolCalls.keys()].filter(id => !completedToolCalls.has(id)); |
There was a problem hiding this comment.
since detection is an edge case, I wonder can we move this code to helper function or separate 'middleware' type utility so we can avoid polluting main AISDKAgent code? This file always tends to do too much.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a step had both completed and incomplete tool calls (e.g. maxOutputTokens allows 1 tool call to finish but truncates the 2nd), the recovery path's `continue` skipped collecting response.messages, causing successful tool results to be lost from conversation history. The model then re-executed already-completed tool calls in a loop. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…runcation-recovery
| output: { | ||
| type: 'text', | ||
| value: [ | ||
| `Error: Your tool call for "${toolCall.name}" was truncated by the output token limit (maxOutputTokens: ${maxOutputTokens}).`, |
There was a problem hiding this comment.
Isn't this too technical for end-user?
Also it should be in i18n strings I think.
Or maybe I misunderstood, and this text is for AI agent instead.
There was a problem hiding this comment.
This message is not shown to end-user. It will be included to the context internally.
There was a problem hiding this comment.
The message is only intended to teach the agent that they have to use smaller tool call args. Thus, it is written in details.
|
sorry but i will merge #53 first. please wait a minutes to fix merge conflicts |
# Conflicts: # packages/server/src/agents/AISDKAgent.ts
masudahiroto
changed the title
masudahiroto
changed the title
|
I fixed the merge conflict caused by refactoring of AISDKAgent. Please re-review it. I apologize for requesting many reviews. |
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Both branches added fields to StepContext and RUN_FINISHED handler. Kept all additions from both sides: - This branch: messageId, hasEmittedTextStart, streamingText/chatId cleanup - PR #50: stepFinishReason, executingTool cleanup on truncation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2 participants
Summary
Fixes a bug where streams truncated mid-tool-input due to token limits (
finishReason: 'length') left the agent in a broken state. When a stream is cut off before a tool call completes, the agent now recovers by injecting synthetic tool result messages with truncated args context, allowing the model to retry with smaller inputs.Also fixes the client-side
executingToolstate not being cleared whenTOOL_CALL_ENDis never received (e.g., due to stream truncation), which would leave the UI stuck in a tool-executing state.Changes
packages/server/src/agents/AISDKAgent.ts— Inline token limit recovery logic: detectfinishReason: 'length'with incomplete tool calls, inject synthetic tool results with truncated args info, and preserve completed tool call results in the same steppackages/server/src/agents/AISDKAgent.test.ts— Add comprehensive tests for token limit recovery: truncated mid-tool-input, maxSteps enforcement, multi-tool truncation, non-length finish reason handling, and mixed complete/incomplete tool callspackages/client/src/hooks/useServerEvents.ts— ClearexecutingToolstate onRUN_FINISHEDandRUN_ERRORevents to handle cases whereTOOL_CALL_ENDis never receivedbun.lock— Bump package versions from 1.9.3 to 1.9.4