feat(py): respond/restart improvements by huangjeff5 · Pull Request #5004 · genkit-ai/genkit

huangjeff5 · 2026-03-26T17:39:42Z

Adds restart support in the Python SDK and other improvements.

Tools (or wrapped tools) can raise Interrupt, which stops the current generation turn and surfaces the pending ToolRequestPart with interrupt metadata.

Now, The execution flow is paused, giving the caller the option to either respond, and restart the tool execution with replaced input and resumed metadata.

See samples for update usage.

Core Changes

Moved interrupt helper off of ToolRunContext. Now you raise Interrupt to trigger interruptions.
Added define_interrupt primitive, which provides the default Interrupt tool, with the option to provide a input schema to constrain the model call.
Renamed tool_response helper to respond_to_interrupt, and added restart_interrupted_tool to support the restart use case.
In the restart use case, use ContextVars to store _resumed_metadata and _replaced_input and propagate those to the tool inside tool_fn_wrapper.
Resume options were flattened and added directly onto the generate() signature.

Related Changes

You can pass tools directly inline (as opposed to requiring a str). tools signature now accepts str, Action, or Callable.
- Flattened prompt signature so that there isn't a separate opts argument; those PromptGenerationOptions are flat kwargs now.
I ran into 4xx issues with Gemini when sending tools with scalar inputs. To solve this, I have the genai plugin convert the input schema into an object with 'value' key. And in the core framework, when handling this, we handle this special case and unwrap it.
Tools were running sequentially, not in parallel. This PR fixes that.

gemini-code-assist

Code Review

This pull request introduces a significant refactor of the tool interrupt and resume mechanism within the Genkit Python SDK. Key changes include replacing the generic tool_responses parameter with more explicit resume_respond, resume_restart, and resume_metadata options in the generate and prompt execution APIs. A new Interrupt exception has been added for tools to signal pauses, complemented by helper functions like respond_to_interrupt and restart_interrupted_tool for resuming execution. Additionally, the ExecutablePrompt API was updated to accept configuration options as keyword arguments rather than a single opts dictionary. The Google AI plugin was also updated to handle scalar tool inputs for Gemini and to correctly report the finish_reason in streaming responses. I have no feedback to provide as there are no review comments to assess.

py/packages/genkit/src/genkit/_ai/_generate.py

py/packages/genkit/src/genkit/_ai/_tools.py

py/plugins/google-genai/src/genkit/plugins/google_genai/models/gemini.py

py/packages/genkit/src/genkit/_core/_registry.py

py/packages/genkit/tests/genkit/ai/prompt_test.py

py/packages/genkit/src/genkit/_core/_registry.py

py/packages/genkit/tests/genkit/ai/prompt_test.py

py/packages/genkit/src/genkit/_ai/_aio.py

huangjeff5 · 2026-03-30T16:01:19Z

py/packages/genkit/src/genkit/_ai/_generate.py


 async def resolve_tool(registry: Registry, tool_name: str) -> Action:
-    """Resolve a tool by name from the registry."""
+    """Resolve a :class:`~genkit.ActionKind.TOOL` action by name from the registry."""


fix comment weirdness

huangjeff5 · 2026-03-30T16:01:55Z

py/packages/genkit/src/genkit/_ai/_generate.py

-    if tool is None:
-        raise ValueError(f'Unable to resolve tool {tool_name}')
-    return tool
+    if tool is not None:


revert this

huangjeff5 · 2026-03-30T17:18:27Z

py/samples/tool-interrupts/src/respond_example.py

+                interrupt=interrupt,
+                metadata={'source': 'cli', 'path': 'respond'},
+            )
+            assert isinstance(interrupt_response, ToolResponsePart)


shouldn't need to assert here

huangjeff5 · 2026-03-30T21:59:49Z

py/packages/genkit/tests/genkit/ai/generate_interrupt_resume_test.py

+        },
+    ]
+
+    pm.responses.append(


does it make sense to have this be in the pm response? I think this is constructed via framework calls

huangjeff5 · 2026-03-30T22:00:00Z

py/packages/genkit/tests/genkit/ai/generate_interrupt_resume_test.py

+
+
+@pytest.mark.asyncio
+async def test_tool_either_interrupts_or_returns() -> None:


this test is unclear

huangjeff5 · 2026-03-30T22:23:48Z

py/packages/genkit/tests/genkit/ai/generate_interrupt_resume_test.py

+                    'toolResponse': {'ref': 'ra', 'name': 'a', 'output': {'done': True}},
+                    'metadata': {'interruptResponse': True},
+                },
+                {'toolResponse': {'ref': 'rb', 'name': 'b', 'output': 'b-done'}},


confirm no metadata is correct on restart case

huangjeff5 · 2026-03-30T22:46:47Z

py/packages/genkit/tests/genkit/ai/generate_interrupt_resume_test.py

+                resume=Resume(),
+            ),
+        )
+    assert 'replies' in ei.value.original_message or 'restarts' in ei.value.original_message


better assert, can't tell this is is verifying the GenkitError

huangjeff5 · 2026-03-30T22:50:37Z

py/packages/genkit/tests/genkit/ai/generate_interrupt_resume_test.py

+
+
+@pytest.mark.asyncio
+async def test_pending_output_trp_yields_tool_response_with_source_pending() -> None:


this scenario doesn't seem realistic because there is no interrupt in this case?

huangjeff5 · 2026-03-30T22:51:07Z

py/packages/genkit/tests/genkit/ai/generate_interrupt_resume_test.py

+                resume=Resume(),
+            ),
+        )
+    assert 'model' in ei.value.original_message.lower()


better assert, can't tell this is is verifying the GenkitError

huangjeff5 · 2026-03-30T22:53:40Z

py/packages/genkit/tests/genkit/ai/_tools_test.py

+    out = pay.restart(None, interrupt=interrupt_trp, resumed_metadata={'k': 'v'})
+    assert isinstance(out, ToolRequestPart)
+    assert out.metadata is not None
+    assert out.metadata.get('resumed') == {'k': 'v'}


should it include: resolvedInterrupt: {'reason': 'hold'}

huangjeff5 · 2026-03-30T23:17:07Z

py/packages/genkit/tests/genkit/ai/_tools_test.py

+    with pytest.raises(GenkitError) as ei:
+        await run_tool_after_restart(action, restart_trp)
+    msg = ei.value.original_message.lower()
+    assert 'restart' in msg or 'nested' in msg or 'not supported' in msg


more clear assert

huangjeff5 · 2026-03-30T23:30:39Z

py/packages/genkit/tests/genkit/ai/_tools_test.py

+async def test_run_tool_after_restart_response_preserves_ref() -> None:
+    """run_tool_after_restart produces a ToolResponsePart whose ref matches the restart TRP's ref."""


Set up a more realistic scenario
confirm that run_tool_after_restart uses the new input

huangjeff5 · 2026-03-30T23:34:46Z

py/packages/genkit/src/genkit/_ai/_aio.py

        system: str | list[Part] | None = None,
        messages: list[Message] | None = None,
-        tools: list[str] | None = None,
+        tools: Sequence[str | Tool] | None = None,


document reasoning for Sequence vs. list.

huangjeff5 · 2026-03-30T23:40:10Z

py/packages/genkit/src/genkit/_ai/_aio.py

-        """Stream generated text, returning a ModelStreamResponse with .stream and .response."""
+        """Stream generated text, returning a ModelStreamResponse with .stream and .response.
+
+        Middleware (``use=``) uses the same signatures as :meth:`generate`:


Unnecessary comment (line 957-960)

huangjeff5 · 2026-03-30T23:40:53Z

py/packages/genkit/src/genkit/_ai/_generate.py

+def tools_to_action_names(
+    tools: Sequence[str | Tool] | None,
+) -> list[str] | None:
+    """Normalize tool arguments to registry names for :class:`GenerateActionOptions`.


Remove :class: and :meth: from comment.

huangjeff5 · 2026-03-30T23:41:57Z

py/packages/genkit/src/genkit/_ai/_generate.py

-async def _resolve_tool_request(tool: Action, tool_request_part: ToolRequestPart) -> tuple[Part | None, Part | None]:
-    """Execute a tool and return (response_part, interrupt_part)."""
+def _interrupt_from_tool_exc(exc: BaseException) -> Interrupt | None:
+    """If ``exc`` is (or wraps) :class:`~genkit._ai._tools.Interrupt`, return that interrupt."""


Remove :class:

huangjeff5 · 2026-03-30T23:43:33Z

py/packages/genkit/src/genkit/_ai/_generate.py

-    """Resolve a single tool request from pending output or resume.respond list."""
+async def _resolve_resumed_tool_request(
+    registry: Registry, raw_request: GenerateActionOptions, tool_request_part: Part
+) -> tuple[Part, Part]:


_resolve_resumed_tool_request should return tuple[ToolRequestPart, ToolResponsePart]. not [Part, Part]

huangjeff5 · 2026-03-30T23:47:14Z

py/packages/genkit/src/genkit/_ai/_tools.py


-P = ParamSpec('P')
-T = TypeVar('T')
+    # ------------------------------------------------------------------


Remove these # ---- Sections

feat(py): respond/restart improvements

99305d7

github-project-automation bot added this to Genkit Backlog Mar 26, 2026

github-actions bot added docs Improvements or additions to documentation python Python config labels Mar 26, 2026

gemini-code-assist bot reviewed Mar 26, 2026

View reviewed changes

cleanup

5538bc5

huangjeff5 commented Mar 26, 2026

View reviewed changes

py/packages/genkit/src/genkit/_ai/_generate.py Outdated Show resolved Hide resolved

huangjeff5 commented Mar 26, 2026

View reviewed changes

py/packages/genkit/src/genkit/_ai/_tools.py Show resolved Hide resolved

huangjeff5 commented Mar 26, 2026

View reviewed changes

py/plugins/google-genai/src/genkit/plugins/google_genai/models/gemini.py Outdated Show resolved Hide resolved

huangjeff5 commented Mar 26, 2026

View reviewed changes

py/packages/genkit/src/genkit/_core/_registry.py Outdated Show resolved Hide resolved

huangjeff5 added 3 commits March 26, 2026 15:15

fix bugs from testing tools w/ gemini with scalar input

0f9ba86

fixes

57510fb

refactor input_schema override

c33cc16

huangjeff5 commented Mar 27, 2026

View reviewed changes

py/packages/genkit/src/genkit/_core/_registry.py Outdated Show resolved Hide resolved

parallelize tool execution

15ddedd

huangjeff5 commented Mar 27, 2026

View reviewed changes

py/packages/genkit/tests/genkit/ai/prompt_test.py Show resolved Hide resolved

huangjeff5 commented Mar 27, 2026

View reviewed changes

py/packages/genkit/src/genkit/_core/_registry.py Outdated Show resolved Hide resolved

jeffdh5 reviewed Mar 27, 2026

View reviewed changes

py/packages/genkit/tests/genkit/ai/prompt_test.py Outdated Show resolved Hide resolved

remove prompts as tools, added tests

cccac65

huangjeff5 commented Mar 27, 2026

View reviewed changes

py/packages/genkit/src/genkit/_ai/_aio.py Outdated Show resolved Hide resolved

huangjeff5 added 2 commits March 27, 2026 21:01

clean up Tool interface

81e2f08

clean up tool design

ed69312

huangjeff5 commented Mar 30, 2026

View reviewed changes



		@pytest.mark.asyncio
		async def test_tool_either_interrupts_or_returns() -> None:



		@pytest.mark.asyncio
		async def test_pending_output_trp_yields_tool_response_with_source_pending() -> None:

		async def test_run_tool_after_restart_response_preserves_ref() -> None:
		"""run_tool_after_restart produces a ToolResponsePart whose ref matches the restart TRP's ref."""

Conversation

huangjeff5 commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Core Changes

Related Changes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

huangjeff5 Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

huangjeff5 Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

huangjeff5 commented Mar 26, 2026 •

edited

Loading

huangjeff5 Mar 30, 2026 •

edited

Loading

huangjeff5 Mar 30, 2026 •

edited

Loading