docs: add comprehensive frame reference pages#650
Merged
markbackman merged 2 commits intomainfrom Mar 25, 2026
Merged
Conversation
Add five new documentation pages covering all 128 frame classes in Pipecat: - overview.mdx: frame categories, properties, direction, mixins, common patterns - data-frames.mdx: audio, image, text, transcription, TTS, transport, DTMF frames - control-frames.mdx: lifecycle, pause/resume, LLM boundaries, function calling, TTS state, settings - system-frames.mdx: interruptions, VAD, user/bot state, mute system, errors, metrics - llm-frames.mdx: context management, thinking, tool config, summarization Also updates docs.json navigation to include the new Frames section. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
markbackman
reviewed
Mar 25, 2026
Contributor
markbackman
left a comment
markbackman
left a comment
There was a problem hiding this comment.
Nice! I made a handful of comments. I'm going to run Claude and see how it fixes things based on this feedback then I'll take a look again.
server/frames/control-frames.mdx
Outdated
| description: "Reference for ControlFrame types: pipeline lifecycle, response boundaries, service settings, and runtime configuration" | ||
| --- | ||
|
|
||
| ControlFrames are queued and processed in order alongside data frames. They signal boundaries, state changes, and configuration updates within the pipeline. Unlike system frames, control frames respect ordering guarantees — they won't skip ahead of data frames already in the queue. Control frames are cancelled on user interruption unless combined with `UninterruptibleFrame`. See the [frames overview](/server/frames/overview) for base class details and the full frame hierarchy. |
Contributor
There was a problem hiding this comment.
There are two types of queues: SystemFrame queues and non-SystemFrame queues. I think we can simplify this wording to reflect that.
Suggested change
| ControlFrames are queued and processed in order alongside data frames. They signal boundaries, state changes, and configuration updates within the pipeline. Unlike system frames, control frames respect ordering guarantees — they won't skip ahead of data frames already in the queue. Control frames are cancelled on user interruption unless combined with `UninterruptibleFrame`. See the [frames overview](/server/frames/overview) for base class details and the full frame hierarchy. | |
| ControlFrames signal boundaries, state changes, and configuration updates within the pipeline. They are queued and processed in order alongside DataFrames. ControlFrames are cancelled on `InterruptionFrame` unless combined with `UninterruptibleFrame`. See the [frames overview](/server/frames/overview) for base class details and the full frame hierarchy. |
Contributor
There was a problem hiding this comment.
Not sure if we need the code styling for references like ControlFrames, but whatever we pick, let's be consistent. Also, please update references like control frame to ControlFrame, etc.
server/frames/control-frames.mdx
Outdated
|
|
||
| ### EndFrame | ||
|
|
||
| Signals graceful pipeline shutdown. The transport stops sending, closes its threads, and the pipeline winds down completely. Inherits from both `ControlFrame` and `UninterruptibleFrame`, so it cannot be cancelled by interruption. |
Contributor
There was a problem hiding this comment.
Clarify a few things.
Suggested change
| Signals graceful pipeline shutdown. The transport stops sending, closes its threads, and the pipeline winds down completely. Inherits from both `ControlFrame` and `UninterruptibleFrame`, so it cannot be cancelled by interruption. | |
| Signals graceful pipeline shutdown. `EndFrame` is queued with other non-SystemFrames, which allows for FrameProcessors to be shutdown in order, allowing queued frames ahead of the EndFrame to be processed first. Inherits from both `ControlFrame` and `UninterruptibleFrame`, so it cannot be cancelled by `InterruptionFrame`. |
server/frames/control-frames.mdx
Outdated
|
|
||
| ### StopFrame | ||
|
|
||
| Stops the pipeline but keeps processors in a running state. Useful when you need to halt frame flow without tearing down the entire processor graph. Inherits from `ControlFrame` and `UninterruptibleFrame`. |
Contributor
There was a problem hiding this comment.
Suggested change
| Stops the pipeline but keeps processors in a running state. Useful when you need to halt frame flow without tearing down the entire processor graph. Inherits from `ControlFrame` and `UninterruptibleFrame`. | |
| Stops the pipeline but keeps processors in a running state. Like `EndFrame`, `StopFrame` is queued with other non-SystemFrames allowing frames preceding it to be processed first. Useful when you need to halt frame flow without tearing down the entire processor graph. Inherits from `ControlFrame` and `UninterruptibleFrame`. |
|
|
||
| ### LLMUpdateSettingsFrame | ||
|
|
||
| Update LLM service settings at runtime. Inherits from `ServiceUpdateSettingsFrame`. |
Contributor
There was a problem hiding this comment.
Many of the frames here are internal frames. Do we need to make it clear which ones can be used to do things in Pipecat? For example, these ServiceUpdateSettingsFrame subclasses.
server/frames/data-frames.mdx
Outdated
|
|
||
| DataFrames carry the main content flowing through a pipeline: audio chunks, text, images, transcriptions, and messages. They are queued and processed in order, and any pending data frames are discarded when a user interrupts. See the [Frames overview](/server/frames/overview) for base class details, mixin fields, and frame properties common to all frames. | ||
|
|
||
| ```python |
Contributor
There was a problem hiding this comment.
What are L10-L12 representing? Perhaps remove?
server/frames/system-frames.mdx
Outdated
|
|
||
| SystemFrames have higher priority than data and control frames and are never cancelled during user interruptions. They carry signals that must always be delivered: pipeline startup and teardown, error notifications, user input, and speaking state changes. See the [frames overview](/server/frames/overview) for base class details, mixin fields, and frame properties common to all frames. | ||
|
|
||
| ```python |
server/frames/system-frames.mdx
Outdated
|
|
||
| ### CancelFrame | ||
|
|
||
| Stops the pipeline immediately, skipping any queued frames. Use this when you need to abort without waiting for pending work to drain. |
Contributor
There was a problem hiding this comment.
Suggested change
| Stops the pipeline immediately, skipping any queued frames. Use this when you need to abort without waiting for pending work to drain. | |
| Stops the pipeline immediately, skipping any queued non-SystemFrames. Use this when you need to abort without waiting for pending work to drain. For example, when the user has left the session. |
server/frames/system-frames.mdx
Outdated
|
|
||
| ## User Speaking State and Mute | ||
|
|
||
| ### User Mute |
Contributor
There was a problem hiding this comment.
It might not be worth rehashing the details about muting. Perhaps we just link to /server/utilities/turn-management/user-mute-strategies for the description?
server/frames/system-frames.mdx
Outdated
|
|
||
| ### User Mute | ||
|
|
||
| User mute is a system for temporarily suppressing user input during specific periods. When muted, the `LLMUserAggregator` drops incoming user frames entirely: `InputAudioRawFrame`, `TranscriptionFrame`, `InterimTranscriptionFrame`, `UserStartedSpeakingFrame`, `UserStoppedSpeakingFrame`, VAD signals, and `InterruptionFrame`. Lifecycle frames (`StartFrame`, `EndFrame`, `CancelFrame`) are never muted. |
Contributor
There was a problem hiding this comment.
Critically, this is only while the bot is speaking. This is about preventing interruptions.
Suggested change
| User mute is a system for temporarily suppressing user input during specific periods. When muted, the `LLMUserAggregator` drops incoming user frames entirely: `InputAudioRawFrame`, `TranscriptionFrame`, `InterimTranscriptionFrame`, `UserStartedSpeakingFrame`, `UserStoppedSpeakingFrame`, VAD signals, and `InterruptionFrame`. Lifecycle frames (`StartFrame`, `EndFrame`, `CancelFrame`) are never muted. | |
| User mute is a system for temporarily suppressing user input while the bot is speaking. When muted, the `LLMUserAggregator` drops incoming user frames entirely: `InputAudioRawFrame`, `TranscriptionFrame`, `InterimTranscriptionFrame`, `UserStartedSpeakingFrame`, `UserStoppedSpeakingFrame`, VAD signals, and `InterruptionFrame`. Lifecycle frames (`StartFrame`, `EndFrame`, `CancelFrame`) are never muted. |
server/frames/system-frames.mdx
Outdated
|
|
||
| Indicates that a user turn has begun. By this point, transcriptions are usually already flowing through the pipeline. | ||
|
|
||
| Inherits from `SystemFrame`. |
Contributor
There was a problem hiding this comment.
Remove? It's in the SystemFrame section, so perhaps that's enough to indicate that this is a SystemFrame? Same for others like this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
src/pipecat/frames/frames.py🤖 Generated with Claude Code