Skip to content

Add request param overrides for pipeline operators#7277

Draft
CamronStaley wants to merge 2 commits intomainfrom
docs/request-param-overrides
Draft

Add request param overrides for pipeline operators#7277
CamronStaley wants to merge 2 commits intomainfrom
docs/request-param-overrides

Conversation

@CamronStaley
Copy link
Copy Markdown
Contributor

@CamronStaley CamronStaley commented Mar 31, 2026

🔗 Related Issues

📋 What changes are proposed in this pull request?

Update docs for request param overrides.

🧪 How is this patch tested? If it is not, please explain why.

n/a

📝 Release Notes

Is this a user-facing change that should be mentioned in the release notes?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release
    notes for FiftyOne users.

Added instructions to docs for how to use request param overrides in pipeline operators.

What areas of FiftyOne does this PR affect?

  • App: FiftyOne application changes
  • Build: Build and test infrastructure changes
  • Core: Core fiftyone Python library changes
  • Documentation: FiftyOne documentation changes
  • Other

Summary by CodeRabbit

  • Documentation
    • Added guidance on overriding pipeline stage-level request parameters using request_params_overrides.
    • Documented parameter inheritance and shadowing behavior for pipeline stages.
    • Included code example demonstrating multi-view pipeline configuration with different execution contexts.

@CamronStaley CamronStaley self-assigned this Mar 31, 2026
@CamronStaley CamronStaley requested a review from a team as a code owner March 31, 2026 15:25
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 31, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2df8dd80-95dc-4684-9deb-4209a799729d

📥 Commits

Reviewing files that changed from the base of the PR and between 6d52a1c and 370eccb.

📒 Files selected for processing (1)
  • docs/source/plugins/developing_plugins.rst

Walkthrough

A documentation section was added to explain how pipeline stages can override request parameters using request_params_overrides on PipelineStage. The section details the inheritance behavior where stages inherit top-level pipeline operator context by default, defines shadowing semantics for overridden keys, specifies supported overridable parameters (view, view_name, filters, and other execution-context parameters), and includes a note restricting params from request_params_overrides while directing users to use PipelineStage.params instead. A Python code example (MultiViewPipeline) demonstrates three pipeline stages with different view contexts using request_params_overrides.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The description is incomplete. While it identifies the change as documentation-only and includes release notes, it lacks detail about what is documented and omits the 'Related Issues' section entirely. Add related issue links (if any exist) and provide more specifics about what request param override behavior/parameters are documented in the PR.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically identifies the main change: adding request parameter override functionality for pipeline operators, matching the documentation additions in the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/request-param-overrides

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@swheaton swheaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm but see what @AdonaiVera wants to do with it
(We have a docs addition but don't want to target today's release which is underway already)

Copy link
Copy Markdown
Member

@AdonaiVera AdonaiVera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just committed some small doc style improvements @CamronStaley, this PR is ready!
Feel free to merge whenever you think it makes sense to have it in production. Once it’s merged, I can add a tag to deploy the docs.

Copy link
Copy Markdown
Member

@brimoor brimoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CamronStaley @swheaton I might have missed the boat on providing feedback on the functional implementation of this feature, but please note I have a small FR on the view parameter 😄

request_params_overrides={"view_name": "val_split"},
)

# Stage 3 runs against the full dataset (no view applied)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a case where where a view is provided?

I imagine a typical case would be something like view=ctx.view[i:j], ie manual batching.


- `view` — a list of view stages to apply
- `view_name` — the name of a saved view to use
- `filters` — a dictionary of filters to apply
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filters is not an officially documented thing. How would users figure out how to successfully use this?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They wouldn't. We can remove this example from the documentation

shadow the corresponding value from the top-level request for that stage only;
all other stages are unaffected.

Overridable parameters include:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can dataset or dataset_name be overridden? That would be powerful, but may be dubious too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about disallowing that but i guess we never did it. Probably should have an allow or block list.

Allowed and potentially useful

  • view_name
  • view (*****)
  • selected
  • current_sample
  • active_fields
  • group_slice
  • num_distributed_tasks

Allowed (not exactly useful but not harmful either)

  • filters
  • extended
  • workspace_name
  • spaces
  • selected_labels
  • extended_selection
  • query_performance

Disallowed

  • dataset_id
  • dataset_name
  • delegated
  • request_delegation
  • results

Unsure

  • delegation target - I'm not sure if this would cause downstream issues. Thinking not, but we may want a more official way to change delegation targets rather than the pipeline operator doing it under the hood

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing delegation target would be useful in the sense that certain stages may need to use GPU.

Certainly it would be an anti-pattern for pipeline operators to hard-code the name of a delegation target in them, however.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this would be a cool feature whenever we have more information about orcs in the collection. Would be cool if they could say something like "delegate_criteria: must have gpu, X storage, blah" and then we could select an appropriate orc on their behalf.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep exactly!


Overridable parameters include:

- `view` — a list of view stages to apply
Copy link
Copy Markdown
Member

@brimoor brimoor Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please support view being provided as a DatasetView object? Which would be serialized into view stages internally by our interface.

I think our public interfaces should always work with FO's main types, ie support DatasetView, not require lists of ViewStages and certainly not requiring lists of serialized view stage dicts.

(it is fwm if the view parameter supports the latter two possibilities as undocumented syntaxes though)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i agree

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed currently you have to serialize it (we don't do that on behalf of the user) which isn't intuitive. I can open a PR to fix that mb

@CamronStaley
Copy link
Copy Markdown
Contributor Author

Going to leave this PR open / undocumented to users until I can address Bmoore's comments in the next release. Current plan is:

  • serialize on behalf of the user if they didn't provide a serialized view
  • clearly document / test more allowed fields and explicitly not allow some fields

@swheaton swheaton marked this pull request as draft April 3, 2026 13:22
@swheaton
Copy link
Copy Markdown
Contributor

swheaton commented Apr 3, 2026

holding off docs until code matches what's expected from above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants