[feat]: add Categorical BDPO

[cmj2002]

No description provided.

[Copilot]

Pull request overview

This PR adds a Categorical BDPO (Behavior-Regularized Diffusion Policy Optimization) variant to the codebase, implementing a distributional version of the BDPO algorithm using categorical value distributions (C51-style) with symlog transformation. The implementation follows the structure of the existing BDPO agent but replaces scalar Q-values with categorical distributions for the temporal value function (vt).

Changes:

• Added CategoricalBDPOAgent with categorical value distributions for the temporal critic (vt) while keeping scalar values for q0
• Introduced CategoricalCriticWithDiscreteTime module for time-conditioned categorical value prediction
• Added configuration files and integration into the existing training pipeline

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
flowrl/agent/offline/bdpo/categorical_bdpo.py	New agent implementation with categorical value distributions, symlog transformation, and C51-style projection
flowrl/module/critic.py	Added CategoricalCriticWithDiscreteTime module for time-conditioned categorical predictions
flowrl/config/offline/algo/categorical_bdpo.py	Configuration dataclasses for the new algorithm
flowrl/config/offline/__init__.py	Registered CategoricalBDPOConfig in the config registry
flowrl/agent/offline/__init__.py	Exported CategoricalBDPOAgent for use in training scripts
examples/offline/main_d4rl.py	Added categorical_bdpo to supported agents
examples/offline/config/d4rl/algo/categorical_bdpo.yaml	Default hyperparameters for D4RL tasks
scripts/d4rl/categorical_bdpo.sh	Experiment script with task-specific hyperparameters for D4RL benchmarks

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.