🪙 COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems

Yifeng Zhang¹, Jieming Chen², Tingguang Zhou¹
Tanishq Duhan¹, Jianghong Dong^3,*, Yuhong Cao¹, Guillaume Sartoretti¹

¹National University of Singapore ²Hong Kong Polytechnic University ³Tsinghua University

Communications in Transportation Research (COMMTR) 2026

Installation

1. Create an environment

conda create -n coin python=3.9
conda activate coin

2. Clone this repository

git clone https://github.com/marmotlab/COIN
cd COIN

3. Install MetaDrive

Install MetaDrive (0.4.2.3) via:

git clone https://github.com/metadriverse/metadrive.git
cd metadrive
pip install -e .

You can verify the installation of MetaDrive via running the testing script:

# Go to a folder where no sub-folder calls metadrive
python -m metadrive.examples.profile_metadrive

Note that please do not run the above command in a folder that has a sub-folder called ./metadrive.

4. Install COIN dependencies

pip install -r requirements.txt

Notes:

• GPU is recommended for training, but the code can fall back to CPU.
• If you already have a working MetaDrive installation, check version compatibility before mixing environments.

Training

Train COIN with the main script:

python td3_main.py --env_name intersection --seed 1000

--env_name supports three scenarios:

• intersection
• roundabout
• bottleneck

Note: For all reported experiments, we train the model with 8 different random seeds: 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000.

Training outputs are saved under a timestamped directory in runs/, including:

• model/ for checkpoints,
• train/ for TensorBoard logs,
• result/ for metrics,
• gifs/ for rendered rollouts.

To monitor training:

tensorboard --logdir runs

Evaluation

Evaluate a trained model with:

python td3_eval.py --env_name intersection --model_path runs/<your_run_dir> --num_eval 20 --gui True

What the evaluation script does:

• loads the best checkpoint from --model_path,
• evaluates multiple traffic densities (e.g., initial agents counts),
• saves summary JSON files into result/,
• exports top-down GIFs into gifs/.

If you are running on a headless server, set:

python td3_eval.py --env_name bottleneck --model_path runs/<your_run_dir> --gui False

Results

For training and evaluating baseline methods, please refer to the CoPO codebase and its configuration setup:

Demos

For detailed qualitative examples and additional demos, please refer to the Project Website.

Acknowledgement

COIN is greatly inspired by the following outstanding contributions to the open-source community: MetaDrive, CoPO.

Citation

If you find this repository useful, please cite the paper:

@article{zhang2026coin,
  title   = {COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems},
  author  = {Zhang, Yifeng and Chen, Jieming and Zhou, Tingguang and Duhan, Tanishq and Dong, Jianghong and Cao, Yuhong and Sartoretti, Guillaume},
  journal = {arXiv preprint arXiv:2603.24931},
  year    = {2026}
}

License

Please follow the licenses of this repository and all upstream dependencies, especially MetaDrive and any assets or external components included in the project tree.