This repository is the official implementation of Solving Video Inverse Problems Using Image Diffusion Models.
Our method demonstrates robustness against various combinations of spatio-temporal degradations, achieving state-of-the-art reconstructions.
Diffusion model-based inverse problem solvers (DIS) enable unconditional diffusion models to solve a wide range of image inverse problems. However, their application to video inverse problems involving temporal degradation remains limited. In response, we introduce an innovative video inverse solver using pre-trained image diffusion models by solving video inverse optimization within the Tweedie denoised manifold. We develop a batch-consistent diffusion sampling strategy to ensure temporal consistency by synchronizing stochastic noise components in image diffusion models. Experimental results confirm that our method demonstrates robustness against various combinations of temporal and spatial degradations, achieving state-of-the-art reconstructions.
This repository is based on openai/guided-diffusion.
- 256x256 diffusion (not class conditional): 256x256_diffusion_uncond.pt
Download the following pre-trained model and move into a folder called models/.
To sample from the models, you can use the TimeDeconv.py script.
Here, we provide flags for sampling from the model.
We assume that you have downloaded the relevant model checkpoints into a folder called models/.
bash sample.sh
We preprocessed DAVIS 2017 train/val dataset into numpy datasets. Here, we provide full preprocessed numpy files used in this work.
Preprocessed 2017 DAVIS train/val dataset.
If you find our method useful, please cite as below or leave a star to this repository.
@inproceedings{
kwon2025solving,
title={Solving Video Inverse Problems Using Image Diffusion Models},
author={Kwon, Taesung and Ye, Jong Chul},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=TRWxFUzK9K}
}