LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning [ICLR 2026]

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning [ICLR 2026]
Nurbek Tastan, Stefanos Laskaridis, Martin Takac, Karthik Nandakumar, Samuel Horvath
The Fourteenth International Conference on Learning Representations (ICLR), 2026

This repository contains two Jupyter notebooks:

1. matrix_factorization.ipynb: demonstrates the synthetic experiment discussed in the paper.
2. hf_implementation.ipynb: provides instructions and examples on how to apply our method using models from Huggingface.

These notebooks are designed to be self-contained and should work seamlessly in up-to-date environments (e.g., numpy, torch, transformers, matplotlib).

You can use these notebooks directly in your existing environments without additional setup.

📖 Citation

If you like our work, please consider citing us:

@inproceedings{tastan2026loft,
    title={{Lo{FT}: Low-Rank Adaptation That Behaves Like Full Fine-Tuning}},
    author={Nurbek Tastan and Stefanos Laskaridis and Martin Tak{\'a}{\v{c}} and Karthik Nandakumar and Samuel Horv{\'a}th},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026},
    url={https://openreview.net/forum?id=86P3sb1dpr}
}