Should you find any issues using this repo, please @ the maintainer @AlbertTan404

[AlbertTan404]

so that GitHub will notify me.

[DJC-GO-SOLO]

In your paper, Colar was trained on an SFT-trained llama3.2-1b-i. I browsed through your open-source model weights and didn't find this model. Can you provide it?@AlbertTan404

[AlbertTan404]

In your paper, Colar was trained on an SFT-trained llama3.2-1b-i. I browsed through your open-source model weights and didn't find this model. Can you provide it?@AlbertTan404

Hi there,

CoLaR-SFT on GSM is at: https://huggingface.co/AlbertTan/CoLaR/tree/main/logs/colar/qsa-gsm/colar-final
and
CoLaR-SFT and CoLaR-RL on MATH are under: https://huggingface.co/AlbertTan/CoLaR/tree/main/logs/colar/qsa-math/

Please download the entire directory (including the hparams.yaml) so that our script could recognize the ckpt.

[DJC-GO-SOLO]

thanking you, let me try it.

[DJC-GO-SOLO]

Sorry, I have another question. Are the weights in colar-final the final trained colar weights, or the COT-SFT weights used for initialization?@AlbertTan404

[AlbertTan404]

Sorry, I have another question. Are the weights in colar-final the final trained colar weights, or the COT-SFT weights used for initialization?@AlbertTan404

Ah I got u, the weights are for CoLaR, not an explicit COT model used for initialization. I would like to upload the weights if needed.

[DJC-GO-SOLO]

Yes, I need it. Thank you very much.

[AlbertTan404]

Yes, I need it. Thank you very much.

Hi DJC, I've uploaded the checkpoints for CoT under https://huggingface.co/AlbertTan/CoLaR/tree/main/logs/cot.

There might be slight performance differences or potential bugs, as the checkpoints are trained before we re-constructed and re-formatted our code. But I believe that those are minor and fixable issues—please just let me know.

[DJC-GO-SOLO]

Thank you so much for your contribution and excellent work!

[DJC-GO-SOLO]

Hello, I noticed that the reproduced CODI model performs significantly worse than reported in the original paper. Could you share your thoughts on what might be causing this discrepancy? @AlbertTan404

[AlbertTan404]

＞Hello, I noticed that the reproduced CODI model performs significantly worse than reported in the original paper. Could you share your thoughts on what might be causing this discrepancy? @AlbertTan404

Hi. As the official code of CODI was not public, we reproduced CODI according to the hyper-parameters in their paper. Moreover, we have also searched over the settings of distillation loss (kl-div and mse), distillation object (logits and hidden states), and initialization (w/ and w/o cot-sft weights). However, we could not reproduce a perfect run with these settings.

Our main concern is that the distillation supervision might be much sparser than the cot teacher, and there could be potential conflict between teacher learning (cot sft) and latent learning (distillation) with such a few amount of the parameters tuned.

The idea of CODI is intuitive and we might make mistakes during the reproduction. We are willing to update our tables when their code is released :)

[DJC-GO-SOLO]

Thank you for your prompt reply—I fully agree with your comments. I also noticed that CODI has open-sourced their model weights. I hope this resource may be helpful to you. https://huggingface.co/zen-E/CODI-llama3.2-1b-Instruct

[AlbertTan404]

Thank you for your prompt reply—I fully agree with your comments. I also noticed that CODI has open-sourced their model weights. I hope this resource may be helpful to you. https://huggingface.co/zen-E/CODI-llama3.2-1b-Instruct

Thank you for this information! We will try their released model.

[YRYangang]

Hi, sorry to bother you @AlbertTan404, can you provide the LoRA target_modules for the checkpoints you uploaded?

Exception has occurred: ValueError
Please specify `target_modules` in `peft_config`

[AlbertTan404]

Hi, sorry to bother you @AlbertTan404, can you provide the LoRA target_modules for the checkpoints you uploaded?

Exception has occurred: ValueError
Please specify target_modules in peft_config

Hi. We use the default setting of LoRA, which should be "q_proj", "v_proj" according to peft constants.

The error could happen if you are using different base model from Llama. This link might help :)

[Tender221]

Hello author, there are many local installation paths in the requirements.txt of this repository, which leads to problems with environment configuration. How should I resolve this?