How to stop generating

[abdullah-cod9]

Can someone clarify how I can stop the generation whenever I want?

[yamikumo-DSD]

Set steam=True and implement the mechanism to interrupt generating loop like observing a flag.

[abdullah-cod9]

That was my first attempt, but the model will continue to generate output in the backend.

[yamikumo-DSD]

Can you copy-and-paste the reproducible code snippet?

[kalle07]

iam also interessted if llama or any chathandler like qwen35 has a break command or at leas ta timeout per answer.
i think its a must if the model hangs in a loop.
so yes you can limit max-tokens but would be also nice to have a time based option.