Search results

Decouple reasoning and evaluation

Status: Open.

#1259 In EvolvingLMMs-Lab/lmms-eval;

· xuzhang0112 opened on Mar 19, 2026

📋 Documentation Enhancement Suggestion

Status: Open.

#1254 In EvolvingLMMs-Lab/lmms-eval;

· croviatrust opened on Mar 15, 2026

[Bug] Internvl_hf: num_frames not applied to videos

bugSomething isn't workingSomething isn't working

Status: Open.

#1242 In EvolvingLMMs-Lab/lmms-eval;

· jong980812 opened on Mar 8, 2026

[Bug] internvl_hf: IndexError when visuals is empty list (video-only inputs)

bugSomething isn't workingSomething isn't working

Status: Open.

#1241 In EvolvingLMMs-Lab/lmms-eval;

· jong980812 opened on Mar 8, 2026

[Bug] NCCL warning on error path: destroy_process_group() not called

Status: Open.

#1240 In EvolvingLMMs-Lab/lmms-eval;

· akawincent opened on Mar 8, 2026

OCRBench eval bug

Status: Open.

#1220 In EvolvingLMMs-Lab/lmms-eval;

· jivercx opened on Mar 3, 2026

'Qwen3_VL' object has no attribute 'fps'

bugSomething isn't workingSomething isn't working

Status: Open.

#1219 In EvolvingLMMs-Lab/lmms-eval;

· dfm021101 opened on Mar 3, 2026

[Installation] lmms-eval --tasks list returns empty list after pip install

bugSomething isn't workingSomething isn't working

Status: Open.

#1215 In EvolvingLMMs-Lab/lmms-eval;

· YirongWho opened on Feb 28, 2026

Integrate with EgoLifeQA Benchmark

Status: Open.

#1214 In EvolvingLMMs-Lab/lmms-eval;

· Artessay opened on Feb 28, 2026

Async evaluation for HF models (async_hf_model backend)

enhancementNew feature or requestNew feature or request

Status: Open.

#1126 In EvolvingLMMs-Lab/lmms-eval;

· GeoffreyChen777 opened on Feb 21, 2026

Issues about MMVP Dataset

Status: Open.

#1018 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Jan 22, 2026

When I tested Qwen-2.5-vl-7B on the Videomme dataset, ValueErrorr: The decoder prompt (length 163270) is longer

Status: Open.

#988 In EvolvingLMMs-Lab/lmms-eval;

· LyuQi127 opened on Jan 12, 2026