Comment by dchichkov

MMMU is not particularly high. Janus-Pro-7B is 41.0, which is only 14 points better than random/frequent choice. I'm pretty sure, their base DeepSeek 7B LLM will get around 41.0 MMMU without access to images, this is a normal number for a roughly GPT4-level LLM base with no access to images.