Description

MMMU is a benchmark for evaluating multimodal models on massive multi-discipline tasks that require college-level subject knowledge and deliberate reasoning. It contains 11.5K carefully collected multimodal questions from exams, quizzes, and textbooks across six core disciplines, 30 subjects and 183 subfields, featuring 30 heterogeneous image types (charts, diagrams, maps, tables, music sheets, chemical structures, etc.) to stress advanced perception and domain-specific reasoning.