Qwen 3.8 Max benchmarks
Highest reported score (ties included)
| Benchmark | Opus4.8 | Fable5 | Gemini3.1-Pro | GPT5.6-Sol | Qwen3.7-Plus | Qwen3.8-Max |
|---|---|---|---|---|---|---|
| Multimodal Reasoning | ||||||
| MMMU-Pro | 75.6 | 81.2 | 80.5 | 83.0 | 79.0 | 82.3 |
| MathVision | 87.1 / 97.1 | 92.7 / 98.6 | 87.4 / 95.7 | 90.8 / 97.8 | 90.3 / -- | 95.2 / 97.7 |
| BabyVision | 28.4 / 81.2 | 42.5 / 90.5 | 55.9 / 68.3 | 65.5 / 88.9 | 64.7 / 70.4 | 82.0 / 91.3 |
| HLE-VL (w/ Tools) | -- | -- | 43.9 | 51.2 | 25.6 | 52.2 |
| ZeroBench (Pass@5) | 17.0 / 34.0 | 20.0 / 46.0 | 17.0 / 23.0 | 22.0 / 35.0 | 19.0 / 19.0 | 24.0 / 49.0 |
| ZeroBench-Sub | 31.1 | 37.1 | 36.5 | 46.7 | 41.0 | 48.5 |
| LogicVista | 76.7 | 85.7 | 82.6 | 89.7 | 84.3 | 91.9 |
| HiPhO | 69.3 | 78.6 | 85.4 | 86.8 | 84.1 | 90.0 |
| PhyX | 54.2 | 71.7 | 79.4 | 79.1 | 80.0 | 83.5 |
| SLAKE | 75.9 | 86.6 | 82.9 | 85.1 | 83.2 | 90.8 |
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
| PMC-VQA | 59.2 | 63.2 | 62.5 | 62.3 | 63.4 | 66.2 |
| Visual Agent & Coding | ||||||
| OSWorld-Verified | 83.4 | 85.0 | 76.2 | 83.2 | 73.3 | 86.1 |
| OSWorld 2.0 | 20.6 / 54.8 | -- / 66.1 | 7.8 / 30.6 | -- / 62.6 | 2.8 / 21.5 | 19.4 / 46.7 |
| ScreenSpot Pro | 82.3 | 87.3 | 68.1 | 81.3 | 79.0 | 84.5 |
| WebArena-Verified | 67.9 | 71.3 | 64.3 | 69.7 | 55.3 | 66.8 |
| AndroidWorld | 75.0 | 88.8 | 70.7 | 77.6 | 81.0 | 85.3 |
| MobileWorld | 67.5 | 85.5 | 58.1 | 76.9 | 51.2 | 77.8 |
| ClawEval-MM | 73.3 / 73.8 | 81.2 / 77.5 | 50.5 / 55.2 | 81.2 / 78.9 | 57.4 / 60.1 | 77.2 / 74.8 |
| Vision2Web | 62.4 | 70.5 | -- | 62.1 | 42.1 | 69.0 |
| QwenBlenderBench | 62.4 | 69.5 | 23.0 | 68.6 | 41.5 | 69.9 |
| Parametric CAD Bench | 85.1 | 87.5 | 73.5 | 86.2 | 73.8 | 91.5 |
| RecreationBench | 48.0 | 56.1 | 16.2 | 47.6 | 30.2 | 51.7 |
| PresentBench | 80.9 | 79.8 | 55.4 | 82.9 | 65.7 | 79.6 |
| Document & Office Intelligence | ||||||
| CharXiv (RQ) | 78.5 / 89.9 | 87.9 / 93.5 | 84.4 / 89.9 | 85.1 / 89.1 | 85.8 / 85.9 | 88.4 / 93.5 |
| OmniDocBench 1.5 | 86.5 | 89.5 | 90.0 | 86.7 | 91.4 | 92.1 |
| OCR-Bench-V2 (EN/ZH) | 53.9 / 55.3 | 65.3 / 58.1 | 64.6 / 58.2 | 69.0 / 57.3 | 70.7 / 67.1 | 74.2 / 68.3 |
| CC-OCR-Bench-V2 | 60.3 | 72.4 | 68.9 | 68.0 | 72.7 | 79.6 |
| MTVQA-Test | 48.1 | 41.6 | 54.3 | 52.7 | 51.2 | 56.6 |
| MADQA | 86.8 | 86.0 | 81.1 | 87.8 | 87.1 | 91.8 |
| QwenVisualOffice | 34.5 | 32.4 | 39.6 | 29.5 | 32.4 | 44.6 |
| Real-World & Spatial Understanding | ||||||
| RealWorldQA | 76.6 | 85.9 | 83.5 | 83.7 | 86.9 | 88.0 |
| ERQA | 57.2 | 70.0 | 68.0 | 70.0 | 69.8 | 77.8 |
| LingoQA | 73.8 | 77.4 | 66.8 | 72.6 | 83.4 | 84.8 |
| SURDS | 62.2 | 79.4 | 64.0 | 63.0 | 77.2 | 77.8 |
| Visual Perception & Grounding | ||||||
| SimpleVQA | 67.3 | 73.4 | 73.1 | 66.6 | 70.3 | 75.0 |
| WorldVQA | 33.9 | 53.5 | 54.0 | 45.1 | 43.9 | 53.2 |
| MMStar | 76.7 | 80.5 | 84.0 | 82.5 | 83.2 | 85.9 |
| PerceptionBench | 47.2 | 57.2 | 56.2 | 59.7 | 51.1 | 63.5 |
| CountQA | 41.3 | 63.1 | 72.8 | 68.6 | 77.0 | 82.4 |
| RefAdv-S | 61.7 | 68.6 | 71.9 | 69.2 | 73.0 | 80.2 |
| Dense200 | 20.8 | 31.1 | 69.7 | 55.3 | 60.7 | 87.0 |
| COCO | 50.7 | 56.4 | 72.4 | 61.2 | 74.2 | 78.7 |
| VisFactor | 30.1 | 54.5 | 39.8 | 62.8 | 42.8 | 60.8 |
| VLMsAreBiased | 43.8 | 61.2 | 74.1 | 59.8 | 36.6 | 88.3 |
| Video Intelligence & Agents | ||||||
| VideoMME (w/ Sub.) | 85.4 | -- | 86.7 | 89.5 | 88.0 | 90.4 |
| VideoMME v2 (w/ Sub.) | 49.0 | 52.2 | 66.9 | 71.1 | 59.7 | 68.3 |
| VideoMMMU | 75.3 | 81.2 | 85.3 | 85.0 | 85.4 | 88.7 |
| MMVU | 67.4 | 72.0 | 77.9 | 81.2 | 76.6 | 82.4 |
| MLVU (M-Avg) | 53.4 | -- | 84.7 | 87.6 | 87.4 | 90.8 |
| TVBench | 61.5 | -- | 73.0 | 83.2 | 78.2 | 81.9 |
| LVBench | 67.3 | -- | 75.1 | 78.8 | 76.2 | 81.8 |
| LVBench (w/ Mem.) | 84.3 | 90.1 | -- | 84.2 | 74.5 | 85.6 |
| EgoLife (w/ Mem.) | 78.3 | 82.3 | -- | 70.8 | 68.8 | 80.3 |
| VideoDR (w/ Search) | 65.6 | 77.1 | -- | 71.3 | 41.0 | 73.2 |
“--” indicates no reported result. Slash-separated results contain multiple metrics; each position is compared independently.