Dimensions overall score 4.0

GitHub Code Pulse

No public code linked for this paper yet.

• The MathGen benchmark is the first comprehensive benchmark dedicated to testing mathematical correctness for text-to-image models.method 95%
• Current text-to-image models perform poorly on mathematical visual generation, with the best closed-source model achieving only 42.0% overall accuracy on the MathGen benchmark.result 90%
• Open-source text-to-image models achieve very low accuracy (approximately 1-11%) on mathematical visual generation tasks, often near 0% on structured tasks.result 90%
• Widely used automatic metrics like CLIP-score and FID are insufficient for assessing mathematical generation tasks because they measure semantic similarity rather than exact structural validity.limitation 85%
• The MathGen benchmark uses a Script-as-a-Judge protocol with executable verifiers for deterministic and objective evaluation of mathematical image generation.method 90%
• Mathematical fidelity, or the ability to render visually correct mathematical solutions, is a major bottleneck for current text-to-image models.limitation 90%
• The benchmark includes both 'clean-scene' and 'open-scene' settings to isolate whether model failures arise from mathematical execution or from interference by compositional scene generation.method 85%
• Among the models tested, Nano Banana Pro achieved the highest overall accuracy (42.0%) on the clean-scene setting of the MathGen benchmark.result 95%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card