LiveMathematicianBench: A Live Benchmark for Mathematician-Level Reasoning with Proof Sketches

Dimensions overall score 7.0

GitHub Code Pulse

No public code linked for this paper yet.

• LiveMathematicianBench is a dynamic multiple-choice benchmark for research-level mathematical reasoning built from recent arXiv papers published after model training cutoffs.method 95%
• The benchmark introduces a thirteen-category logical taxonomy of theorem types enabling fine-grained evaluation across reasoning forms.technical 90%
• Gemini-3.1-pro-preview, the best model, achieves only 43.5% accuracy on LiveMathematicianBench.result 95%
• Under substitution-resistant evaluation, GPT-5.4 scores highest at 30.6%, while Gemini-3.1-pro-preview falls to 17.6%, below the 20% random baseline.result 95%
• A dual-mode protocol reveals that proof-sketch access yields consistent accuracy gains, suggesting models can leverage high-level proof strategies for reasoning.result 90%
• Existing mathematical reasoning benchmarks are limited by synthetic settings and data contamination.limitation 85%
• The benchmark employs a proof-sketch-guided distractor pipeline that uses high-level proof strategies to construct plausible but invalid answer choices.technical 90%
• LiveMathematicianBench offers a scalable, contamination-resistant testbed for studying research-level mathematical reasoning in LLMs.market 85%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card