Dimensions overall score 8.0

• AdaRubric generates task-specific evaluation rubrics on the fly from task descriptions for LLM agent evaluation.method 95%
• AdaRubric achieves Pearson r=0.79 human correlation on WebArena and ToolBench, which is +0.16 over the best static baseline.result 95%
• DPO agents trained on AdaRubric preference pairs gain +6.8 to +8.5 percentage points task success over Prometheus across three benchmarks.result 90%
• AdaRubric's DimensionAwareFilter is a provably necessary condition for preventing high-scoring dimensions from masking dimension-level failures.technical 85%
• AdaRubric achieves deployment-grade reliability with Krippendorff's α=0.83.result 90%
• Gains from AdaRubric-trained agents transfer to SWE-bench code repair with +4.9 percentage point improvement.result 85%
• AdaRubric accelerates PPO convergence by +6.6 percentage points at 5K steps without any rubric engineering.result 85%
• LLM-as-Judge evaluation fails agent tasks because a fixed rubric cannot capture what matters for different task types.limitation 80%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card