Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• with Scalable Horizons and Controllable Difficulty under Lightweight Environments Wang Yang1, Chaoda Song1, Xinpeng Li1, Debargha Ganguly1, Chuang Ma2 Shouren Wang1, Zhihao Dou1, Yuli Zhou3, Vipin Chaudhary1claim 72%
• of executing complex, multi-step tasks across diverse real-world domains(Zhang et al., 2025; Schneider, 2025). To keep pace with this progress, a rich ecosystem of agent benchmarks has emerged(Yehudai et al., 2025)—including WebArena(Zhou et al., 2023), TAU2-Bench(Barres et al., 2025; Yao et al., 2024)claim 72%
• Qwen3.5-9B MiniMax-M2 Qwen3.5-4B MiroThinker-1.7-mini Qwen3.5-2B Qwen3.5-0.8B 94.4 81.5 81.5 83.3 77.8 90.7 84.9 81.5 74.1 66.7 77.8 74.1 75.9 75.0 79.6 79.6 72.2 74.1 74.1 68.5 74.7 68.5 74.1 68.5 74.1 68.5 61.1 69.1 57.4 55.6 48.1 68.5 61.1 61.1 58.6 59.3 53.7 53.7 57.4 53.7 64.8 57.1 44.4 46.3 48.1 51.9 40.7 50claim 72%
• DataExamples Time1Time2Time3MonTue Time1Time2Time3MonTue Time1Time2Time3MonTue Time1Time2Time3MonTue DecoyBudget=0 DecoyBudget=1 Hidden=1 Hidden=2 Environmentclaim 72%
• Tools: Interaction AgentRole MessageUserSolve the schedule of... AssistantI will query the candidate... ToolQuery_attribute(....)…AssistantI think I finish ... ToolDone() SlotConstraints:•Difficulty≤4GlobalConstrains•Total credits ≥ 85•Same teacher ≤ 5 timesCandidate Pool…… Figure 3: Overview of ACE-Bench frameworkclaim 72%
• time, making them prohibitively expensive for large-scale or training-time evaluation. Second, existing benchmarks are highly imbalanced: task horizons vary widely across instances (e.g., from under 10 to over 100 interaction steps)claim 72%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card