MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants

Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• MiniAppBench is the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation.method 90%
  Evidencepartial

  To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation.

  Implicationpartial

  Directly stated in the abstract with explicit 'first comprehensive benchmark' phrasing

  Verificationpartial

  partial

• Existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture capabilities required for interactive HTML-based applications.limitation 85%
  Evidencepartial

  However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm.

  Implicationpartial

  Directly stated in abstract as a motivation for the new benchmark

  Verificationpartial

  partial

• MiniAppBench contains 500 tasks across six domains including Games, Science, and Tools.method 90%
  Evidencepartial

  MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools).

  Implicationpartial

  Directly stated in abstract with specific numbers and domain examples

  Verificationpartial

  partial

• MiniAppBench is sourced from a real-world application with 10M+ generations.method 90%
  Evidencepartial

  Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains

  Implicationpartial

  Directly stated in abstract with specific numeric evidence

  Verificationpartial

  partial

• MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research.result 85%
  Evidencepartial

  MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research.

  Implicationpartial

  Directly stated in abstract as a result of experiments

  Verificationpartial

  partial

• Current LLMs still face significant challenges in generating high-quality MiniApps.result 85%
  Evidencepartial

  Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps

  Implicationpartial

  Directly stated in abstract as a key finding from experiments

  Verificationpartial

  partial

• MiniAppEval performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic.method 85%
  Evidencepartial

  Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic.

  Implicationpartial

  Directly stated in abstract with specific dimensions mentioned

  Verificationpartial

  partial

• Human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications called MiniApps.technical 80%
  Evidencepartial

  human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps.

  Implicationpartial

  Directly stated in abstract as the premise of the research

  Verificationpartial

  partial

Startup potential card