GNNVerifier: Graph-based Verifier for LLM Task Planning

Dimensions overall score 8.0

• GNNVerifier achieves significant gains in improving plan quality across diverse datasets, backbone LLMs, and planners.result 90%
• LLM-based verifiers struggle to detect failures caused by structural relations across steps, such as type mismatches, missing intermediates, or broken dependencies.limitation 85%
• The method represents a plan as a directed graph with enriched attributes, where nodes denote sub-tasks and edges encode execution order and dependency constraints.method 95%
• A graph neural network performs structural evaluation and diagnosis, producing graph-level plausibility scores and node/edge-level risk scores.method 90%
• The method constructs controllable perturbations from ground truth plan graphs to automatically generate training data with fine-grained annotations.method 90%
• The verifier enables an LLM to conduct local edits (e.g., tool replacement or insertion) to correct plans when graph-level scores are insufficient.method 85%
• The approach requires structured plan representations that may not exist in all applications.limitation 80%
• The method adds computational overhead that could slow real-time systems.limitation 75%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card