Reducing Mental Workload through On-Demand Human Assistance for Physical Action Failures in LLM-based Multi-Robot Coordination

Dimensions overall score 7.0

• The REPAIR framework, which integrates on-demand human assistance for physical failures, significantly improves task progress in LLM-based multi-robot coordination compared to fully autonomous methods.result 90%
  Evidencepartial

  Evaluations using a multi-robot trash collection task in a real-world environment confirmed that REPAIR significantly improves task progress (the number of items cleared within a time limit) compared to fully autonomous methods.

  Implicationpartial

  Directly stated in the abstract with evaluation confirmation; the abstract says 'confirmed that REPAIR significantly improves task progress'.

  Verificationpartial

  partial

• For easily collectable items, the REPAIR framework achieves task progress equivalent to full remote control.result 85%
  Evidencepartial

  Furthermore, for easily collectable items, it achieved task progress equivalent to full remote control.

  Implicationpartial

  Explicitly stated in the abstract as a result of the evaluation.

  Verificationpartial

  partial

• The mental workload on the operator in the REPAIR framework differs in terms of physical demand and effort compared to other control methods.result 75%
  Evidencepartial

  The results also suggested that the mental workload on the operator may differ in terms of physical demand and effort.

  Implicationpartial

  Strongly suggested in the abstract and contributions section, but specific comparative results are not detailed in the provided excerpts.

  Verificationpartial

  partial

• REPAIR incorporates a human-in-the-loop mechanism where an LLM requests operator assistance when an irrecoverable physical failure occurs, enabling task continuity through remote intervention.method 95%
  Evidencepartial

  In this method, robots execute tasks autonomously; however, when an irrecoverable failure occurs, the LLM requests assistance from an operator, enabling task continuity through remote intervention.

  Implicationpartial

  Core method is explicitly described in the abstract and method section with clear operational steps.

  Verificationpartial

  partial

• Recovering from physical execution failures in LLM-based multi-robot coordination remains difficult, often causing tasks to stagnate due to repetition of unsuccessful actions.limitation 90%
  Evidencepartial

  However, recovering from physical execution failures remains difficult, and tasks often stagnate due to the repetition of the same unsuccessful actions.

  Implicationpartial

  Directly stated as a key problem in the abstract and introduction.

  Verificationpartial

  partial

• In multi-robot coordination, the failure of one robot can stall overall progress because tasks are often interdependent.technical 85%
  Evidencepartial

  In multi-robot coordination, tasks are often interdependent, and the failure of one robot can stall the progress

  Implicationpartial

  Explicitly stated as a challenge in the introduction section.

  Verificationpartial

  partial

• The REPAIR framework consists of two components: (1) LLM-based task decomposition, allocation, and action planning, and (2) remote error resolution triggered by operator assistance.method 95%
  Evidencepartial

  REPAIR consists of two components: (1) task decomposition, allocation, and action planning for multiple robots based on an LLM, and (2) remote error resolution, whereby an operator responds remotely when a robot fails to perform a

  Implicationpartial

  Explicitly and clearly defined in the proposed method section.

  Verificationpartial

  partial

• There have been few attempts to implement remote error resolution specifically for physical failures in multi-robot environments, despite proposed frameworks for remote operation using Mixed Reality.technical 80%
  Evidencepartial

  While frameworks for remote robot operation using Mixed Reality were proposed, there have been few attempts to implement remote error resolution specifically for physical failures in multi-robot environments.

  Implicationpartial

  Directly stated in the abstract as a gap in prior work, though it references other proposed frameworks not detailed in the excerpts.

  Verificationpartial

  partial

Competitive landscape

A human-in-the-loop framework that integrates remote error resolution into LLM-based multi-robot planning to overcome physical execution failures.