Gradually Excavating External Knowledge for Implicit Complex Question Answering

Dimensions overall score 8.0

• The proposed method uses less than 6% of the parameters of its competitors while achieving its results.result 95%
  Evidencepartial

  with less than 6% parameters of its competitors

  Implicationpartial

  Directly stated in the abstract with a specific numeric comparison.

  Verificationpartial

  partial

• The proposed method sets a new state-of-the-art (SOTA) for approximately 10B-scale large language models.result 90%
  Evidencepartial

  setting new SOTA for ~10B-scale LLMs.

  Implicationpartial

  Directly stated in the abstract as a performance claim, though the exact scale ('~10B') is approximate.

  Verificationpartial

  partial

• Large language models (LLMs) are not the ultimate solution for open-domain implicit question-answering problems due to uncovered or out-of-date domain knowledge.limitation 90%
  Evidencepartial

  LLMs may not be the ultimate solution due to the reasons of: 1) uncovered or out-of-date domain knowledge

  Implicationpartial

  Directly stated in the abstract as reason 1, explicitly identifying a limitation of LLMs.

  Verificationpartial

  partial

• Large language models (LLMs) are not the ultimate solution for open-domain implicit question-answering problems due to one-shot generation and hence restricted comprehensiveness.limitation 90%
  Evidencepartial

  2) one-shot generation and hence restricted comprehensiveness.

  Implicationpartial

  Directly stated in the abstract as reason 2, explicitly identifying a limitation of LLMs.

  Verificationpartial

  partial

• This work proposes a gradual knowledge excavation framework for open-domain complex question answering where LLMs iteratively and actively acquire external information.method 95%
  Evidencepartial

  this work proposes a gradual knowledge excavation framework for open-domain complex question answering, where LLMs iteratively and actively acquire external information

  Implicationpartial

  Directly stated in the abstract as the core methodological contribution.

  Verificationpartial

  partial

• The proposed method can effectively leverage plug-and-play external knowledge and dynamically adjust the strategy for solving complex questions.technical 85%
  Evidencepartial

  Our method can effectively leverage plug-and-play external knowledge and dynamically adjust the strategy for solving complex questions.

  Implicationpartial

  Directly stated in the abstract as a capability of the method, though 'effectively' is qualitative.

  Verificationpartial

  partial

• The proposed method achieves 78.17% accuracy on the StrategyQA dataset.result 95%
  Evidencepartial

  Evaluated on the StrategyQA dataset, our method achieves 78.17% accuracy

  Implicationpartial

  Directly stated in the abstract with a specific numeric result.

  Verificationpartial

  partial

• During each step of the solving process, the model selects an action (e.g., querying external knowledge or performing logical reasoning) to gradually progress toward a final answer.method 90%
  Evidencepartial

  during each step of the solving process, the model selects an action to execute, such as querying external knowledge or performing a single logical reasoning step, to gradually progress toward a final answer.

  Implicationpartial

  Directly stated in the abstract describing the specific operational mechanism of the method.

  Verificationpartial

  partial

Competitive landscape

A framework for open-domain complex question answering that iteratively acquires external information and reasons based on historical knowledge, achieving SOTA results with smaller models.