ARXIV:2605.14038 · LLM AGENTS · SUBMITTED 15 MAY · 20:12 UTC · FRESHNESS FRESH

VerifiedSource: PDF linkedVerifiedPaperPack: citation fields availablePartialProof: unverified proof status

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Yize Cheng · Chenrui Fan · Mahdi JafariRaviz · Keivan Rezaei · Soheil Feiz · arXiv

Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.

Ship in 2-4 weeks›Score6.0Evidence unverified

Opportunity summary

Pain Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.

Evidence 0 refs | 0 sources | 0% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition. when to invoke external tools.

METHOD

Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools.

Full abstract

Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools. Prior work studying adaptive tool use has largely treated tool necessity as a model-agnostic property, annotated by human or LLM judge, and mostly cover cases where the answer is obvious (e.g., fetching the weather vs. paraphrasing text). However, tool necessity in the wild is more nuanced due to the divergence of capability boundaries across models: a problem solvable by a strong model on its own may still require tools for a weaker one. In this work, we introduce a model-adaptive definition of tool-necessity, grounded in each model's empirical performance. Following this definition, we compare the necessity against observed tool-call behavior across four models on arithmetic and factual QA dataset, and find substantial mismatches of 26.5-54.0% and 30.8-41.8%, respectively. To diagnose the failure, we decompose tool use into two stages: an internal cognition stage that reflects whether a model believes a tool is necessary, and an execution stage that determines whether the model actually makes a tool-call action. By probing the LLM hidden states, we find that both signals are often linearly decodable, yet their probe directions become nearly orthogonal in the late-layer, last-token regime that drives the next-token action. By tracing the trajectory of samples in the two-stage process, we further discover that the majority of mismatch is concentrated in the cognition-to-action transition, not in cognition itself. These results reveal a knowing-doing gap in LLM tool-use: improving tool-use reliability requires not only better recognition of when tools are needed, but also better translation of that recognition into action.

RESULT

ScienceToStartup currently rates this 6.0/10 on the public viability pass. These results reveal a knowing-doing gap in LLM tool-use: improving tool-use reliability requires not only better recognition of when tools are needed, but also…

WHY NOW

LLM Agents moved forward this cycle; last verified May 2026. Public score 6.0/10. Implementation evidence is present through a linked repository.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score6.0

PainDiagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.

Evidence0 refs | 0 sources | 0% coverage

Blockerno shell-level blocker reported

Analysis summary

Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.

VerifiedSource: PDF linkedVerifiedPaperPack: citation fields availablePartialProof: unverified proof status

Competitive landscape

Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.

Segment

LLM Agents

Adoption evidence

Public code linked for build inspection

Commercial read

6.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

{ "contract_version": "paper-r2", "paper_id": "2c379879-5115-4470-bb40-e6a8dd510569", "arxiv_id": "2605.14038", "canonical_route": "/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use", "active_tab": "synced from current hash by the drawer client", "selected_artifact": "model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use", "endpoints": { "paper_pack": "/api/v1/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use/paper-pack", "build_passport": "/api/v1/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use/build-passport", "mcp_resource": "sciencetostartup://surfaces/paper-workspace" } }

{ "surface": "paper", "mode": "paper", "query": "Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use", "normalized_query": "2605.14038", "route": "/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use", "paper_ref": "model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use", "topic_slug": null, "benchmark_ref": null, "dataset_ref": null }

{ "@context": "https://schema.org", "@graph": [ { "@type": "WebPage", "@id": "https://sciencetostartup.com/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use#webpage", "url": "https://sciencetostartup.com/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use", "name": "Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use", "description": "Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.", "isPartOf": { "@id": "https://sciencetostartup.com/#website" } }, { "@type": "ScholarlyArticle", "@id": "https://sciencetostartup.com/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use#scholarlyArticle", "headline": "Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use", "description": "Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.", "url": "https://sciencetostartup.com/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use", "sameAs": "https://arxiv.org/abs/2605.14038", "identifier": { "@type": "PropertyValue", "propertyID": "arXiv", "value": "2605.14038" }, "isAccessibleForFree": true, "isPartOf": { "@id": "https://sciencetostartup.com/#website" }, "datePublished": "2026-05-13T18:59:28.000Z", "author": [ { "@type": "Person", "name": "Yize Cheng" }, { "@type": "Person", "name": "Chenrui Fan" }, { "@type": "Person", "name": "Mahdi JafariRaviz" }, { "@type": "Person", "name": "Keivan Rezaei" }, { "@type": "Person", "name": "Soheil Feiz" } ], "codeRepository": "https://github.com/chengez/Tool-Cognition-Action", "additionalProperty": [ { "@type": "PropertyValue", "propertyID": "viabilityScore", "value": 6 }, { "@type": "PropertyValue", "propertyID": "researchDomain", "value": "LLM Agents" }, { "@type": "PropertyValue", "propertyID": "commercialReadiness", "value": "code, repo url" } ] }, { "@type": "SoftwareSourceCode", "@id": "https://sciencetostartup.com/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use#software", "name": "Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use - Source Code", "description": "Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.", "codeRepository": "https://github.com/chengez/Tool-Cognition-Action", "url": "https://github.com/chengez/Tool-Cognition-Action" }, { "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://sciencetostartup.com" }, { "@type": "ListItem", "position": 2, "name": "LLM Agents", "item": "https://sciencetostartup.com/topics" }, { "@type": "ListItem", "position": 3, "name": "Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap ", "item": "https://sciencetostartup.com/paper/model-adaptive-tool-necessity-reveals-the-knowing-doing-gap-in-llm-tool-use" } ] } ] }

Competitive landscape

Diagnosing the 'knowing-doing gap' in LLM tool use by introducing a model-adaptive definition of tool necessity and analyzing the cognition-to-action transition.

Segment

LLM Agents

Adoption evidence

Public code linked for build inspection

Commercial read

6.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Related Resources

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Related Resources

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline