Clawed and Dangerous: Can We Trust Open Agentic Systems?

Dimensions overall score 4.0

• Open agentic systems combine LLM-based planning with external capabilities, persistent memory, and privileged execution.technical 95%
  Evidencepartial

  Open agentic systems combine LLM-based planning with external capabilities, persistent memory, and privileged execution.

  Implicationpartial

  This is a direct definition provided in the abstract.

  Verificationpartial

  partial

• The security challenge of open agentic systems is fundamentally different from traditional software due to probabilistic planning, untrusted natural-language inputs, uncertain execution environments, and delegated authority.technical 95%
  Evidencepartial

  Without much attention yet, their security challenge is fundamentally different from that of traditional software that relies on predictable execution and well-defined control flow. In open agentic systems, everything is ''probabilistic'': plans are generated at runtime, key decisions may be shaped by untrusted natural-language inputs and tool outputs, execution unfolds in uncertain environments, and actions are taken under authority delegated by human users.

  Implicationpartial

  The abstract explicitly states this difference and elaborates on the reasons.

  Verificationpartial

  partial

• The literature is mature in attack characterization and benchmark construction for open agentic systems.result 90%
  Evidencepartial

  Our review shows that the literature is relatively mature in attack characterization and benchmark construction, but remains weak in deployment controls, operational governance, persistent-memory integrity, and capability revocation.

  Implicationpartial

  The abstract explicitly states this finding from their synthesis.

  Verificationpartial

  partial

• The literature remains weak in deployment controls, operational governance, persistent-memory integrity, and capability revocation for open agentic systems.limitation 90%
  Evidencepartial

  Our review shows that the literature is relatively mature in attack characterization and benchmark construction, but remains weak in deployment controls, operational governance, persistent-memory integrity, and capability revocation.

  Implicationpartial

  The abstract explicitly states this finding from their synthesis.

  Verificationpartial

  partial

• Memory integrity is completely absent in the defense coverage analysis of open agentic systems.limitation 90%
  Evidencepartial

  limitation becomes especially visible in the defense coverage analy- sis below: memory integrity is completely absent, and

  Implicationpartial

  The text explicitly states this absence in the defense coverage analysis.

  Verificationpartial

  partial

• No current benchmarks evaluate whether tool use exceeds declared scope because none model permission manifests.limitation 90%
  Evidencepartial

  Capability Overreach Rate(COR): 0 of 10 benchmarks evaluate whether tool use exceeds declared scope, because none models per- mission manifests.

  Implicationpartial

  The text explicitly states this limitation of existing benchmarks.

  Verificationpartial

  partial

• LLM-generated plans should be treated as proposals and pass through deterministic policy checks and typed tool interfaces before execution.method 90%
  Evidencepartial

  LLM outputs are context-dependent, and vulnerable to indirect prompt injection. They should be treated as proposals rather than privileged actions. The key requirement is that model-generated plans pass through deterministic policy checks and typed tool interfaces before any side-effecting operation is executed.

  Implicationpartial

  This is presented as a key requirement and principle derived from the literature.

  Verificationpartial

  partial

• Tool publication, installation, updates, and revocation should be treated as supply-chain events, with clients verifying signed manifests.method 90%
  Evidencepartial

  The requirement is to move trust checks earlier. Tool publica- tion, installation, updates, and revocation should all be treated as supply-chain events. Mature precedents already exist: Sigstore [17] supports signing and transparency, while in-toto [77] ties artifacts to verifiable build provenance. In an OpenClaw/MCP setting, clients should verify signed manifests

  Implicationpartial

  This is presented as a requirement for moving trust checks earlier in the process, drawing parallels to existing precedents.

  Verificationpartial

  partial

Competitive landscape

A framework for systematically evaluating and securing open agentic systems by providing a taxonomy, doctrine, and scorecard for governable and resilient agent platforms.