ORACAL: A Robust and Explainable Multimodal Framework for Smart Contract Vulnerability Detection with Causal Graph Enrichment

Dimensions overall score 8.0

• ORACAL achieves state-of-the-art performance for smart contract vulnerability detection, with a peak Macro F1 of 91.28% on the primary benchmark.result 95%
  Evidencepartial

  Experiments on large-scale datasets demonstrate that ORACAL achieves state-of-the-art performance, outperforming MANDO-HGT, MTVHunter, GNN-SC, and SCVHunter by up to 39.6 percentage points, with a peak Macro F1 of 91.28% on the primary benchmark.

  Implicationpartial

  Explicitly stated numeric result in the abstract with clear comparison to other methods.

  Verificationpartial

  partial

• ORACAL maintains strong generalization on out-of-distribution datasets, achieving 91.8% Macro F1 on CGT Weakness and 77.1% on DAppScan.result 95%
  Evidencepartial

  ORACAL maintains strong generalization on out-of-distribution datasets with 91.8% on CGT Weakness and 77.1% on DAppScan.

  Implicationpartial

  Directly stated numeric results for specific datasets in the abstract.

  Verificationpartial

  partial

• ORACAL is more robust against adversarial attacks than prior methods, limiting performance degradation to a 2.35% F1 decrease with an Attack Success Rate of only 3%.result 90%
  Evidencepartial

  Under adversarial attacks, ORACAL limits performance degradation to approximately 2.35% F1 decrease with an Attack Success Rate (ASR) of only 3%, surpassing SCVHunter and MANDO-HGT which exhibit ASRs ranging from 10.91% to 18.73%.

  Implicationpartial

  Specific numeric results for adversarial robustness are provided in the abstract, with a direct comparison to other methods.

  Verificationpartial

  partial

• ORACAL integrates a heterogeneous multimodal graph (CFG, DFG, and Call Graph) and enriches critical subgraphs using an LLM-based RAG pipeline to inject expert-level security knowledge.method 90%
  Evidencepartial

  We propose a novel method to construct a heterogeneous graph (combining CFG, DFG, and Call Graph) and enrich its critical subgraphs using an LLM-based RAG pipeline. This injects expert-level security knowledge directly into the graph structure

  Implicationpartial

  This is a core methodological contribution explicitly stated in the key contributions section.

  Verificationpartial

  partial

• ORACAL employs a causal attention mechanism to disentangle true vulnerability indicators from spurious correlations.method 85%
  Evidencepartial

  ORACAL selectively enriches critical subgraphs with expert-level security context from Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs), and employs a causal attention mechanism to disentangle true vulnerability indicators from spurious correlations.

  Implicationpartial

  Directly stated in the abstract as a core component of the framework.

  Verificationpartial

  partial

• The framework provides explainable evidence for its predictions by using PGExplainer to generate subgraph-level explanations identifying vulnerability triggering paths.method 90%
  Evidencepartial

  For transparency, the framework adopts PGExplainer to generate subgraph-level explanations identifying vulnerability triggering paths.

  Implicationpartial

  Explicitly stated as a key feature in both the abstract and the methodology description.

  Verificationpartial

  partial

• In explainability evaluation, PGExplainer achieves 32.51% Mean Intersection over Union (MIoU) against manually annotated vulnerability triggering paths.result 90%
  Evidencepartial

  In explainability evaluation, PGExplainer achieves 32.51% Mean Intersection over Union (MIoU) against manually annotated vulnerability triggering paths.

  Implicationpartial

  Specific numeric result for explainability performance is provided in the abstract.

  Verificationpartial

  partial

• Existing static analysis tools (like Slither and Oyente) are limited by handcrafted rules and struggle to generalize to unseen or evolving attack strategies.limitation 85%
  Evidencepartial

  Nevertheless, their detection capability is inherently bounded by handcrafted rules and predefined signatures. As a result, they primarily identify known vulnerability patterns and struggle to generalize to unseen or evolving attack strategies.

  Implicationpartial

  Directly stated as a limitation of prior work, providing motivation for ORACAL.

  Verificationpartial

  partial

Competitive landscape

ORACAL is a multimodal graph learning framework that uses RAG and LLMs to detect smart contract vulnerabilities with explainable causal reasoning, significantly outperforming existing methods.