VulnScout-C: A Lightweight Transformer for C Code Vulnerability Detection

Dimensions overall score 7.0

• VulnScout-C, with 693M total parameters (353M active during inference), outperforms all evaluated baselines including state-of-the-art reasoning LLMs and commercial static analysis tools on a standardized C vulnerability detection benchmark.result 90%
  Evidencepartial

  Evaluated on a standardized C vulnerability detection benchmark, VULNSCOUT-C outperforms all evaluated baselines, including state-of-the-art reasoning LLMs and commercial static analysis tools

  Implicationpartial

  Directly stated in the abstract with performance comparison implied; specific parameter counts and benchmark outperformance are core claims.

  Verificationpartial

  partial

• VulnScout-C processes samples at 4.97 ms each (batch size 32, 201.1 samples/s), significantly faster than 7B-scale generative LLMs and DeepSeek R1.technical 95%
  Evidencepartial

  processing samples at 4.97 ms each (batch size 32, 201.1 samples/s), significantly faster than 7B-scale generative LLMs and DeepSeek R1

  Implicationpartial

  Explicit numeric performance metric provided in the analysis excerpt.

  Verificationpartial

  partial

• The VULNSCOUT dataset contains 33,565 labeled C code samples (19,239 vulnerable, 14,326 safe) generated through a multi-agent pipeline with a dual-verification protocol using ESBMC bounded model checking and a GPT-OSS-120B verifier.method 92%
  Evidencepartial

  We construct a new dataset of 33,565 labeled C code samples (19,239 vulnerable, 14,326 safe) spanning a wide range of CWE categories. Samples are generated through a multi-agent pipeline and retained only when a dual-verification protocol, combining ESBMC bounded model checking and a GPT-OSS-120B verifier, yields identical verdicts

  Implicationpartial

  Specific dataset size, composition, and generation methodology are directly stated.

  Verificationpartial

  partial

• The model uses a weighted BCE loss that prioritizes detection of high-severity CWEs according to the MITRE Top 25 ranking.method 88%
  Evidencepartial

  We propose a weighted BCE loss that prioritizes detection of high-severity CWEs according to the MITRE Top 25 ranking.

  Implicationpartial

  Directly stated as a proposed method in the analysis excerpt.

  Verificationpartial

  partial

• Large language models (LLMs) for vulnerability detection face limitations including inconsistent performance across vulnerability types, high computational costs, hallucinations producing false positives, and long inference times.limitation 85%
  Evidencepartial

  they face several limitations: (1) inconsistent performance across different vulnerability types, (2) high computational and memory costs, (3) hallucinations that can produce false positives, and (4) long inference times

  Implicationpartial

  Directly stated as limitations of existing approaches in the analysis excerpt.

  Verificationpartial

  partial

• The VULNSCOUT dataset is designed to fill coverage gaps in existing benchmarks where several CWEs are sparsely represented or entirely absent.method 87%
  Evidencepartial

  This conservative filtering addresses coverage gaps in existing benchmarks where several CWEs are sparsely represented or entirely absent.

  Implicationpartial

  Explicitly stated as a motivation and contribution of the new dataset.

  Verificationpartial

  partial

• Task-specialized compact architectures like VulnScout-C can match or even outperform the detection capability of models orders of magnitude larger, making continuous, low-latency vulnerability analysis practical.result 82%
  Evidencepartial

  These results demonstrate that task-specialized compact architectures can match or even outperform the detection capability of models orders of magnitude larger, making continuous, low-latency vulnerability analysis practical within real-world development workflows.

  Implicationpartial

  Directly stated conclusion in the abstract, though 'match or even outperform' is a comparative claim supported by the benchmark result.

  Verificationpartial

  partial

Competitive landscape

A compact transformer model and curated dataset for efficient and effective C code vulnerability detection, outperforming larger models and commercial tools.