Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies

Dimensions overall score 7.0

• The Symbolic-Neural Consistency Audit (SNCA) framework measures the gap between LLMs' self-stated safety policies and their actual behavior, revealing systematic compliance gaps. Existing benchmarks evaluate models against external standards but do not measure whether models understand and enforce their own stated boundaries.problem 35%
  Evidencepartial

  The Symbolic-Neural Consistency Audit (SNCA) framework measures the gap between LLMs' self-stated safety policies and their actual behavior, revealing systematic compliance gaps. Existing benchmarks evaluate models against external standards but do not measure whether models understand and enforce their own stated boundaries.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• LLMs internalize safety policies through RLHF, yet these policies are never formally specified and remain difficult to inspect. Existing benchmarks evaluate models against external standards but do not measure whether models understand and enforce their own stated boundaries.method 35%
  Evidencepartial

  LLMs internalize safety policies through RLHF, yet these policies are never formally specified and remain difficult to inspect. Existing benchmarks evaluate models against external standards but do not measure whether models understand and enforce their own stated boundaries.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 7.0/10 on the public viability pass. Evaluating four frontier models across 45 harm categories and 47,496 observations reveals systematic gaps between stated policy and observed behavior: models claiming absolute refusal frequently comply with harmful prompts, reasoning models achieve the highest self-consistency but fail to articulate policies for 29% of categories, and cross-model agreement on rule types is remarkably low (11%). Code availability is flagged in the production record; the public repository link still needs proof alignment.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 7.0/10 on the public viability pass. Evaluating four frontier models across 45 harm categories and 47,496 observations reveals systematic gaps between stated policy and observed behavior: models claiming absolute refusal frequently comply with harmful prompts, reasoning models achieve the highest self-consistency but fail to articulate policies for 29% of categories, and cross-model agreement on rule types is remarkably low (11%). Code availability is flagged in the production record; the public repository link still needs proof alignment.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• LLM Safety moved forward this cycle; last verified April 2026. Public score 7.0/10. Production flags indicate code availability.why now 35%
  Evidencepartial

  LLM Safety moved forward this cycle; last verified April 2026. Public score 7.0/10. Production flags indicate code availability.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

The Symbolic-Neural Consistency Audit (SNCA) framework measures the gap between LLMs' self-stated safety policies and their actual behavior, revealing systematic compliance gaps.