De Jure: Iterative LLM Self-Refinement for Structured Extraction of Regulatory Rules

Dimensions overall score 7.0

• De Jure is a fully automated, domain-agnostic pipeline for extracting structured regulatory rules from raw documents, requiring no human annotation, domain-specific prompting, or annotated gold data.method 95%
  Evidencepartial

  We present De Jure, a fully automated, domain-agnostic pipeline for extracting structured regulatory rules from raw documents, requiring no human annotation, domain-specific prompting, or annotated gold data.

  Implicationpartial

  Explicitly stated in the abstract as a core contribution of the paper

  Verificationpartial

  partial

• De Jure operates through four sequential stages: normalization, semantic decomposition, multi-criteria LLM-as-a-judge evaluation, and iterative repair.method 90%
  Evidencepartial

  De Jure operates through four sequential stages: normalization of source documents into structured Markdown; LLM-driven semantic decomposition into structured rule units; multi-criteria LLM-as-a-judge evaluation across 19 dimensions spanning metadata, definitions, and rule semantics; and iterative repair of low-scoring extractions within a bounded regeneration budget

  Implicationpartial

  Directly described in the abstract as the pipeline's operational stages

  Verificationpartial

  partial

• On the finance domain, De Jure yields consistent and monotonic improvement in extraction quality, reaching peak performance within three judge-guided iterations.result 90%
  Evidencepartial

  On the finance domain, De Jure yields consistent and monotonic improvement in extraction quality, reaching peak performance within three judge-guided iterations.

  Implicationpartial

  Explicitly stated in the abstract with specific performance characteristics

  Verificationpartial

  partial

• De Jure generalizes effectively to healthcare and AI governance domains, maintaining high performance across both open- and closed-source models.result 85%
  Evidencepartial

  De Jure generalizes effectively to healthcare and AI governance, maintaining high performance across both open- and closed-source models.

  Implicationpartial

  Directly stated in the abstract with domain generalization claims

  Verificationpartial

  partial

• In downstream compliance QA via RAG, responses grounded in De Jure extracted rules are preferred over prior work in 73.8% of cases at single-rule retrieval depth, rising to 84.0% under broader retrieval.result 95%
  Evidencepartial

  In a downstream compliance question-answering evaluation via RAG, responses grounded in De Jure extracted rules are preferred over prior work in 73.8% of cases at single-rule retrieval depth, rising to 84.0% under broader retrieval

  Implicationpartial

  Explicitly stated with precise numeric results in the abstract

  Verificationpartial

  partial

• Explicit, interpretable evaluation criteria can substitute for human annotation in complex regulatory domains.technical 80%
  Evidencepartial

  These results demonstrate that explicit, interpretable evaluation criteria can substitute for human annotation in complex regulatory domains

  Implicationpartial

  Strongly implied conclusion from the abstract's final statement

  Verificationpartial

  partial

• De Jure's multi-criteria LLM-as-a-judge evaluation spans 19 dimensions covering metadata, definitions, and rule semantics.method 90%
  Evidencepartial

  multi-criteria LLM-as-a-judge evaluation across 19 dimensions spanning metadata, definitions, and rule semantics

  Implicationpartial

  Directly stated in the abstract with specific detail about evaluation dimensions

  Verificationpartial

  partial

• Extraction fidelity from De Jure translates directly into downstream utility for compliance applications.result 85%
  Evidencepartial

  confirming that extraction fidelity translates directly into downstream utility

  Implicationpartial

  Directly stated conclusion based on the RAG evaluation results

  Verificationpartial

  partial

Competitive landscape

An automated pipeline for extracting structured regulatory rules from legal documents using LLM self-refinement, improving downstream compliance question-answering.