ARXIV:2602.05523 · CYBERSECURITY EVALUATION · SUBMITTED 02 APR · 02:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

arXiv

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Blocked on Code›Score4.0Evidence unverified

Opportunity summary

Pain A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Evidence 0 refs | 0 sources | 17% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks. However, existing pointwise benchmarks have limited ability to shed light on the robustness and generalisation abilities of agents across alternative versions…

METHOD

Full abstract

Agentic large language models (LLMs) are increasingly evaluated on cybersecurity tasks using capture-the-flag (CTF) benchmarks. However, existing pointwise benchmarks have limited ability to shed light on the robustness and generalisation abilities of agents across alternative versions of the source code. We introduce CTF challenge families, whereby a single CTF is used as the basis for generating a family of semantically-equivalent challenges via semantics-preserving program transformations. This enables controlled evaluation of agent robustness to source code transformations while keeping the underlying exploit strategy fixed. We introduce a new tool, Evolve-CTF, that generates CTF families from Python challenges using a range of transformations. Using Evolve-CTF to derive families from Cybench and Intercode challenges, we evaluate 13 agentic LLM configurations with tool access. We find that models are remarkably robust to intrusive renaming and code insertion-based transformations, but that composed transformations and deeper obfuscation affect performance by requiring more sophisticated use of tools. We also find that enabling explicit reasoning has little effect on solution success rates across challenge families. Our work contributes a valuable technique and tool for future LLM evaluations, and a large dataset characterising the capabilities of current state-of-the-art models in this domain.

RESULT

ScienceToStartup currently rates this 4.0/10 on the public viability pass. This enables controlled evaluation of agent robustness to source code transformations while keeping the underlying exploit strategy fixed.

WHY NOW

Cybersecurity Evaluation moved forward this cycle; last verified April 2026. Public score 4.0/10.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score4.0

PainA tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Evidence0 refs | 0 sources | 17% coverage

Blockermissing authors

Analysis summary

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

ARXIV:2602.05523 · CYBERSECURITY EVALUATION · SUBMITTED 02 APR · 02:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

arXiv

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Blocked on Code›Score4.0Evidence unverified

Opportunity summary

Pain A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Evidence 0 refs | 0 sources | 17% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

METHOD

Full abstract

RESULT

WHY NOW

Cybersecurity Evaluation moved forward this cycle; last verified April 2026. Public score 4.0/10.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score4.0

PainA tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Evidence0 refs | 0 sources | 17% coverage

Blockermissing authors

Analysis summary

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Paper Pack

10.48550/arXiv.2602.05523

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Abstract

Source availability

PDF linked

The paper record includes a public PDF URL.

Extraction status

Derived fallback

Read summaries are estimated from adjacent metadata, not verified extraction rows.

Proof status

unverified

0 refs; 0 sources; 17% coverage.

What was readable

linkedon filenot materializedderived fallback32 indexednot indexed

Derived fallback: Estimated from adjacent evidence; not verified from source.

Viability

4.0

Time to MVP

MVP estimate missing

Commercial

No commercial flags on file

Export

Preparing verified analysis

lens / founder

PROBLEM

METHOD

RESULT

WHY NOW

Cybersecurity Evaluation moved forward this cycle; last verified April 2026. Public score 4.0/10.

Claim map

Abstract-backed public claims while anchored extraction refreshes.

Strong 0Mixed 0Weak 4

Evidencepartial
A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks. However, existing pointwise benchmarks have limited ability to shed light on the robustness and generalisation abilities of agents across alternative versions of the source code.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial
Evidencepartial
Agentic large language models (LLMs) are increasingly evaluated on cybersecurity tasks using capture-the-flag (CTF) benchmarks. However, existing pointwise benchmarks have limited ability to shed light on the robustness and generalisation abilities of agents across alternative versions of the source code.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial
Evidencepartial
ScienceToStartup currently rates this 4.0/10 on the public viability pass. This enables controlled evaluation of agent robustness to source code transformations while keeping the underlying exploit strategy fixed.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial
Evidencepartial
Cybersecurity Evaluation moved forward this cycle; last verified April 2026. Public score 4.0/10.
Implicationpartial
Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.
Verificationpartial
partial

Constellation map

Paper-native neighborhood for concepts, methods, materials, markets, and competitors. Missing lanes stay labeled instead of disappearing behind commercialization gates.

Open full Signal Canvas

Concepts

not indexed

Methods

Materials

PDF linked

Markets

Cybersecurity Evaluation

Competitors

not indexed

Competitive landscape

A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.

Segment

Cybersecurity Evaluation

Adoption evidence

No public code link in the paper record yet

Commercial read

4.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

Buzz

No indexed public discussion is attached to 2602.05523 yet. That is a visibility signal, not a blank module: the monitor is watching the public channels below.

Hacker News

Not indexed yet

Bluesky

Not indexed yet

PDF

Preview the source document here, or use the hero PDF action for a new tab.

References(32)

Reference metadata pending (fca7f7d057c42b5e1dcbeaff586d640d40a127a0)

Reference metadata pending (63c14f6fb71680ba8145235ccbce14124fe56ff6)

Reference metadata pending (47e8b78761ffa188bc6759cdea3a116363c789a2)

Reference metadata pending (da914ca43ad394a172b4280b390ff7afad06421d)

Reference metadata pending (018eb3ee241c1a69bbe0bc5914980b39215f1717)

Reference metadata pending (e9036d90dc99fd5ee2f071c7c3402208c784d205)

Reference metadata pending (8ff77f8da2e2b044eb2fce0e045044b78e8ad75d)

Reference metadata pending (16099ac2baa6568bea525fb67eb79a246f529903)

Reference metadata pending (425d16205b28ce175c8429965a964d19b6f390c1)

Reference metadata pending (e0e7d91b3db485482a0f2138d16d6615a3ab78ee)

Reference metadata pending (8d8921ceea5bcb73a458a6f92f46a042514f2a98)

Reference metadata pending (05506581cade1a8ef6372616cec20b81a3d5c366)

Reference metadata pending (cb4b0bba67466c22bbc99bbf973dce5e1d9a48b6)

Reference metadata pending (0587ba2247e1e140ce8b776517ee931ad72e7156)

Reference metadata pending (7fc1757dbeb51349e9be7f20a3b3273cd87c7565)

Reference metadata pending (6cdedb971475321c62d1af5b3038db2f9e608fab)

Reference metadata pending (31babc068383c3966d322579eaaf79edc6a6bef4)

Reference metadata pending (f8e52bf582f21343d7be2c6773ae31a9e7b61d70)

Reference metadata pending (ef62f95c16f668f031d649799cbd79081c6d2b0f)

Reference metadata pending (22467a50298439854d44a40100bf03c6ce6fa001)

Showing 20 of 32 references

CITED BY

No citing papers are indexed in the public S2S graph yet. This is an explicit zero-signal state, not a hidden lookup.

Foundation

Prior WorkSame Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation

4.0

Extension

none indexed

Commercially relevant

Higher ViabilityDo Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture The Flag Challenges

7.0

Higher ViabilityCyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps

7.0

Higher ViabilityA Unified Framework for the Evaluation of LLM Agentic Capabilities

7.0

Higher ViabilityCyberThreat-Eval: Can Large Language Models Automate Real-World Threat Research?

9.0

Higher ViabilitySemantic Invariance in Agentic AI

5.0

Higher ViabilitySoK: Robustness in Large Language Models against Jailbreak Attacks

7.0

Higher ViabilityQuantifying LLM Safety Degradation Under Repeated Attacks Using Survival Analysis

7.0

Higher ViabilityCCTU: A Benchmark for Tool Use under Complex Constraints

7.0

Higher ViabilityFrom Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning

5.0

Conflicting

none indexed

Owned Distribution

Subscribe to the weekly brief

Get the weekly shortlist of commercializable papers, benchmark movers, and proof receipts that matter for product execution.

Agent drawer

5 surfaces preserved for agents. Humans can ignore.

Developer contracts, payload previews, evidence maps, and run controls stay here instead of the Read, Build, and Track workspace.

Run context

Paper: 2602.05523
Route: /paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations
Active tab: read
Artifact: capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

Available agents

Read extractor
Build planner
Track monitor
Competitive mapper
Related-paper scout

API/MCP endpoints

REST paper pack API/api/v1/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations/paper-pack
REST build passport API/api/v1/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations/build-passport
REST OpenAPI/api/openapi.json
MCP descriptor/api/mcp
MCP resourcesciencetostartup://surfaces/paper-workspace

Tool contracts

paper_packbuild_passportopportunity_kernelforesightsource_proofevidence_state

Payload preview

Inspect payload

{
  "contract_version": "paper-r2",
  "paper_id": "8ed3fb7b-bd37-4b9f-b7bf-5dc21b201977",
  "arxiv_id": "2602.05523",
  "canonical_route": "/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations",
  "active_tab": "synced from current hash by the drawer client",
  "selected_artifact": "capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations",
  "endpoints": {
    "paper_pack": "/api/v1/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations/paper-pack",
    "build_passport": "/api/v1/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations/build-passport",
    "mcp_resource": "sciencetostartup://surfaces/paper-workspace"
  }
}

Schema validation

paper-r2 contract: present
JSON-LD twin: SSR emitted
OpenAPI path parity: /api/openapi.json
MCP resource parity: paper-workspace

Job trace

queued: drawer opened by user action
running: inspect or copy payload
succeeded: payload available in SSR
failed: route errors appear in evidence cards

Evidence map

sources used: page freshness, source proof anchors, JSON-LD
missing sources: exposed by PaperPack and EvidenceState chips
derived fallbacks: marked unverified before handoff

Page Freshness

Canonical route, proof status, last verified, refs, sources, and coverage.

Page Freshness

Paper proof surface

Canonical route: /paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

stale

Proof freshness: stale
Proof status: unverified
Display score: 4/10
Last proof check: 2026-04-02
Score updated: 2026-04-02
Score fresh until: 2026-05-02
References: 0
Source count: 0
Coverage: 17%

This page is showing the last landed evidence receipt and score bundle because the latest proof data is outside the freshness window.

OpenAlex: pending — this preprint is not yet indexed by OpenAlex.

Agent Handoff

Endpoint list, payload shape, route context, and copyable handoff data.

Agent Handoff

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

Canonical ID capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations | Route /paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

REST example

curl https://sciencetostartup.com/api/v1/agent-handoff/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

MCP example

{
  "tool": "get_paper",
  "arguments": {
    "arxiv_id": "2602.05523"
  }
}

source_context

{
  "surface": "paper",
  "mode": "paper",
  "query": "Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations",
  "normalized_query": "2602.05523",
  "route": "/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations",
  "paper_ref": "capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations",
  "topic_slug": null,
  "benchmark_ref": null,
  "dataset_ref": null
}

Buildability Receipt

Verdict, compute envelope, blockers, signature state, and receipt links.

Paper proof page receipt window

Not build-ready: Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

/buildability/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

Ignoreblocked

Subject: Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

Verdict

Ignore

Verdict is Ignore because current viability and proof state do not clear the buildability gate.

Time to first demo

Insufficient data

No first-demo timestamp, owner estimate, or elapsed demo receipt is attached to this surface.

Compute envelope

Structured compute envelope

Insufficient data

No data, compute, hardware, memory, latency, dependency, or serving requirement receipt is attached.

Evidence ids

Receipt path

/buildability/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

Paper ref

capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations

arXiv id

2602.05523

Freshness

Generated at

2026-04-02T02:30:40.136Z

Evidence freshness

stale

Last verification

2026-04-02T02:30:40.136Z

Sources

References

Coverage

17%

Hash state

Lineage hash

45ea1443989bdba4966b2f94e0e7210e247fcd2c3a7d459b7375bd7f24318692

Canonical opportunity-kernel lineage hash.

Signature state

External signature

unsigned_external

No founder, registry, pilot, or production-adoption signature is attached to this receipt.

Verification

not_verified

Verification is blocked until an external signature is provided.

Blockers

Missing: repo_url
Missing: references
Missing: proof_status
Missing: distribution_readiness_scores
Missing: paper_extraction_scorecards
Unknown: distribution readiness has not been computed yet
Unknown: proof verification has not been recorded yet

Verification pending / evidence receipt incomplete

repo_url

references

Missing proof, requirement, signature, approval, adoption, or telemetry fields are blockers and must not be inferred.

Open receipt API receipt Build Loop Signal Canvas Proof divergence Divergence API Brier outcomes API

Source Proof anchors

Visual citations from the paper document graph.

JSON-LD twin

The application/ld+json payload rendered for agents.

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "WebPage",
      "@id": "https://sciencetostartup.com/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations#webpage",
      "url": "https://sciencetostartup.com/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations",
      "name": "Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations",
      "description": "A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.",
      "isPartOf": {
        "@id": "https://sciencetostartup.com/#website"
      }
    },
    {
      "@type": "ScholarlyArticle",
      "@id": "https://sciencetostartup.com/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations#scholarlyArticle",
      "headline": "Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations",
      "description": "A tool for evaluating LLM robustness using semantically-preserving program transformations in cybersecurity tasks.",
      "url": "https://sciencetostartup.com/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations",
      "sameAs": "https://arxiv.org/abs/2602.05523",
      "identifier": {
        "@type": "PropertyValue",
        "propertyID": "arXiv",
        "value": "2602.05523"
      },
      "isAccessibleForFree": true,
      "isPartOf": {
        "@id": "https://sciencetostartup.com/#website"
      },
      "datePublished": "2026-02-05T10:30:57.000Z",
      "citation": [
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "fca7f7d057c42b5e1dcbeaff586d640d40a127a0"
          },
          "url": "https://www.semanticscholar.org/paper/fca7f7d057c42b5e1dcbeaff586d640d40a127a0"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "63c14f6fb71680ba8145235ccbce14124fe56ff6"
          },
          "url": "https://www.semanticscholar.org/paper/63c14f6fb71680ba8145235ccbce14124fe56ff6"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "47e8b78761ffa188bc6759cdea3a116363c789a2"
          },
          "url": "https://www.semanticscholar.org/paper/47e8b78761ffa188bc6759cdea3a116363c789a2"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "da914ca43ad394a172b4280b390ff7afad06421d"
          },
          "url": "https://www.semanticscholar.org/paper/da914ca43ad394a172b4280b390ff7afad06421d"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "018eb3ee241c1a69bbe0bc5914980b39215f1717"
          },
          "url": "https://www.semanticscholar.org/paper/018eb3ee241c1a69bbe0bc5914980b39215f1717"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "e9036d90dc99fd5ee2f071c7c3402208c784d205"
          },
          "url": "https://www.semanticscholar.org/paper/e9036d90dc99fd5ee2f071c7c3402208c784d205"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "8ff77f8da2e2b044eb2fce0e045044b78e8ad75d"
          },
          "url": "https://www.semanticscholar.org/paper/8ff77f8da2e2b044eb2fce0e045044b78e8ad75d"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "16099ac2baa6568bea525fb67eb79a246f529903"
          },
          "url": "https://www.semanticscholar.org/paper/16099ac2baa6568bea525fb67eb79a246f529903"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "425d16205b28ce175c8429965a964d19b6f390c1"
          },
          "url": "https://www.semanticscholar.org/paper/425d16205b28ce175c8429965a964d19b6f390c1"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "e0e7d91b3db485482a0f2138d16d6615a3ab78ee"
          },
          "url": "https://www.semanticscholar.org/paper/e0e7d91b3db485482a0f2138d16d6615a3ab78ee"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "8d8921ceea5bcb73a458a6f92f46a042514f2a98"
          },
          "url": "https://www.semanticscholar.org/paper/8d8921ceea5bcb73a458a6f92f46a042514f2a98"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "05506581cade1a8ef6372616cec20b81a3d5c366"
          },
          "url": "https://www.semanticscholar.org/paper/05506581cade1a8ef6372616cec20b81a3d5c366"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "cb4b0bba67466c22bbc99bbf973dce5e1d9a48b6"
          },
          "url": "https://www.semanticscholar.org/paper/cb4b0bba67466c22bbc99bbf973dce5e1d9a48b6"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "0587ba2247e1e140ce8b776517ee931ad72e7156"
          },
          "url": "https://www.semanticscholar.org/paper/0587ba2247e1e140ce8b776517ee931ad72e7156"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "7fc1757dbeb51349e9be7f20a3b3273cd87c7565"
          },
          "url": "https://www.semanticscholar.org/paper/7fc1757dbeb51349e9be7f20a3b3273cd87c7565"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "6cdedb971475321c62d1af5b3038db2f9e608fab"
          },
          "url": "https://www.semanticscholar.org/paper/6cdedb971475321c62d1af5b3038db2f9e608fab"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "31babc068383c3966d322579eaaf79edc6a6bef4"
          },
          "url": "https://www.semanticscholar.org/paper/31babc068383c3966d322579eaaf79edc6a6bef4"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "f8e52bf582f21343d7be2c6773ae31a9e7b61d70"
          },
          "url": "https://www.semanticscholar.org/paper/f8e52bf582f21343d7be2c6773ae31a9e7b61d70"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "ef62f95c16f668f031d649799cbd79081c6d2b0f"
          },
          "url": "https://www.semanticscholar.org/paper/ef62f95c16f668f031d649799cbd79081c6d2b0f"
        },
        {
          "@type": "ScholarlyArticle",
          "identifier": {
            "@type": "PropertyValue",
            "propertyID": "SemanticScholar",
            "value": "22467a50298439854d44a40100bf03c6ce6fa001"
          },
          "url": "https://www.semanticscholar.org/paper/22467a50298439854d44a40100bf03c6ce6fa001"
        }
      ],
      "additionalProperty": [
        {
          "@type": "PropertyValue",
          "propertyID": "viabilityScore",
          "value": 4
        },
        {
          "@type": "PropertyValue",
          "propertyID": "researchDomain",
          "value": "Cybersecurity Evaluation"
        }
      ]
    },
    {
      "@type": "BreadcrumbList",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://sciencetostartup.com"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Cybersecurity Evaluation",
          "item": "https://sciencetostartup.com/topics"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "Capture the Flags: Family-Based Evaluation of Agentic LLMs v",
          "item": "https://sciencetostartup.com/paper/capture-the-flags-family-based-evaluation-of-agentic-llms-via-semantics-preserving-transformations"
        }
      ]
    }
  ]
}

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

Claim map

Constellation map

Competitive landscape

Buzz

PDF

References(32)

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

References(32)

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline