ARXIV:2604.19321 · LLM ADAPTATION · SUBMITTED 22 APR · 02:13 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedVerifiedPaperPack: citation fields availablePartialProof: unverified proof status

RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models

Yusuf Çelebi · Yağız Asker · Özay Ezerceli · Mahmoud ElHussieni · Selva Taş · Reyhan Bayraktar · +1 at arXiv

A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

Ship in 2-4 weeks›Score7.0Evidence unverified

Opportunity summary

Pain A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

Evidence 0 refs | 3 sources | 50% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters. We model the evolution of hidden states as a high-dimensional geometric…

METHOD

Full abstract

Fine-tuning Large Language Models (LLMs) remains structurally uncertain despite parameter-efficient methods such as Low-Rank Adaptation (LoRA), as the layer-specific roles of internal representations are poorly understood, leading to heuristic decisions about where adaptation should be applied. We model the evolution of hidden states as a high-dimensional geometric trajectory and propose using the Ramer-Douglas-Peucker (RDP) algorithm, a parameter-free and training-free polygon simplification method that preserves global structural transitions while eliminating locally redundant changes, to identify critical breakpoints along the representation path. Crucially, we use these geometric pivots not merely for analysis, but as a direct decision signal for determining which layers should be adapted during parameter-efficient fine-tuning. By integrating this geometry-aware layer selection strategy into LoRA fine-tuning of Qwen3-8B-Base, we achieve superior performance on MMLU-Math using only 13 RDP-selected layers (81.67%), significantly outperforming both full 36-layer adaptation (79.32%) and random 13-layer selection (75.56%), as well as the baseline Qwen3-8B-Base model (74.25%). These results demonstrate that leveraging the intrinsic geometry of representation trajectories provides a robust, interpretable, and training-free signal for optimizing layer selection during model adaptation.

RESULT

ScienceToStartup currently rates this 7.0/10 on the public viability pass. By integrating this geometry-aware layer selection strategy into LoRA fine-tuning of Qwen3-8B-Base, we achieve superior performance on MMLU-Math using only 13 RDP-selected layers (81.67%),…

WHY NOW

LLM Adaptation moved forward this cycle; last verified April 2026. Public score 7.0/10. Production flags indicate code availability.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score7.0

PainA novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

Evidence0 refs | 3 sources | 50% coverage

Blockerno shell-level blocker reported

Analysis summary

A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

VerifiedSource: PDF linkedVerifiedPaperPack: citation fields availablePartialProof: unverified proof status

RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models

Yusuf Çelebi · Yağız Asker · Özay Ezerceli · Mahmoud ElHussieni · Selva Taş · Reyhan Bayraktar · +1 at arXiv

A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

Competitive landscape

A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

Segment

LLM Adaptation

Adoption evidence

No public code link in the paper record yet

Commercial read

7.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

{ "contract_version": "paper-r2", "paper_id": "c0e26132-fa42-4a69-aa95-30d9b59044ca", "arxiv_id": "2604.19321", "canonical_route": "/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models", "active_tab": "synced from current hash by the drawer client", "selected_artifact": "rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models", "endpoints": { "paper_pack": "/api/v1/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models/paper-pack", "build_passport": "/api/v1/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models/build-passport", "mcp_resource": "sciencetostartup://surfaces/paper-workspace" } }

{ "surface": "paper", "mode": "paper", "query": "RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models", "normalized_query": "2604.19321", "route": "/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models", "paper_ref": "rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models", "topic_slug": null, "benchmark_ref": null, "dataset_ref": null }

{ "@context": "https://schema.org", "@graph": [ { "@type": "WebPage", "@id": "https://sciencetostartup.com/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models#webpage", "url": "https://sciencetostartup.com/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models", "name": "RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models", "description": "A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.", "isPartOf": { "@id": "https://sciencetostartup.com/#website" } }, { "@type": "ScholarlyArticle", "@id": "https://sciencetostartup.com/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models#scholarlyArticle", "headline": "RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models", "description": "A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.", "url": "https://sciencetostartup.com/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models", "sameAs": "https://arxiv.org/abs/2604.19321", "identifier": { "@type": "PropertyValue", "propertyID": "arXiv", "value": "2604.19321" }, "isAccessibleForFree": true, "isPartOf": { "@id": "https://sciencetostartup.com/#website" }, "datePublished": "2026-04-21T10:29:42.000Z", "author": [ { "@type": "Person", "name": "Yusuf Çelebi" }, { "@type": "Person", "name": "Yağız Asker" }, { "@type": "Person", "name": "Özay Ezerceli" }, { "@type": "Person", "name": "Mahmoud ElHussieni" }, { "@type": "Person", "name": "Selva Taş" }, { "@type": "Person", "name": "Reyhan Bayraktar" }, { "@type": "Person", "name": "Fatma Betül Terzioğlu" } ], "additionalProperty": [ { "@type": "PropertyValue", "propertyID": "viabilityScore", "value": 7 }, { "@type": "PropertyValue", "propertyID": "researchDomain", "value": "LLM Adaptation" }, { "@type": "PropertyValue", "propertyID": "commercialReadiness", "value": "code" } ] }, { "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://sciencetostartup.com" }, { "@type": "ListItem", "position": 2, "name": "LLM Adaptation", "item": "https://sciencetostartup.com/topics" }, { "@type": "ListItem", "position": 3, "name": "RDP LoRA: Geometry-Driven Identification for Parameter-Effic", "item": "https://sciencetostartup.com/paper/rdp-lora-geometry-driven-identification-for-parameter-efficient-adaptation-in-large-language-models" } ] } ] }

Competitive landscape

A novel method using geometric analysis of LLM representations to identify critical layers for parameter-efficient fine-tuning, significantly improving performance with fewer adapted parameters.

Segment

LLM Adaptation

Adoption evidence

No public code link in the paper record yet

Commercial read

7.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models

RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Related Resources

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Related Resources

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline