ARXIV:2602.08333 · NN TRAINING DYNAMICS · SUBMITTED 02 APR · 02:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training

arXiv

Research on neural network training dynamics without immediate product implications.

Blocked on Code›Score3.0Evidence unverified

Opportunity summary

Pain Research on neural network training dynamics without immediate product implications.

Evidence 0 refs | 0 sources | 17% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

Research on neural network training dynamics without immediate product implications. In ReLU-based models, the activation pattern induced by a given input determines the piecewise-linear region in which the network behaves affinely.

METHOD

Full abstract

Despite the empirical success of DNN, their internal training dynamics remain difficult to characterize. In ReLU-based models, the activation pattern induced by a given input determines the piecewise-linear region in which the network behaves affinely. Motivated by this geometry, we investigate whether training exhibits a two-timescale behavior: an early stage with substantial changes in activation patterns and a later stage where weight updates predominantly refine the model within largely stable activation regimes. We first prove a local stability property: outside measure-zero sets of parameters and inputs, sufficiently small parameter perturbations preserve the activation pattern of a fixed input, implying locally affine behavior within activation regions. We then empirically track per-iteration changes in weights and activation patterns across fully-connected and convolutional architectures, as well as Transformer-based models, where activation patterns are recorded in the ReLU feed-forward (MLP/FFN) submodules, using fixed validation subsets. Across the evaluated settings, activation-pattern changes decay 3 times earlier than weight-update magnitudes, showing that late-stage training often proceeds within relatively stable activation regimes. These findings provide a concrete, architecture-agnostic instrument for monitoring training dynamics and motivate further study of decoupled optimization strategies for piecewise-linear networks. For reproducibility, code and experiment configurations will be released upon acceptance.

RESULT

ScienceToStartup currently rates this 3.0/10 on the public viability pass. For reproducibility, code and experiment configurations will be released upon acceptance.

WHY NOW

NN Training Dynamics moved forward this cycle; last verified April 2026. Public score 3.0/10.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score3.0

PainResearch on neural network training dynamics without immediate product implications.

Evidence0 refs | 0 sources | 17% coverage

Blockermissing authors

Analysis summary

Research on neural network training dynamics without immediate product implications.

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

{ "contract_version": "paper-r2", "paper_id": "bfdcdc19-acdf-4b10-8338-d8a90cf59b84", "arxiv_id": "2602.08333", "canonical_route": "/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training", "active_tab": "synced from current hash by the drawer client", "selected_artifact": "regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training", "endpoints": { "paper_pack": "/api/v1/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training/paper-pack", "build_passport": "/api/v1/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training/build-passport", "mcp_resource": "sciencetostartup://surfaces/paper-workspace" } }

{ "surface": "paper", "mode": "paper", "query": "Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training", "normalized_query": "2602.08333", "route": "/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training", "paper_ref": "regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training", "topic_slug": null, "benchmark_ref": null, "dataset_ref": null }

{ "@context": "https://schema.org", "@graph": [ { "@type": "WebPage", "@id": "https://sciencetostartup.com/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training#webpage", "url": "https://sciencetostartup.com/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training", "name": "Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training", "description": "Research on neural network training dynamics without immediate product implications.", "isPartOf": { "@id": "https://sciencetostartup.com/#website" } }, { "@type": "ScholarlyArticle", "@id": "https://sciencetostartup.com/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training#scholarlyArticle", "headline": "Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training", "description": "Research on neural network training dynamics without immediate product implications.", "url": "https://sciencetostartup.com/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training", "sameAs": "https://arxiv.org/abs/2602.08333", "identifier": { "@type": "PropertyValue", "propertyID": "arXiv", "value": "2602.08333" }, "isAccessibleForFree": true, "isPartOf": { "@id": "https://sciencetostartup.com/#website" }, "datePublished": "2026-02-09T07:14:28.000Z", "additionalProperty": [ { "@type": "PropertyValue", "propertyID": "viabilityScore", "value": 3 }, { "@type": "PropertyValue", "propertyID": "researchDomain", "value": "NN Training Dynamics" } ] }, { "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://sciencetostartup.com" }, { "@type": "ListItem", "position": 2, "name": "NN Training Dynamics", "item": "https://sciencetostartup.com/topics" }, { "@type": "ListItem", "position": 3, "name": "Regime Change Hypothesis: Foundations for Decoupled Dynamics", "item": "https://sciencetostartup.com/paper/regime-change-hypothesis-foundations-for-decoupled-dynamics-in-neural-network-training" } ] } ] }

Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training

Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline