Can Heterogeneous Language Models Be Fused? | Signal Canvas | ScienceToStartup

Can Heterogeneous Language Models Be Fused? | Signal Canvas | ScienceToStartup

Page Freshness

Signal Canvas proof surface

Canonical route: /signal-canvas/can-heterogeneous-language-models-be-fused

stale

Proof freshness: stale
Proof status: unverified
Display score: 4/10
Last proof check: 2026-04-03
Score updated: 2026-04-03
Score fresh until: 2026-05-03
References: 0
Source count: 0
Coverage: 33%

This page is showing the last landed evidence receipt and score bundle because the latest proof data is outside the freshness window.

Agent Handoff

Can Heterogeneous Language Models Be Fused?

Canonical ID can-heterogeneous-language-models-be-fused | Route /signal-canvas/can-heterogeneous-language-models-be-fused

REST example

curl https://sciencetostartup.com/api/v1/agent-handoff/signal-canvas/can-heterogeneous-language-models-be-fused

MCP example

{
  "tool": "search_signal_canvas",
  "arguments": {
    "mode": "paper",
    "paper_ref": "can-heterogeneous-language-models-be-fused",
    "query_text": "Summarize Can Heterogeneous Language Models Be Fused?"
  }
}

source_context

{
  "surface": "signal_canvas",
  "mode": "paper",
  "query": "Can Heterogeneous Language Models Be Fused?",
  "normalized_query": "2604.01674",
  "route": "/signal-canvas/can-heterogeneous-language-models-be-fused",
  "paper_ref": "can-heterogeneous-language-models-be-fused",
  "topic_slug": null,
  "benchmark_ref": null,
  "dataset_ref": null
}

Evidence Receipt

Route status: building

Claims: 8

References: Pending verification

Proof: Verification pending

Freshness state: computing

Source paper: Can Heterogeneous Language Models Be Fused?

PDF: https://arxiv.org/pdf/2604.01674v1

Source count: Pending verification

Coverage: 33%

Last proof check: 2026-04-03T20:50:40.820Z

Signal Canvas receipt window

Not build-ready: Can Heterogeneous Language Models Be Fused?

/buildability/can-heterogeneous-language-models-be-fused

Ignoreblocked

Subject: Can Heterogeneous Language Models Be Fused?

Verdict

Ignore

Verdict is Ignore because current viability and proof state do not clear the buildability gate.

Time to first demo

Insufficient data

No first-demo timestamp, owner estimate, or elapsed demo receipt is attached to this surface.

Compute envelope

Structured compute envelope

Insufficient data

No data, compute, hardware, memory, latency, dependency, or serving requirement receipt is attached.

Evidence ids

Preparing verified analysis

GitHub Code Pulse

No public code linked for this paper yet.

Claim map

Strong 8Mixed 0Weak 0

Evidencepartial
Recent progress has shown that merging can be highly effective when all source models are \emph{homogeneous}, i.e., derived from the same pretrained backbone
Implicationpartial
Directly stated in abstract with clear definition of homogeneous models
Verificationpartial
partial
Evidencepartial
In such \emph{heterogeneous} settings, direct weight-space fusion becomes ill-posed due to architectural mismatch, latent basis misalignment, and amplified cross-source conflict.
Implicationpartial
Directly stated in abstract as the core problem being addressed
Verificationpartial
partial
Evidencepartial
We address this problem with \texttt{HeteroFusion} for heterogeneous language model fusion, which consists of two key components: topology-based alignment that transfers knowledge across heterogeneous backbones by matching functional module structures instead of raw tensor coordinates, and conflict-aware denoising that suppresses incompatible or noisy transfer signals during fusion.
Implicationpartial
Explicitly stated in abstract with component names
Verificationpartial
partial
Evidencepartial
topology-based alignment that transfers knowledge across heterogeneous backbones by matching functional module structures instead of raw tensor coordinates
Implicationpartial
Directly stated in abstract as part of method description
Verificationpartial
partial
Evidencepartial
conflict-aware denoising that suppresses incompatible or noisy transfer signals during fusion
Implicationpartial
Directly stated in abstract as part of method description
Verificationpartial
partial
Evidencepartial
We further provide analytical justification showing that preserving the target adapter basis while predicting structured updates leads to a stable and well-conditioned transfer process.
Implicationpartial
Directly stated in abstract as analytical justification
Verificationpartial
partial
Evidencepartial
Across heterogeneous transfer, multi-source fusion, noisy-source robustness, and cross-family generalization settings, \texttt{HeteroFusion} consistently outperforms strong merging, fusion, and ensemble baselines.
Implicationpartial
Directly stated in abstract but requires inference that results support this claim
Verificationpartial
partial
Evidencepartial
Yet this assumption is increasingly unrealistic in open model ecosystems, where useful experts are often built on different families such as Llama, Qwen, and Mistral.
Implicationpartial
Directly stated in abstract but represents an observation about the field rather than a specific finding
Verificationpartial
partial

Author intelligence and commercialization panels stay hidden until the proof receipt is verified, cites at least 3 references, includes at least 2 sources, and clears 50% coverage. The paper narrative and citation surfaces remain public while verification is pending.