Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weights

Dimensions overall score 5.0

GitHub Code Pulse

No public code linked for this paper yet.

• Model merging enables effective instruction-following behavior in new languages for LLMs without requiring language-specific instruction data.result 90%
• Model merging supports multilingual capability through the combination of multiple language-specific models.result 85%
• Model merging achieves competitive performance while greatly reducing computational cost compared to traditional adaptation methods for low-resource languages.technical 88%
• The DARE (Drop And REscale) method sparsifies task vectors by randomly dropping a proportion p of parameters and rescaling the remaining ones to reduce interference between tasks during model merging.method 92%
• The TIES (Trim and Elect) method addresses update redundancy and parameter sign disagreement challenges in model merging by masking low-magnitude values and computing aggregate sign vectors.method 90%
• Merging an instruction-tuned LLM with language-specific base models eliminates the need for language-specific instructions and repeated fine-tuning processes when stronger instructed variants become available.technical 87%
• Existing adaptation approaches like continual pre-training demand significant computational resources and require high-quality instruction data that is often inaccessible for low-resource language communities.limitation 85%
• Large Language Models remain heavily centered on English with limited performance in low-resource languages.limitation 90%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card