Dimensions overall score 8.0

• Standard BPE tokenizers struggle to process complex Abugida scripts because they break complex conjuncts into meaningless sub-character units.technical 90%
• The proposed WWHO architecture and SGPE algorithm separate linguistic rules from statistical compression while enabling seamless multilingual tokenization.method 90%
• For Sinhala, SGPE achieves a Token to Word Ratio (TWR) of 1.274, representing a 61.7 percent reduction in tokens compared to OpenAI's o200k base.result 95%
• For Hindi, SGPE achieves a TWR of 1.181, representing a 27.0 percent reduction in tokens compared to OpenAI's o200k base.result 95%
• On a mixed-script dataset, SGPE achieves an overall TWR of 1.240, representing token reductions of 36.7%, 39.6%, and 60.2% relative to o200k base, Llama 4 Scout, and DeepSeek V3, respectively.result 95%
• The SGPE method effectively extends the usable context window by up to 4.38 times for Abugida languages.result 90%
• The SGPE method ensures a Linguistic Zero-Breakage Guarantee, meaning no valid syllable is ever split across multiple tokens.technical 90%
• The degradation of LLM reasoning efficiency and increased inference costs due to standard BPE tokenizers on Abugida scripts results in a significant 'Token Tax' for the Global South.market 85%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card