Dimensions overall score 3.0

GitHub Code Pulse

No public code linked for this paper yet.

• A novel node-matching algorithm is proposed that leverages node-word alignments inherent in Uniform Meaning Representations (UMR) for comparing multiple UMRs of a single sentence.method 95%
• The proposed word alignment-based evaluation method is more intuitive and interpretable than existing approaches like smatch.method 90%
• The proposed method avoids the NP-hard search problem inherent in smatch.method 90%
• The proposed method, 'juːmæʧ', achieved an F1 score of 25.05 when evaluated on English UMR files converted from dependency trees, compared to smatch's 32.39 and AnCast's 18.46.result 95%
• The percentage of unaligned nodes in UMR 2.0 varies significantly across languages, ranging from 10% in Czech to 53% in Sanapana.result 90%
• A script implementing the proposed word alignment-based evaluation method is freely available.technical 95%
• Varying alignment strategies across different annotation projects, annotators, and languages can introduce inconsistencies that affect evaluation and cross-linguistic comparison.limitation 85%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card