Translation-Aware Contamination Detection

Gold definitionUpdated Apr 2, 2026

Definition

Translation-Aware Contamination Detection is a method designed to identify data contamination in Large Language Models across multilingual contexts. It extends existing techniques like Tested Slot Guessing with choice-reordering and integrates Min-K% probability analysis to capture subtle behavioral and distributional signals, even when translation suppresses conventional indicators.

At a glance

Executive summary

Translation-Aware Contamination Detection is a new way to find hidden cheating in AI language models, especially when they're used with multiple languages. It helps uncover when a model has secretly memorized test answers, even if those answers were translated, ensuring that we can trust how well these models truly understand and generate language.

TL;DR

It's a method to catch AI language models that secretly memorize test answers, even when the tests are translated into different languages.

Key points

Extends Tested Slot Guessing with choice-reordering and integrates Min-K% probability analysis to detect behavioral and distributional contamination signals.
Addresses the blind spot of multilingual data contamination in LLMs, where translation can mask conventional detection indicators.
Used by researchers and ML engineers evaluating and developing robust, generalizable Large Language Models, especially for multilingual applications.
Unlike prior methods limited to English, this approach specifically tackles contamination in multilingual settings where translation suppresses traditional signals.
Focuses on ensuring evaluation integrity and trustworthiness of LLMs, particularly as models become more multilingual and widely deployed.

Use cases

Benchmarking Multilingual LLMs: Ensuring the validity of evaluation benchmarks for models like mBERT or XLM-R by detecting if they've seen translated test data.
Pre-training Data Curation: Identifying and removing contaminated samples from large, diverse multilingual datasets used for pre-training foundation models.
Fine-tuning Quality Control: Verifying that LLMs fine-tuned on domain-specific multilingual data are genuinely learning and not just memorizing translated examples.
Responsible AI Development: Supporting the development of fair and robust AI by ensuring that models' reported performance truly reflects their generalization capabilities across languages.

Also known as

TACD, Multilingual Contamination Detection, Cross-lingual Contamination Detection