Macro-F1 Score

Gold definitionUpdated Apr 2, 2026

Definition

Macro-F1 Score is an evaluation metric that calculates the F1-score for each class independently and then averages them, giving equal weight to each class. It is particularly useful for assessing model performance on imbalanced datasets, as it prevents dominant classes from skewing the overall score.

At a glance

Executive summary

Macro-F1 Score is a metric used to evaluate how well a machine learning model performs across different categories, especially when some categories have much less data than others. It calculates a performance score for each category separately and then averages them, ensuring that rare categories are not overlooked.

TL;DR

Macro-F1 Score measures a model's accuracy and completeness for each type of output equally, which is great for datasets where some output types are rare.

Key points

Calculates F1-score for each class independently and then averages them.
Provides a robust evaluation metric for classification tasks with imbalanced datasets, preventing majority classes from dominating the score.
Used by researchers and ML engineers in NLP (e.g., hallucination detection, sentiment analysis), computer vision (e.g., object detection), and medical diagnosis.
Unlike micro-F1, which aggregates true positives, false positives, and false negatives globally before calculating F1, macro-F1 calculates F1 per class and then averages, giving equal weight to each class.
Increasingly used in benchmarks for complex, multi-class tasks like hallucination detection in LLMs, especially for low-resource languages, to ensure comprehensive model evaluation.

Use cases

Evaluating models for hallucination detection in LLMs, as seen in the ViHallu Challenge, to ensure robust performance across different hallucination types.
Assessing the performance of medical diagnosis models for rare diseases, where a high Macro-F1 ensures critical rare conditions are identified accurately.
Evaluating fraud detection systems to ensure effective identification of various, often rare, types of fraudulent activities.
Measuring model performance in sentiment analysis across positive, negative, and neutral sentiments, particularly if one sentiment category is less frequent.
Benchmarking multi-label image classification models to ensure balanced performance across all object categories, including less common ones.

Also known as

Macro F1, Macro-averaged F1, Macro-F1-score