Dimensions overall score 7.0

• Existing evaluation methods for LLM-generated time series explanations are limited because reference-based metrics require ground truth explanations and traditional time series methods cannot assess free-form textual reasoning.limitation 95%
• No general purpose method exists to directly verify whether a time series explanation is faithful to underlying data without predefined references or task-specific rules.limitation 90%
• The paper proposes using LLMs as evaluators of time series explanations in a reference-free setting, assigning ternary correctness labels based on pattern identification, numeric accuracy, and answer faithfulness.method 92%
• The researchers constructed a synthetic benchmark of 350 time series cases across seven query types, each paired with correct, partially correct, and incorrect explanations.method 93%
• LLM performance on explanation generation is highly pattern dependent with systematic failures on certain query types, with accuracies ranging from 0.00 to 0.12 for Seasonal Drop and Volatility Shift, to 0.94 to 0.96 for Structural Break.result 94%
• LLMs are more stable as evaluators than as generators, correctly ranking and scoring explanations even when their own generated outputs are incorrect.result 91%
• The findings demonstrate the feasibility of data-grounded LLM-based evaluation for time series explanations.result 88%
• LLMs have potential as reliable evaluators of data-grounded reasoning in the time series domain.result 85%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card