QuitoBench: A High-Quality Open Time Series Forecasting Benchmark

Dimensions overall score 5.0

• QuitoBench is a benchmark for time series forecasting that covers eight trend x seasonality x forecastability (TSF) regimes.method 95%
  Evidencepartial

  we introduce QuitoBench, a regime-balanced benchmark for time series forecasting with coverage across eight trend×seasonality×forecastability (TSF) regimes

  Implicationpartial

  This is a core definition of the benchmark presented in the abstract and introduction.

  Verificationpartial

  partial

• QuitoBench is built upon Quito, a billion-scale time series corpus of application traffic from Alipay.method 95%
  Evidencepartial

  The benchmark is built upon Quito, a billion-scale time series corpus of application traffic from Alipay

  Implicationpartial

  This describes the data source for the benchmark, explicitly stated in the abstract.

  Verificationpartial

  partial

• Deep learning models lead in time series forecasting at short context lengths (L=96), while foundation models dominate at long context lengths (L >= 576).result 90%
  Evidencepartial

  a context-length crossover where deep learning models lead at short context (L=96) but foundation models dominate at long context (L ≥ 576)

  Implicationpartial

  This is one of the four key findings reported in the abstract and elaborated in the results section.

  Verificationpartial

  partial

• Forecastability is the dominant driver of difficulty in time series forecasting, leading to a 3.64x MAE gap across regimes.result 90%
  Evidencepartial

  forecastability is the dominant difficulty driver, producing a 3.64 × MAE gap across regimes

  Implicationpartial

  This is another key finding explicitly stated in the abstract with a quantitative measure.

  Verificationpartial

  partial

• Deep learning models achieve comparable or superior performance to foundation models with 59 times fewer parameters.result 90%
  Evidencepartial

  deep learning models match or surpass foundation models at 59 × fewer parameters

  Implicationpartial

  This is a key finding regarding model efficiency, clearly stated in the abstract.

  Verificationpartial

  partial

• Scaling the amount of training data provides greater performance benefits than scaling model size for both deep learning and foundation models.result 90%
  Evidencepartial

  scaling the amount of training data provides substantially greater benefit than scaling model size for both model families

  Implicationpartial

  This is a key finding about the impact of data and model scaling, explicitly stated in the abstract.

  Verificationpartial

  partial

• Existing time series benchmarks suffer from coarse categorization by application domain, which does not systematically predict forecasting difficulty.limitation 85%
  Evidencepartial

  existing benchmarks group series by application domain (e.g., electricity, traffic, weather), yet there is no systematic justification for why these domain labels should predict forecasting difficulty.

  Implicationpartial

  This is presented as a weakness of prior benchmarks, motivating the need for QuitoBench.

  Verificationpartial

  partial

• QuitoBench is a benchmark for time series forecasting that covers eight trend x seasonality x forecastability (TSF) regimes.method 95%
  Evidencepartial

  we introduce QuitoBench, a regime-balanced benchmark for time series forecasting with coverage across eight trend×seasonality×forecastability (TSF) regimes

  Implicationpartial

  The abstract explicitly states the benchmark's coverage of TSF regimes.

  Verificationpartial

  partial

• QuitoBench is built upon Quito, a billion-scale time series corpus of application traffic from Alipay.method 95%
  Evidencepartial

  The benchmark is built upon Quito, a billion-scale time series corpus of application traffic from Alipay

  Implicationpartial

  The abstract clearly states the origin and scale of the benchmark's data.

  Verificationpartial

  partial

• Deep learning models lead in time series forecasting at short context lengths (L=96), while foundation models dominate at long context lengths (L >= 576).result 90%
  Evidencepartial

  a context-length crossover where deep learning models lead at short context (L=96) but foundation models dominate at long context (L ≥ 576)

  Implicationpartial

  This is presented as a key finding with specific context lengths mentioned.

  Verificationpartial

  partial

• Forecastability is the dominant driver of difficulty in time series forecasting, leading to a 3.64x MAE gap across regimes.result 90%
  Evidencepartial

  forecastability is the dominant difficulty driver, producing a 3.64 × MAE gap across regimes

  Implicationpartial

  This is stated as a key finding with a specific quantitative measure.

  Verificationpartial

  partial

• Deep learning models achieve comparable or superior performance to foundation models with 59 times fewer parameters.result 90%
  Evidencepartial

  deep learning models match or surpass foundation models at 59 × fewer parameters

  Implicationpartial

  This is presented as a key finding with a specific quantitative comparison.

  Verificationpartial

  partial

Competitive landscape

A new benchmark for time series forecasting that reveals performance trade-offs between deep learning and foundation models across different data regimes.