Benchmarks — dataset sources

This directory holds the third-party time-series benchmarks used to evaluate TS-Align against peer models, together with our own OOS test set. The table below documents the source of every dataset except TS_Caption_test (that one is our own — see the repo root / results/ts-align/ README).

Dataset	Source	What it is
TSQA	Time-MQA — arXiv:2503.01875 · HF: Time-MQA (ACL 2025 Findings)	The TSQA dataset from Time-MQA: ~200k QA pairs over 12 real-world domains × 5 task types (forecasting, imputation, anomaly detection, classification, open-ended reasoning). We use the open-ended reasoning split (true/false, multiple-choice, open-ended).
ChatTS_test	ChatTS — arXiv:2412.03104 · GitHub: NetManAIOps/ChatTS · data: Zenodo record `14349206` (VLDB 2025, Tsinghua × ByteDance)	The official `dataset_a` / `dataset_b` evaluation sets released with ChatTS (aligning time series with LLMs via synthetic data).
MTBench	MTBench — arXiv:2503.16858 · HF: GGLabYale/MTBench_* (Yale GGLab)	A multimodal TS benchmark pairing time series with text: finance (news + stock movements) and weather (reports + temperature). Tasks: forecasting, trend analysis, news-driven QA. Used here as an OOD / cross-domain probe.
TimeSeriesExam	TimeSeriesExam — arXiv:2410.14752 (Cai, Choudhry, Goswami, Dubrawski — Auton Lab, CMU; NeurIPS 2024 workshop)	A configurable multiple-choice exam of time-series understanding across 5 categories: pattern recognition, noise, similarity, anomaly detection, causality. Encoder-only style, no textual leakage.

Not third-party

TS_Caption_test/ — our own OOS test set (metric-QA + captioning), de-duplicated. Source/provenance documented in the project's TS_Caption dataset release, not here.
TS_Caption_test_duped/ — the pre-de-duplication version of the above, kept locally for reference only; git-ignored, not published.

Notes

Each dataset is consumed through its benchmark_eval/adapters/<name>_adapter.py (requests built locally, data path passed via CLI). See the bundle run.sh / _infer/ for the exact invocation.
MTBench caveat: most finance/weather QA tasks embed long news text → they are largely text questions; only weather_rain is near-pure time series.