# TSQA Adapter for ChatTime

This folder adapts ChatTime to the **Time-MQA TSQA** multi-task time-series QA
benchmark (`/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp`). It is the TSQA analog
of `rats40k_adapter` and is meant to fill in the **ChatTime TSQA baseline**.

The dataset has four source groups with free-text answers:
`anomaly_detection`, `classification`, `forecasting`, `open_ended` (the
`open_ended` group mixes `multiple_choice`, `true_false` and open questions).

## How ChatTime is fed

ChatTime has no chat template, so prompts use ChatTime's native analysis
template (`utils.prompt.getPrompt`):

* `### Instruction` = the TSQA `question`, kept **verbatim** (including any
  numbers embedded in the text, so forecasting keeps its absolute scale).
* `### Input` = the `time_series` field discretised + serialised into ChatTime's
  native time-series tokens (`###value###`).
* `### Response` = the free-text `answer` (the SFT target).

Some context-enhanced forecasting questions are much longer than ChatTime's
4096-token context (e.g. an earnings-call transcript with a tiny series).
Prompts are therefore **left-truncated** — keeping the serialized series, the
trailing ask and (in training) the full response — instead of failing.

## Metrics

Evaluation reuses the **canonical Time-MQA evaluator**
(`MQA/data_utils.compute_group_metrics`), so the numbers are directly comparable
to the other TSQA baselines (MQA / ITFormer). Per group:

* `anomaly_detection`: accuracy (normal vs anomaly label parsing)
* `classification`: accuracy (class-label parsing)
* `forecasting`: MSE / MAE over parsed forecast values
* `open_ended`: accuracy (multiple-choice / true-false / semantic / numeric)

`eval_tsqa.py` writes `predictions.jsonl` and `metrics.json` (with a `by_group`
block) under the eval output directory.

## Required inputs

- `MODEL_PATH`: local ChatTime model dir. Default
  `/mnt/share01/sqk/models/ChatTime-1-7B-Chat`.
- `PYTHON_BIN`: Python executable. Defaults to
  `/dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python` (override if your env
  lives elsewhere, e.g. `/home/suiqk/anaconda3/envs/scalerag-ts-v4/bin/python`).
- `MQA_DIR`: defaults to `/mnt/share01/sqk/MQA` (provides the shared evaluator).
- `DATA_ROOT`: defaults to `/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp`.

The defaults target the **4× V100** machine: FP16 LoRA with `LOAD_IN_4BIT=0`,
`PER_DEVICE_TRAIN_BATCH_SIZE=1`, `GRADIENT_ACCUMULATION_STEPS=16`, the 4-GPU
accelerate config at `/mnt/share01/sqk/ITFormer/accelerate_config.yaml`.
Generation defaults: `MAX_NEW_TOKENS=256`, `MAX_INPUT_TOKENS=3840`,
`MAX_SEQ_LENGTH=4096`.

## SFT + Eval

```bash
cd /mnt/share01/sqk/ChatTime
MODEL_PATH=/mnt/share01/sqk/models/ChatTime-1-7B-Chat \
bash tsqa_adapter/run_sft_4gpu.sh
```

## Smoke test

```bash
cd /mnt/share01/sqk/ChatTime
MAX_TRAIN_SAMPLES=128 MAX_EVAL_SAMPLES=64 bash tsqa_adapter/run_sft_4gpu.sh
```

Outputs land in `tsqa_adapter/outputs/sft_<RUN_ID>/{adapter,eval}` and logs in
`tsqa_adapter/logs/`.