RATS_UNI_ADAPTATION.md

# Adapting AnomSeer to the Time-RA RATs-Uni (univariate) dataset

This documents how AnomSeer (TimerPO) is adapted to train / evaluate on the
**Time-RA RATs-Uni** univariate dataset
(`RATs40K/RATs-Uni-TSImage_Reason.json`).

## Why an adaptation is needed

AnomSeer's native task is **anomaly localization (intervals) + classification +
explanation**, and ~70% of its reward comes from interval-overlap /
affiliation metrics (see `verl/utils/reward_score/anol.py`).

The RATs-Uni dataset provides, per univariate series:

| Field | Meaning |
|-------|---------|
| `Observation` | raw series (list of floats, length 32 / 64 / 128) |
| `FigurePath`  | rendered plot, `figures/{train,test}/{idx}.jpg` |
| `ActionID` / `Action` | anomaly class, **15-way** (0 = normal) |
| `Label` | `Normal` / `Anomaly` |
| `Thought` | expert chain-of-thought explanation |

It has **no anomaly intervals**. We therefore retarget AnomSeer to a
**classification + reasoning** task over the 15-class taxonomy and drop the
localization term. Everything else in the TimerPO pipeline (GRPO, the OT-based
expert-CoT semantic-alignment advantage) is kept intact — the dataset's
`Thought` is reused as the expert CoT (`numtext`) that TimerPO aligns against.

## What was added (no existing files behaviorally changed)

| File | Purpose |
|------|---------|
| `multimodal_data_processing/rats_uni.py` | Convert RATs-Uni JSON → AnomSeer parquet |
| `verl/utils/reward_score/rats.py` | 15-class classification + format reward |
| `verl/utils/reward_score/__init__.py` | Route `data_source == "timeseries_rats"` to `rats.compute_score` (one `elif` added) |
| `example/timerpo_trainer/run_anomseer_rats.sh` | Train / eval launcher pointed at the RATs-Uni parquet |

### Parquet row schema (produced by the converter)

```text
data_source  : "timeseries_rats"
prompt       : [{"role": "user", "content": "<image>\n ... <class> ..."}]
images       : [{"bytes": <jpg bytes>, "path": <abs path>}]
ability      : "time_series_anomaly_detection"
reward_model : {"style": "rule", "ground_truth": <canonical class name>}
extra_info   : {index, category, source, split, anomaly_type, action_id,
                label, series_length, image_path, expcot, numtext}
```

`extra_info.numtext` (= the dataset `Thought`) is **required** by TimerPO's
`compute_hidden_states_of_hint` (`verl/workers/fsdp_workers.py`), which is called
unconditionally in the GRPO training loop.

### Reward (`rats.compute_score`)

```text
final = 0.8 * class_acc    # exact 15-class match
      + 0.1 * binary_acc   # normal-vs-anomaly correct (partial credit)
      + 0.1 * fmt_score    # <think>...</think> present AND a parseable <class>
```

The class parser accepts `<class>Sudden Spike Anomaly</class>`, a bare id
(`<class>12</class>`), short aliases (`spike`), JSON (`{"ActionID": 12}`), and a
raw `ActionID: 12` line. The return signature matches the existing
`AnomalyTimeSeriesReward` manager, so per-class metrics (`class acc`) and the
validation path keep working unchanged.

## How to run

1) Prepare data (run inside the `anomseer` env, or any env with
`pandas`+`pyarrow`+`Pillow`):

```bash
python multimodal_data_processing/rats_uni.py \
    --json_path /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json \
    --out_dir   ./data/rats_uni_processed
# -> ./data/rats_uni_processed/{train_full,test_full}.parquet
```

Coverage: train = 30,266 usable; test = 6,034 usable (11 null JSON entries are
skipped; 1,552 test entries whose `FigurePath` points at a non-existent `.pdf`
are recovered via the same-index `.jpg`).

2) Train (4 GPUs by default):

```bash
bash example/timerpo_trainer/run_anomseer_rats.sh
# pick the backbone with MODEL_PATH=Qwen/Qwen2.5-VL-3B-Instruct (or 7B)
```

3) Evaluate (val only):

```bash
EVAL=True bash example/timerpo_trainer/run_anomseer_rats.sh
```

Override data paths without editing the script via `TRAIN_FILE` / `VAL_FILE`.

## LoRA fine-tuning (optional)

By default training is **full-parameter** (the validated path; this older veRL
fork has no native RL-path LoRA). A `lora_rank` hyperparameter was added to switch
the **actor** to LoRA; the reference policy stays the full base model.

```bash
# LoRA training on 2 GPUs (rank 16). LR auto-bumps to 1e-4; override with LR=.
LORA_RANK=16 STAGE=train bash example/timerpo_trainer/run_rats_2gpu.sh

# tune the adapter
LORA_RANK=32 LORA_ALPHA=64 LORA_DROPOUT=0.05 \
LORA_TARGET_MODULES="q_proj,k_proj,v_proj,o_proj" \
    bash example/timerpo_trainer/run_rats_2gpu.sh
```

What it touches (all guarded by `lora_rank>0`; `lora_rank=0` is byte-for-byte the
old full-FT path):

| File | Change |
|------|--------|
| `verl/trainer/config/ppo_trainer.yaml` | `actor_rollout_ref.model.{lora_rank,lora_alpha,lora_dropout,lora_target_modules}` |
| `verl/workers/fsdp_workers.py` | Wrap actor with PEFT, `use_orig_params=True`, `is_lora` wrap policy, force `hf` rollout sync |
| `verl/workers/sharding_manager/fsdp_vllm.py` | Merge LoRA into base + rename to HF keys before the vLLM weight sync |
| `example/timerpo_trainer/run_rats_2gpu.sh` | `LORA_RANK` / `LORA_ALPHA` / `LORA_DROPOUT` / `LORA_TARGET_MODULES` / auto-`LR` |

**vLLM sync**: vLLM can't load PEFT-named/adapter weights, so on every rollout the
sharding manager gathers full params and pushes merged `W + (α/r)·B·A` weights
under standard HF names. This merge was unit-tested to match PEFT's own
`merge_and_unload()` bit-for-bit (max|Δ|=0) and to reproduce the base model's
exact parameter names.

> ⚠️ **Experimental — not yet verified on GPU.** The merge math and key mapping are
> tested, but the full FSDP + PEFT + vLLM integration on Qwen2.5-VL has not been run
> on hardware. Two known caveats:
> - **Offline checkpoint eval**: `scripts/model_merger.py` expects base-model weights,
>   so the saved LoRA checkpoint won't merge cleanly offline. Use the in-training
>   validation (`test_freq`) for LoRA, or export with `peft` separately. (Full-FT
>   `STAGE=train_eval` is unaffected.)
> - **target modules**: PEFT `all-linear` errors on Qwen2.5-VL (it tries to wrap whole
>   `Qwen2_5_VLVisionBlock` modules), so the default is the explicit list
>   `q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj` (LLM attn+MLP, plus the
>   vision MLP which shares those names). Narrow to `q_proj,k_proj,v_proj,o_proj` for
>   attention-only.
> - **parallelism**: `TP=2` by default (tensor-parallel — splits vLLM's 7B copy across
>   both cards). GRPO also keeps the actor and a reference policy (both 7B, FSDP-sharded
>   across the 2 cards regardless of `TP`) plus vLLM's own 7B copy, so `TP=2` is the safer
>   choice on 48 GB even for LoRA. Set `TP=1` for data-parallel rollout only if you have
>   headroom.

## Evaluation — Time-RA-comparable metrics (`eval_rats_uni.py`)

A standalone vLLM eval that reports the **same metrics as Time-RA / ITFormer** for
direct comparison (the metric computation is copied verbatim from
`ITFormer/inference_rats40k.py::compute_rats40k_metrics`, so the numbers line up):

```text
num_samples, num_valid, num_invalid, valid_rate,
binary_{accuracy,precision_macro,recall_macro,f1_macro},   # Normal vs Anomaly
type_{accuracy,precision_macro,recall_macro,f1_macro},     # 15-class, labels 0-14
thought_rouge_l, thought_bleu                              # reasoning vs GT Thought
```

`--model` is a HF model directory. **Zero-shot baseline** (run now):

```bash
python eval_rats_uni.py \
    --model /mnt/share01/sqk/models/Qwen2.5-VL-3B-Instruct \
    --data  /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json \
    --out   ./eval_results/rats_uni_base.json --tp 2
```

Metric口径 (matches ITFormer): classification metrics are over all test samples,
unparseable predictions count as label `-1` (always wrong) and lower `valid_rate`;
type macro-P/R/F1 use labels 0-14; `thought_rouge_l` = rougeL fmeasure, `thought_bleu`
= nltk corpus BLEU (method-1 smoothing). The `<class>`/`<think>` parser tolerates the
doubled-tag output the model sometimes emits.

**Evaluating the trained LoRA model**: the checkpoint is FSDP-sharded PEFT, so first
merge it to a HF model (reuse the repo's now-working merge:
`verl/workers/sharding_manager/fsdp_vllm.py::_merge_peft_state_dict` +
`_convert_to_hf_checkpoint_keys`), then point `--model` at the merged dir. (A turnkey
`merge_lora_ckpt.py` can be added once a checkpoint exists to test against.)

## 2-GPU launcher defaults (`run_rats_2gpu.sh`)

Configured for this box (2× RTX 6000 Ada, 48 GB each, no NVLink):

- `PYTHON_BIN=/home/suiqk/anaconda3/envs/scalerag-ts-v4/bin/python`
- `MODEL_PATH=/mnt/share01/sqk/models/qwen2.5-vl-7b-instruct` (7B)
- `TP=2` (vLLM splits the 7B across both cards), `GPU_MEM_UTIL=0.4`, `MICRO_BSZ=2`

**7B memory reality on 2×48 GB**: full-parameter fine-tuning will not fit without
offload. Two working recipes:

```bash
# (a) recommended — LoRA (comfortable on 48 GB):
LORA_RANK=16 bash example/timerpo_trainer/run_rats_2gpu.sh

# (b) full-FT with CPU offload (slower; uses the box's 251 GB RAM):
PARAM_OFFLOAD=True OPTIM_OFFLOAD=True bash example/timerpo_trainer/run_rats_2gpu.sh
```

> The env has vLLM **0.8.5** → veRL uses its native SPMD path (`vllm_version=None`),
> so the LoRA weight sync goes through `model.load_weights` (guarded so gathered
> full tensors and sharded DTensors both work). Ensure ray/vLLM/peft/flash-attn are
> importable in `scalerag-ts-v4` (verified: torch 2.6, vLLM 0.8.5, peft 0.14,
> flash-attn 2.7.1, ray 2.55).

## Notes / knobs

- **Taxonomy**: full 15-class Time-RA taxonomy (not AnomSeer's 5-class). To
  change reward weights, edit the `final_score` line in
  `verl/utils/reward_score/rats.py`.
- **Prompt length**: the 15-class prompt + an 800×300 figure is ≈ 0.4k tokens,
  comfortably under `data.max_prompt_length=1024`.
- **Localization**: intentionally dropped (no GT intervals in the data). If GT
  intervals are added later, switch `data_source` back to `timeseries_anol`.