RATS_UNI_ADAPTATION.md

Adapting AnomSeer to the Time-RA RATs-Uni (univariate) dataset

This documents how AnomSeer (TimerPO) is adapted to train / evaluate on the Time-RA RATs-Uni univariate dataset (RATs40K/RATs-Uni-TSImage_Reason.json).

Why an adaptation is needed

AnomSeer's native task is anomaly localization (intervals) + classification + explanation, and ~70% of its reward comes from interval-overlap / affiliation metrics (see verl/utils/reward_score/anol.py).

The RATs-Uni dataset provides, per univariate series:

Field	Meaning
Observation	raw series (list of floats, length 32 / 64 / 128)
FigurePath	rendered plot, figures/{train,test}/{idx}.jpg
ActionID / Action	anomaly class, 15-way (0 = normal)
Label	Normal / Anomaly
Thought	expert chain-of-thought explanation

It has no anomaly intervals. We therefore retarget AnomSeer to a classification + reasoning task over the 15-class taxonomy and drop the localization term. Everything else in the TimerPO pipeline (GRPO, the OT-based expert-CoT semantic-alignment advantage) is kept intact — the dataset's Thought is reused as the expert CoT (numtext) that TimerPO aligns against.

What was added (no existing files behaviorally changed)

File	Purpose
multimodal_data_processing/rats_uni.py	Convert RATs-Uni JSON → AnomSeer parquet
verl/utils/reward_score/rats.py	15-class classification + format reward
verl/utils/reward_score/__init__.py	Route data_source == "timeseries_rats" to rats.compute_score (one elif added)
example/timerpo_trainer/run_anomseer_rats.sh	Train / eval launcher pointed at the RATs-Uni parquet

Parquet row schema (produced by the converter)

data_source  : "timeseries_rats"
prompt       : [{"role": "user", "content": "<image>\n ... <class> ..."}]
images       : [{"bytes": <jpg bytes>, "path": <abs path>}]
ability      : "time_series_anomaly_detection"
reward_model : {"style": "rule", "ground_truth": <canonical class name>}
extra_info   : {index, category, source, split, anomaly_type, action_id,
                label, series_length, image_path, expcot, numtext}

extra_info.numtext (= the dataset Thought) is required by TimerPO's compute_hidden_states_of_hint (verl/workers/fsdp_workers.py), which is called unconditionally in the GRPO training loop.

Reward (rats.compute_score)

final = 0.8 * class_acc    # exact 15-class match
      + 0.1 * binary_acc   # normal-vs-anomaly correct (partial credit)
      + 0.1 * fmt_score    # <think>...</think> present AND a parseable <class>

The class parser accepts <class>Sudden Spike Anomaly</class>, a bare id (<class>12</class>), short aliases (spike), JSON ({"ActionID": 12}), and a raw ActionID: 12 line. The return signature matches the existing AnomalyTimeSeriesReward manager, so per-class metrics (class acc) and the validation path keep working unchanged.

How to run

1. Prepare data (run inside the anomseer env, or any env with pandas+pyarrow+Pillow):

python multimodal_data_processing/rats_uni.py \
    --json_path /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json \
    --out_dir   ./data/rats_uni_processed
# -> ./data/rats_uni_processed/{train_full,test_full}.parquet

Coverage: train = 30,266 usable; test = 6,034 usable (11 null JSON entries are skipped; 1,552 test entries whose FigurePath points at a non-existent .pdf are recovered via the same-index .jpg).

2. Train (4 GPUs by default):

bash example/timerpo_trainer/run_anomseer_rats.sh
# pick the backbone with MODEL_PATH=Qwen/Qwen2.5-VL-3B-Instruct (or 7B)

3. Evaluate (val only):

EVAL=True bash example/timerpo_trainer/run_anomseer_rats.sh

Override data paths without editing the script via TRAIN_FILE / VAL_FILE.

LoRA fine-tuning (optional)

By default training is full-parameter (the validated path; this older veRL fork has no native RL-path LoRA). A lora_rank hyperparameter was added to switch the actor to LoRA; the reference policy stays the full base model.

# LoRA training on 2 GPUs (rank 16). LR auto-bumps to 1e-4; override with LR=.
LORA_RANK=16 STAGE=train bash example/timerpo_trainer/run_rats_2gpu.sh

# tune the adapter
LORA_RANK=32 LORA_ALPHA=64 LORA_DROPOUT=0.05 \
LORA_TARGET_MODULES="q_proj,k_proj,v_proj,o_proj" \
    bash example/timerpo_trainer/run_rats_2gpu.sh

What it touches (all guarded by lora_rank>0; lora_rank=0 is byte-for-byte the old full-FT path):

File	Change
verl/trainer/config/ppo_trainer.yaml	actor_rollout_ref.model.{lora_rank,lora_alpha,lora_dropout,lora_target_modules}
verl/workers/fsdp_workers.py	Wrap actor with PEFT, use_orig_params=True, is_lora wrap policy, force hf rollout sync
verl/workers/sharding_manager/fsdp_vllm.py	Merge LoRA into base + rename to HF keys before the vLLM weight sync
example/timerpo_trainer/run_rats_2gpu.sh	LORA_RANK / LORA_ALPHA / LORA_DROPOUT / LORA_TARGET_MODULES / auto-LR

vLLM sync: vLLM can't load PEFT-named/adapter weights, so on every rollout the sharding manager gathers full params and pushes merged W + (α/r)·B·A weights under standard HF names. This merge was unit-tested to match PEFT's own merge_and_unload() bit-for-bit (max|Δ|=0) and to reproduce the base model's exact parameter names.

⚠️ Experimental — not yet verified on GPU. The merge math and key mapping are tested, but the full FSDP + PEFT + vLLM integration on Qwen2.5-VL has not been run on hardware. Two known caveats:

• Offline checkpoint eval: scripts/model_merger.py expects base-model weights, so the saved LoRA checkpoint won't merge cleanly offline. Use the in-training validation (test_freq) for LoRA, or export with peft separately. (Full-FT STAGE=train_eval is unaffected.)
• target modules: PEFT all-linear errors on Qwen2.5-VL (it tries to wrap whole Qwen2_5_VLVisionBlock modules), so the default is the explicit list q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj (LLM attn+MLP, plus the vision MLP which shares those names). Narrow to q_proj,k_proj,v_proj,o_proj for attention-only.
• parallelism: TP=2 by default (tensor-parallel — splits vLLM's 7B copy across both cards). GRPO also keeps the actor and a reference policy (both 7B, FSDP-sharded across the 2 cards regardless of TP) plus vLLM's own 7B copy, so TP=2 is the safer choice on 48 GB even for LoRA. Set TP=1 for data-parallel rollout only if you have headroom.

Evaluation — Time-RA-comparable metrics (eval_rats_uni.py)

A standalone vLLM eval that reports the same metrics as Time-RA / ITFormer for direct comparison (the metric computation is copied verbatim from ITFormer/inference_rats40k.py::compute_rats40k_metrics, so the numbers line up):

num_samples, num_valid, num_invalid, valid_rate,
binary_{accuracy,precision_macro,recall_macro,f1_macro},   # Normal vs Anomaly
type_{accuracy,precision_macro,recall_macro,f1_macro},     # 15-class, labels 0-14
thought_rouge_l, thought_bleu                              # reasoning vs GT Thought

--model is a HF model directory. Zero-shot baseline (run now):

python eval_rats_uni.py \
    --model /mnt/share01/sqk/models/Qwen2.5-VL-3B-Instruct \
    --data  /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json \
    --out   ./eval_results/rats_uni_base.json --tp 2

Metric口径 (matches ITFormer): classification metrics are over all test samples, unparseable predictions count as label -1 (always wrong) and lower valid_rate; type macro-P/R/F1 use labels 0-14; thought_rouge_l = rougeL fmeasure, thought_bleu = nltk corpus BLEU (method-1 smoothing). The <class>/<think> parser tolerates the doubled-tag output the model sometimes emits.

Evaluating the trained LoRA model: the checkpoint is FSDP-sharded PEFT, so first merge it to a HF model (reuse the repo's now-working merge: verl/workers/sharding_manager/fsdp_vllm.py::_merge_peft_state_dict + _convert_to_hf_checkpoint_keys), then point --model at the merged dir. (A turnkey merge_lora_ckpt.py can be added once a checkpoint exists to test against.)

2-GPU launcher defaults (run_rats_2gpu.sh)

Configured for this box (2× RTX 6000 Ada, 48 GB each, no NVLink):

• PYTHON_BIN=/home/suiqk/anaconda3/envs/scalerag-ts-v4/bin/python
• MODEL_PATH=/mnt/share01/sqk/models/qwen2.5-vl-7b-instruct (7B)
• TP=2 (vLLM splits the 7B across both cards), GPU_MEM_UTIL=0.4, MICRO_BSZ=2

7B memory reality on 2×48 GB: full-parameter fine-tuning will not fit without offload. Two working recipes:

# (a) recommended — LoRA (comfortable on 48 GB):
LORA_RANK=16 bash example/timerpo_trainer/run_rats_2gpu.sh

# (b) full-FT with CPU offload (slower; uses the box's 251 GB RAM):
PARAM_OFFLOAD=True OPTIM_OFFLOAD=True bash example/timerpo_trainer/run_rats_2gpu.sh

The env has vLLM 0.8.5 → veRL uses its native SPMD path (vllm_version=None), so the LoRA weight sync goes through model.load_weights (guarded so gathered full tensors and sharded DTensors both work). Ensure ray/vLLM/peft/flash-attn are importable in scalerag-ts-v4 (verified: torch 2.6, vLLM 0.8.5, peft 0.14, flash-attn 2.7.1, ray 2.55).

Notes / knobs

• Taxonomy: full 15-class Time-RA taxonomy (not AnomSeer's 5-class). To change reward weights, edit the final_score line in verl/utils/reward_score/rats.py.
• Prompt length: the 15-class prompt + an 800×300 figure is ≈ 0.4k tokens, comfortably under data.max_prompt_length=1024.
• Localization: intentionally dropped (no GT intervals in the data). If GT intervals are added later, switch data_source back to timeseries_anol.