Hugging Face's logo

sdzt
/
forensics-grpo

Video-Text-to-Text
Safetensors
video
forensics
grpo
temporal-grounding
Model card Files
xet
 Community
forensics-grpo / code /tempsamp_r1 /README.md
sdzt's picture
sdzt
Add source code
33569f9 verified
preview code
|
Raw
History Blame Contribute Delete
2.96 kB

TempSamp-R1 (no-CoT) baseline on ActivityForensics

TempSamp-R1's official code is not released. The paper's algorithm, however, is a strict subset of what forensics_grpo already implements:

TempSamp-R1 component Where it lives in forensics_grpo
Mix-policy sampling (G−1 on-policy + 1 GT) src/open_r1/trainer/grpo_trainer_video_GT_soft.py lines ~509, 786-787
Non-linear reward shaping (paper Eq. 4) transform_rewards at grpo_trainer_video_GT_soft.py:692-704
Group-relative advantage (paper Eq. 2) grpo_trainer_video_GT_soft.py lines 1102-1106 (default branch)
IoU reward src/open_r1/reward.py::hungarian_iou_reward (multi-segment) / forensics_iou

So this baseline = forensics_grpo trainer + no-CoT prompt + IoU-only reward

  • all forensics-specific extras disabled. The whole baseline is therefore a ~60-line wrapper plus one launch script — no algorithm is re-implemented.

What this enforces

src/tempsamp_main.py sets FORENSICS_COT=false at import time, then asserts that none of these forensics-specific switches is enabled (they would diverge from the paper algorithm):

  • FORENSICS_DECOMP_ADV – set-level GT-segment advantage decomposition
  • FORENSICS_HUNGARIAN_DECOMP – per-PRED-segment token-level advantage
  • FORENSICS_CTA_GRPO – counterfactual token-level advantage
  • FORENSICS_FORGERY_AWARE – external forgery-head reward shaping
  • FORENSICS_USE_EXTERNAL_VERIFIER, FORENSICS_VERIFIER_CONTEXT

If any of these is true at startup, the script raises.

On the "use their IoU" requirement

TempSamp-R1's paper IoU is single-span (Charades-style). ActivityForensics is inherently multi-segment, so this baseline uses the natural multi-segment extension hungarian_iou_reward. The reward FORMULA is the same — set-IoU via Hungarian matching reduces to plain IoU when |GT|=|pred|=1. Algorithm (mix-policy + non-linear shaping) is unchanged.

If you want strict single-span IoU instead, switch --reward_funcs to iou and add a per-segment splitting step to the dataset (mirroring the data_forensics/ adapter in ../time_r1/).

Files

  • src/tempsamp_main.py entry point (locks no-CoT, asserts no extras)
  • scripts/run_tempsamp_no_cot.sh launch script
  • this README

Running 

cd /mnt/local-fast/zhangt/baselines/tempsamp_r1
# edit MODEL_PATH / ANNOT_DIR / VIDEO_ROOT / PREPROC_DIR inside the script first
bash scripts/run_tempsamp_no_cot.sh

FORENSICS_GRPO_ROOT (default /mnt/local-fast/zhangt/forensics_grpo) is on sys.path so the wrapper imports src.open_r1.grpo_forensics:main directly.