Remove accidental run and build artifacts

code/orarl.egg-info/PKG-INFO

@@ -1,230 +0,0 @@
-Metadata-Version: 2.4
-Name: orarl
-Version: 0.1.0
-Summary: Annotations as rollouts for unified video MLLM reinforcement learning
-Author: Yunheng Li, Guohong Mu, Hao Li, Shengsheng Qian, Dingwen Zhang, Qibin Hou, Ming-Ming Cheng
-License-Expression: Apache-2.0
-Project-URL: Homepage, https://orarl.github.io/
-Project-URL: Documentation, https://orarl.github.io/
-Keywords: reinforcement-learning,video-understanding,multimodal,grpo
-Classifier: Development Status :: 3 - Alpha
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-License-File: NOTICE
-Requires-Dist: numpy
-Requires-Dist: PyYAML
-Requires-Dist: torch
-Provides-Extra: test
-Requires-Dist: pytest; extra == "test"
-Provides-Extra: lint
-Requires-Dist: ruff; extra == "lint"
-Provides-Extra: hf
-Requires-Dist: huggingface_hub; extra == "hf"
-Dynamic: license-file
-
-<p align="right"><a href="README_zh.md">简体中文</a></p>
-
-<div align="center">
-
-# OraRL
-
-### Annotations as Rollouts
-
-**Efficient and scalable reinforcement learning for unified video MLLMs**
-
-Yunheng Li · Guohong Mu · Hao Li · Shengsheng Qian · Dingwen Zhang ·
-Qibin Hou · Ming-Ming Cheng
-
-<p>
-  <a href="https://arxiv.org/abs/2608.20492">📄 Paper</a>
-  &nbsp;&nbsp;·&nbsp;&nbsp;
-  <a href="https://orarl.github.io/">🌐 Project Page</a>
-  &nbsp;&nbsp;·&nbsp;&nbsp;
-  <a href="#models">🤗 Models (4B / 9B)</a>
-</p>
-<p>
-  <a href="docs/environment.md">⚙️ Environment</a>
-  &nbsp;&nbsp;·&nbsp;&nbsp;
-  <a href="docs/training.md">🚀 Training</a>
-  &nbsp;&nbsp;·&nbsp;&nbsp;
-  <a href="docs/evaluation.md">📊 Evaluation</a>
-  &nbsp;&nbsp;·&nbsp;&nbsp;
-  <a href="LICENSE">⚖️ License</a>
-</p>
-
-<a href="https://orarl.github.io/assets/orarl-teaser.mp4">
-  <img src="assets/orarl-hero.gif"
-       alt="Animated OraRL method preview" width="92%">
-</a>
-
-**▶ Click the image to watch the 1:38 project overview.**
-
-</div>
-
-## Why OraRL
-
-- **Annotation-as-rollout:** annotations become reliable positive rollouts while
-  policy samples retain an on-policy baseline.
-- **Seven task families:** one update rule covers temporal and spatial grounding,
-  segmentation, tracking, spatial-temporal grounding, video QA, and spatial
-  intelligence.
-- **Efficient training (4B):** sign-balanced pruning delivers **1.48× faster
-  updates** (**92.5 → 62.4 s/step**) while reducing peak per-GPU memory from
-  **62.4 to 50.9 GB**.
-- **Efficient inference:** on one H20 with vLLM in BF16, weight loading occupies
-  **8.6 GiB (4B)** and **17.6 GiB (9B)**. On ten-minute, 2-fps videos,
-  answer-only decoding cuts median post-TTFT latency from **4.78 s to 130 ms**
-  and total latency from **29.03 to 24.30 s**.
-- **Multimodal veRL infrastructure:** a unified video contract carries cached
-  artifacts, raw paths, or inline frame tensors through vLLM rollouts and FSDP
-  updates, with decode-once frame reuse, temporal metadata, task-grouped
-  batching, asynchronous Ray rewards, and safe hybrid-engine cache handling.
-
-## OraRL in One Update
-
-<p align="center">
-  <img src="assets/orarl-method.gif"
-       alt="Animated OraRL framework" width="96%">
-</p>
-
-An OraRL update separates reliable annotation guidance from on-policy
-normalization:
-
-1. **Build the group:** append one serialized annotation rollout to the policy
-   samples generated for the same prompt.
-2. **Keep the baseline on-policy:** estimate the group baseline from policy
-   rewards only.
-3. **Guide and select:** convert the annotation-policy reward gap into a
-   correction, then retain a sign-balanced subset for the update.
-
-This design uses task-native annotations directly and requires no
-chain-of-thought supervision or decoding.
-
-## Video-ORA Results
-
-<p align="center">
-  <img src="assets/paper-results.png"
-       alt="Video-ORA-9B results across seven task families" width="100%">
-</p>
-
-### Dataset-Level Results
-
-<picture>
-  <source media="(prefers-color-scheme: dark)"
-          srcset="assets/video_ora_benchmark_matrix_dark.svg">
-  <source media="(prefers-color-scheme: light)"
-          srcset="assets/video_ora_benchmark_matrix_light.svg">
-  <img src="assets/video_ora_benchmark_matrix_light.svg"
-       alt="Dataset-level benchmark matrix comparing Video-ORA with multimodal baselines"
-       width="100%">
-</picture>
-
-Video-ORA-9B leads the matched seven-family comparison without CoT decoding.
-Best and second-best values are highlighted per row; `†` denotes an
-original-report value whose frame, prompt, split, or decoding settings may
-differ. Averages require complete family coverage.
-
-<!-- <details>
-<summary>Benchmark sources</summary>
-
-Unmarked values come from Tables 1–8 and Appendix Table 20 of the latest
-[OraRL paper](https://arxiv.org/abs/2608.20492). External entries follow the original
-[LLaVA-OneVision-2](https://arxiv.org/abs/2605.25979),
-[VideoChat3](https://github.com/MCG-NJU/VideoChat3), and
-[OneThinker](https://arxiv.org/abs/2512.03043) reports. OneThinker is cited only
-as the source of public Qwen3-VL scores. ReVSI uses each model's reported frame
-setting; the paper's three-benchmark spatial-intelligence average excludes it.
-
-</details> -->
-
-### Model Scaling
-
-<p align="center">
-  <img src="assets/orarl-model-scaling.gif"
-       alt="Animated Video-ORA model scaling from 0.8B to 9B" width="100%">
-</p>
-
-### Data Scaling
-
-<p align="center">
-  <img src="assets/orarl-data-scaling.gif"
-       alt="Animated OraRL data scaling and reward dynamics" width="100%">
-</p>
-
-## Models
-
-| Model | Backbone | Released recipe | Weights |
-| --- | --- | --- | --- |
-| **Video-ORA-9B** | Qwen3.5-9B | `orarl_9b.yaml` | [Hugging Face](https://huggingface.co/OraRL/Video-ORA-9B) |
-| **Video-ORA-4B** | Qwen3.5-4B | `orarl_4b.yaml` | Hugging Face (coming soon) |
-
-### vLLM Serving
-
-Both Video-ORA checkpoints load directly with **vLLM 0.19.1** for
-OpenAI-compatible serving:
-
-```bash
-MODEL=OraRL/Video-ORA-9B
-
-vllm serve "$MODEL" \
-  --served-model-name Video-ORA-9B \
-  --trust-remote-code \
-  --dtype bfloat16 \
-  --tensor-parallel-size 1 \
-  --max-model-len 131072 \
-  --limit-mm-per-prompt '{"image": 1, "video": 1}'
-```
-
-Set `--tensor-parallel-size` to the GPU count for multi-GPU deployment and
-lower `--max-model-len` on smaller-memory devices. Use
-`enable_thinking=false` in the chat template for answer-only inference.
-
-## Use OraRL
-
-The release is organized around three user-facing workflows:
-
-1. **[Environment](docs/environment.md):** install the pinned CUDA stack that
-   covers both the bundled trainer and the evaluators.
-2. **[Training](docs/training.md):** prepare licensed local training data and
-   launch GRPO or OraRL on one or multiple nodes.
-3. **[Evaluation](docs/evaluation.md):** download Video-ORA and OraRL-Data,
-   then run a smoke test or the complete paper suite.
-
-Training and evaluation are dry runs by default; inspect the resolved command
-before adding `--run`. Checkpoints and evaluation media are hosted under the
-[OraRL Hugging Face organization](https://huggingface.co/OraRL).
-
-## Acknowledgements
-
-OraRL is built on [veRL](https://github.com/volcengine/verl) — a
-high-performance RL framework with HybridEngine. We thank its authors and
-contributors for open-sourcing the training infrastructure.
-
-## License
-
-OraRL source is released under [Apache-2.0](LICENSE). Datasets, models,
-benchmarks, and optional dependencies retain their original licenses; see
-[NOTICE](NOTICE).
-
-## Citation
-
-If you find OraRL useful, please consider giving this repository a ⭐ and
-citing our [paper](https://arxiv.org/abs/2608.20492).
-
-```bibtex
-@article{li2026orarl,
-  title   = {Annotations as Rollouts: Efficient and Scalable
-             Reinforcement Learning for Video MLLMs},
-  author  = {Li, Yunheng and Mu, Guohong and Li, Hao and
-             Qian, Shengsheng and Zhang, Dingwen and Hou, Qibin
-             and Cheng, Ming-Ming},
-  journal = {arXiv preprint arXiv:2608.20492},
-  year    = {2026},
-  url     = {https://arxiv.org/abs/2608.20492}
-}
-```

code/orarl.egg-info/SOURCES.txt

@@ -1,196 +0,0 @@
-LICENSE
-MANIFEST.in
-NOTICE
-README.md
-environment.yml
-pyproject.toml
-requirements-cu129.txt
-configs/data_sources.example.yaml
-configs/grpo_4b.yaml
-configs/grpo_9b.yaml
-configs/orarl_4b.yaml
-configs/orarl_9b.yaml
-data/eval/README.md
-data/eval/datasets.jsonl
-docs/environment.md
-docs/environment_zh.md
-docs/evaluation.md
-docs/evaluation_zh.md
-docs/training.md
-docs/training_zh.md
-eval/README.md
-eval/task/canonical_data.py
-eval/task/eval.sh
-eval/task/eval_image_mc_vllm.py
-eval/task/eval_prompt.py
-eval/task/eval_revsi_multinode.sh
-eval/task/eval_vllm.py
-eval/task/qwenvl_decord_patch.py
-eval/task/mindcube/data_utils.py
-eval/task/mmsi/eval_mmsi_transformers.py
-eval/task/mmsi/run_eval_transformers.sh
-eval/task/revsi/eval_revsi_vllm.py
-eval/task/revsi/merge_multinode_shards.py
-eval/task/revsi/run_eval_vllm.sh
-eval/task/segmentation/eval_seg_vllm.py
-eval/task/segmentation/post_sam2.py
-eval/task/segmentation/run_eval_vllm.sh
-eval/task/spatial_grounding/_grounding_utils.py
-eval/task/spatial_grounding/eval_refcoco_vllm.py
-eval/task/spatial_temporal_grounding/eval_stvg_vllm.py
-eval/task/temporal_grounding/eval_timelens_hf.py
-eval/task/temporal_grounding/run_eval.sh
-eval/task/tracking/eval_tracking_vllm.py
-orarl/__init__.py
-orarl/resources.py
-orarl.egg-info/PKG-INFO
-orarl.egg-info/SOURCES.txt
-orarl.egg-info/dependency_links.txt
-orarl.egg-info/entry_points.txt
-orarl.egg-info/requires.txt
-orarl.egg-info/top_level.txt
-orarl/algorithm/__init__.py
-orarl/algorithm/_utils.py
-orarl/algorithm/advantages.py
-orarl/algorithm/config.py
-orarl/algorithm/correction.py
-orarl/algorithm/selection.py
-orarl/cli/__init__.py
-orarl/cli/eval_data.py
-orarl/cli/evaluate.py
-orarl/cli/prepare.py
-orarl/cli/train.py
-orarl/data/__init__.py
-orarl/data/build.py
-orarl/data/identity.py
-orarl/data/schema.py
-orarl/evaluation/__init__.py
-orarl/evaluation/card.py
-orarl/evaluation/converters.py
-orarl/evaluation/hub.py
-orarl/evaluation/layout.py
-orarl/evaluation/manifest.py
-orarl/evaluation/schema.py
-orarl/evaluation/sources.py
-orarl/evaluation/staging.py
-orarl/rewards/__init__.py
-orarl/rewards/router.py
-orarl/rewards/types.py
-orarl/rewards/adapters/__init__.py
-orarl/rewards/adapters/_common.py
-orarl/rewards/adapters/segmentation.py
-orarl/rewards/adapters/spatial_grounding.py
-orarl/rewards/adapters/spatial_intelligence.py
-orarl/rewards/adapters/spatial_temporal_grounding.py
-orarl/rewards/adapters/temporal_grounding.py
-orarl/rewards/adapters/tracking.py
-orarl/rewards/adapters/video_qa.py
-scripts/check_environment.py
-scripts/check_release.py
-scripts/create_conda_env.sh
-scripts/create_eval_source_manifest.py
-scripts/evaluate.sh
-scripts/finalize_eval_index.py
-scripts/install_conda_runtime_hook.sh
-scripts/launch_multinode.sh
-scripts/model_merger.py
-scripts/organize_evaluation_outputs.py
-scripts/prepare_data.sh
-scripts/smoke_evaluation.sh
-scripts/smoke_training.sh
-scripts/train_grpo.sh
-scripts/train_orarl.sh
-tests/test_algorithm_advantages.py
-tests/test_algorithm_correction.py
-tests/test_algorithm_selection.py
-tests/test_builtin_reward_adapters.py
-tests/test_cli_evaluate.py
-tests/test_cli_metadata.py
-tests/test_cli_train.py
-tests/test_data_build.py
-tests/test_data_schema.py
-tests/test_environment_spec.py
-tests/test_eval_canonical_data.py
-tests/test_eval_source_discovery.py
-tests/test_evaluation_manifest.py
-tests/test_evaluation_release.py
-tests/test_evaluation_runtime.py
-tests/test_evaluation_schema.py
-tests/test_evaluation_staging.py
-tests/test_organize_evaluation_outputs.py
-tests/test_release_hygiene.py
-tests/test_resources.py
-tests/test_rewards.py
-tests/test_runtime_reward_entrypoint.py
-tests/test_training_jsonl_dataset.py
-tests/test_training_runtime.py
-verl/__init__.py
-verl/protocol.py
-verl/models/__init__.py
-verl/models/monkey_patch.py
-verl/models/transformers/__init__.py
-verl/models/transformers/flash_attention_utils.py
-verl/models/transformers/qwen2_vl.py
-verl/models/transformers/qwen3_5.py
-verl/models/transformers/qwen3_vl.py
-verl/single_controller/__init__.py
-verl/single_controller/base/__init__.py
-verl/single_controller/base/decorator.py
-verl/single_controller/base/worker.py
-verl/single_controller/base/worker_group.py
-verl/single_controller/base/register_center/__init__.py
-verl/single_controller/base/register_center/ray.py
-verl/single_controller/ray/__init__.py
-verl/single_controller/ray/base.py
-verl/trainer/__init__.py
-verl/trainer/config.py
-verl/trainer/core_algos.py
-verl/trainer/data_loader.py
-verl/trainer/main.py
-verl/trainer/metrics.py
-verl/trainer/orarl_config.py
-verl/trainer/orarl_post_selection.py
-verl/trainer/orarl_selection.py
-verl/trainer/ray_trainer.py
-verl/utils/__init__.py
-verl/utils/dataset.py
-verl/utils/flops_counter.py
-verl/utils/fsdp_utils.py
-verl/utils/model_utils.py
-verl/utils/multimodal_contract.py
-verl/utils/prompt_template.py
-verl/utils/py_functional.py
-verl/utils/seqlen_balancing.py
-verl/utils/tokenizer.py
-verl/utils/torch_dtypes.py
-verl/utils/torch_functional.py
-verl/utils/ulysses.py
-verl/utils/checkpoint/__init__.py
-verl/utils/checkpoint/checkpoint_manager.py
-verl/utils/checkpoint/fsdp_checkpoint_manager.py
-verl/utils/logger/__init__.py
-verl/utils/logger/gen_logger.py
-verl/utils/logger/logger.py
-verl/workers/__init__.py
-verl/workers/config.py
-verl/workers/fsdp_workers.py
-verl/workers/actor/__init__.py
-verl/workers/actor/base.py
-verl/workers/actor/config.py
-verl/workers/actor/dp_actor.py
-verl/workers/critic/__init__.py
-verl/workers/critic/base.py
-verl/workers/critic/config.py
-verl/workers/critic/dp_critic.py
-verl/workers/reward/__init__.py
-verl/workers/reward/config.py
-verl/workers/reward/function.py
-verl/workers/rollout/__init__.py
-verl/workers/rollout/base.py
-verl/workers/rollout/config.py
-verl/workers/rollout/hf_rollout.py
-verl/workers/rollout/vllm_rollout_spmd.py
-verl/workers/sharding_manager/__init__.py
-verl/workers/sharding_manager/base.py
-verl/workers/sharding_manager/fsdp_ulysses.py
-verl/workers/sharding_manager/fsdp_vllm.py

code/orarl.egg-info/dependency_links.txt

@@ -1 +0,0 @@
-

code/orarl.egg-info/entry_points.txt

@@ -1,5 +0,0 @@
-[console_scripts]
-orarl-eval = orarl.cli.evaluate:main
-orarl-eval-data = orarl.cli.eval_data:main
-orarl-prepare = orarl.cli.prepare:main
-orarl-train = orarl.cli.train:main

code/orarl.egg-info/requires.txt

@@ -1,12 +0,0 @@
-numpy
-PyYAML
-torch
-
-[hf]
-huggingface_hub
-
-[lint]
-ruff
-
-[test]
-pytest

code/orarl.egg-info/top_level.txt

@@ -1,2 +0,0 @@
-orarl
-verl

code/runs/smoke-training-9b-20260825_174304/smoke.log

@@ -1,467 +0,0 @@
-OraRL root:       /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official
-Python:           /data/miniconda3/envs/orarl/bin/python
-Model:            /apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B
-Train data:       /mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl
-Val data:         /mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl
-Recipe scale:     9b
-World size:       1 x 8
-Steps per run:    1
-Results:          /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_174304
-Execute:          1
-0, NVIDIA H20, 0 MiB, 97871 MiB
-1, NVIDIA H20, 0 MiB, 97871 MiB
-2, NVIDIA H20, 0 MiB, 97871 MiB
-3, NVIDIA H20, 0 MiB, 97871 MiB
-4, NVIDIA H20, 0 MiB, 97871 MiB
-5, NVIDIA H20, 0 MiB, 97871 MiB
-6, NVIDIA H20, 0 MiB, 97871 MiB
-7, NVIDIA H20, 0 MiB, 97871 MiB
-/data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-  warnings.warn(
-dependency OK: numpy 2.2.6
-dependency OK: torch 2.10.0+cu129
-dependency OK: transformers 5.5.4
-dependency OK: vllm 0.19.1
-dependency OK: ray 2.54.0
-dependency OK: tensordict 0.12.2
-dependency OK: codetiming 1.4.0
-dependency OK: omegaconf 2.3.0
-dependency OK: verl.trainer.main /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py
-dependency OK: orarl.rewards /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/orarl/rewards/__init__.py
-
-=== grpo smoke (1 update(s)) ===
-/data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-  warnings.warn(
-2026-08-25 17:43:40,058	INFO worker.py:2004 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265 
-/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
-  warnings.warn(
-(pid=1944439) W0825 17:43:56.478000 1944439 site-packages/torch/utils/cpp_extension.py:118] No CUDA runtime is found, using CUDA_HOME='/data/miniconda3/envs/orarl'
-(Runner pid=1944439) {
-(Runner pid=1944439)   "data": {
-(Runner pid=1944439)     "train_files": "/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl",
-(Runner pid=1944439)     "val_files": "/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl",
-(Runner pid=1944439)     "prompt_key": "problem",
-(Runner pid=1944439)     "answer_key": "answer",
-(Runner pid=1944439)     "image_key": "images",
-(Runner pid=1944439)     "video_key": "videos",
-(Runner pid=1944439)     "image_dir": "/",
-(Runner pid=1944439)     "video_fps": 2.0,
-(Runner pid=1944439)     "video_max_frames": 128,
-(Runner pid=1944439)     "max_prompt_length": 24576,
-(Runner pid=1944439)     "max_response_length": 4096,
-(Runner pid=1944439)     "rollout_batch_size": 8,
-(Runner pid=1944439)     "mini_rollout_batch_size": null,
-(Runner pid=1944439)     "val_batch_size": 16,
-(Runner pid=1944439)     "format_prompt": null,
-(Runner pid=1944439)     "override_chat_template": null,
-(Runner pid=1944439)     "enable_thinking": false,
-(Runner pid=1944439)     "response_prefix": "",
-(Runner pid=1944439)     "shuffle": true,
-(Runner pid=1944439)     "seed": 42,
-(Runner pid=1944439)     "min_pixels": 4096,
-(Runner pid=1944439)     "max_pixels": 4194304,
-(Runner pid=1944439)     "image_min_pixels": 4096,
-(Runner pid=1944439)     "image_max_pixels": 1048576,
-(Runner pid=1944439)     "video_min_pixels": 4096,
-(Runner pid=1944439)     "video_max_pixels": 786432,
-(Runner pid=1944439)     "video_total_pixels": 8388608,
-(Runner pid=1944439)     "val_video_fps": 2.0,
-(Runner pid=1944439)     "val_video_max_frames": 128,
-(Runner pid=1944439)     "val_video_min_pixels": 4096,
-(Runner pid=1944439)     "val_video_max_pixels": 786432,
-(Runner pid=1944439)     "val_video_total_pixels": 8388608,
-(Runner pid=1944439)     "filter_overlong_prompts": false,
-(Runner pid=1944439)     "filter_overlong_prompts_workers": 16,
-(Runner pid=1944439)     "use_preprocessed_videos": false,
-(Runner pid=1944439)     "video_source_mode": "realtime_only",
-(Runner pid=1944439)     "preprocessed_video_dir": null,
-(Runner pid=1944439)     "val_preprocessed_video_dir": null,
-(Runner pid=1944439)     "val_video_source_mode": "realtime_only",
-(Runner pid=1944439)     "inline_video_tensors": true,
-(Runner pid=1944439)     "group_by_task": true,
-(Runner pid=1944439)     "group_by_task_key": "problem_type",
-(Runner pid=1944439)     "dataloader_num_workers": 1
-(Runner pid=1944439)   },
-(Runner pid=1944439)   "worker": {
-(Runner pid=1944439)     "hybrid_engine": true,
-(Runner pid=1944439)     "actor": {
-(Runner pid=1944439)       "strategy": "fsdp",
-(Runner pid=1944439)       "global_batch_size": 8,
-(Runner pid=1944439)       "micro_batch_size_per_device_for_update": 1,
-(Runner pid=1944439)       "micro_batch_size_per_device_for_experience": 1,
-(Runner pid=1944439)       "max_grad_norm": 1.0,
-(Runner pid=1944439)       "clip_ratio_low": 0.2,
-(Runner pid=1944439)       "clip_ratio_high": 0.2,
-(Runner pid=1944439)       "clip_ratio_dual": 3.0,
-(Runner pid=1944439)       "loss_avg_mode": "token",
-(Runner pid=1944439)       "loss_type": "default",
-(Runner pid=1944439)       "ppo_epochs": 1,
-(Runner pid=1944439)       "padding_free": true,
-(Runner pid=1944439)       "dynamic_batching": true,
-(Runner pid=1944439)       "max_token_len_per_gpu": 32768,
-(Runner pid=1944439)       "ulysses_size": 1,
-(Runner pid=1944439)       "use_torch_compile": false,
-(Runner pid=1944439)       "model": {
-(Runner pid=1944439)         "model_path": "/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B",
-(Runner pid=1944439)         "tokenizer_path": "/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B",
-(Runner pid=1944439)         "override_config": {},
-(Runner pid=1944439)         "enable_gradient_checkpointing": true,
-(Runner pid=1944439)         "trust_remote_code": false,
-(Runner pid=1944439)         "freeze_vision_tower": true,
-(Runner pid=1944439)         "train_vision_merger": false
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "optim": {
-(Runner pid=1944439)         "lr": 1e-06,
-(Runner pid=1944439)         "betas": [
-(Runner pid=1944439)           0.9,
-(Runner pid=1944439)           0.999
-(Runner pid=1944439)         ],
-(Runner pid=1944439)         "weight_decay": 0.0,
-(Runner pid=1944439)         "strategy": "adamw",
-(Runner pid=1944439)         "lr_warmup_ratio": 0.0,
-(Runner pid=1944439)         "lr_warmup_steps": null,
-(Runner pid=1944439)         "min_lr_ratio": null,
-(Runner pid=1944439)         "lr_scheduler_type": "constant",
-(Runner pid=1944439)         "training_steps": -1
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "fsdp": {
-(Runner pid=1944439)         "enable_full_shard": true,
-(Runner pid=1944439)         "enable_cpu_offload": false,
-(Runner pid=1944439)         "enable_rank0_init": true,
-(Runner pid=1944439)         "use_orig_params": false,
-(Runner pid=1944439)         "torch_dtype": "bf16",
-(Runner pid=1944439)         "fsdp_size": -1,
-(Runner pid=1944439)         "mp_param_dtype": "bf16",
-(Runner pid=1944439)         "mp_reduce_dtype": "fp32",
-(Runner pid=1944439)         "mp_buffer_dtype": "fp32"
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "offload": {
-(Runner pid=1944439)         "offload_params": false,
-(Runner pid=1944439)         "offload_optimizer": false
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "global_batch_size_per_device": -1,
-(Runner pid=1944439)       "disable_kl": true,
-(Runner pid=1944439)       "use_kl_loss": false,
-(Runner pid=1944439)       "kl_penalty": "low_var_kl",
-(Runner pid=1944439)       "kl_coef": 0.0
-(Runner pid=1944439)     },
-(Runner pid=1944439)     "critic": {
-(Runner pid=1944439)       "strategy": "fsdp",
-(Runner pid=1944439)       "global_batch_size": 256,
-(Runner pid=1944439)       "micro_batch_size_per_device_for_update": 4,
-(Runner pid=1944439)       "micro_batch_size_per_device_for_experience": 16,
-(Runner pid=1944439)       "max_grad_norm": 1.0,
-(Runner pid=1944439)       "cliprange_value": 0.5,
-(Runner pid=1944439)       "loss_avg_mode": "token",
-(Runner pid=1944439)       "ppo_epochs": 1,
-(Runner pid=1944439)       "padding_free": false,
-(Runner pid=1944439)       "dynamic_batching": true,
-(Runner pid=1944439)       "max_token_len_per_gpu": null,
-(Runner pid=1944439)       "ulysses_size": 1,
-(Runner pid=1944439)       "model": {
-(Runner pid=1944439)         "model_path": null,
-(Runner pid=1944439)         "tokenizer_path": null,
-(Runner pid=1944439)         "override_config": {},
-(Runner pid=1944439)         "enable_gradient_checkpointing": true,
-(Runner pid=1944439)         "trust_remote_code": true,
-(Runner pid=1944439)         "freeze_vision_tower": false,
-(Runner pid=1944439)         "train_vision_merger": false
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "optim": {
-(Runner pid=1944439)         "lr": 1e-06,
-(Runner pid=1944439)         "betas": [
-(Runner pid=1944439)           0.9,(Runner pid=1944439) The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-(Runner pid=1944439) 
-Traceback (most recent call last):
-  File "<frozen runpy>", line 198, in _run_module_as_main
-  File "<frozen runpy>", line 88, in _run_code
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py", line 137, in <module>
-    main()
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py", line 129, in main
-    ray.get(runner.run.remote(ppo_config))
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
-    return fn(*args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
-    return func(*args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/worker.py", line 2981, in get
-    values, debugger_breakpoint = worker.get_objects(
-                                  ^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/worker.py", line 1012, in get_objects
-    raise value.as_instanceof_cause()
-ray.exceptions.RayTaskError(DatasetGenerationError): ray::Runner.run() (pid=1944439, ip=29.127.69.4, actor_id=e981ae3ef4ecbc78705cfeea01000000, repr=<main.Runner object at 0x7f3a507bda10>)
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/packaged_modules/json/json.py", line 265, in _generate_tables
-    self._cast_table(pa_table, json_field_paths=json_field_paths),
-    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/packaged_modules/json/json.py", line 120, in _cast_table
-    pa_table = table_cast(pa_table, self.info.features.arrow_schema)
-               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/table.py", line 2272, in table_cast
-    return cast_table_to_schema(table, schema)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/table.py", line 2218, in cast_table_to_schema
-    raise CastError(
-datasets.table.CastError: Couldn't cast
-problem: string
-answer: string
-ground_truth: string
-problem_type: string
-videos: list<item: string>
-  child 0, item: string
-data_type: string
-task_source: string
-segmentation_output: string
-meta: string
-images: list<item: string>
-  child 0, item: string
-fps: int64
-video_second: double
-resolution: struct<width: int64, height: int64>
-  child 0, width: int64
-  child 1, height: int64
-path: string
-data_source: string
-messages: list<item: struct<role: string, content: string>>
-  child 0, item: struct<role: string, content: string>
-      child 0, role: string
-      child 1, content: string
-solution: string
-difficulty: double
-pred_iou: double
-pred_span: list<item: double>
-  child 0, item: double
-pred_text: string
-checkpoint_score: double
-hardness_band: string
-selection_role: string
-selection_checkpoint: string
-identity_key: string
-media_anchors: list<item: string>
-  child 0, item: string
-selection_temporal_bucket: string
-question_type: string
-source_file: string
-source_id: string
-selection_weak_buckets: list<item: string>
-  child 0, item: string
-selection_bucket: string
-source_block: string
-selection_scores: struct<spatial_relation: int64, object_relation: int64, ego_exo_match: int64, relative_direction: in (... 100 chars omitted)
-  child 0, spatial_relation: int64
-  child 1, object_relation: int64
-  child 2, ego_exo_match: int64
-  child 3, relative_direction: int64
-  child 4, multi_step_scene: int64
-  child 5, cardinal_scene: int64
-  child 6, temporal_order: int64
-  child 7, viewpoint_motion: int64
-has_offline_trajectory: bool
-offline_output: string
-bbox_area: double
-problem_id: int64
-question_type_orig: string
-unit_orig: string
-unit_is_eval: bool
-scene_name: string
-preprocessed_video: string
-to
-{'problem': Value('string'), 'answer': Value('string'), 'ground_truth': Value('string'), 'problem_type': Value('string'), 'data_type': Value('string'), 'task_source': Value('string'), 'videos': List(Value('string')), 'images': List(Value('string')), 'preprocessed_video': Value('string'), 'segmentation_output': Json(decode=True), 'meta': Json(decode=True), 'fps': Value('int64'), 'video_second': Value('float64'), 'resolution': {'width': Value('int64'), 'height': Value('int64')}, 'path': Value('string'), 'data_source': Value('string')}
-because column names don't match
-
-The above exception was the direct cause of the following exception:
-
-ray::Runner.run() (pid=1944439, ip=29.127.69.4, actor_id=e981ae3ef4ecbc78705cfeea01000000, repr=<main.Runner object at 0x7f3a507bda10>)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py", line 74, in run
-    train_dataloader, val_dataloader = create_dataloader(
-                                       ^^^^^^^^^^^^^^^^^^
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/data_loader.py", line 41, in create_dataloader
-    train_dataset = RLHFDataset(
-                    ^^^^^^^^^^^^
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/utils/dataset.py", line 444, in __init__
-    self.dataset = load_dataset(file_type, data_files=data_path, split=data_split)
-                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/load.py", line 1708, in load_dataset
-    builder_instance.download_and_prepare(
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/builder.py", line 884, in download_and_prepare
-    self._download_and_prepare(
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/builder.py", line 947, in _download_and_prepare
-    self._prepare_split(split_generator, **prepare_split_kwargs)
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/builder.py", line 1739, in _prepare_split
-    for job_id, done, content in self._prepare_split_single(
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/datasets/builder.py", line 1922, in _prepare_split_single
-    raise DatasetGenerationError("An error occurred while generating the dataset") from e
-datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset
-
-(Runner pid=1944439)           0.999
-(Runner pid=1944439)         ],
-(Runner pid=1944439)         "weight_decay": 0.01,
-(Runner pid=1944439)         "strategy": "adamw",
-(Runner pid=1944439)         "lr_warmup_ratio": 0.0,
-(Runner pid=1944439)         "lr_warmup_steps": null,
-(Runner pid=1944439)         "min_lr_ratio": null,
-(Runner pid=1944439)         "lr_scheduler_type": "constant",
-(Runner pid=1944439)         "training_steps": -1
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "fsdp": {
-(Runner pid=1944439)         "enable_full_shard": true,
-(Runner pid=1944439)         "enable_cpu_offload": false,
-(Runner pid=1944439)         "enable_rank0_init": true,
-(Runner pid=1944439)         "use_orig_params": false,
-(Runner pid=1944439)         "torch_dtype": null,
-(Runner pid=1944439)         "fsdp_size": -1,
-(Runner pid=1944439)         "mp_param_dtype": "bf16",
-(Runner pid=1944439)         "mp_reduce_dtype": "fp32",
-(Runner pid=1944439)         "mp_buffer_dtype": "fp32"
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "offload": {
-(Runner pid=1944439)         "offload_params": false,
-(Runner pid=1944439)         "offload_optimizer": false
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "global_batch_size_per_device": -1
-(Runner pid=1944439)     },
-(Runner pid=1944439)     "ref": {
-(Runner pid=1944439)       "strategy": "fsdp",
-(Runner pid=1944439)       "fsdp": {
-(Runner pid=1944439)         "enable_full_shard": true,
-(Runner pid=1944439)         "enable_cpu_offload": false,
-(Runner pid=1944439)         "enable_rank0_init": true,
-(Runner pid=1944439)         "use_orig_params": false,
-(Runner pid=1944439)         "torch_dtype": "bf16",
-(Runner pid=1944439)         "fsdp_size": -1,
-(Runner pid=1944439)         "mp_param_dtype": "bf16",
-(Runner pid=1944439)         "mp_reduce_dtype": "fp32",
-(Runner pid=1944439)         "mp_buffer_dtype": "fp32"
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "offload": {
-(Runner pid=1944439)         "offload_params": false,
-(Runner pid=1944439)         "offload_optimizer": false
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "micro_batch_size_per_device_for_experience": 1,
-(Runner pid=1944439)       "padding_free": true,
-(Runner pid=1944439)       "dynamic_batching": true,
-(Runner pid=1944439)       "max_token_len_per_gpu": 32768,
-(Runner pid=1944439)       "ulysses_size": 1,
-(Runner pid=1944439)       "use_torch_compile": false
-(Runner pid=1944439)     },
-(Runner pid=1944439)     "reward": {
-(Runner pid=1944439)       "reward_function": "orarl.rewards",
-(Runner pid=1944439)       "reward_function_kwargs": {},
-(Runner pid=1944439)       "skip_special_tokens": true,
-(Runner pid=1944439)       "num_cpus": 1,
-(Runner pid=1944439)       "reward_function_name": "compute_score",
-(Runner pid=1944439)       "reward_function_is_module": true
-(Runner pid=1944439)     },
-(Runner pid=1944439)     "rollout": {
-(Runner pid=1944439)       "name": "vllm",
-(Runner pid=1944439)       "n": 8,
-(Runner pid=1944439)       "temperature": 1.0,
-(Runner pid=1944439)       "top_p": 0.85,
-(Runner pid=1944439)       "top_k": -1,
-(Runner pid=1944439)       "seed": 42,
-(Runner pid=1944439)       "limit_images": 10,
-(Runner pid=1944439)       "dtype": "bf16",
-(Runner pid=1944439)       "gpu_memory_utilization": 0.45,
-(Runner pid=1944439)       "ignore_eos": false,
-(Runner pid=1944439)       "enforce_eager": false,
-(Runner pid=1944439)       "enable_chunked_prefill": true,
-(Runner pid=1944439)       "tensor_parallel_size": 1,
-(Runner pid=1944439)       "max_model_len": null,
-(Runner pid=1944439)       "max_num_batched_tokens": 32768,
-(Runner pid=1944439)       "disable_log_stats": true,
-(Runner pid=1944439)       "disable_tqdm": true,
-(Runner pid=1944439)       "val_override_config": {
-(Runner pid=1944439)         "temperature": 0.7,
-(Runner pid=1944439)         "top_p": 0.95,
-(Runner pid=1944439)         "n": 1
-(Runner pid=1944439)       },
-(Runner pid=1944439)       "kv_cache_dtype": "auto",
-(Runner pid=1944439)       "calculate_log_probs": false,
-(Runner pid=1944439)       "collect_seq_logprob_for_filter": false,
-(Runner pid=1944439)       "prompt_length": 24576,
-(Runner pid=1944439)       "response_length": 4096,
-(Runner pid=1944439)       "trust_remote_code": false
-(Runner pid=1944439)     }
-(Runner pid=1944439)   },
-(Runner pid=1944439)   "algorithm": {
-(Runner pid=1944439)     "name": "grpo",
-(Runner pid=1944439)     "gamma": 1.0,
-(Runner pid=1944439)     "lam": 1.0,
-(Runner pid=1944439)     "adv_estimator": "grpo",
-(Runner pid=1944439)     "scale_rewards": true,
-(Runner pid=1944439)     "disable_kl": true,
-(Runner pid=1944439)     "use_kl_loss": false,
-(Runner pid=1944439)     "kl_penalty": "low_var_kl",
-(Runner pid=1944439)     "kl_coef": 0.0,
-(Runner pid=1944439)     "kl_type": "fixed",
-(Runner pid=1944439)     "kl_horizon": 10000.0,
-(Runner pid=1944439)     "kl_target": 0.1
-(Runner pid=1944439)   },
-(Runner pid=1944439)   "trainer": {
-(Runner pid=1944439)     "total_epochs": 1,
-(Runner pid=1944439)     "max_steps": 1,
-(Runner pid=1944439)     "project_name": "OraRL",
-(Runner pid=1944439)     "experiment_name": "smoke-grpo-9b",
-(Runner pid=1944439)     "logger": [
-(Runner pid=1944439)       "console"
-(Runner pid=1944439)     ],
-(Runner pid=1944439)     "nnodes": 1,
-(Runner pid=1944439)     "n_gpus_per_node": 8,
-(Runner pid=1944439)     "critic_warmup": 0,
-(Runner pid=1944439)     "val_freq": -1,
-(Runner pid=1944439)     "val_before_train": false,
-(Runner pid=1944439)     "val_only": false,
-(Runner pid=1944439)     "val_generations_to_log": 0,
-(Runner pid=1944439)     "save_freq": 1,
-(Runner pid=1944439)     "save_limit": 3,
-(Runner pid=1944439)     "save_model_only": false,
-(Runner pid=1944439)     "keep_optim_only_latest": true,
-(Runner pid=1944439)     "save_checkpoint_path": "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_174304/grpo",
-(Runner pid=1944439)     "load_checkpoint_path": null,
-(Runner pid=1944439)     "ray_timeline": null,
-(Runner pid=1944439)     "find_last_checkpoint": false,
-(Runner pid=1944439)     "keep_best_train_ckpt": false,
-(Runner pid=1944439)     "best_train_metric_key": "reward/overall",
-(Runner pid=1944439)     "best_train_smooth_window": 5,
-(Runner pid=1944439)     "best_train_min_step": 10
-(Runner pid=1944439)   }
-(Runner pid=1944439) }
-(AutoRewardManager pid=1946036) Using reward function `compute_score` from `orarl.rewards`.
-(AutoRewardManager pid=1946036) Reward name: orarl, reward type: batch.
-(AutoRewardManager pid=1946038) Using reward function `compute_score` from `orarl.rewards`.
-(AutoRewardManager pid=1946038) Reward name: orarl, reward type: batch.
-method=grpo mode=run
-/data/miniconda3/envs/orarl/bin/python -m verl.trainer.main config=/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/configs/grpo_9b.yaml trainer.max_steps=1 trainer.save_freq=1 trainer.val_before_train=false trainer.experiment_name=smoke-grpo-9b data.rollout_batch_size=8 worker.actor.global_batch_size=8 worker.actor.model.model_path=/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B worker.actor.model.tokenizer_path=/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B data.train_files=/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl data.val_files=/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl trainer.save_checkpoint_path=/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_174304/grpo trainer.nnodes=1 trainer.n_gpus_per_node=8

code/runs/smoke-training-9b-20260825_175213/smoke.log

@@ -1,827 +0,0 @@
-OraRL root:       /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official
-Python:           /data/miniconda3/envs/orarl/bin/python
-Model:            /apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B
-Train data:       /mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl
-Val data:         /mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl
-Recipe scale:     9b
-World size:       1 x 8
-Steps per run:    1
-Results:          /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_175213
-Execute:          1
-0, NVIDIA H20, 0 MiB, 97871 MiB
-1, NVIDIA H20, 0 MiB, 97871 MiB
-2, NVIDIA H20, 0 MiB, 97871 MiB
-3, NVIDIA H20, 0 MiB, 97871 MiB
-4, NVIDIA H20, 0 MiB, 97871 MiB
-5, NVIDIA H20, 0 MiB, 97871 MiB
-6, NVIDIA H20, 0 MiB, 97871 MiB
-7, NVIDIA H20, 0 MiB, 97871 MiB
-/data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-  warnings.warn(
-dependency OK: numpy 2.2.6
-dependency OK: torch 2.10.0+cu129
-dependency OK: transformers 5.5.4
-dependency OK: vllm 0.19.1
-dependency OK: ray 2.54.0
-dependency OK: tensordict 0.12.2
-dependency OK: codetiming 1.4.0
-dependency OK: omegaconf 2.3.0
-dependency OK: verl.trainer.main /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py
-dependency OK: orarl.rewards /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/orarl/rewards/__init__.py
-
-=== grpo smoke (1 update(s)) ===
-/data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-  warnings.warn(
-2026-08-25 17:52:41,284	INFO worker.py:2004 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265 
-/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
-  warnings.warn(
-(pid=1984436) W0825 17:52:57.110000 1984436 site-packages/torch/utils/cpp_extension.py:118] No CUDA runtime is found, using CUDA_HOME='/data/miniconda3/envs/orarl'
-(Runner pid=1984436) {
-(Runner pid=1984436)   "data": {
-(Runner pid=1984436)     "train_files": "/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl",
-(Runner pid=1984436)     "val_files": "/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl",
-(Runner pid=1984436)     "prompt_key": "problem",
-(Runner pid=1984436)     "answer_key": "answer",
-(Runner pid=1984436)     "image_key": "images",
-(Runner pid=1984436)     "video_key": "videos",
-(Runner pid=1984436)     "image_dir": "/",
-(Runner pid=1984436)     "video_fps": 2.0,
-(Runner pid=1984436)     "video_max_frames": 128,
-(Runner pid=1984436)     "max_prompt_length": 24576,
-(Runner pid=1984436)     "max_response_length": 4096,
-(Runner pid=1984436)     "rollout_batch_size": 8,
-(Runner pid=1984436)     "mini_rollout_batch_size": null,
-(Runner pid=1984436)     "val_batch_size": 16,
-(Runner pid=1984436)     "format_prompt": null,
-(Runner pid=1984436)     "override_chat_template": null,
-(Runner pid=1984436)     "enable_thinking": false,
-(Runner pid=1984436)     "response_prefix": "",
-(Runner pid=1984436)     "shuffle": true,
-(Runner pid=1984436)     "seed": 42,
-(Runner pid=1984436)     "min_pixels": 4096,
-(Runner pid=1984436)     "max_pixels": 4194304,
-(Runner pid=1984436)     "image_min_pixels": 4096,
-(Runner pid=1984436)     "image_max_pixels": 1048576,
-(Runner pid=1984436)     "video_min_pixels": 4096,
-(Runner pid=1984436)     "video_max_pixels": 786432,
-(Runner pid=1984436)     "video_total_pixels": 8388608,
-(Runner pid=1984436)     "val_video_fps": 2.0,
-(Runner pid=1984436)     "val_video_max_frames": 128,
-(Runner pid=1984436)     "val_video_min_pixels": 4096,
-(Runner pid=1984436)     "val_video_max_pixels": 786432,
-(Runner pid=1984436)     "val_video_total_pixels": 8388608,
-(Runner pid=1984436)     "filter_overlong_prompts": false,
-(Runner pid=1984436)     "filter_overlong_prompts_workers": 16,
-(Runner pid=1984436)     "use_preprocessed_videos": false,
-(Runner pid=1984436)     "video_source_mode": "realtime_only",
-(Runner pid=1984436)     "preprocessed_video_dir": null,
-(Runner pid=1984436)     "val_preprocessed_video_dir": null,
-(Runner pid=1984436)     "val_video_source_mode": "realtime_only",
-(Runner pid=1984436)     "inline_video_tensors": true,
-(Runner pid=1984436)     "group_by_task": true,
-(Runner pid=1984436)     "group_by_task_key": "problem_type",
-(Runner pid=1984436)     "dataloader_num_workers": 1
-(Runner pid=1984436)   },
-(Runner pid=1984436)   "worker": {
-(Runner pid=1984436)     "hybrid_engine": true,
-(Runner pid=1984436)     "actor": {
-(Runner pid=1984436)       "strategy": "fsdp",
-(Runner pid=1984436)       "global_batch_size": 8,
-(Runner pid=1984436)       "micro_batch_size_per_device_for_update": 1,
-(Runner pid=1984436)       "micro_batch_size_per_device_for_experience": 1,
-(Runner pid=1984436)       "max_grad_norm": 1.0,
-(Runner pid=1984436)       "clip_ratio_low": 0.2,
-(Runner pid=1984436)       "clip_ratio_high": 0.2,
-(Runner pid=1984436)       "clip_ratio_dual": 3.0,
-(Runner pid=1984436)       "loss_avg_mode": "token",
-(Runner pid=1984436)       "loss_type": "default",
-(Runner pid=1984436)       "ppo_epochs": 1,
-(Runner pid=1984436)       "padding_free": true,
-(Runner pid=1984436)       "dynamic_batching": true,
-(Runner pid=1984436)       "max_token_len_per_gpu": 32768,
-(Runner pid=1984436)       "ulysses_size": 1,
-(Runner pid=1984436)       "use_torch_compile": false,
-(Runner pid=1984436)       "model": {
-(Runner pid=1984436)         "model_path": "/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B",
-(Runner pid=1984436)         "tokenizer_path": "/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B",
-(Runner pid=1984436)         "override_config": {},
-(Runner pid=1984436)         "enable_gradient_checkpointing": true,
-(Runner pid=1984436)         "trust_remote_code": false,
-(Runner pid=1984436)         "freeze_vision_tower": true,
-(Runner pid=1984436)         "train_vision_merger": false
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "optim": {
-(Runner pid=1984436)         "lr": 1e-06,
-(Runner pid=1984436)         "betas": [
-(Runner pid=1984436)           0.9,
-(Runner pid=1984436)           0.999
-(Runner pid=1984436)         ],
-(Runner pid=1984436)         "weight_decay": 0.0,
-(Runner pid=1984436)         "strategy": "adamw",
-(Runner pid=1984436)         "lr_warmup_ratio": 0.0,
-(Runner pid=1984436)         "lr_warmup_steps": null,
-(Runner pid=1984436)         "min_lr_ratio": null,
-(Runner pid=1984436)         "lr_scheduler_type": "constant",
-(Runner pid=1984436)         "training_steps": -1
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "fsdp": {
-(Runner pid=1984436)         "enable_full_shard": true,
-(Runner pid=1984436)         "enable_cpu_offload": false,
-(Runner pid=1984436)         "enable_rank0_init": true,
-(Runner pid=1984436)         "use_orig_params": false,
-(Runner pid=1984436)         "torch_dtype": "bf16",
-(Runner pid=1984436)         "fsdp_size": -1,
-(Runner pid=1984436)         "mp_param_dtype": "bf16",
-(Runner pid=1984436)         "mp_reduce_dtype": "fp32",
-(Runner pid=1984436)         "mp_buffer_dtype": "fp32"
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "offload": {
-(Runner pid=1984436)         "offload_params": false,
-(Runner pid=1984436)         "offload_optimizer": false
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "global_batch_size_per_device": -1,
-(Runner pid=1984436)       "disable_kl": true,
-(Runner pid=1984436)       "use_kl_loss": false,
-(Runner pid=1984436)       "kl_penalty": "low_var_kl",
-(Runner pid=1984436)       "kl_coef": 0.0
-(Runner pid=1984436)     },
-(Runner pid=1984436)     "critic": {
-(Runner pid=1984436)       "strategy": "fsdp",
-(Runner pid=1984436)       "global_batch_size": 256,
-(Runner pid=1984436)       "micro_batch_size_per_device_for_update": 4,
-(Runner pid=1984436)       "micro_batch_size_per_device_for_experience": 16,
-(Runner pid=1984436)       "max_grad_norm": 1.0,
-(Runner pid=1984436)       "cliprange_value": 0.5,
-(Runner pid=1984436)       "loss_avg_mode": "token",
-(Runner pid=1984436)       "ppo_epochs": 1,
-(Runner pid=1984436)       "padding_free": false,
-(Runner pid=1984436)       "dynamic_batching": true,
-(Runner pid=1984436)       "max_token_len_per_gpu": null,
-(Runner pid=1984436)       "ulysses_size": 1,
-(Runner pid=1984436)       "model": {
-(Runner pid=1984436)         "model_path": null,
-(Runner pid=1984436)         "tokenizer_path": null,
-(Runner pid=1984436)         "override_config": {},
-(Runner pid=1984436)         "enable_gradient_checkpointing": true,
-(Runner pid=1984436)         "trust_remote_code": true,
-(Runner pid=1984436)         "freeze_vision_tower": false,
-(Runner pid=1984436)         "train_vision_merger": false
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "optim": {
-(Runner pid=1984436)         "lr": 1e-06,
-(Runner pid=1984436)         "betas": [
-(Runner pid=1984436)           0.9,(Runner pid=1984436) The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
-
-(Runner pid=1984436)           0.999
-(Runner pid=1984436)         ],
-(Runner pid=1984436)         "weight_decay": 0.01,
-(Runner pid=1984436)         "strategy": "adamw",
-(Runner pid=1984436)         "lr_warmup_ratio": 0.0,
-(Runner pid=1984436)         "lr_warmup_steps": null,
-(Runner pid=1984436)         "min_lr_ratio": null,
-(Runner pid=1984436)         "lr_scheduler_type": "constant",
-(Runner pid=1984436)         "training_steps": -1
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "fsdp": {
-(Runner pid=1984436)         "enable_full_shard": true,
-(Runner pid=1984436)         "enable_cpu_offload": false,
-(Runner pid=1984436)         "enable_rank0_init": true,
-(Runner pid=1984436)         "use_orig_params": false,
-(Runner pid=1984436)         "torch_dtype": null,
-(Runner pid=1984436)         "fsdp_size": -1,
-(Runner pid=1984436)         "mp_param_dtype": "bf16",
-(Runner pid=1984436)         "mp_reduce_dtype": "fp32",
-(Runner pid=1984436)         "mp_buffer_dtype": "fp32"
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "offload": {
-(Runner pid=1984436)         "offload_params": false,
-(Runner pid=1984436)         "offload_optimizer": false
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "global_batch_size_per_device": -1
-(Runner pid=1984436)     },
-(Runner pid=1984436)     "ref": {
-(Runner pid=1984436)       "strategy": "fsdp",
-(Runner pid=1984436)       "fsdp": {
-(Runner pid=1984436)         "enable_full_shard": true,
-(Runner pid=1984436)         "enable_cpu_offload": false,
-(Runner pid=1984436)         "enable_rank0_init": true,
-(Runner pid=1984436)         "use_orig_params": false,
-(Runner pid=1984436)         "torch_dtype": "bf16",
-(Runner pid=1984436)         "fsdp_size": -1,
-(Runner pid=1984436)         "mp_param_dtype": "bf16",
-(Runner pid=1984436)         "mp_reduce_dtype": "fp32",
-(Runner pid=1984436)         "mp_buffer_dtype": "fp32"
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "offload": {
-(Runner pid=1984436)         "offload_params": false,
-(Runner pid=1984436)         "offload_optimizer": false
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "micro_batch_size_per_device_for_experience": 1,
-(Runner pid=1984436)       "padding_free": true,
-(Runner pid=1984436)       "dynamic_batching": true,
-(Runner pid=1984436)       "max_token_len_per_gpu": 32768,
-(Runner pid=1984436)       "ulysses_size": 1,
-(Runner pid=1984436)       "use_torch_compile": false
-(Runner pid=1984436)     },
-(Runner pid=1984436)     "reward": {
-(Runner pid=1984436)       "reward_function": "orarl.rewards",
-(Runner pid=1984436)       "reward_function_kwargs": {},
-(Runner pid=1984436)       "skip_special_tokens": true,
-(Runner pid=1984436)       "num_cpus": 1,
-(Runner pid=1984436)       "reward_function_name": "compute_score",
-(Runner pid=1984436)       "reward_function_is_module": true
-(Runner pid=1984436)     },
-(Runner pid=1984436)     "rollout": {
-(Runner pid=1984436)       "name": "vllm",
-(Runner pid=1984436)       "n": 8,
-(Runner pid=1984436)       "temperature": 1.0,
-(Runner pid=1984436)       "top_p": 0.85,
-(Runner pid=1984436)       "top_k": -1,
-(Runner pid=1984436)       "seed": 42,
-(Runner pid=1984436)       "limit_images": 10,
-(Runner pid=1984436)       "dtype": "bf16",
-(Runner pid=1984436)       "gpu_memory_utilization": 0.45,
-(Runner pid=1984436)       "ignore_eos": false,
-(Runner pid=1984436)       "enforce_eager": false,
-(Runner pid=1984436)       "enable_chunked_prefill": true,
-(Runner pid=1984436)       "tensor_parallel_size": 1,
-(Runner pid=1984436)       "max_model_len": null,
-(Runner pid=1984436)       "max_num_batched_tokens": 32768,
-(Runner pid=1984436)       "disable_log_stats": true,
-(Runner pid=1984436)       "disable_tqdm": true,
-(Runner pid=1984436)       "val_override_config": {
-(Runner pid=1984436)         "temperature": 0.7,
-(Runner pid=1984436)         "top_p": 0.95,
-(Runner pid=1984436)         "n": 1
-(Runner pid=1984436)       },
-(Runner pid=1984436)       "kv_cache_dtype": "auto",
-(Runner pid=1984436)       "calculate_log_probs": false,
-(Runner pid=1984436)       "collect_seq_logprob_for_filter": false,
-(Runner pid=1984436)       "prompt_length": 24576,
-(Runner pid=1984436)       "response_length": 4096,
-(Runner pid=1984436)       "trust_remote_code": false
-(Runner pid=1984436)     }
-(Runner pid=1984436)   },
-(Runner pid=1984436)   "algorithm": {
-(Runner pid=1984436)     "name": "grpo",
-(Runner pid=1984436)     "gamma": 1.0,
-(Runner pid=1984436)     "lam": 1.0,
-(Runner pid=1984436)     "adv_estimator": "grpo",
-(Runner pid=1984436)     "scale_rewards": true,
-(Runner pid=1984436)     "disable_kl": true,
-(Runner pid=1984436)     "use_kl_loss": false,
-(Runner pid=1984436)     "kl_penalty": "low_var_kl",
-(Runner pid=1984436)     "kl_coef": 0.0,
-(Runner pid=1984436)     "kl_type": "fixed",
-(Runner pid=1984436)     "kl_horizon": 10000.0,
-(Runner pid=1984436)     "kl_target": 0.1
-(Runner pid=1984436)   },
-(Runner pid=1984436)   "trainer": {
-(Runner pid=1984436)     "total_epochs": 1,
-(Runner pid=1984436)     "max_steps": 1,
-(Runner pid=1984436)     "project_name": "OraRL",
-(Runner pid=1984436)     "experiment_name": "smoke-grpo-9b",
-(Runner pid=1984436)     "logger": [
-(Runner pid=1984436)       "console"
-(Runner pid=1984436)     ],
-(Runner pid=1984436)     "nnodes": 1,
-(Runner pid=1984436)     "n_gpus_per_node": 8,
-(Runner pid=1984436)     "critic_warmup": 0,
-(Runner pid=1984436)     "val_freq": -1,
-(Runner pid=1984436)     "val_before_train": false,
-(Runner pid=1984436)     "val_only": false,
-(Runner pid=1984436)     "val_generations_to_log": 0,
-(Runner pid=1984436)     "save_freq": 1,
-(Runner pid=1984436)     "save_limit": 3,
-(Runner pid=1984436)     "save_model_only": false,
-(Runner pid=1984436)     "keep_optim_only_latest": true,
-(Runner pid=1984436)     "save_checkpoint_path": "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_175213/grpo",
-(Runner pid=1984436)     "load_checkpoint_path": null,
-(Runner pid=1984436)     "ray_timeline": null,
-(Runner pid=1984436)     "find_last_checkpoint": false,
-(Runner pid=1984436)     "keep_best_train_ckpt": false,
-(Runner pid=1984436)     "best_train_metric_key": "reward/overall",
-(Runner pid=1984436)     "best_train_smooth_window": 5,
-(Runner pid=1984436)     "best_train_min_step": 10
-(Runner pid=1984436)   }
-(Runner pid=1984436) }
-(Runner pid=1984436) [TaskGroupedBatchSampler] 15 groups: image_sequence_mc_answer_only(10304), obj_appearance_order(64), object_abs_distance(1344), object_counting(960), object_rel_direction(448), object_rel_distance(640), object_size_estimation(1408), room_size_estimation(960), route_planning(960), segmentation(12032), spatial grounding(7040), spatial-temporal grounding(9536), temporal grounding(20096), tracking(13952), video_qa_mc(20288) | batch_size=8 | total_batches=12504
-(Runner pid=1984436) Size of train dataloader: 12504
-(Runner pid=1984436) Size of val dataloader: 32
-(Runner pid=1984436) KL is disabled, no KL metrics will be logged. Please set `kl_coef=0` to log KL metrics.(pid=1986239) /data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-(pid=1986239)   warnings.warn(
-(pid=1986747) /data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-(pid=1986747)   warnings.warn(
-(pid=1986750) /data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed.
-(pid=1986750)   warnings.warn(
-(WorkerDict pid=1986750) [W825 17:53:20.598420573 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
-(pid=1986745) /data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/utils/import_utils.py:1006: FutureWarning: `is_flash_attn_greater_or_equal_2_10` is deprecated and will be removed in v5.8. Please use `is_flash_attn_greater_or_equal(library_version='2.1.0')` instead if needed. [repeated 5x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
-(pid=1986745)   warnings.warn( [repeated 5x across cluster]
-(WorkerDict pid=1986750) The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
-(WorkerDict pid=1986750) `torch_dtype` is deprecated! Use `dtype` instead!
-(WorkerDict pid=1986745) [W825 17:53:22.611308342 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) [repeated 7x across cluster]
-(WorkerDict pid=1986750) The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
-(WorkerDict pid=1986750) [rank7]:[W825 17:53:25.252368567 ProcessGroupNCCL.cpp:5138] Guessing device ID based on global rank. This can cause a hang if rank to GPU mapping is heterogeneous. You can specify device_id in init_process_group()
-(WorkerDict pid=1986239) The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`. [repeated 7x across cluster]
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) /data/miniconda3/envs/orarl/lib/python3.11/site-packages/torch/distributed/c10d_logger.py:83: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning.
-(WorkerDict pid=1986239)   return func(*args, **kwargs)
-(WorkerDict pid=1986750) `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
-(WorkerDict pid=1986748) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 6x across cluster]
-(WorkerDict pid=1986748) The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d [repeated 7x across cluster]
-(WorkerDict pid=1986239) [rank0]:[W825 17:53:28.954233606 ProcessGroupNCCL.cpp:5138] Guessing device ID based on global rank. This can cause a hang if rank to GPU mapping is heterogeneous. You can specify device_id in init_process_group() [repeated 7x across cluster]
-(WorkerDict pid=1986747) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
-(WorkerDict pid=1986747) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
-(WorkerDict pid=1986749) `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead. [repeated 7x across cluster]
-(WorkerDict pid=1986239) 2026-08-25 17:54:33,940 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
-(WorkerDict pid=1986239) 2026-08-25 17:54:33,958 - INFO - autotuner.py:268 - flashinfer.jit: [Autotuner]: Autotuning process ends
-(WorkerDict pid=1986748) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead. [repeated 7x across cluster]
-(WorkerDict pid=1986748) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead. [repeated 7x across cluster]
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986750) 2026-08-25 17:54:33,938 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
-(WorkerDict pid=1986750) 2026-08-25 17:54:33,956 - INFO - autotuner.py:268 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) 
-
-(Runner pid=1984436) [GRPO] Scaled advantages ON: A=(r-mean)/(std+eps).
-(Runner pid=1984436) Total training steps: 1
-(AutoRewardManager pid=1986029) Using reward function `compute_score` from `orarl.rewards`.
-(AutoRewardManager pid=1986029) Reward name: orarl, reward type: batch.
-(WorkerDict pid=1986239) actor will use global batch size 64.
-(AutoRewardManager pid=1986031) Using reward function `compute_score` from `orarl.rewards`.
-(AutoRewardManager pid=1986031) Reward name: orarl, reward type: batch.
-(WorkerDict pid=1986239) Model config: Qwen3_5Config {
-(WorkerDict pid=1986239)   "architectures": [
-(WorkerDict pid=1986239)     "Qwen3_5ForConditionalGeneration"
-(WorkerDict pid=1986239)   ],
-(WorkerDict pid=1986239)   "dtype": "bfloat16",
-(WorkerDict pid=1986239)   "image_token_id": 248056,
-(WorkerDict pid=1986239)   "model_type": "qwen3_5",
-(WorkerDict pid=1986239)   "text_config": {
-(WorkerDict pid=1986239)     "attention_bias": false,
-(WorkerDict pid=1986239)     "attention_dropout": 0.0,
-(WorkerDict pid=1986239)     "attn_output_gate": true,
-(WorkerDict pid=1986239)     "bos_token_id": null,
-(WorkerDict pid=1986239)     "dtype": "bfloat16",
-(WorkerDict pid=1986239)     "eos_token_id": 248044,
-(WorkerDict pid=1986239)     "full_attention_interval": 4,
-(WorkerDict pid=1986239)     "head_dim": 256,
-(WorkerDict pid=1986239)     "hidden_act": "silu",
-(WorkerDict pid=1986239)     "hidden_size": 4096,
-(WorkerDict pid=1986239)     "initializer_range": 0.02,
-(WorkerDict pid=1986239)     "intermediate_size": 12288,
-(WorkerDict pid=1986239)     "layer_types": [
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "linear_attention",
-(WorkerDict pid=1986239)       "full_attention"
-(WorkerDict pid=1986239)     ],
-(WorkerDict pid=1986239)     "linear_conv_kernel_dim": 4,
-(WorkerDict pid=1986239)     "linear_key_head_dim": 128,
-(WorkerDict pid=1986239)     "linear_num_key_heads": 16,
-(WorkerDict pid=1986239)     "linear_num_value_heads": 32,
-(WorkerDict pid=1986239)     "linear_value_head_dim": 128,
-(WorkerDict pid=1986239)     "mamba_ssm_dtype": "float32",
-(WorkerDict pid=1986239)     "max_position_embeddings": 262144,
-(WorkerDict pid=1986239)     "mlp_only_layers": [],
-(WorkerDict pid=1986239)     "model_type": "qwen3_5_text",
-(WorkerDict pid=1986239)     "mtp_num_hidden_layers": 0,
-(WorkerDict pid=1986239)     "mtp_use_dedicated_embeddings": false,
-(WorkerDict pid=1986239)     "num_attention_heads": 16,
-(WorkerDict pid=1986239)     "num_hidden_layers": 32,
-(WorkerDict pid=1986239)     "num_key_value_heads": 4,
-(WorkerDict pid=1986239)     "pad_token_id": null,
-(WorkerDict pid=1986239)     "partial_rotary_factor": 0.25,
-(WorkerDict pid=1986239)     "rms_norm_eps": 1e-06,
-(WorkerDict pid=1986239)     "rope_parameters": {
-(WorkerDict pid=1986239)       "mrope_interleaved": true,
-(WorkerDict pid=1986239)       "mrope_section": [
-(WorkerDict pid=1986239)         11,
-(WorkerDict pid=1986239)         11,
-(WorkerDict pid=1986239)         10
-(WorkerDict pid=1986239)       ],
-(WorkerDict pid=1986239)       "partial_rotary_factor": 0.25,
-(WorkerDict pid=1986239)       "rope_theta": 10000000,
-(WorkerDict pid=1986239)       "rope_type": "default"
-(WorkerDict pid=1986239)     },
-(WorkerDict pid=1986239)     "tie_word_embeddings": false,
-(WorkerDict pid=1986239)     "use_cache": true,
-(WorkerDict pid=1986239)     "vocab_size": 248320
-(WorkerDict pid=1986239)   },
-(WorkerDict pid=1986239)   "tie_word_embeddings": false,
-(WorkerDict pid=1986239)   "transformers_version": "5.5.4",
-(WorkerDict pid=1986239)   "video_token_id": 248057,
-(WorkerDict pid=1986239)   "vision_config": {
-(WorkerDict pid=1986239)     "deepstack_visual_indexes": [],
-(WorkerDict pid=1986239)     "depth": 27,
-(WorkerDict pid=1986239)     "dtype": "bfloat16",
-(WorkerDict pid=1986239)     "hidden_act": "gelu_pytorch_tanh",
-(WorkerDict pid=1986239)     "hidden_size": 1152,
-(WorkerDict pid=1986239)     "in_channels": 3,
-(WorkerDict pid=1986239)     "initializer_range": 0.02,
-(WorkerDict pid=1986239)     "intermediate_size": 4304,
-(WorkerDict pid=1986239)     "model_type": "qwen3_5",
-(WorkerDict pid=1986239)     "num_heads": 16,
-(WorkerDict pid=1986239)     "num_position_embeddings": 2304,
-(WorkerDict pid=1986239)     "out_hidden_size": 4096,
-(WorkerDict pid=1986239)     "patch_size": 16,
-(WorkerDict pid=1986239)     "spatial_merge_size": 2,
-(WorkerDict pid=1986239)     "temporal_patch_size": 2
-(WorkerDict pid=1986239)   },
-(WorkerDict pid=1986239)   "vision_end_token_id": 248054,
-(WorkerDict pid=1986239)   "vision_start_token_id": 248053
-(WorkerDict pid=1986239) }
-(WorkerDict pid=1986239) 
-(WorkerDict pid=1986239) Ulysses patch applied!
-(WorkerDict pid=1986239) Vision backbone is set to not trainable.
-(WorkerDict pid=1986239) Qwen3_5ForConditionalGeneration contains 9.41B parameters.
-(WorkerDict pid=1986239) After huggingface model init: 1.17 GB / 95.00 GB.
-(WorkerDict pid=1986239) FSDP wrap policy: functools.partial(<function transformer_auto_wrap_policy at 0x7f3e365eb600>, transformer_layer_cls={<class 'transformers.models.qwen3_5.modeling_qwen3_5.Qwen3_5VisionBlock'>, <class 'transformers.models.qwen3_5.modeling_qwen3_5.Qwen3_5DecoderLayer'>}).
-(WorkerDict pid=1986239) After FSDP module init: 15.42 GB / 95.00 GB.
-(WorkerDict pid=1986239) After optimizer init: 15.42 GB / 95.00 GB.
-(WorkerDict pid=1986744) Sampling params: {'max_tokens': 4096, 'detokenize': False, 'logit_bias': {248056: -100, 248057: -100}, 'n': 8, 'temperature': 1.0, 'top_p': 0.85, 'top_k': -1, 'ignore_eos': False}.
-(WorkerDict pid=1986239) After vllm init: 7.76 GB / 95.00 GB.
-(Runner pid=1984436) Config
-(Runner pid=1984436) algorithm:
-(Runner pid=1984436)   adv_estimator: grpo
-(Runner pid=1984436)   disable_kl: true
-(Runner pid=1984436)   gamma: 1.0
-(Runner pid=1984436)   kl_coef: 0.0
-(Runner pid=1984436)   kl_horizon: 10000.0
-(Runner pid=1984436)   kl_penalty: low_var_kl
-(Runner pid=1984436)   kl_target: 0.1
-(Runner pid=1984436)   kl_type: fixed
-(Runner pid=1984436)   lam: 1.0
-(Runner pid=1984436)   name: grpo
-(Runner pid=1984436)   scale_rewards: true
-(Runner pid=1984436)   use_kl_loss: false
-(Runner pid=1984436) data:
-(Runner pid=1984436)   answer_key: answer
-(Runner pid=1984436)   dataloader_num_workers: 1
-(Runner pid=1984436)   enable_thinking: false
-(Runner pid=1984436)   filter_overlong_prompts: false
-(Runner pid=1984436)   filter_overlong_prompts_workers: 16
-(Runner pid=1984436)   format_prompt: null
-(Runner pid=1984436)   group_by_task: true
-(Runner pid=1984436)   group_by_task_key: problem_type
-(Runner pid=1984436)   image_dir: /
-(Runner pid=1984436)   image_key: images
-(Runner pid=1984436)   image_max_pixels: 1048576
-(Runner pid=1984436)   image_min_pixels: 4096
-(Runner pid=1984436)   inline_video_tensors: true
-(Runner pid=1984436)   max_pixels: 4194304
-(Runner pid=1984436)   max_prompt_length: 24576
-(Runner pid=1984436)   max_response_length: 4096
-(Runner pid=1984436)   min_pixels: 4096
-(Runner pid=1984436)   mini_rollout_batch_size: null
-(Runner pid=1984436)   override_chat_template: null
-(Runner pid=1984436)   preprocessed_video_dir: null
-(Runner pid=1984436)   prompt_key: problem
-(Runner pid=1984436)   response_prefix: ''
-(Runner pid=1984436)   rollout_batch_size: 8
-(Runner pid=1984436)   seed: 42
-(Runner pid=1984436)   shuffle: true
-(Runner pid=1984436)   train_files: /mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl
-(Runner pid=1984436)   use_preprocessed_videos: false
-(Runner pid=1984436)   val_batch_size: 16
-(Runner pid=1984436)   val_files: /mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl
-(Runner pid=1984436)   val_preprocessed_video_dir: null
-(Runner pid=1984436)   val_video_fps: 2.0
-(Runner pid=1984436)   val_video_max_frames: 128
-(Runner pid=1984436)   val_video_max_pixels: 786432
-(Runner pid=1984436)   val_video_min_pixels: 4096
-(Runner pid=1984436)   val_video_source_mode: realtime_only
-(Runner pid=1984436)   val_video_total_pixels: 8388608
-(Runner pid=1984436)   video_fps: 2.0
-(Runner pid=1984436)   video_key: videos
-(Runner pid=1984436)   video_max_frames: 128
-(Runner pid=1984436)   video_max_pixels: 786432
-(Runner pid=1984436)   video_min_pixels: 4096
-(Runner pid=1984436)   video_source_mode: realtime_only
-(Runner pid=1984436)   video_total_pixels: 8388608
-(Runner pid=1984436) trainer:
-(Runner pid=1984436)   best_train_metric_key: reward/overall
-(Runner pid=1984436)   best_train_min_step: 10
-(Runner pid=1984436)   best_train_smooth_window: 5
-(Runner pid=1984436)   critic_warmup: 0
-(Runner pid=1984436)   experiment_name: smoke-grpo-9b
-(Runner pid=1984436)   find_last_checkpoint: false
-(Runner pid=1984436)   keep_best_train_ckpt: false
-(Runner pid=1984436)   keep_optim_only_latest: true
-(Runner pid=1984436)   load_checkpoint_path: null
-(Runner pid=1984436)   logger:
-(Runner pid=1984436)   - console
-(Runner pid=1984436)   max_steps: 1
-(Runner pid=1984436)   n_gpus_per_node: 8
-(Runner pid=1984436)   nnodes: 1
-(Runner pid=1984436)   project_name: OraRL
-(Runner pid=1984436)   ray_timeline: null
-(Runner pid=1984436)   save_checkpoint_path: /mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_175213/grpo
-(Runner pid=1984436)   save_freq: 1
-(Runner pid=1984436)   save_limit: 3
-(Runner pid=1984436)   save_model_only: false
-(Runner pid=1984436)   total_epochs: 1
-(Runner pid=1984436)   val_before_train: false
-(Runner pid=1984436)   val_freq: -1
-(Runner pid=1984436)   val_generations_to_log: 0
-(Runner pid=1984436)   val_only: false
-(Runner pid=1984436) worker:
-(Runner pid=1984436)   actor:
-(Runner pid=1984436)     clip_ratio_dual: 3.0
-(Runner pid=1984436)     clip_ratio_high: 0.2
-(Runner pid=1984436)     clip_ratio_low: 0.2
-(Runner pid=1984436)     disable_kl: true
-(Runner pid=1984436)     dynamic_batching: true
-(Runner pid=1984436)     fsdp:
-(Runner pid=1984436)       enable_cpu_offload: false
-(Runner pid=1984436)       enable_full_shard: true
-(Runner pid=1984436)       enable_rank0_init: true
-(Runner pid=1984436)       fsdp_size: -1
-(Runner pid=1984436)       mp_buffer_dtype: fp32
-(Runner pid=1984436)       mp_param_dtype: bf16
-(Runner pid=1984436)       mp_reduce_dtype: fp32
-(Runner pid=1984436)       torch_dtype: bf16
-(Runner pid=1984436)       use_orig_params: false
-(Runner pid=1984436)     global_batch_size: 8
-(Runner pid=1984436)     global_batch_size_per_device: -1
-(Runner pid=1984436)     kl_coef: 0.0
-(Runner pid=1984436)     kl_penalty: low_var_kl
-(Runner pid=1984436)     loss_avg_mode: token
-(Runner pid=1984436)     loss_type: default
-(Runner pid=1984436)     max_grad_norm: 1.0
-(Runner pid=1984436)     max_token_len_per_gpu: 32768
-(Runner pid=1984436)     micro_batch_size_per_device_for_experience: 1
-(Runner pid=1984436)     micro_batch_size_per_device_for_update: 1
-(Runner pid=1984436)     model:
-(Runner pid=1984436)       enable_gradient_checkpointing: true
-(Runner pid=1984436)       freeze_vision_tower: true
-(Runner pid=1984436)       model_path: /apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B
-(Runner pid=1984436)       override_config: {}
-(Runner pid=1984436)       tokenizer_path: /apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B
-(Runner pid=1984436)       train_vision_merger: false
-(Runner pid=1984436)       trust_remote_code: false
-(Runner pid=1984436)     offload:
-(Runner pid=1984436)       offload_optimizer: false
-(Runner pid=1984436)       offload_params: false
-(Runner pid=1984436)     optim:
-(Runner pid=1984436)       betas:
-(Runner pid=1984436)       - 0.9
-(Runner pid=1984436)       - 0.999
-(Runner pid=1984436)       lr: 1.0e-06
-(Runner pid=1984436)       lr_scheduler_type: constant
-(Runner pid=1984436)       lr_warmup_ratio: 0.0
-(Runner pid=1984436)       lr_warmup_steps: null
-(Runner pid=1984436)       min_lr_ratio: null
-(Runner pid=1984436)       strategy: adamw
-(Runner pid=1984436)       training_steps: 1
-(Runner pid=1984436)       weight_decay: 0.0
-(Runner pid=1984436)     padding_free: true
-(Runner pid=1984436)     ppo_epochs: 1
-(Runner pid=1984436)     strategy: fsdp
-(Runner pid=1984436)     ulysses_size: 1
-(Runner pid=1984436)     use_kl_loss: false
-(Runner pid=1984436)     use_torch_compile: false
-(Runner pid=1984436)   critic:
-(Runner pid=1984436)     cliprange_value: 0.5
-(Runner pid=1984436)     dynamic_batching: true
-(Runner pid=1984436)     fsdp:
-(Runner pid=1984436)       enable_cpu_offload: false
-(Runner pid=1984436)       enable_full_shard: true
-(Runner pid=1984436)       enable_rank0_init: true
-(Runner pid=1984436)       fsdp_size: -1
-(Runner pid=1984436)       mp_buffer_dtype: fp32
-(Runner pid=1984436)       mp_param_dtype: bf16
-(Runner pid=1984436)       mp_reduce_dtype: fp32
-(Runner pid=1984436)       torch_dtype: null
-(Runner pid=1984436)       use_orig_params: false
-(Runner pid=1984436)     global_batch_size: 256
-(Runner pid=1984436)     global_batch_size_per_device: -1
-(Runner pid=1984436)     loss_avg_mode: token
-(Runner pid=1984436)     max_grad_norm: 1.0
-(Runner pid=1984436)     max_token_len_per_gpu: null
-(Runner pid=1984436)     micro_batch_size_per_device_for_experience: 16
-(Runner pid=1984436)     micro_batch_size_per_device_for_update: 4
-(Runner pid=1984436)     model:
-(Runner pid=1984436)       enable_gradient_checkpointing: true
-(Runner pid=1984436)       freeze_vision_tower: false
-(Runner pid=1984436)       model_path: null
-(Runner pid=1984436)       override_config: {}
-(Runner pid=1984436)       tokenizer_path: null
-(Runner pid=1984436)       train_vision_merger: false
-(Runner pid=1984436)       trust_remote_code: true
-(Runner pid=1984436)     offload:
-(Runner pid=1984436)       offload_optimizer: false
-(Runner pid=1984436)       offload_params: false
-(Runner pid=1984436)     optim:
-(Runner pid=1984436)       betas:
-(Runner pid=1984436)       - 0.9
-(Runner pid=1984436)       - 0.999
-(Runner pid=1984436)       lr: 1.0e-06
-(Runner pid=1984436)       lr_scheduler_type: constant
-(Runner pid=1984436)       lr_warmup_ratio: 0.0
-(Runner pid=1984436)       lr_warmup_steps: null
-(Runner pid=1984436)       min_lr_ratio: null
-(Runner pid=1984436)       strategy: adamw
-(Runner pid=1984436)       training_steps: 1
-(Runner pid=1984436)       weight_decay: 0.01
-(Runner pid=1984436)     padding_free: false
-(Runner pid=1984436)     ppo_epochs: 1
-(Runner pid=1984436)     strategy: fsdp
-(Runner pid=1984436)     ulysses_size: 1
-(Runner pid=1984436)   hybrid_engine: true
-(Runner pid=1984436)   ref:
-(Runner pid=1984436)     dynamic_batching: true
-(Runner pid=1984436)     fsdp:
-(Runner pid=1984436)       enable_cpu_offload: false
-(Runner pid=1984436)       enable_full_shard: true
-(Runner pid=1984436)       enable_rank0_init: true
-(Runner pid=1984436)       fsdp_size: -1
-(Runner pid=1984436)       mp_buffer_dtype: fp32
-(Runner pid=1984436)       mp_param_dtype: bf16
-(Runner pid=1984436)       mp_reduce_dtype: fp32
-(Runner pid=1984436)       torch_dtype: bf16
-(Runner pid=1984436)       use_orig_params: false
-(Runner pid=1984436)     max_token_len_per_gpu: 32768
-(Runner pid=1984436)     micro_batch_size_per_device_for_experience: 1
-(Runner pid=1984436)     offload:
-(Runner pid=1984436)       offload_optimizer: false
-(Runner pid=1984436)       offload_params: false
-(Runner pid=1984436)     padding_free: true
-(Runner pid=1984436)     strategy: fsdp
-(Runner pid=1984436)     ulysses_size: 1
-(Runner pid=1984436)     use_torch_compile: false
-(Runner pid=1984436)   reward:
-(Runner pid=1984436)     num_cpus: 1
-(Runner pid=1984436)     reward_function: orarl.rewards
-(Runner pid=1984436)     reward_function_is_module: true
-(Runner pid=1984436)     reward_function_kwargs: {}
-(Runner pid=1984436)     reward_function_name: compute_score
-(Runner pid=1984436)     skip_special_tokens: true
-(Runner pid=1984436)   rollout:
-(Runner pid=1984436)     calculate_log_probs: false
-(Runner pid=1984436)     collect_seq_logprob_for_filter: false
-(Runner pid=1984436)     disable_log_stats: true
-(Runner pid=1984436)     disable_tqdm: true
-(Runner pid=1984436)     dtype: bf16
-(Runner pid=1984436)     enable_chunked_prefill: true
-(Runner pid=1984436)     enforce_eager: false
-(Runner pid=1984436)     gpu_memory_utilization: 0.45
-(Runner pid=1984436)     ignore_eos: false
-(Runner pid=1984436)     kv_cache_dtype: auto
-(Runner pid=1984436)     limit_images: 10
-(Runner pid=1984436)     max_model_len: null
-(Runner pid=1984436)     max_num_batched_tokens: 32768
-(Runner pid=1984436)     n: 8
-(Runner pid=1984436)     name: vllm
-(Runner pid=1984436)     prompt_length: 24576
-(Runner pid=1984436)     response_length: 4096
-(Runner pid=1984436)     seed: 42
-(Runner pid=1984436)     temperature: 1.0
-(Runner pid=1984436)     tensor_parallel_size: 1
-(Runner pid=1984436)     top_k: -1
-(Runner pid=1984436)     top_p: 0.85
-(Runner pid=1984436)     trust_remote_code: false
-(Runner pid=1984436)     val_override_config:
-(Runner pid=1984436)       n: 1
-(Runner pid=1984436)       temperature: 0.7
-(Runner pid=1984436)       top_p: 0.95
-(Runner pid=1984436) 
-
                                                                       
                                                                       
                                                                       
                                                                       
                                                                       
-  File "<frozen runpy>", line 198, in _run_module_as_main
-  File "<frozen runpy>", line 88, in _run_code
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py", line 137, in <module>
-    main()
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py", line 129, in main
-    ray.get(runner.run.remote(ppo_config))
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
-    return fn(*args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
-    return func(*args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/worker.py", line 2981, in get
-    values, debugger_breakpoint = worker.get_objects(
-                                  ^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/ray/_private/worker.py", line 1012, in get_objects
-    raise value.as_instanceof_cause()
-ray.exceptions.RayTaskError(IndexError): ray::Runner.run() (pid=1984436, ip=29.127.69.4, actor_id=2700fe7dc5d33e51d56c429c01000000, repr=<main.Runner object at 0x7ff5f0413990>)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/main.py", line 92, in run
-    trainer.fit()
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/ray_trainer.py", line 1870, in fit
-    batch = self._make_batch_data(metrics=metrics)
-            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/trainer/ray_trainer.py", line 1662, in _make_batch_data
-    batch_dict = next(self.data_iterator)
-                 ^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torchdata/stateful_dataloader/stateful_dataloader.py", line 450, in __next__
-    return super().__next__()
-           ^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 741, in __next__
-    data = self._next_data()
-           ^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torchdata/stateful_dataloader/stateful_dataloader.py", line 1456, in _next_data
-    return self._process_data(data, worker_id, state_dict)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torchdata/stateful_dataloader/stateful_dataloader.py", line 1543, in _process_data
-    data.reraise()
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torch/_utils.py", line 775, in reraise
-    raise exception
-IndexError: Caught IndexError in DataLoader worker process 0.
-Original Traceback (most recent call last):
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torchdata/stateful_dataloader/worker.py", line 242, in _worker_loop
-    data = fetcher.fetch(index)  # type: ignore[union-attr]
-           ^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
-    data = [self.dataset[idx] for idx in possibly_batched_index]
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 54, in <listcomp>
-    data = [self.dataset[idx] for idx in possibly_batched_index]
-            ~~~~~~~~~~~~^^^^^
-  File "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/verl/utils/dataset.py", line 826, in __getitem__
-    model_inputs = self.processor(processed_images, [prompt], add_special_tokens=False, return_tensors="pt")
-                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/data/miniconda3/envs/orarl/lib/python3.11/site-packages/transformers/models/qwen3_vl/processing_qwen3_vl.py", line 132, in __call__
-    num_image_tokens = image_grid_thw[index].prod() // merge_length
-                       ~~~~~~~~~~~~~~^^^^^^^
-IndexError: index 8 is out of bounds for dimension 0 with size 8
-
                                                                       
-
                                                                       
-(WorkerDict pid=1986239) After gather model weights in sharding manager: 31.26 GB / 95.00 GB.
-(Runner pid=1984436) Start generating batch...
-(WorkerDict pid=1986239) After sync model weights in sharding manager: 25.44 GB / 95.00 GB.
-(WorkerDict pid=1986239) After vllm wake up in sharding manager: 45.79 GB / 95.00 GB.
-(WorkerDict pid=1986746) Sampling params: {'max_tokens': 4096, 'detokenize': False, 'logit_bias': {248056: -100, 248057: -100}, 'n': 8, 'temperature': 1.0, 'top_p': 0.85, 'top_k': -1, 'ignore_eos': False}. [repeated 7x across cluster]
-
-method=grpo mode=run
-/data/miniconda3/envs/orarl/bin/python -m verl.trainer.main config=/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/configs/grpo_9b.yaml trainer.max_steps=1 trainer.save_freq=1 trainer.val_before_train=false trainer.experiment_name=smoke-grpo-9b data.rollout_batch_size=8 worker.actor.global_batch_size=8 worker.actor.model.model_path=/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B worker.actor.model.tokenizer_path=/apdcephfs_gy7/share_305340263/hunyuan/yunhengli/checkpoint/Video-ORA-9B data.train_files=/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_joint_100k_maskaware.jsonl data.val_files=/mnt/lyh/TempSamp/EasyVideoR1/data/joint_final/rl_phase3_49152.canary.jsonl trainer.save_checkpoint_path=/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_175213/grpo trainer.nnodes=1 trainer.n_gpus_per_node=8

code/runs/smoke-training-9b-20260825_175949/grpo/checkpoint_tracker.json

@@ -1,6 +0,0 @@
-{
-  "best_global_step": null,
-  "best_val_reward_score": -1.0,
-  "last_global_step": 1,
-  "last_actor_path": "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor"
-}

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/huggingface/chat_template.jinja

@@ -1,154 +0,0 @@
-{%- set image_count = namespace(value=0) %}
-{%- set video_count = namespace(value=0) %}
-{%- macro render_content(content, do_vision_count, is_system_content=false) %}
-    {%- if content is string %}
-        {{- content }}
-    {%- elif content is iterable and content is not mapping %}
-        {%- for item in content %}
-            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
-                {%- if is_system_content %}
-                    {{- raise_exception('System message cannot contain images.') }}
-                {%- endif %}
-                {%- if do_vision_count %}
-                    {%- set image_count.value = image_count.value + 1 %}
-                {%- endif %}
-                {%- if add_vision_id %}
-                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
-                {%- endif %}
-                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
-            {%- elif 'video' in item or item.type == 'video' %}
-                {%- if is_system_content %}
-                    {{- raise_exception('System message cannot contain videos.') }}
-                {%- endif %}
-                {%- if do_vision_count %}
-                    {%- set video_count.value = video_count.value + 1 %}
-                {%- endif %}
-                {%- if add_vision_id %}
-                    {{- 'Video ' ~ video_count.value ~ ': ' }}
-                {%- endif %}
-                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
-            {%- elif 'text' in item %}
-                {{- item.text }}
-            {%- else %}
-                {{- raise_exception('Unexpected item type in content.') }}
-            {%- endif %}
-        {%- endfor %}
-    {%- elif content is none or content is undefined %}
-        {{- '' }}
-    {%- else %}
-        {{- raise_exception('Unexpected content type.') }}
-    {%- endif %}
-{%- endmacro %}
-{%- if not messages %}
-    {{- raise_exception('No messages provided.') }}
-{%- endif %}
-{%- if tools and tools is iterable and tools is not mapping %}
-    {{- '<|im_start|>system\n' }}
-    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
-    {%- for tool in tools %}
-        {{- "\n" }}
-        {{- tool | tojson }}
-    {%- endfor %}
-    {{- "\n</tools>" }}
-    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
-    {%- if messages[0].role == 'system' %}
-        {%- set content = render_content(messages[0].content, false, true)|trim %}
-        {%- if content %}
-            {{- '\n\n' + content }}
-        {%- endif %}
-    {%- endif %}
-    {{- '<|im_end|>\n' }}
-{%- else %}
-    {%- if messages[0].role == 'system' %}
-        {%- set content = render_content(messages[0].content, false, true)|trim %}
-        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
-    {%- endif %}
-{%- endif %}
-{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
-{%- for message in messages[::-1] %}
-    {%- set index = (messages|length - 1) - loop.index0 %}
-    {%- if ns.multi_step_tool and message.role == "user" %}
-        {%- set content = render_content(message.content, false)|trim %}
-        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
-            {%- set ns.multi_step_tool = false %}
-            {%- set ns.last_query_index = index %}
-        {%- endif %}
-    {%- endif %}
-{%- endfor %}
-{%- if ns.multi_step_tool %}
-    {{- raise_exception('No user query found in messages.') }}
-{%- endif %}
-{%- for message in messages %}
-    {%- set content = render_content(message.content, true)|trim %}
-    {%- if message.role == "system" %}
-        {%- if not loop.first %}
-            {{- raise_exception('System message must be at the beginning.') }}
-        {%- endif %}
-    {%- elif message.role == "user" %}
-        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
-    {%- elif message.role == "assistant" %}
-        {%- set reasoning_content = '' %}
-        {%- if message.reasoning_content is string %}
-            {%- set reasoning_content = message.reasoning_content %}
-        {%- else %}
-            {%- if '</think>' in content %}
-                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
-                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
-            {%- endif %}
-        {%- endif %}
-        {%- set reasoning_content = reasoning_content|trim %}
-        {%- if loop.index0 > ns.last_query_index %}
-            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
-        {%- else %}
-            {{- '<|im_start|>' + message.role + '\n' + content }}
-        {%- endif %}
-        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
-            {%- for tool_call in message.tool_calls %}
-                {%- if tool_call.function is defined %}
-                    {%- set tool_call = tool_call.function %}
-                {%- endif %}
-                {%- if loop.first %}
-                    {%- if content|trim %}
-                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
-                    {%- else %}
-                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
-                    {%- endif %}
-                {%- else %}
-                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
-                {%- endif %}
-                {%- if tool_call.arguments is defined %}
-                    {%- for args_name, args_value in tool_call.arguments|items %}
-                        {{- '<parameter=' + args_name + '>\n' }}
-                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
-                        {{- args_value }}
-                        {{- '\n</parameter>\n' }}
-                    {%- endfor %}
-                {%- endif %}
-                {{- '</function>\n</tool_call>' }}
-            {%- endfor %}
-        {%- endif %}
-        {{- '<|im_end|>\n' }}
-    {%- elif message.role == "tool" %}
-        {%- if loop.previtem and loop.previtem.role != "tool" %}
-            {{- '<|im_start|>user' }}
-        {%- endif %}
-        {{- '\n<tool_response>\n' }}
-        {{- content }}
-        {{- '\n</tool_response>' }}
-        {%- if not loop.last and loop.nextitem.role != "tool" %}
-            {{- '<|im_end|>\n' }}
-        {%- elif loop.last %}
-            {{- '<|im_end|>\n' }}
-        {%- endif %}
-    {%- else %}
-        {{- raise_exception('Unexpected message role.') }}
-    {%- endif %}
-{%- endfor %}
-{%- if add_generation_prompt %}
-    {{- '<|im_start|>assistant\n' }}
-    {%- if enable_thinking is defined and enable_thinking is false %}
-        {{- '<think>\n\n</think>\n\n' }}
-    {%- else %}
-        {{- '<think>\n' }}
-    {%- endif %}
-{%- endif %}

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/huggingface/config.json

@@ -1,109 +0,0 @@
-{
-  "architectures": [
-    "Qwen3_5ForConditionalGeneration"
-  ],
-  "dtype": "bfloat16",
-  "image_token_id": 248056,
-  "model_type": "qwen3_5",
-  "text_config": {
-    "attention_bias": false,
-    "attention_dropout": 0.0,
-    "attn_output_gate": true,
-    "bos_token_id": null,
-    "dtype": "bfloat16",
-    "eos_token_id": 248044,
-    "full_attention_interval": 4,
-    "head_dim": 256,
-    "hidden_act": "silu",
-    "hidden_size": 4096,
-    "initializer_range": 0.02,
-    "intermediate_size": 12288,
-    "layer_types": [
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention",
-      "linear_attention",
-      "linear_attention",
-      "linear_attention",
-      "full_attention"
-    ],
-    "linear_conv_kernel_dim": 4,
-    "linear_key_head_dim": 128,
-    "linear_num_key_heads": 16,
-    "linear_num_value_heads": 32,
-    "linear_value_head_dim": 128,
-    "mamba_ssm_dtype": "float32",
-    "max_position_embeddings": 262144,
-    "mlp_only_layers": [],
-    "model_type": "qwen3_5_text",
-    "mtp_num_hidden_layers": 0,
-    "mtp_use_dedicated_embeddings": false,
-    "num_attention_heads": 16,
-    "num_hidden_layers": 32,
-    "num_key_value_heads": 4,
-    "pad_token_id": null,
-    "partial_rotary_factor": 0.25,
-    "rms_norm_eps": 1e-06,
-    "rope_parameters": {
-      "mrope_interleaved": true,
-      "mrope_section": [
-        11,
-        11,
-        10
-      ],
-      "partial_rotary_factor": 0.25,
-      "rope_theta": 10000000,
-      "rope_type": "default"
-    },
-    "tie_word_embeddings": false,
-    "use_cache": true,
-    "vocab_size": 248320
-  },
-  "tie_word_embeddings": false,
-  "transformers_version": "5.5.4",
-  "video_token_id": 248057,
-  "vision_config": {
-    "deepstack_visual_indexes": [],
-    "depth": 27,
-    "dtype": "bfloat16",
-    "hidden_act": "gelu_pytorch_tanh",
-    "hidden_size": 1152,
-    "in_channels": 3,
-    "initializer_range": 0.02,
-    "intermediate_size": 4304,
-    "model_type": "qwen3_5",
-    "num_heads": 16,
-    "num_position_embeddings": 2304,
-    "out_hidden_size": 4096,
-    "patch_size": 16,
-    "spatial_merge_size": 2,
-    "temporal_patch_size": 2
-  },
-  "vision_end_token_id": 248054,
-  "vision_start_token_id": 248053
-}

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/huggingface/generation_config.json

@@ -1,13 +0,0 @@
-{
-  "bos_token_id": 248044,
-  "do_sample": true,
-  "eos_token_id": [
-    248046,
-    248044
-  ],
-  "pad_token_id": 248044,
-  "temperature": 1.0,
-  "top_k": 20,
-  "top_p": 0.95,
-  "transformers_version": "5.5.4"
-}

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/huggingface/processor_config.json

@@ -1,60 +0,0 @@
-{
-  "image_processor": {
-    "do_convert_rgb": true,
-    "do_normalize": true,
-    "do_rescale": true,
-    "do_resize": true,
-    "image_mean": [
-      0.5,
-      0.5,
-      0.5
-    ],
-    "image_processor_type": "Qwen2VLImageProcessor",
-    "image_std": [
-      0.5,
-      0.5,
-      0.5
-    ],
-    "merge_size": 2,
-    "patch_size": 16,
-    "resample": 3,
-    "rescale_factor": 0.00392156862745098,
-    "size": {
-      "longest_edge": 16777216,
-      "shortest_edge": 65536
-    },
-    "temporal_patch_size": 2
-  },
-  "processor_class": "Qwen3VLProcessor",
-  "video_processor": {
-    "do_convert_rgb": true,
-    "do_normalize": true,
-    "do_rescale": true,
-    "do_resize": true,
-    "do_sample_frames": true,
-    "fps": 2,
-    "image_mean": [
-      0.5,
-      0.5,
-      0.5
-    ],
-    "image_std": [
-      0.5,
-      0.5,
-      0.5
-    ],
-    "max_frames": 768,
-    "merge_size": 2,
-    "min_frames": 4,
-    "patch_size": 16,
-    "resample": 3,
-    "rescale_factor": 0.00392156862745098,
-    "return_metadata": false,
-    "size": {
-      "longest_edge": 25165824,
-      "shortest_edge": 4096
-    },
-    "temporal_patch_size": 2,
-    "video_processor_type": "Qwen3VLVideoProcessor"
-  }
-}

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/huggingface/tokenizer.json

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
-size 19989343

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/huggingface/tokenizer_config.json

@@ -1,32 +0,0 @@
-{
-  "add_prefix_space": false,
-  "audio_bos_token": "<|audio_start|>",
-  "audio_eos_token": "<|audio_end|>",
-  "audio_token": "<|audio_pad|>",
-  "backend": "tokenizers",
-  "bos_token": null,
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<|im_end|>",
-  "errors": "replace",
-  "image_token": "<|image_pad|>",
-  "is_local": true,
-  "model_max_length": 262144,
-  "model_specific_special_tokens": {
-    "audio_bos_token": "<|audio_start|>",
-    "audio_eos_token": "<|audio_end|>",
-    "audio_token": "<|audio_pad|>",
-    "image_token": "<|image_pad|>",
-    "video_token": "<|video_pad|>",
-    "vision_bos_token": "<|vision_start|>",
-    "vision_eos_token": "<|vision_end|>"
-  },
-  "pad_token": "<|endoftext|>",
-  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
-  "processor_class": "Qwen3VLProcessor",
-  "split_special_tokens": false,
-  "tokenizer_class": "TokenizersBackend",
-  "unk_token": null,
-  "video_token": "<|video_pad|>",
-  "vision_bos_token": "<|vision_start|>",
-  "vision_eos_token": "<|vision_end|>"
-}

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_0.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e7a112d3ef8e9efd0c7404343835e8d1cacb68f1bf47ec9e70a6035d628bfeae
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_1.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:929bde074b4648a5e40ea9d59f1b37f587f605044b3013c2d9663496ff270997
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_2.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5f191a9b777c259be21b12b8ed02585b790915f74c547f0ac2600a45ce3a598a
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_3.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:ff8e73e975bbe1ea882acbebe7533b050a9c127beb1c39b8a5d3cfd72ffbc17c
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_4.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:49a0e15c74448d5debb04be1f773463910f08df82e32e33fc164e1e674a1d3e2
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_5.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:84a14b9ff89d4c3769d06de5e98af7f71b0b2c7d82226c3e9761d37af480a7db
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_6.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:53206b46e825c183bee092c93d115c7863e368745f0b4a9340f477845fff6521
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/model_world_size_8_rank_7.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5f69907e184a8e128ebd0087e9e47ebd62f2cfa47dd93b356b2b03df20317cda
-size 2352797236

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_0.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:dc1018349737fd19af9fe2ae532ee8aed53a1cd2a6289bd51efc0e893092b5e7
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_1.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:72b7df9ff763ae286ab239e33b2d6711bff9f19a16f9043418b8c68accc08cb2
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_2.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:fad4ce51bcfbc75a0bd746f41af31c8c44cd57974acc1a36adcc4153bf303a6b
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_3.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:11ed1bcf8aa7199792a3f094d8844b6840cdd53efb3fcdfd7f1ea6931b07ea75
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_4.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:ccb3231b0406316170091c991dd49652d43e89bcc3f8544cbe436742baf2e960
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_5.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:8b1e07b9349724c6f93fd95d24146c217483921a201c48db3cd1742e22527b70
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_6.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:92c3605f87040e3ee654db866052d94620101854bc2ee697c10a0536bc5b3dbb
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/grpo/global_step_1/actor/optim_world_size_8_rank_7.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:c2f8b77f8faef58fce850c407f800d2cd0e7c9e4388e7c7a08f6465357fb9891
-size 4477429805

code/runs/smoke-training-9b-20260825_175949/orarl/checkpoint_tracker.json

@@ -1,6 +0,0 @@
-{
-  "best_global_step": null,
-  "best_val_reward_score": -1.0,
-  "last_global_step": 1,
-  "last_actor_path": "/mnt/lyh/TempSamp/EasyVideoR1/OraRL-official/runs/smoke-training-9b-20260825_175949/orarl/global_step_1/actor"
-}

code/runs/smoke-training-9b-20260825_175949/orarl/global_step_1/actor/huggingface/chat_template.jinja

@@ -1,154 +0,0 @@
-{%- set image_count = namespace(value=0) %}
-{%- set video_count = namespace(value=0) %}
-{%- macro render_content(content, do_vision_count, is_system_content=false) %}
-    {%- if content is string %}
-        {{- content }}
-    {%- elif content is iterable and content is not mapping %}
-        {%- for item in content %}
-            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
-                {%- if is_system_content %}
-                    {{- raise_exception('System message cannot contain images.') }}
-                {%- endif %}
-                {%- if do_vision_count %}
-                    {%- set image_count.value = image_count.value + 1 %}
-                {%- endif %}
-                {%- if add_vision_id %}
-                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
-                {%- endif %}
-                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
-            {%- elif 'video' in item or item.type == 'video' %}
-                {%- if is_system_content %}
-                    {{- raise_exception('System message cannot contain videos.') }}
-                {%- endif %}
-                {%- if do_vision_count %}
-                    {%- set video_count.value = video_count.value + 1 %}
-                {%- endif %}
-                {%- if add_vision_id %}
-                    {{- 'Video ' ~ video_count.value ~ ': ' }}
-                {%- endif %}
-                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
-            {%- elif 'text' in item %}
-                {{- item.text }}
-            {%- else %}
-                {{- raise_exception('Unexpected item type in content.') }}
-            {%- endif %}
-        {%- endfor %}
-    {%- elif content is none or content is undefined %}
-        {{- '' }}
-    {%- else %}
-        {{- raise_exception('Unexpected content type.') }}
-    {%- endif %}
-{%- endmacro %}
-{%- if not messages %}
-    {{- raise_exception('No messages provided.') }}
-{%- endif %}
-{%- if tools and tools is iterable and tools is not mapping %}
-    {{- '<|im_start|>system\n' }}
-    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
-    {%- for tool in tools %}
-        {{- "\n" }}
-        {{- tool | tojson }}
-    {%- endfor %}
-    {{- "\n</tools>" }}
-    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
-    {%- if messages[0].role == 'system' %}
-        {%- set content = render_content(messages[0].content, false, true)|trim %}
-        {%- if content %}
-            {{- '\n\n' + content }}
-        {%- endif %}
-    {%- endif %}
-    {{- '<|im_end|>\n' }}
-{%- else %}
-    {%- if messages[0].role == 'system' %}
-        {%- set content = render_content(messages[0].content, false, true)|trim %}
-        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
-    {%- endif %}
-{%- endif %}
-{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
-{%- for message in messages[::-1] %}
-    {%- set index = (messages|length - 1) - loop.index0 %}
-    {%- if ns.multi_step_tool and message.role == "user" %}
-        {%- set content = render_content(message.content, false)|trim %}
-        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
-            {%- set ns.multi_step_tool = false %}
-            {%- set ns.last_query_index = index %}
-        {%- endif %}
-    {%- endif %}
-{%- endfor %}
-{%- if ns.multi_step_tool %}
-    {{- raise_exception('No user query found in messages.') }}
-{%- endif %}
-{%- for message in messages %}
-    {%- set content = render_content(message.content, true)|trim %}
-    {%- if message.role == "system" %}
-        {%- if not loop.first %}
-            {{- raise_exception('System message must be at the beginning.') }}
-        {%- endif %}
-    {%- elif message.role == "user" %}
-        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
-    {%- elif message.role == "assistant" %}
-        {%- set reasoning_content = '' %}
-        {%- if message.reasoning_content is string %}
-            {%- set reasoning_content = message.reasoning_content %}
-        {%- else %}
-            {%- if '</think>' in content %}
-                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
-                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
-            {%- endif %}
-        {%- endif %}
-        {%- set reasoning_content = reasoning_content|trim %}
-        {%- if loop.index0 > ns.last_query_index %}
-            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
-        {%- else %}
-            {{- '<|im_start|>' + message.role + '\n' + content }}
-        {%- endif %}
-        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
-            {%- for tool_call in message.tool_calls %}
-                {%- if tool_call.function is defined %}
-                    {%- set tool_call = tool_call.function %}
-                {%- endif %}
-                {%- if loop.first %}
-                    {%- if content|trim %}
-                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
-                    {%- else %}
-                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
-                    {%- endif %}
-                {%- else %}
-                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
-                {%- endif %}
-                {%- if tool_call.arguments is defined %}
-                    {%- for args_name, args_value in tool_call.arguments|items %}
-                        {{- '<parameter=' + args_name + '>\n' }}
-                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
-                        {{- args_value }}
-                        {{- '\n</parameter>\n' }}
-                    {%- endfor %}
-                {%- endif %}
-                {{- '</function>\n</tool_call>' }}
-            {%- endfor %}
-        {%- endif %}
-        {{- '<|im_end|>\n' }}
-    {%- elif message.role == "tool" %}
-        {%- if loop.previtem and loop.previtem.role != "tool" %}
-            {{- '<|im_start|>user' }}
-        {%- endif %}
-        {{- '\n<tool_response>\n' }}
-        {{- content }}
-        {{- '\n</tool_response>' }}
-        {%- if not loop.last and loop.nextitem.role != "tool" %}
-            {{- '<|im_end|>\n' }}
-        {%- elif loop.last %}
-            {{- '<|im_end|>\n' }}
-        {%- endif %}
-    {%- else %}
-        {{- raise_exception('Unexpected message role.') }}
-    {%- endif %}
-{%- endfor %}
-{%- if add_generation_prompt %}
-    {{- '<|im_start|>assistant\n' }}
-    {%- if enable_thinking is defined and enable_thinking is false %}
-        {{- '<think>\n\n</think>\n\n' }}
-    {%- else %}
-        {{- '<think>\n' }}
-    {%- endif %}
-{%- endif %}