Add Apache 2.0 License and other relevant metadata

README.md

@@ -3,11 +3,19 @@ base_model:
 - deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
 datasets:
 - Skywork/Skywork-OR1-RL-Data
+library_name: transformers
+license: apache-2.0
+pipeline_tag: text-generation
 ---
+
 <div align="center">
 
 # 🤔 Skywork-OR1 (Open Reasoner 1)
 
+<div>
+✊ Unleashing the Power of Reinforcement Learning for Math and Code Reasoners 🤖
+</div>
+
 </div>
 <div>
 <br>
@@ -26,6 +34,11 @@ datasets:
 
 ## 🔥 News
 
+- **May 29, 2025**: Our [Skywork Open Reasoner 1 Technical Report](https://arxiv.org/abs/2505.22312) has been released on arXiv. We provide further details on the training pipeline, investigation and mitigation to the entropy collapse phenomenon, and extensive analysis and ablation studies.
+- **May 13, 2025**: We release our final version of **`Skywork-OR1`** series of models：**`Skywork-OR1-32B`** and **`Skywork-OR1-7B`**.
+  - **[`Skywork-OR1-32B`](https://huggingface.co/Skywork/Skywork-OR1-32B)**  outperforms Deepseek-R1 and Qwen3-32B on math tasks (AIME24 and AIME25) and delivers comparable performance on coding tasks (LiveCodeBench).
+  - **[`Skywork-OR1-7B`](https://huggingface.co/Skywork/Skywork-OR1-7B)** exhibits competitive performance compared to similarly sized models in both math and coding scenarios.
+- **April 15, 2025**: We release our rl training dataset [`Skywork-OR1-RL-Data`](https://huggingface.co/datasets/Skywork/Skywork-OR1-RL-Data).
 - **April 13, 2025**: We release the **`Skywork-OR1`** (Open Reasoner 1) series of models, including **`Skywork-OR1-Math-7B`**, **`Skywork-OR1-32B-Preview`**, and **`Skywork-OR1-7B-Preview`**. We open-source
   - 🤗 Model weights: [`Skywork-OR1-Math-7B`](https://huggingface.co/Skywork/Skywork-OR1-Math-7B),  [`Skywork-OR1-32B-Preview`](https://huggingface.co/Skywork/Skywork-OR1-32B-Preview), [`Skywork-OR1-7B-Preview`](https://huggingface.co/Skywork/Skywork-OR1-7B-Preview)
   - 🤗 Training data: [`Skywork-OR1-RL-Data`](https://huggingface.co/datasets/Skywork/Skywork-OR1-RL-Data)
@@ -35,58 +48,183 @@ datasets:
 ## 📖 Overview
 
 <div align="center">
-<img src="./assets/skywork-or1-math-7b-multi-stage.png" width="60%"/>
+<img src="./assets/32b_perf.jpg" width="100%"/>
 
-<sub>The AIME24 scores versus training steps of Skywork-OR1-Math-7B in our multi-stage training pipeline.</sub>
+<sub>The AIME24 and AIME225 scores versus training steps of Skywork-OR1-32B in our multi-stage training pipeline.</sub>
 </div>
 
-The **`Skywork-OR1`** (Open Reasoner 1) model series consists of powerful math and code reasoning models trained using large-scale rule-based reinforcement learning with carefully designed datasets and training recipes. This series includes two general-purpose reasoning modelsl, **`Skywork-OR1-7B-Preview`** and **`Skywork-OR1-32B-Preview`**, along with a math-specialized model, **`Skywork-OR1-Math-7B`**.
-
-- **[`Skywork-OR1-Math-7B`](https://huggingface.co/Skywork/Skywork-OR1-Math-7B)** is specifically optimized for mathematical reasoning, scoring **69.8** on AIME24 and **52.3** on AIME25 — well ahead of all models of similar size.
-- **[`Skywork-OR1-32B-Preview`](https://huggingface.co/Skywork/Skywork-OR1-32B-Preview)** delivers the 671B-parameter Deepseek-R1 performance on math tasks (AIME24 and AIME25) and coding tasks (LiveCodeBench).
-- **[`Skywork-OR1-7B-Preview`](https://huggingface.co/Skywork/Skywork-OR1-7B-Preview)** outperforms all similarly sized models in both math and coding scenarios.
+The **`Skywork-OR1`** (Open Reasoner 1) model series consists of powerful math and code reasoning models trained using large-scale rule-based reinforcement learning with carefully designed datasets and training recipes. This series includes two general-purpose reasoning modelsl, **`Skywork-OR1-7B`** and **`Skywork-OR1-32B`**.
 
-The final release version will be available in two weeks.
+- **[`Skywork-OR1-32B`](https://huggingface.co/Skywork/Skywork-OR1-32B)** outperforms Deepseek-R1 and Qwen3-32B on math tasks (AIME24 and AIME25) and delivers comparable performance on coding tasks (LiveCodeBench).
+- **[`Skywork-OR1-7B`](https://huggingface.co/Skywork/Skywork-OR1-7B)** exhibits competitive performance compared to similarly sized models in both math and coding scenarios.
 
 ## 📊 Evaluation
 
 <div align="center">
-<img src="./assets/32b_perf.png" width="75%"/>
-<img src="./assets/7b_perf.png" width="75%"/>
+<img src="./assets/32b_eval.jpg" width="75%"/>
+<img src="./assets/7b_eval.jpg" width="75%"/>
 </div>
 </div>
+<br>
 
 We evaluate our models on AIME24, AIME25, and LiveCodeBench. Instead of using Pass@1, which is common in prior work, we introduce Avg@K as the primary metric. This metric robustly measures a model's average performance across K independent attempts, reducing the impact of randomness and enhancing the reliability of the results. We believe that Avg@K provides a better reflection of a model's stability and reasoning consistency.
 
 We include the detailed results in the following table.
 
-| Model | AIME24 (Avg@32) | AIME25 (Avg@32) | LiveCodeBench (8/1/24-2/1/25) (Avg@4) |
-|-------|---------|---------|--------------|
-| DeepSeek-R1-Distill-Qwen-7B | 55.5 | 39.2| 37.6 |
-| Light-R1-7B-DS | 59.1 | 44.3| 39.5 |
-| DeepSeek-R1-Distill-Qwen-32B | 72.9 | 59.0| 57.2 |
-| TinyR1-32B-Preview | 78.1| 65.3| 61.6 |
-| QwQ-32B | 79.5 | 65.3| 61.6 |
-| DeepSeek-R1 | 79.8 | 70.0| 65.9 |
-| **Skywork-OR1-Math-7B** | 69.8 | 52.3 | 43.6 |
-| **Skywork-OR1-7B-Preview** | 63.6 | 45.8 | 43.9 |
-| **Skywork-OR1-32B-Preview** | 79.7 | 69.0 | 63.9 |
+| Model                        | AIME24 (Avg@32) | AIME25 (Avg@32) | LiveCodeBench (8/1/24-2/1/25) (Avg@4) |
+| ---------------------------- | --------------- | --------------- | ------------------------------------- |
+| DeepSeek-R1-Distill-Qwen-7B  | 55.5            | 39.2            | 37.6                                  |
+| Light-R1-7B-DS               | 59.1            | 44.3            | 39.5                                  |
+| **Skywork-OR1-7B**           | 70.2            | 54.6            | 47.6                                  |
+| DeepSeek-R1-Distill-Qwen-32B | 72.9            | 59.0            | 57.2                                  |
+| TinyR1-32B-Preview           | 78.1            | 65.3            | 61.6                                  |
+| QwQ-32B                      | 79.5            | 65.3            | 61.6                                  |
+| Qwen3-32B                    | 81.4            | 72.9            | 65.7                                  |
+| DeepSeek-R1                  | 79.8            | 70.0            | 65.9                                  |
+| **Skywork-OR1-32B**          | 82.2            | 73.3            | 63.0                                  |
+
+## 🎯 Getting Started
+
+### Installation
+
+Docker environment:
+
+```bash
+docker pull whatcanyousee/verl:vemlp-th2.4.0-cu124-vllm0.6.3-ray2.10-te2.0-megatron0.11.0-v0.0.6
+
+# Launch the desired Docker image:
+docker run --runtime=nvidia -it --rm --shm-size="10g" --cap-add=SYS_ADMIN -v <image:tag>
+
+# Inside the container, install Skywork-OR1
+git clone https://github.com/SkyworkAI/Skywork-OR1.git && cd Skywork-OR1 && pip3 install -e .
+```
+
+Conda environment:
+
+```bash
+# Installing Python 3.10 Environment.
+conda create -n verl python==3.10
+conda activate verl
+
+# Installing RLLM dependencies.
+pip3 install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu124
+pip3 install flash-attn --no-build-isolation
+git clone https://github.com/SkyworkAI/Skywork-OR1.git
+cd Skywork-OR1
+pip3 install -e .
+```
+
+### Training ⚙️
+
+We provide training scripts and data to reproduce the results of the “Skywork-OR1-Series”.
+
+### Training Data Preparation
+
+To prepare the training data, we provide a script to download the data from Hugging Face and filter the problems based on the difficulty level with respect to a particular model (i.e., DeepSeek-R1-Distill-Qwen-{1.5,7,32}B).
+
+```bash
+model_size=32b  # 1p5b, 7b
+python ./or1_scripts/data_preprocess/download_and_filter_data_${model_size}.py --local_dir ./or1_data/train
+```
+
+This will generate the training data in the following format:
+
+```bash
+./or1_data/train/train_${model_size}_math.pkl
+./or1_data/train/train_${model_size}_code.pkl
+```
+
+### Train Script
+
+By default, we only provide evaluation on AIME datasets. If you would like to evaluate on LiveCodeBench, please refer to the section [**Evaluation Data Preparation**](#evaluation-data-preparation)  and set `LIVECODEBENCH_DATA_PATH` to `./or1_data/eval/livecodebench/livecodebench_2408_2502`.
+
+```bash
+# Note: You must provide CODE_PATH and MODEL_PATH
+model_size=7b # or 32b
+train_seq_len=8 # or 16, 32
+export CODE_PATH=./
+export MODEL_PATH=
+bash ./or1_scripts/train/${model_size}_${train_seq_len}k.sh
+```
+
+### Using Ray for Multi-Node Training
+
+If you plan to perform **multi-node training**, you need to **start and connect all nodes using Ray** before launching the training script. Here's a quick guide to set up Ray across machines:
+
+#### Step 1: Start Ray on the Head Node (node0)
+
+On the first node (typically called `node0`), run:
+
+```bash
+ray start --head --dashboard-host=0.0.0.0
+```
+
+After running the command, you will see a message like:
+
+```
+Ray runtime started.
+Next steps
+To add another node to this Ray cluster, run
+    ray start --address='10.94.16.4:6379'
+```
+
+Note down the IP address (in this example, `10.94.16.4`).
+
+#### Step 2: Connect Other Nodes (e.g., node1)
+
+On each additional worker node (e.g., `node1`), run the following, replacing the IP with that of your head node:
+
+```bash
+ray start --address='10.94.16.4:6379'
+```
+
+#### Step 3: Check Cluster Status
+
+On `node0`, run:
+
+```bash
+ray status
+```
+
+You should see output showing all connected nodes and available resources (e.g., CPUs, GPUs, memory). For example:
+
+```
+Resources
+---------------------------------------------------------------
+Usage:
+ 0.0/360.0 CPU
+ 0.0/16.0 GPU
+...
+```
+
+Once the Ray cluster is up and running, you can launch the training script as usual. The script will automatically utilize the connected nodes.
+
+### Evaluation ⚖️
+
+We provide evaluation scripts to reproduce the results of the `Skywork-OR1-Series`.
+
+#### Evaluation Data Preparation
 
-## ⚙️ Training Recipe
+Evaluation data for AIME24 and AIME25 is already available in our GitHub repository.
 
-We offer a brief overview of our data and training pipeline below. For more details, please refer to our Notion Blog [here](https://capricious-hydrogen-41c.notion.site/Skywork-Open-Reaonser-Series-1d0bc9ae823a80459b46c149e4f51680).
+For LiveCodeBench, please download the data from [Hugging Face](https://huggingface.co/datasets/Skywork/LiveCodeBench).
 
-### Data
+```bash
+# Download LiveCodeBench
+huggingface-cli download Skywork/LiveCodeBench --repo-type=dataset --local-dir ./or1_data/eval/livecodebench
+unzip ./or1_data/eval/livecodebench/livecodebench.zip -d ./or1_data/eval/livecodebench/
+mv ./or1_data/eval/livecodebench/livecodebench/* ./or1_data/eval/livecodebench/
+```
 
-- We select, clean, and curate **a dataset of 110K verifiable, challenging, and diverse math problems and 14K coding questions** from open-source datasets.
-- We perform **model-aware difficulty estimation** for each problem and model and conduct **rigorous quality assessment prior to training** to ensure training efficiency and effectiveness.
+#### Evaluation Start
 
-### Training
+```bash
+bash ./or1_scripts/eval/eval_7b.sh
 
-We develop a customized version of GRPO that leverages both data-wise and training-wise improvements:
+bash ./or1_scripts/eval/eval_32b.sh
+```
 
-- We perform both **offline and online difficulty-based filtering** and **rejection sampling** to improve training efficiency.
-- We incorporate a **multi-stage training pipeline** coupled with **adaptive entropy control** and other techniques to enhance exploration and stability.
+The evaluation results will be automatically saved to [outputs/evalation/pass.csv](outputs/evalation/pass.csv)
 
 ## 📄 Technical Report
 
@@ -111,7 +249,7 @@ Please cite the following:
 
 @misc{skywork-or1-2025,
   title={Skywork Open Reasoner Series},
-  author = {He, Jujie and Liu, Jiacai and Liu, Chris Yuhao and Yan, Rui and Wang, Chaojie and Cheng, Peng and Zhang, Xiaoyu and Zhang, Fuxiang and Xu, Jiacheng and Shen, Wei and Li, Siyuan and Zeng, Liang and Wei, Tianwen and Cheng, Cheng and Liu, Yang and Zhou, Yahui},
+  author = {He, Jujie and Liu, Jiacai and Liu, Chris Yuhao and Yan, Rui and Wang, Chaojie and Cheng, Peng and Zhang, Xiaoyu and Zhang, Fuxiang and Xu, Jiacheng and Shen, Wei and Li, Siyuan and Zeng, Liang and Wei, Tianwen and Cheng, Cheng and An, Bo and Liu, Yang and Zhou, Yahui},
   howpublished={\url{https://capricious-hydrogen-41c.notion.site/Skywork-Open-Reaonser-Series-1d0bc9ae823a80459b46c149e4f51680}},
   note={Notion Blog},
   year={2025}