Instructions to use SeongryongJung/Qwen-4b-base-RLSD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SeongryongJung/Qwen-4b-base-RLSD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SeongryongJung/Qwen-4b-base-RLSD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SeongryongJung/Qwen-4b-base-RLSD")
model = AutoModelForCausalLM.from_pretrained("SeongryongJung/Qwen-4b-base-RLSD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SeongryongJung/Qwen-4b-base-RLSD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SeongryongJung/Qwen-4b-base-RLSD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SeongryongJung/Qwen-4b-base-RLSD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SeongryongJung/Qwen-4b-base-RLSD

SGLang

How to use SeongryongJung/Qwen-4b-base-RLSD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SeongryongJung/Qwen-4b-base-RLSD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SeongryongJung/Qwen-4b-base-RLSD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SeongryongJung/Qwen-4b-base-RLSD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SeongryongJung/Qwen-4b-base-RLSD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SeongryongJung/Qwen-4b-base-RLSD with Docker Model Runner:
```
docker model run hf.co/SeongryongJung/Qwen-4b-base-RLSD
```

Qwen-4b-base-RLSD / README.md

SeongryongJung

Add model card and training score plot

a0ee6a2 verified 21 days ago

preview code

Raw

History Blame Contribute Delete

2.73 kB

	---
	license: apache-2.0
	base_model: Qwen/Qwen3-4B-Base
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- qwen3
	- math
	- reinforcement-learning
	- rlsd
	- verl
	---

	# Qwen3-4B-Base RLSD

	RLSD self-distillation reinforcement learning on the local math training split.

	This repository contains the final merged Hugging Face checkpoint from `global_step_100`.
	The training checkpoint was saved from FSDP shards and merged to safetensors for this upload.

	## Training Method

	- Policy loss mode: `rlsd`.
	- Self-distillation uses reprompt feedback and token reweighting.
	- Token reweighting: lambda 0.5, eps_w 0.2, decay steps 50.
	- Advantage estimator remains `grpo` in the trainer config.
	- Reward function: local math `compute_score` reward manager.
	- Fine-tuning type: full-parameter FSDP training, not LoRA.

	## Training Hyperparameters

	\| Field \| Value \|
	\|---\|---:\|
	\| Base model \| `Qwen/Qwen3-4B-Base` \|
	\| Train file \| `/home1/irteam/SDPO/self-distillation-analysis/data/math/train.parquet` \|
	\| Validation file \| `/home1/irteam/SDPO/self-distillation-analysis/data/math/evaluation/aime24.parquet` \|
	\| Train max samples \| 25600 \|
	\| Train batch size \| 256 \|
	\| Rollouts per prompt \| 8 \|
	\| PPO mini batch size \| 128 \|
	\| PPO micro batch size per GPU \| 1 \|
	\| Optimizer \| AdamW \|
	\| Learning rate \| 1e-06 \|
	\| Weight decay \| 0.01 \|
	\| LR warmup steps \| 10 \|
	\| Total training steps \| 100 \|
	\| Save frequency \| every 10 steps \|
	\| Validation frequency \| every 10 steps \|
	\| Max prompt length \| 2048 \|
	\| Max response length \| 20480 \|
	\| Rollout backend \| vllm \|
	\| Rollout temperature \| 1 \|
	\| Rollout top_p \| 1 \|
	\| vLLM GPU memory utilization \| 0.75 \|
	\| Actor strategy \| fsdp \|
	\| Dtype \| bfloat16 \|
	\| Advantage estimator \| grpo \|
	\| Gamma / Lambda \| 1 / 1 \|
	\| KL loss enabled \| False \|
	\| KL loss coefficient \| 0.001 \|
	\| Checkpoint uploaded \| `math-RLSD-Qwen3-4B-Base-128-train256-rollout8-lr1e-6-vllm0.75-modelQwen-Qwen3-4B-Base/global_step_100` \|
	\| W&B run id \| `3tuehy90` \|

	## Training Score

	The plot below shows `critic/score/mean` logged during training.

	![Training score](training_score.png)

	CSV data is included in [`training_score.csv`](training_score.csv).

	\| Metric \| Value \|
	\|---\|---:\|
	\| Final training step \| 100 \|
	\| Final `critic/score/mean` \| 0.304199 \|
	\| Final `val-core/math_dapo/acc/mean@1` \| 0.1 \|

	## Intended Use

	This model is intended for internal research and analysis of math-focused RL fine-tuning methods.
	It has not been broadly safety evaluated for production use.

	## Limitations

	The model was trained for 100 optimization steps on a local math dataset split.
	Reported scores are training-time reward/validation metrics from the same experiment setup and should not be treated as broad benchmark results.

	---
	license: apache-2.0
	base_model: Qwen/Qwen3-4B-Base
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- qwen3
	- math
	- reinforcement-learning
	- rlsd
	- verl
	---

	# Qwen3-4B-Base RLSD

	RLSD self-distillation reinforcement learning on the local math training split.

	This repository contains the final merged Hugging Face checkpoint from `global_step_100`.
	The training checkpoint was saved from FSDP shards and merged to safetensors for this upload.

	## Training Method

	- Policy loss mode: `rlsd`.
	- Self-distillation uses reprompt feedback and token reweighting.
	- Token reweighting: lambda 0.5, eps_w 0.2, decay steps 50.
	- Advantage estimator remains `grpo` in the trainer config.
	- Reward function: local math `compute_score` reward manager.
	- Fine-tuning type: full-parameter FSDP training, not LoRA.

	## Training Hyperparameters

	\| Field \| Value \|
	\|---\|---:\|
	\| Base model \| `Qwen/Qwen3-4B-Base` \|
	\| Train file \| `/home1/irteam/SDPO/self-distillation-analysis/data/math/train.parquet` \|
	\| Validation file \| `/home1/irteam/SDPO/self-distillation-analysis/data/math/evaluation/aime24.parquet` \|
	\| Train max samples \| 25600 \|
	\| Train batch size \| 256 \|
	\| Rollouts per prompt \| 8 \|
	\| PPO mini batch size \| 128 \|
	\| PPO micro batch size per GPU \| 1 \|
	\| Optimizer \| AdamW \|
	\| Learning rate \| 1e-06 \|
	\| Weight decay \| 0.01 \|
	\| LR warmup steps \| 10 \|
	\| Total training steps \| 100 \|
	\| Save frequency \| every 10 steps \|
	\| Validation frequency \| every 10 steps \|
	\| Max prompt length \| 2048 \|
	\| Max response length \| 20480 \|
	\| Rollout backend \| vllm \|
	\| Rollout temperature \| 1 \|
	\| Rollout top_p \| 1 \|
	\| vLLM GPU memory utilization \| 0.75 \|
	\| Actor strategy \| fsdp \|
	\| Dtype \| bfloat16 \|
	\| Advantage estimator \| grpo \|
	\| Gamma / Lambda \| 1 / 1 \|
	\| KL loss enabled \| False \|
	\| KL loss coefficient \| 0.001 \|
	\| Checkpoint uploaded \| `math-RLSD-Qwen3-4B-Base-128-train256-rollout8-lr1e-6-vllm0.75-modelQwen-Qwen3-4B-Base/global_step_100` \|
	\| W&B run id \| `3tuehy90` \|

	## Training Score

	The plot below shows `critic/score/mean` logged during training.

	![Training score](training_score.png)

	CSV data is included in [`training_score.csv`](training_score.csv).

	\| Metric \| Value \|
	\|---\|---:\|
	\| Final training step \| 100 \|
	\| Final `critic/score/mean` \| 0.304199 \|
	\| Final `val-core/math_dapo/acc/mean@1` \| 0.1 \|

	## Intended Use

	This model is intended for internal research and analysis of math-focused RL fine-tuning methods.
	It has not been broadly safety evaluated for production use.

	## Limitations

	The model was trained for 100 optimization steps on a local math dataset split.
	Reported scores are training-time reward/validation metrics from the same experiment setup and should not be treated as broad benchmark results.