File size: 1,645 Bytes
e5b3986 09b3d0a e5b3986 09b3d0a e5b3986 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | ---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-0.8B
datasets:
- PrimeIntellect/Reverse-Text-SFT
library_name: transformers
tags:
- prime-rl
- ci
---
# Qwen3.5-0.8B-Reverse-Text-SFT
A short SFT fine-tune of [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B) on [`PrimeIntellect/Reverse-Text-SFT`](https://huggingface.co/datasets/PrimeIntellect/Reverse-Text-SFT).
It is a small, deliberately under-trained starting point for `reverse-text` RL, made for the planned move of the [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) CI RL tests from `PrimeIntellect/Qwen3-0.6B-Reverse-Text-SFT` to Qwen3.5. It is not meant for general use.
## Recipe
- Code: prime-rl commit [`21814b401`](https://github.com/PrimeIntellect-ai/prime-rl/commit/21814b401e29240e8ca4f161dafa0bba6d1425ba) (branch `ci/qwen3_5-ci`, contains the Qwen3.5 tied `lm_head` fix #3863 and the CP fix #3864).
- Config (`uv run sft @ sft.toml`), 1 H200:
```toml
max_steps = 10
[model]
name = "Qwen/Qwen3.5-0.8B"
[data]
name = "PrimeIntellect/Reverse-Text-SFT"
seq_len = 4096
batch_size = 32
[optim]
lr = 2e-5
```
- Chat template: unchanged Qwen3.5 template, thinking off (the 0.8B default). Completions are rendered with the empty `<think>\n\n</think>\n\n` prefix, the same as RL generation.
- Weights include the (frozen, unchanged) vision tower, so the checkpoint loads as `Qwen3_5ForConditionalGeneration` like the base model.
## Numbers
- SFT loss, steps 1-10: 4.98, 6.16, 5.59, 4.96, 4.53, 4.22, 3.96, 3.62, 3.27, 2.92.
- `reverse-text` eval reward (LCS ratio, 256 prompts, temperature 1, 128 max tokens): **0.31** (base model: 0.03 on 32 prompts).
|