README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-8B-Base
tags:
- qwen3
- search-r1
- lora
- reinforcement-learning
- merged
library_name: transformers
---

# sdft-search-lora-iter160

Qwen3-8B SDFT checkpoint with a search-R1 RL **LoRA adapter merged into the base weights**
(full dense model, not an adapter — load it like any Qwen3-8B).

## What this is

| | |
|---|---|
| Base | Qwen3-8B-Base, SDFT'd on an oracle-mix (`Qwen3-8B-oracle-mix-SFT-balance`) |
| RL | search-R1 style multi-turn retrieval RL, trained with [slime](https://github.com/THUDM/slime) (GSPO) |
| Adapter | LoRA rank 16, alpha 32 (scaling 2.0) on `q/k/v`, `o_proj`, `gate/up_proj`, `down_proj` of all 36 layers |
| Iteration | RL step 160 |
| Merge | `W += (alpha/r) * (B @ A)` applied in Megatron parameter space, then converted to HF |

## Provenance / honesty notes

- The adapter at this iteration is **verified trained**: all 144 LoRA modules have non-zero
  `lora_B` (max |lora_B| = 1.46e-4). Earlier iterations of this project silently saved
  *untrained* adapters (`lora_B` bit-exactly 0) because a TIS rejection-sampling veto zeroed
  the gradient for effectively every sequence; those are not published.
- The merged delta is **small**: relative Frobenius norm ||ΔW||/||W|| per LoRA'd matrix is
  min 5.2e-5, median 9.6e-5, max 2.4e-4. This is "SDFT base + a real but modest RL delta",
  not a heavily-RL'd model. Treat it accordingly in evaluations.
- Verified after merge: 0 non-finite tensors, embedding shape (151936, 4096), greedy decoding
  sane (`The capital of France is` -> ` Paris`, `12*8=` -> `96`).
- Tokenizer/config assets come from Qwen3-8B-Base (unchanged).

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "willamazon1/sdft-search-lora-iter160"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
```