README.md

---
base_model: Jackrong/Qwopus3.5-9B-v3
tags: [qwen3.5, reasoning, grpo, dapo, dpo, self-correction, verifiable-rewards]
license: apache-2.0
language: [en]
pipeline_tag: text-generation
---

# Qwopus3.5-9B-v4

## Training Pipeline
```
Jackrong/Qwopus3.5-9B-v3 (clean base, 87.8% HumanEval)
 -> Phase 1: DAPO-GRPO 150 steps (native TRL)
     loss=dapo, clip=[0.2, 0.28], beta=0, mask_truncated=True
     Multiplicative reward: tags as entry fee, filler penalized
 -> Phase 2: SAI-DPO Round 1 (self-generated pairs)
 -> Phase 3: SAI-DPO Round 2 (hard example mining)
 -> This Model
```

## Key Innovation: Multiplicative Rewards
- No tags + correct = score x 0.1 (format bad)
- Filler thinking = score x 0.15 - 0.5 (canned phrase penalty)
- Full tags + correct = score x 1.0 + bonuses (full credit)

## DAPO Features (Native TRL)
- Token-level loss (no length bias)
- Asymmetric clipping (epsilon_high=0.28)
- Overlong filtering (mask_truncated=True)
- No KL penalty (beta=0)

## Files
- `merged-model/` — Full merged safetensors

## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    'MeridianVector/Qwopus3.5-9B-v4', subfolder='merged-model',
    torch_dtype='auto', device_map='auto', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(
    'MeridianVector/Qwopus3.5-9B-v4', subfolder='merged-model', trust_remote_code=True)
```

## Acknowledgements
- [Jackrong](https://huggingface.co/Jackrong) for Qwopus3.5-9B-v3
- [Qwen Team](https://huggingface.co/Qwen) for Qwen3.5-9B

Trained on Google Colab G4 (RTX PRO 6000 Blackwell, 96GB).
Benchmarks pending.