---
license: other
base_model: Qwen/Qwen3.5-4B
tags:
  - uncertainty-quantification
  - bayesian
  - grpo
  - lora
---

# BayesClue-Latent checkpoints (private backup)

Latent-belief RL on the passive BayesClue detective game. Base: Qwen3.5-4B.
Belief is read from forced-choice probe **logits** (never verbalized).

## Contents

| path | what | key metric |
|------|------|------------|
| `sft_final/adapter/` | Stage-1 distributional-KL SFT adapter (LoRA r64). Soft-target match `KL(p* ‖ softmax(logits[label_ids]))` at the `Answer:` token. **R1**. | held-out belief KL **0.0066**, top1 0.93 |
| `rl_lr1e5_step400/lora_adapter/` | Stage-2 logit-probe GRPO LoRA delta (lr 1e-5, step 400, 346 modules). Reasoning-only response mask → probe gets zero gradient. | — |
| `rl_lr1e5_step400/merged/` | R2 = SFT⊕RL correctly merged into base (full fp model). | post-reasoning **world_kl 0.0245** (= proper-scoring entropy floor), q_entropy→p_entropy, world_top1 0.71 |

## Notes
- Merge the LoRA with `credal_verl08/remerge_sft_qwen35.py` (the `model.layers.`→`model.language_model.layers.` remap); the stock `verl.model_merger` writes a base copy.
- Reward `R = α·(−CE(p*_H,q_H)) + (1−α)·mean_s(−CE(p*_R,q_R))`, α=0.6. The −1.225 reward plateau IS the optimum `−H(p*)`, not a truncation artifact.