File size: 1,751 Bytes
7a36681 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | ---
license: apache-2.0
base_model: Dream-org/Dream-v0-Instruct-7B
tags:
- diffusion
- reasoning
- reversethought
- dream
datasets:
- ianncity/KIMI-K2.5-1000000x
pipeline_tag: text-generation
---
# Bridge-7b-Diffusion
A fine-tuned [DREAM 7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) masked diffusion language model trained with the **ReverseThought** objective.
## What is ReverseThought?
Given a question and its answer, the model learns to produce the step-by-step reasoning chain that bridges the question to the answer. This trains the model to generate coherent chain-of-thought reasoning via DREAM's masked diffusion process.
- **Input**: Question + Answer
- **Output**: Detailed reasoning trace connecting them
## Training Details
- **Base model**: Dream-org/Dream-v0-Instruct-7B
- **Training data**: 75,000 examples from [KIMI-K2.5-1000000x](https://huggingface.co/datasets/ianncity/KIMI-K2.5-1000000x) (General-Distillation subset)
- **Objective**: DREAM masked diffusion with CART time reweighting
- **Hardware**: 8x NVIDIA H100 80GB
- **Epochs**: 3
- **Batch size**: 128
- **Learning rate**: 2e-6 (cosine schedule)
- **Max sequence length**: 2048 tokens
- **Precision**: bf16 mixed precision (FSDP)
## Usage
```python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("WilhelmH/Bridge-7b-Diffusion", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("WilhelmH/Bridge-7b-Diffusion", trust_remote_code=True)
```
## Architecture
This is a **masked diffusion language model** (not autoregressive). It uses bidirectional attention and generates text by iteratively denoising masked tokens. See the [DREAM paper](https://arxiv.org/abs/2508.15487) for details.
|