Blame for README.md

---
license: apache-2.0
base_model: ricursiveAI/Ripple-65B-Preview
tags:
- dflash2
- speculative-decoding
- ripple
---
# Ripple-65B Preview DFlash2

<div align="center">
<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://framerusercontent.com/images/rSaJGoPP1Saln5kpScZNGCw.png" width="30%">
  <source media="(prefers-color-scheme: light)" srcset="https://framerusercontent.com/images/3rFANSmoZuSBhjRwxZolyEU5guo.png" width="30%">
  <img alt="Ricursive Logo" src="https://framerusercontent.com/images/3rFANSmoZuSBhjRwxZolyEU5guo.png" width="30%">
</picture>
</div>

<p align="center">
  <a href="https://huggingface.co/ricursiveAI/Ripple-65B-Preview">BF16</a> |
  <a href="https://huggingface.co/ricursiveAI/Ripple-65B-Preview-NVFP4">NVFP4</a> |
  <a href="https://huggingface.co/ricursiveAI/Ripple-65B-Base-Preview">Base Model</a> |
  <a href="https://huggingface.co/ricursiveAI/Ripple-65B-Preview-DFlash2">DFlash2</a> |
  <a href="https://www.ricursive.com/ripple">Launch Blog</a> | 
  License: <a href="https://huggingface.co/ricursiveAI/Ripple-65B-Preview/blob/main/LICENSE">Apache 2.0</a>
</p>

**Ripple-65B Preview DFlash2** is a draft model for speculative decoding with [Ripple-65B Preview](https://huggingface.co/ricursiveAI/Ripple-65B-Preview) or [Ripple-65B Preview NVFP4](https://huggingface.co/ricursiveAI/Ripple-65B-Preview-NVFP4). **This repo contains the BF16 drafter.** It requires the target model to generate text.

The drafter proposes multiple tokens at once, conditioned on intermediate hidden states from Ripple. The target then verifies the proposals. A serving implementation with correct speculative rejection sampling preserves the target model's sampling distribution.

## Model

DFlash2 combines parallel block drafting, dynamic convolutions, and a candidate selector. This checkpoint drafts blocks of 16 tokens, retains 32 candidates per position, and uses a rank-256 selector. Its dynamic convolutions have kernel size 4 and group size 16.

The draft has four sliding-attention layers with a 4,096-token window, followed by a full-attention layer without rotary position embeddings (NoPE). The full-attention layer can attend beyond the sliding window.

The checkpoint contains **4.04B parameters** (4,037,752,064), all stored in BF16. This count is derived from the tensors in `model.safetensors`; it excludes the target model's shared token embedding and output head, which are not stored in this repository.

<div align="center">

| Property | Value |
| --- | --- |
| Architecture | `DFlash2DraftModel` |
| Draft layers | 5 |
| Hidden size | 5,120 |
| Query / KV heads | 80 / 8 |
| Head dimension | 128 |
| MLP | SwiGLU, intermediate size 40,960 |
| Attention pattern | 4 sliding-attention layers + 1 full-attention layer |
| Sliding window | 4,096 tokens |
| NoPE layer | Draft layer 4 (zero-based) |
| RoPE theta | 10,000 on rotary-enabled layers |
| Target hidden-state layers | 1, 20, 39, 58, 77 (zero-based) |
| Target depth | 80 layers |
| Vocabulary size | 100,352 |
| Draft block size | 16 |
| Dynamic convolution | Kernel size 4, group size 16 |
| Candidate selector | Top-k 32, rank 256 |
| Configured maximum positions | 131,072 |
| Weight precision | BF16 |

</div>

## Serve

Load this repository as the **speculative draft model** alongside Ripple-65B Preview. Use the target model's tokenizer, chat template, token embeddings, and output head.

The serving runtime must support Ripple and `DFlash2DraftModel`, including this checkpoint's per-layer attention pattern, NoPE layer, four-tap convolutions, and target hidden-state taps. Preserve these settings from `config.json` when loading the draft.

For SGLang-based serving, the draft model settings are:

```bash
--speculative-algorithm DFLASH \
--speculative-draft-model-path ricursiveAI/Ripple-65B-Preview-DFlash2 \
--speculative-draft-model-quantization unquant \
--speculative-num-draft-tokens 16
```

These flags supplement a Ripple-compatible server launch; they are not a standalone launch command. The export's recorded validation used the `ripple-sglang` runtime on eight H100 GPUs with the RL-step-500 target checkpoint. That validation does not establish compatibility or performance for other target revisions or runtime versions.

## Citation

```bibtex
@misc{ricursive2026ripple65b,
      title={Ripple-65B: Pushing the Frontier of Local Intelligence},
      author={Ricursive Intelligence},
      year={2026},
}
```

## Contact

For questions and comments, please contact us at [info@ricursive.com](mailto:info@ricursive.com). To learn more, please visit our [website](https://www.ricursive.com).