Instructions to use ELBennett/Weevil-Recurrent-16M-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ELBennett/Weevil-Recurrent-16M-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ELBennett/Weevil-Recurrent-16M-Base", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ELBennett/Weevil-Recurrent-16M-Base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ELBennett/Weevil-Recurrent-16M-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ELBennett/Weevil-Recurrent-16M-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ELBennett/Weevil-Recurrent-16M-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ELBennett/Weevil-Recurrent-16M-Base

SGLang

How to use ELBennett/Weevil-Recurrent-16M-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ELBennett/Weevil-Recurrent-16M-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ELBennett/Weevil-Recurrent-16M-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ELBennett/Weevil-Recurrent-16M-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ELBennett/Weevil-Recurrent-16M-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ELBennett/Weevil-Recurrent-16M-Base with Docker Model Runner:
```
docker model run hf.co/ELBennett/Weevil-Recurrent-16M-Base
```

Weevil-Recurrent-16M-Base / README.md

ELBennett

Show the scoring prompt as a multiline string

893c1c2 verified 11 days ago

preview code

Raw History Blame Contribute Delete

6.83 kB

---
library_name: transformers
tags:
  - weevil
  - recurrent
  - gated-deltanet
language:
  - en
pipeline_tag: text-generation
datasets:
  - HuggingFaceFW/fineweb-edu
  - OptimalScale/ClimbMix
  - IFM/TxT360-v2
  - HuggingFaceTB/finemath
  - HuggingFaceTB/smollm-corpus
  - nvidia/OpenMathInstruct-2
  - codeparrot/codeparrot-clean
---
# Weevil-Recurrent-16M-Base

A 16,085,568-parameter recurrent decoder with post-decay pretraining weights. [Weevil-Recurrent-16M-QA](https://huggingface.co/ELBennett/Weevil-Recurrent-16M-QA) is this model fine-tuned for multiple-choice questions.

## Checkpoints

| Model | Description |
|---|---|
| [ELBennett/Weevil-Recurrent-16M-Base](https://huggingface.co/ELBennett/Weevil-Recurrent-16M-Base) | Post-decay pretraining checkpoint. |
| [ELBennett/Weevil-Recurrent-16M-QA](https://huggingface.co/ELBennett/Weevil-Recurrent-16M-QA) | The Base checkpoint fine-tuned for multiple-choice questions. |

## Model details

| Property | Value |
|---|---|
| Parameters | 16,085,568 |
| Architecture | Causal Gated DeltaNet-2 |
| Vocabulary | 50,368 |
| Factorized embedding | 128 |
| Hidden size | 256 |
| Stack | 2-layer prelude + 4-layer core + 2-layer coda |
| Pretraining | 9.75B first-mix tokens + the entire 750M decay stage = 10.5B content tokens |
| Precision | Float32 stored weights; bfloat16 linears |
| Pause token | `[pause]`, token id 50285 |

## How to use

```bash
pip install torch transformers
```

### Generate

Pass unpadded rows: right-padding is consumed by the convolution.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    "ELBennett/Weevil-Recurrent-16M-Base", trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("ELBennett/Weevil-Recurrent-16M-Base")
model.eval()
prompt = "The capital of France is"
prompt_ids = torch.tensor([tokenizer.encode(prompt, add_special_tokens=False)])
output_ids = model.generate(prompt_ids, max_new_tokens=16)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
```

### Score choices

Continue with the model and tokenizer above.

```python
choices = ["cat", "dog"]
prompt = """Which animal barks?
A. cat
B. dog
Answer:"""
pause_id = tokenizer.convert_tokens_to_ids("[pause]")
passes, n_pause = 16, 10
prompt_ids = torch.tensor(
    [tokenizer.encode(prompt, add_special_tokens=False) + [pause_id] * n_pause]
)
answers = [tokenizer.encode(" " + choice, add_special_tokens=False) for choice in choices]
with torch.no_grad():
    scores = model.score_continuations(prompt_ids, answers, core_passes=passes)
print(choices[int(scores.argmax())], [round(score, 3) for score in scores.tolist()])
```

The prompt and pauses are prefilled once. Each choice is scored from its own copy of the recurrent state.

### Reuse the recurrent state

This continues the `prompt_ids` from the scoring example. `generate` prefills the GDN2 state and convolution windows, then advances them one token at a time. `past_key_values` and `recurrent_state` refer to the same object: one state matrix and one convolution window per mixer, with a separate copy for every core pass.

```python
out = model(prompt_ids, use_cache=True)
next_id = out.logits[:, -1:].argmax(dim=-1)
nxt = model(next_id, past_key_values=out.recurrent_state, use_cache=True)
print(out.recurrent_state.seq_length, nxt.logits.shape)
```

Use `recurrent_state.clone()` or `recurrent_state.repeat(n)` to copy the state when computing token log-probabilities yourself.

### Inference speed

On an H100 at 16 core passes, after the prompt is prefilled, one new token takes 2.52 ms at batch 1. Compiling each block and replaying the token step from a CUDA graph reaches 259,000 tokens/s at batch 16,384 (100% GPU utilization, about 532 W, 31 GB). Prefill of a long prompt still runs the chunk schedule; decode does not recompute it.

## Pretraining mix

This checkpoint trained **9.75 billion tokens** on the first mix. Ten percent of the positions in each pretraining sequence were `[pause]` tokens, and those positions are not part of the content count.

Weight is the probability of drawing the next document from a dataset. Tokens are that weight times 9.75 billion. Each packed document also contributes one end-of-text token toward the count.

| Dataset | Subset | Weight | Tokens |
|---|---|---:|---:|
| [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) | | 25% | 2.4375B |
| [ClimbMix](https://huggingface.co/datasets/OptimalScale/ClimbMix) | | 30% | 2.925B |
| [TxT360-v2](https://huggingface.co/datasets/IFM/TxT360-v2) | `web-high-medium` | 20% | 1.95B |
| [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) | `finemath-4plus` | 10% | 975M |
| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) | `cosmopedia-v2` | 8% | 780M |
| [OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2) | problem and generated solution | 5% | 487.5M |
| [codeparrot-clean](https://huggingface.co/datasets/codeparrot/codeparrot-clean) | Python source, autogenerated files skipped | 2% | 195M |
| **First mix** | | **100%** | **9.75B** |

## Decay mix

The entire decay stage trained **750 million tokens**, shifting weight toward ClimbMix, WikiHow-style tutorials, and math. Tokens are the weight times 750 million.

WikiHow and math rows are filters of [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus): `format = wikihow`, and `seed_data = auto_math_text`.

| Dataset | Subset | Weight | Tokens |
|---|---|---:|---:|
| [ClimbMix](https://huggingface.co/datasets/OptimalScale/ClimbMix) | | 40% | 300M |
| [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) | | 15% | 112.5M |
| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) | WikiHow tutorials | 15% | 112.5M |
| [OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2) | problem and generated solution | 12% | 90M |
| [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) | `finemath-4plus` | 10% | 75M |
| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) | math (`auto_math_text`) | 8% | 60M |
| **Decay stage** | | **100%** | **750M** |

Together that is **10.5 billion** content tokens: 9.75 billion from the first mix and 750 million from the decay stage.

## Architecture

`core_passes` repeats only the core. Each core pass adds `input_scale * prelude_output` back onto the residual. Each mixer and MLP branch has a 1-lane read/write gate. Next-token logits go back through the factorized embedding. Linears run in bfloat16.

`[pause]` is token id 50285. It is a special token, so the word "pause" in ordinary text is not rewritten to it. The model does not learn to emit pauses. Put them in the input.

	---
	library_name: transformers
	tags:
	- weevil
	- recurrent
	- gated-deltanet
	language:
	- en
	pipeline_tag: text-generation
	datasets:
	- HuggingFaceFW/fineweb-edu
	- OptimalScale/ClimbMix
	- IFM/TxT360-v2
	- HuggingFaceTB/finemath
	- HuggingFaceTB/smollm-corpus
	- nvidia/OpenMathInstruct-2
	- codeparrot/codeparrot-clean
	---

	# Weevil-Recurrent-16M-Base

	A 16,085,568-parameter recurrent decoder with post-decay pretraining weights. [Weevil-Recurrent-16M-QA](https://huggingface.co/ELBennett/Weevil-Recurrent-16M-QA) is this model fine-tuned for multiple-choice questions.

	## Checkpoints

	\| Model \| Description \|
	\|---\|---\|
	\| [ELBennett/Weevil-Recurrent-16M-Base](https://huggingface.co/ELBennett/Weevil-Recurrent-16M-Base) \| Post-decay pretraining checkpoint. \|
	\| [ELBennett/Weevil-Recurrent-16M-QA](https://huggingface.co/ELBennett/Weevil-Recurrent-16M-QA) \| The Base checkpoint fine-tuned for multiple-choice questions. \|

	## Model details

	\| Property \| Value \|
	\|---\|---\|
	\| Parameters \| 16,085,568 \|
	\| Architecture \| Causal Gated DeltaNet-2 \|
	\| Vocabulary \| 50,368 \|
	\| Factorized embedding \| 128 \|
	\| Hidden size \| 256 \|
	\| Stack \| 2-layer prelude + 4-layer core + 2-layer coda \|
	\| Pretraining \| 9.75B first-mix tokens + the entire 750M decay stage = 10.5B content tokens \|
	\| Precision \| Float32 stored weights; bfloat16 linears \|
	\| Pause token \| `[pause]`, token id 50285 \|

	## How to use

	```bash
	pip install torch transformers
	```

	### Generate

	Pass unpadded rows: right-padding is consumed by the convolution.

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model = AutoModelForCausalLM.from_pretrained(
	"ELBennett/Weevil-Recurrent-16M-Base", trust_remote_code=True
	)
	tokenizer = AutoTokenizer.from_pretrained("ELBennett/Weevil-Recurrent-16M-Base")
	model.eval()

	prompt = "The capital of France is"
	prompt_ids = torch.tensor([tokenizer.encode(prompt, add_special_tokens=False)])
	output_ids = model.generate(prompt_ids, max_new_tokens=16)
	print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
	```

	### Score choices

	Continue with the model and tokenizer above.

	```python
	choices = ["cat", "dog"]
	prompt = """Which animal barks?

	A. cat
	B. dog
	Answer:"""
	pause_id = tokenizer.convert_tokens_to_ids("[pause]")
	passes, n_pause = 16, 10

	prompt_ids = torch.tensor(
	[tokenizer.encode(prompt, add_special_tokens=False) + [pause_id] * n_pause]
	)
	answers = [tokenizer.encode(" " + choice, add_special_tokens=False) for choice in choices]
	with torch.no_grad():
	scores = model.score_continuations(prompt_ids, answers, core_passes=passes)
	print(choices[int(scores.argmax())], [round(score, 3) for score in scores.tolist()])
	```

	The prompt and pauses are prefilled once. Each choice is scored from its own copy of the recurrent state.

	### Reuse the recurrent state

	This continues the `prompt_ids` from the scoring example. `generate` prefills the GDN2 state and convolution windows, then advances them one token at a time. `past_key_values` and `recurrent_state` refer to the same object: one state matrix and one convolution window per mixer, with a separate copy for every core pass.

	```python
	out = model(prompt_ids, use_cache=True)
	next_id = out.logits[:, -1:].argmax(dim=-1)
	nxt = model(next_id, past_key_values=out.recurrent_state, use_cache=True)
	print(out.recurrent_state.seq_length, nxt.logits.shape)
	```

	Use `recurrent_state.clone()` or `recurrent_state.repeat(n)` to copy the state when computing token log-probabilities yourself.

	### Inference speed

	On an H100 at 16 core passes, after the prompt is prefilled, one new token takes 2.52 ms at batch 1. Compiling each block and replaying the token step from a CUDA graph reaches 259,000 tokens/s at batch 16,384 (100% GPU utilization, about 532 W, 31 GB). Prefill of a long prompt still runs the chunk schedule; decode does not recompute it.

	## Pretraining mix

	This checkpoint trained 9.75 billion tokens on the first mix. Ten percent of the positions in each pretraining sequence were `[pause]` tokens, and those positions are not part of the content count.

	Weight is the probability of drawing the next document from a dataset. Tokens are that weight times 9.75 billion. Each packed document also contributes one end-of-text token toward the count.

	\| Dataset \| Subset \| Weight \| Tokens \|
	\|---\|---\|---:\|---:\|
	\| [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) \| \| 25% \| 2.4375B \|
	\| [ClimbMix](https://huggingface.co/datasets/OptimalScale/ClimbMix) \| \| 30% \| 2.925B \|
	\| [TxT360-v2](https://huggingface.co/datasets/IFM/TxT360-v2) \| `web-high-medium` \| 20% \| 1.95B \|
	\| [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) \| `finemath-4plus` \| 10% \| 975M \|
	\| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) \| `cosmopedia-v2` \| 8% \| 780M \|
	\| [OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2) \| problem and generated solution \| 5% \| 487.5M \|
	\| [codeparrot-clean](https://huggingface.co/datasets/codeparrot/codeparrot-clean) \| Python source, autogenerated files skipped \| 2% \| 195M \|
	\| First mix \| \| 100% \| 9.75B \|

	## Decay mix

	The entire decay stage trained 750 million tokens, shifting weight toward ClimbMix, WikiHow-style tutorials, and math. Tokens are the weight times 750 million.

	WikiHow and math rows are filters of [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus): `format = wikihow`, and `seed_data = auto_math_text`.

	\| Dataset \| Subset \| Weight \| Tokens \|
	\|---\|---\|---:\|---:\|
	\| [ClimbMix](https://huggingface.co/datasets/OptimalScale/ClimbMix) \| \| 40% \| 300M \|
	\| [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) \| \| 15% \| 112.5M \|
	\| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) \| WikiHow tutorials \| 15% \| 112.5M \|
	\| [OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2) \| problem and generated solution \| 12% \| 90M \|
	\| [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) \| `finemath-4plus` \| 10% \| 75M \|
	\| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) \| math (`auto_math_text`) \| 8% \| 60M \|
	\| Decay stage \| \| 100% \| 750M \|

	Together that is 10.5 billion content tokens: 9.75 billion from the first mix and 750 million from the decay stage.

	## Architecture

	`core_passes` repeats only the core. Each core pass adds `input_scale * prelude_output` back onto the residual. Each mixer and MLP branch has a 1-lane read/write gate. Next-token logits go back through the factorized embedding. Linears run in bfloat16.

	`[pause]` is token id 50285. It is a special token, so the word "pause" in ordinary text is not rewritten to it. The model does not learn to emit pauses. Put them in the input.