Instructions to use Diffuspace/Clover-35B-A3B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Diffuspace/Clover-35B-A3B-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Diffuspace/Clover-35B-A3B-Preview", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Diffuspace/Clover-35B-A3B-Preview", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Diffuspace/Clover-35B-A3B-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Diffuspace/Clover-35B-A3B-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview

SGLang

How to use Diffuspace/Clover-35B-A3B-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Diffuspace/Clover-35B-A3B-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Diffuspace/Clover-35B-A3B-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Diffuspace/Clover-35B-A3B-Preview with Docker Model Runner:
```
docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview
```

Clover-35B-A3B-Preview / README.md

Sansa

ZiruiWu

Add Clover-35B-A3B-Preview release

4b0402b about 18 hours ago

preview code

Raw History Blame Contribute Delete

7.48 kB

---
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
library_name: transformers
pipeline_tag: text-generation
tags:
  - clover
  - mixture-of-experts
  - diffusion
  - text-generation
  - custom-code
---
<p align="center">
  <a href="https://diffuspace.ai/">
    <img src="https://assets.diffuspace.ai/brand/logofull.svg" alt="DiffuSpace" width="360" />
  </a>
</p>
# Clover-35B-A3B-Preview

**Clover-35B-A3B-Preview** is the first model released by **DiffuSpace**, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our [launch blog](https://diffuspace.ai/blog/clover-35b/). Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a **128K context window**.

[Blog](https://diffuspace.ai/blog/clover-35b/) · [Code & inference backends](https://github.com/DiffuSpace/Clover-35B-A3B-Preview) · [Hugging Face](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview)

## Performance

[![Clover agentic benchmark results](https://diffuspace.ai/assets/blog/clover/agentic-benchmarks.jpg)](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks)

See the [benchmark discussion](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) for evaluation results, settings, and comparisons.

## Architecture and decoding

Clover is a text-only model derived from [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), retaining its hybrid Gated DeltaNet (GDN) / full-attention MoE backbone with 40 layers, 256 routed experts, top-8 token routing, and one shared expert per layer.

- **Bidirectional block attention:** additional learned GDN parameters, including right-convolution weights and softmax controls, let tokens attend within a noisy block while preserving a causal prefix.
- **128K context:** up to 131,072 tokens for the prompt and generated response together.
- **Block diffusion with LowConfidence decoding:** this Hugging Face implementation starts each block with masked tokens and resolves confident positions in parallel. Positions with confidence above the threshold are filled; if none qualify, the most confident position is filled to ensure progress. Once resolved, the block becomes part of the prefix for the next block. Defaults are `block_size=32` and `threshold=0.95`.

For **self-speculative decoding** with our SGLang and llama.cpp implementations, see the [official code repository](https://github.com/DiffuSpace/Clover-35B-A3B-Preview).

## Quick start

Install PyTorch with CUDA support, then the Hugging Face dependencies. The example below uses Transformers 5.19.0 and PyTorch 2.10 or later.

```bash
pip install "transformers==5.19.0" accelerate safetensors
```

Use a GPU with enough memory for the BF16 weights (approximately 69 GB) plus inference state.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Diffuspace/Clover-35B-A3B-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="cuda:0",
).eval()
messages = [{"role": "user", "content": "What is diffusion?"}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to("cuda:0")
output = model.generate(
    **inputs,
    max_new_tokens=24576,
    block_size=32,
    threshold=0.95,
    do_sample=False,
)
completion = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(completion, skip_special_tokens=True))
```

You can also set `model_id` to a local model directory. Generation defaults are loaded automatically from `generation_config.json`; keyword arguments to `generate()` override them. Requests exceeding the 128K context limit are rejected, including space needed to resolve the final full block.

## Generation controls

We observe that very long generations can sometimes become repetitive. The model includes repetition controls and gradual biases toward ending the response or thinking phase. **The defaults are already configured**, so no extra arguments are needed for normal use. They also apply during greedy decoding.

| Parameter | Default | Purpose |
| --- | --- | --- |
| `presence_penalty` | `0.9` | Discourages tokens that have already appeared in the generated response. |
| `frequency_penalty` | `0.0` | Penalizes tokens in proportion to their occurrence count; disabled by default. |
| `repetition_penalty` | `1.05` | Reduces the preference for previously generated tokens; `1.0` disables it. |
| `dllm_length_penalty` | `10.0` | Gradually increases the preference for end-of-response tokens. |
| `dllm_think_end_bias` | `10.0` | Gradually encourages `</think>` to close the thinking phase. |

Length control begins at `dllm_length_penalty_start=8192` response tokens and reaches its full strength over `dllm_length_penalty_window=8192` more tokens. Thinking control uses `dllm_think_end_start=4096` and `dllm_think_end_window=4096`. These are soft controls; `max_new_tokens` sets the hard output limit. Repetition controls operate on generated response history, excluding the prompt.

## Expert selection

During block denoising, Clover can restrict routing to a shared subset of experts for each block. This reduces the number of distinct experts used across the block. The relevant model configuration settings are:

| Setting | Default | Purpose |
| --- | --- | --- |
| `moe_block_coreset_enabled` | `true` | Enables block-level expert selection during denoising. |
| `moe_block_coreset_top_p` | `0.9` | Sets the cumulative routing-mass threshold used to select the expert subset. |

The selector aggregates the routing scores of each token's top-8 candidate experts across the block, ranks experts by their aggregate score, and keeps enough to cover the requested fraction of that mass. A value of `0.9` means **90% of the aggregated routing mass**, not 90% of all experts. Lower values generally select fewer experts, with a possible quality tradeoff. This setting is separate from token-sampling `top_p` and does not reduce the size of the stored or loaded model weights.

## Serving and evaluation

The Hugging Face implementation provides a standalone PyTorch inference path. Small numerical differences between its operations and SGLang's kernels can lead to different generated text, even with the same decoding settings.

**We recommend our official SGLang implementation for serving and evaluation.** Setup instructions and supported decoding modes are available in the [official repository](https://github.com/DiffuSpace/Clover-35B-A3B-Preview).

## Upstream attribution and modifications

Clover is derived from Qwen3.6-35B-A3B. Upstream model assets are Copyright 2026 Alibaba Cloud and distributed under Apache-2.0; the accompanying [LICENSE](LICENSE) preserves the upstream license and copyright statement.

The model implementation derives from the Hugging Face Transformers Qwen3.5-MoE implementation used for the Qwen3.6 architecture, Copyright 2025 The Qwen Team and The HuggingFace Inc. team. Clover adds text-only block diffusion inference, bidirectional GDN parameters, and generation controls. See [ATTRIBUTIONS.md](ATTRIBUTIONS.md) for file-level provenance and modification notices.

	---
	license: apache-2.0
	base_model: Qwen/Qwen3.6-35B-A3B
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- clover
	- mixture-of-experts
	- diffusion
	- text-generation
	- custom-code
	---

	<p align="center">
	<a href="https://diffuspace.ai/">
	<img src="https://assets.diffuspace.ai/brand/logofull.svg" alt="DiffuSpace" width="360" />
	</a>
	</p>

	# Clover-35B-A3B-Preview

	Clover-35B-A3B-Preview is the first model released by DiffuSpace, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our [launch blog](https://diffuspace.ai/blog/clover-35b/). Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a 128K context window.

	[Blog](https://diffuspace.ai/blog/clover-35b/) · [Code & inference backends](https://github.com/DiffuSpace/Clover-35B-A3B-Preview) · [Hugging Face](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview)

	## Performance

	[![Clover agentic benchmark results](https://diffuspace.ai/assets/blog/clover/agentic-benchmarks.jpg)](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks)

	See the [benchmark discussion](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) for evaluation results, settings, and comparisons.

	## Architecture and decoding

	Clover is a text-only model derived from [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), retaining its hybrid Gated DeltaNet (GDN) / full-attention MoE backbone with 40 layers, 256 routed experts, top-8 token routing, and one shared expert per layer.

	- Bidirectional block attention: additional learned GDN parameters, including right-convolution weights and softmax controls, let tokens attend within a noisy block while preserving a causal prefix.
	- 128K context: up to 131,072 tokens for the prompt and generated response together.
	- Block diffusion with LowConfidence decoding: this Hugging Face implementation starts each block with masked tokens and resolves confident positions in parallel. Positions with confidence above the threshold are filled; if none qualify, the most confident position is filled to ensure progress. Once resolved, the block becomes part of the prefix for the next block. Defaults are `block_size=32` and `threshold=0.95`.

	For self-speculative decoding with our SGLang and llama.cpp implementations, see the [official code repository](https://github.com/DiffuSpace/Clover-35B-A3B-Preview).

	## Quick start

	Install PyTorch with CUDA support, then the Hugging Face dependencies. The example below uses Transformers 5.19.0 and PyTorch 2.10 or later.

	```bash
	pip install "transformers==5.19.0" accelerate safetensors
	```

	Use a GPU with enough memory for the BF16 weights (approximately 69 GB) plus inference state.

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model_id = "Diffuspace/Clover-35B-A3B-Preview"
	tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained(
	model_id,
	trust_remote_code=True,
	dtype=torch.bfloat16,
	device_map="cuda:0",
	).eval()

	messages = [{"role": "user", "content": "What is diffusion?"}]
	prompt = tokenizer.apply_chat_template(
	messages,
	tokenize=False,
	add_generation_prompt=True,
	enable_thinking=True,
	)
	inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to("cuda:0")
	output = model.generate(
	**inputs,
	max_new_tokens=24576,
	block_size=32,
	threshold=0.95,
	do_sample=False,
	)
	completion = output[0, inputs["input_ids"].shape[1]:]
	print(tokenizer.decode(completion, skip_special_tokens=True))
	```

	You can also set `model_id` to a local model directory. Generation defaults are loaded automatically from `generation_config.json`; keyword arguments to `generate()` override them. Requests exceeding the 128K context limit are rejected, including space needed to resolve the final full block.

	## Generation controls

	We observe that very long generations can sometimes become repetitive. The model includes repetition controls and gradual biases toward ending the response or thinking phase. The defaults are already configured, so no extra arguments are needed for normal use. They also apply during greedy decoding.

	\| Parameter \| Default \| Purpose \|
	\| --- \| --- \| --- \|
	\| `presence_penalty` \| `0.9` \| Discourages tokens that have already appeared in the generated response. \|
	\| `frequency_penalty` \| `0.0` \| Penalizes tokens in proportion to their occurrence count; disabled by default. \|
	\| `repetition_penalty` \| `1.05` \| Reduces the preference for previously generated tokens; `1.0` disables it. \|
	\| `dllm_length_penalty` \| `10.0` \| Gradually increases the preference for end-of-response tokens. \|
	\| `dllm_think_end_bias` \| `10.0` \| Gradually encourages `</think>` to close the thinking phase. \|

	Length control begins at `dllm_length_penalty_start=8192` response tokens and reaches its full strength over `dllm_length_penalty_window=8192` more tokens. Thinking control uses `dllm_think_end_start=4096` and `dllm_think_end_window=4096`. These are soft controls; `max_new_tokens` sets the hard output limit. Repetition controls operate on generated response history, excluding the prompt.

	## Expert selection

	During block denoising, Clover can restrict routing to a shared subset of experts for each block. This reduces the number of distinct experts used across the block. The relevant model configuration settings are:

	\| Setting \| Default \| Purpose \|
	\| --- \| --- \| --- \|
	\| `moe_block_coreset_enabled` \| `true` \| Enables block-level expert selection during denoising. \|
	\| `moe_block_coreset_top_p` \| `0.9` \| Sets the cumulative routing-mass threshold used to select the expert subset. \|

	The selector aggregates the routing scores of each token's top-8 candidate experts across the block, ranks experts by their aggregate score, and keeps enough to cover the requested fraction of that mass. A value of `0.9` means 90% of the aggregated routing mass, not 90% of all experts. Lower values generally select fewer experts, with a possible quality tradeoff. This setting is separate from token-sampling `top_p` and does not reduce the size of the stored or loaded model weights.

	## Serving and evaluation

	The Hugging Face implementation provides a standalone PyTorch inference path. Small numerical differences between its operations and SGLang's kernels can lead to different generated text, even with the same decoding settings.

	We recommend our official SGLang implementation for serving and evaluation. Setup instructions and supported decoding modes are available in the [official repository](https://github.com/DiffuSpace/Clover-35B-A3B-Preview).

	## Upstream attribution and modifications

	Clover is derived from Qwen3.6-35B-A3B. Upstream model assets are Copyright 2026 Alibaba Cloud and distributed under Apache-2.0; the accompanying [LICENSE](LICENSE) preserves the upstream license and copyright statement.

	The model implementation derives from the Hugging Face Transformers Qwen3.5-MoE implementation used for the Qwen3.6 architecture, Copyright 2025 The Qwen Team and The HuggingFace Inc. team. Clover adds text-only block diffusion inference, bidirectional GDN parameters, and generation controls. See [ATTRIBUTIONS.md](ATTRIBUTIONS.md) for file-level provenance and modification notices.