---
license: apache-2.0
base_model: Cloudflare/clef-flash
tags:
- system-one
- decision-model
- bitsandbytes
- 4-bit
---

# clef-flash-4bit

A pre-quantized 4-bit copy of [Cloudflare/clef-flash](https://huggingface.co/Cloudflare/clef-flash), a 9B System One decision model (Qwen3.5-9B backbone + joint schema head) with a Jev/SystemOne-compatible interface. It exists so that GPUs without bf16 and with little memory (for example a Colab T4) can download about a third of the original size and load the model without quantizing it on the fly.

- **Backbone:** quantized with bitsandbytes NF4, compute dtype float16 (the quantization settings are stored in `config.json`).
- **Joint schema head, loader code (`joint_schema_model.py`), processor and tokenizer:** copied unchanged from the original repository.
- **License:** Apache-2.0, like the original. All credit for the model goes to Cloudflare; see the [original model card](https://huggingface.co/Cloudflare/clef-flash) for its description, evaluations and limitations.

## Usage

Exactly as the original, without passing a quantization config:

```python
import sys, torch
from huggingface_hub import snapshot_download

path = snapshot_download("irodkin/clef-flash-4bit")
sys.path.insert(0, path)
from joint_schema_model import load_release_model, systemone

model, processor = load_release_model(path, device="cuda", dtype=torch.float16)
response = systemone(model, processor, {
    "model": "clef-flash",
    "state": "Customer says the parcel arrived damaged and wants their money back.",
    "questions": {"refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"}},
})
print(response["answers"])
```

Needs `transformers` ≥ 5.10, `bitsandbytes`, `torchvision` and `pillow`.

## Check

Against quantizing the original on the fly with the same settings, the pre-quantized checkpoint gives identical decisions (maximum probability difference 0.0000 on 16 noul/choice questions) and loads with 7.9 GiB of GPU memory. Download size: about 8.5 GB instead of 19 GB (the embeddings and the vision encoder stay in 16-bit). Used as the retrieval controller in a course lab, it matched on-the-fly 4-bit quantization exactly (accuracy 1.00 / 1.00 / 0.87 on single-hop / 2-hop / 3-hop questions).