---
license: apache-2.0
language:
- en
tags:
- mobiquant
- quantization
- mixed-precision
- llama
---

# MoBiQuant Checkpoints

Calibrated OmniQuant and router parameters for **MoBiQuant** on four LLaMA base models.

**Code:** [SimWangArizona/MoBiQuant](https://github.com/SimWangArizona/MoBiQuant)

## Available checkpoints

| Directory | Base model |
|---|---|
| `llama2-7b` | Llama-2-7B |
| `llama3-8b` | Llama-3-8B |
| `llama3-3b` | Llama-3.2-3B |
| `llama3-1b` | Llama-3.2-1B |

Each directory contains:

- `omni_parameters.pth`: learned quantization parameters.
- `router_parameters.pth`: learned router parameters.

These are supplemental parameter checkpoints. Original pretrained model weights and tokenizer files are required separately; this repository cannot be loaded directly with `AutoModelForCausalLM.from_pretrained()`.

## Quantization configuration

- Weight-only quantization with 16-bit activations.
- Group size: **128**.
- Bit slices: **[2, 2, 2, 2]** — a 2-bit base plus three routed residual slices.
- Routing covers Attention q/k/v/o and MLP gate/up/down projections.
- Effective average bits depend on routing and the evaluation target; Attention is not fixed at 4 bits.

## Download

```python
from huggingface_hub import snapshot_download

checkpoint_dir = snapshot_download(
    repo_id="Simwang/MoBiQuant",
    allow_patterns=["llama3-1b/*.pth"],
)
```

Replace `llama3-1b` with another directory above to download its parameters. See the code repository for evaluation instructions. The current directory-resume workflow additionally expects `router_stats.pth`; that file is not included in this release, so the downloaded pair alone is not a complete directory-resume bundle.

## Limitations

The implementation evaluates quantization using floating-point simulation. These files do not constitute a packed low-bit deployment format or establish inference speedups.

The Apache-2.0 metadata applies to this release; obtain and use the original base models under their respective model licenses and access conditions.