README.md

---
language: en
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- longcat
license: mit
base_model: meituan-longcat/LongCat-2.0-FP8
---

# pipenetwork/LongCat-2.0-2bit

2-bit (2.501 bits/weight) MLX quantization of
[meituan-longcat/LongCat-2.0](https://huggingface.co/meituan-longcat/LongCat-2.0-FP8),
a 1.6T-parameter / ~48B-active MoE (MLA attention + LongCat sparse-attention indexer +
identity experts + n-gram embeddings). Converted from the FP8 source with `mlx-lm`.
Router classifiers are kept at 8-bit (mixed precision); MTP layers are dropped.

**Size:** ~477 GB. This exceeds a 512 GB unified-memory ceiling in practice — intended for
larger-memory or sharded/multi-node MLX inference, not a single 512 GB machine.

## Requires mlx-lm PR #1464

LongCat-2.0 (`model_type: longcat2`) support is not yet in a released `mlx-lm`. Install from
the PR branch:

```bash
pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/1464/head
```

## Use

```python
from mlx_lm import load, generate

model, tokenizer = load("pipenetwork/LongCat-2.0-2bit")
messages = [{"role": "user", "content": "Who is Albert Einstein?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True))
```

For large builds, use sharded/distributed generation (`mlx.launch` + `sharded_generate.py`).