---
library_name: transformers
pipeline_tag: text-generation
base_model:
- meituan-longcat/LongCat-Flash-Lite-Sparse
---

This is a randomly initialized tiny model based on [meituan-longcat/LongCat-Flash-Lite-Sparse](https://huggingface.co/meituan-longcat/LongCat-Flash-Lite-Sparse) for testing and debugging.

| Feature | Source | This repo |
|---|---|---|
| Checkpoint tensor elements | 69.11B BF16, 17.26M F32 | 11.72M BF16, 5.76K F32 |
| Safetensors weight files | 138.29 GB (26 files) | 23.53 MB (model.safetensors) |
| Logical layers (attention sublayers) | 14 (28) | 4 (8) |
| Hidden width / dense MLP / expert MLP | 3,072 / 6,144 / 1,024 | 8 / 64 / 32 |
| Q heads / MLA nope-rope-v head dims | 32 / 128-64-128 | 8 / 64-64-128 |
| MLA q-lora / kv-lora rank | 1,536 / 512 | 32 / 256 |
| Routed / identity zero experts, top-k | 256 / 128, 12 | 32 / 128, 12 |
| LSA index heads / head dim / top-k | 16 / 128 / 2,048 | Same |
| CLI factor / local + init tokens | 2 / 1,024 + 16 | Same |
| Over-embedding tables | 12 tables, ~10.22M rows each | 2 tables, ~524k rows |
| MTP/NextN draft layers | 1 (with LSA indexer) | Same |
| Attention / RoPE | LSA + deepseek-yarn, 1M context | Same |
| Vocabulary rows / tying | 131,072 / untied | Same |
| Routers / correction bias | FP32 | Same |
| Stored precision | BF16 | Same |

SGLang-only: the source ships no Transformers code and the tensors follow SGLang's `longcat_flash` schema. The n-gram hash modulus is internal embedding machinery, so it follows [tiny-random/longcat-flash-lite](https://huggingface.co/tiny-random/longcat-flash-lite).

| File path | Size |
|------|------|
| model.safetensors | 23.5MB |

### Example usage

#### SGLang serving (unverified)

LSA is implemented in SGLang (main branch; the LongCat Sparse adaptation landed via PR #32918). The checkpoint keeps the source's LongcatCausalLM architecture and serves text only, like the upstream deployment guide.

```bash
python3 -m sglang.launch_server \
  --trust-remote-code \
  --model scratch/longcat-flash-lite-sparse \
  --host 0.0.0.0 \
  --port 30000 \
  --tp-size 4 \
  --max-running-requests 64 \
  --mem-fraction-static 0.93 \
  --chunked-prefill-size 2048 \
  --dsa-prefill-backend fa3 \
  --kv-cache-dtype bfloat16

curl http://127.0.0.1:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages": [{"role": "user", "content": "Hello, world!"}], "max_tokens": 16}'
```

### Creating this repo

- Code: [.meta/create_model.py](.meta/create_model.py)
- Model structure: [.meta/model.log](.meta/model.log)
- Environment: [.meta/env.json](.meta/env.json)

### Change log

- 2026-11-03: Initial version: direct SGLang-schema LSA checkpoint (shortcut-MoE, CLI indexers, ngram over-embeddings and one NextN MTP draft).

*This repo is generated by GPT-6.1-Sol.*