Enhance model card with library, tags, usage example, and detailed description
Browse filesThis PR significantly improves the model card for BlockFFN-3B-SFT by:
- Adding `library_name: transformers` to the metadata, which enables the "How to use" widget and ensures proper categorization on the Hub.
- Adding the `moe` tag, as the model is based on a Mixture-of-Experts architecture, enhancing discoverability.
- Expanding the model description with a concise summary from the paper's abstract, providing more context to users.
- Including a runnable Python code snippet for `AutoTokenizer` and `AutoModelForCausalLM`, making it easier for users to get started with the model.
- Adding a link to the associated Hugging Face Models Collection (`SparseLLM`) for better project navigation.
The original Arxiv paper link has been retained as per guidance.
|
@@ -1,17 +1,46 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
- zh
|
|
|
|
| 6 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
# BlockFFN-3B-SFT
|
| 10 |
|
| 11 |
This is the original 3B BlockFFN checkpoint used in the paper *BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity* for acceleration tests.
|
| 12 |
-
You can load and use this model simply by using `AutoTokenizer` and `AutoModelForCausalLM`.
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
### Citation
|
| 17 |
|
|
@@ -25,4 +54,4 @@ If you find our work useful for your research, please kindly cite our paper as f
|
|
| 25 |
year={2025},
|
| 26 |
url={https://arxiv.org/pdf/2507.08771},
|
| 27 |
}
|
| 28 |
-
```
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
- zh
|
| 5 |
+
license: apache-2.0
|
| 6 |
pipeline_tag: text-generation
|
| 7 |
+
library_name: transformers
|
| 8 |
+
tags:
|
| 9 |
+
- moe
|
| 10 |
---
|
| 11 |
|
| 12 |
# BlockFFN-3B-SFT
|
| 13 |
|
| 14 |
This is the original 3B BlockFFN checkpoint used in the paper *BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity* for acceleration tests.
|
|
|
|
| 15 |
|
| 16 |
+
BlockFFN introduces a novel Mixture-of-Experts (MoE) architecture designed to alleviate the computational burden of large language models (LLMs) by promoting both token-level sparsity (TLS) and chunk-level sparsity (CLS). It features a new router integrating ReLU activation and RMSNorm for differentiable and flexible routing. CLS-aware training objectives are designed to enhance acceleration-friendliness, particularly for low-resource conditions like end-side devices. The model also integrates efficient acceleration kernels, combining activation sparsity and speculative decoding for the first time. Experimental results demonstrate BlockFFN's superior performance, achieving high TLS and CLS, and significant speedups on real end-side devices compared to dense models.
|
| 17 |
+
|
| 18 |
+
Links: [[Paper](https://arxiv.org/pdf/2507.08771)] [[Codes](https://github.com/thunlp/BlockFFN)] [[Models Collection](https://huggingface.co/SparseLLM)]
|
| 19 |
+
|
| 20 |
+
### How to use
|
| 21 |
+
|
| 22 |
+
You can load and use this model simply by using `AutoTokenizer` and `AutoModelForCausalLM` from the `transformers` library:
|
| 23 |
+
|
| 24 |
+
```python
|
| 25 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 26 |
+
import torch
|
| 27 |
+
|
| 28 |
+
model_name = "SparseLLM/BlockFFN-3B-SFT"
|
| 29 |
+
|
| 30 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 31 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 32 |
+
model_name,
|
| 33 |
+
torch_dtype=torch.bfloat16,
|
| 34 |
+
device_map="auto",
|
| 35 |
+
trust_remote_code=True
|
| 36 |
+
)
|
| 37 |
+
|
| 38 |
+
text = "Hello, my name is"
|
| 39 |
+
input_ids = tokenizer(text, return_tensors="pt").input_ids.to(model.device)
|
| 40 |
+
|
| 41 |
+
outputs = model.generate(input_ids, max_new_tokens=20, do_sample=True, top_p=0.8, temperature=0.8)
|
| 42 |
+
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 43 |
+
```
|
| 44 |
|
| 45 |
### Citation
|
| 46 |
|
|
|
|
| 54 |
year={2025},
|
| 55 |
url={https://arxiv.org/pdf/2507.08771},
|
| 56 |
}
|
| 57 |
+
```
|