Update README.md
Browse files
README.md
CHANGED
|
@@ -1,72 +1,106 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
-
license: apache-2.0 # 或者你选择的开源许可证
|
| 4 |
language:
|
| 5 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
library_name: transformers
|
| 7 |
-
base_model: Qwen/Qwen2-0.5B # 填写你使用的原始基础模型 ID
|
| 8 |
-
|
| 9 |
-
# 评测结果 - 这是展示 SOTA 的核心部分!
|
| 10 |
-
results:
|
| 11 |
-
- task:
|
| 12 |
-
type: text-generation
|
| 13 |
-
name: "中文大模型安全评测 (BFCL)"
|
| 14 |
-
dataset:
|
| 15 |
-
type: EvalPlus/bfcl # 假设这是BFCL在HF上的数据集ID,如果不是,请写真实名称
|
| 16 |
-
name: BFCL
|
| 17 |
-
split: test
|
| 18 |
-
metrics:
|
| 19 |
-
- type: bfcl_score # 自定义一个指标类型
|
| 20 |
-
value: 95.8 # 你的SOTA分数!
|
| 21 |
-
name: "BFCL Score"
|
| 22 |
-
verified: false # 如果结果被官方验证,可以改为 true
|
| 23 |
---
|
| 24 |
|
|
|
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
-
|
| 29 |
-
我们在 [BFCL (中文大模型安全评测基准)](https://github.com/EvalPlus/bfcl) 上取得了最先进的(SOTA)结果。
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
* **硬件**: 2 * A100 (80GB)
|
| 37 |
-
* **框架**: transformers, peft (LoRA)
|
| 38 |
-
* **超参数**: learning_rate=2e-5, epochs=3, batch_size=32 ...
|
| 39 |
|
| 40 |
-
|
| 41 |
|
| 42 |
-
|
| 43 |
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
|
|
|
| 49 |
|
| 50 |
-
|
| 51 |
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
-
|
|
|
|
|
|
|
| 55 |
|
| 56 |
```python
|
| 57 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
|
|
|
| 3 |
language:
|
| 4 |
+
- en
|
| 5 |
+
- zh
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
+
base_model: Qwen/Qwen3-1.7B
|
| 8 |
+
tags:
|
| 9 |
+
- chat
|
| 10 |
+
- function-calling
|
| 11 |
+
- tool-use
|
| 12 |
+
- star-method
|
| 13 |
+
- sota
|
| 14 |
library_name: transformers
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# STAR-1b7
|
| 18 |
|
| 19 |
+
## Introduction
|
| 20 |
|
| 21 |
+
**STAR-1b7** is a highly capable 1.7B parameter language model specialized in function calling, achieving excellent performances on the [Berkeley Function Calling Leaderboard (BFCL)](https://huggingface.co/spaces/gorilla-llm/berkeley-function-calling-leaderboard) for models in its size class.
|
|
|
|
| 22 |
|
| 23 |
+
This model is the result of fine-tuning the `Qwen/Qwen3-1.7B` base model using the novel **STAR (Similarity-guided Teacher-Assisted Refinement)** framework. STAR is a holistic training curriculum designed to effectively transfer the advanced capabilities of large language models (LLMs) into "super-tiny" models, making them powerful, accessible, and efficient for real-world agentic applications.
|
| 24 |
|
| 25 |
+
The key innovations of the STAR framework include:
|
| 26 |
+
- **Similarity-guided RL (Sim-RL)**: A reinforcement learning mechanism that uses a fine-grained, similarity-based reward signal. This provides a more robust and continuous signal for policy optimization compared to simple binary rewards, which is crucial for complex, multi-solution tasks like function calling.
|
| 27 |
+
- **Constrained Knowledge Distillation (CKD)**: An advanced training objective that augments top-k forward KL divergence to suppress confidently incorrect predictions. This ensures training stability while preserving the model's exploration capacity, creating a strong foundation for the subsequent RL phase.
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
+
Our STAR-1b7 model significantly outperforms other open models under 1B parameters and even surpasses several larger models, demonstrating the effectiveness of the STAR methodology.
|
| 30 |
|
| 31 |
+
## Model Details
|
| 32 |
|
| 33 |
+
- **Model Type**: Causal Language Model, fine-tuned for function calling.
|
| 34 |
+
- **Base Model**: `Qwen/Qwen3-1.7B`
|
| 35 |
+
- **Training Framework**: STAR (CKD + Sim-RL)
|
| 36 |
+
- **Architecture**: Transformer with RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
|
| 37 |
+
- **Number of Parameters**: ~1.7B
|
| 38 |
+
- **Context Length**: Supports up to 32,768 tokens.
|
| 39 |
|
| 40 |
+
## Requirements
|
| 41 |
|
| 42 |
+
The code of Qwen3 has been in the latest Hugging Face `transformers` and we advise you to use the latest version of `transformers`.
|
| 43 |
+
With `transformers<4.51.0`, you will encounter the following error:
|
| 44 |
+
```
|
| 45 |
+
KeyError: 'qwen3'
|
| 46 |
+
```
|
| 47 |
|
| 48 |
+
## Quickstart
|
| 49 |
+
|
| 50 |
+
Here is a code snippet showing how to load STAR-1b7 and use it for a chat-based task.
|
| 51 |
|
| 52 |
```python
|
| 53 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 54 |
|
| 55 |
+
model_name = "star-lab/STAR-1b7"
|
| 56 |
+
|
| 57 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 58 |
+
model_name,
|
| 59 |
+
torch_dtype="auto",
|
| 60 |
+
device_map="auto"
|
| 61 |
+
)
|
| 62 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 63 |
+
|
| 64 |
+
# Example prompt that could trigger a function call
|
| 65 |
+
prompt = "What is the current weather in San Francisco?"
|
| 66 |
+
messages = [
|
| 67 |
+
{"role": "system", "content": "You are a helpful assistant with access to external tools."},
|
| 68 |
+
{"role": "user", "content": prompt}
|
| 69 |
+
]
|
| 70 |
+
text = tokenizer.apply_chat_template(
|
| 71 |
+
messages,
|
| 72 |
+
tokenize=False,
|
| 73 |
+
add_generation_prompt=True
|
| 74 |
+
)
|
| 75 |
+
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
| 76 |
+
|
| 77 |
+
generated_ids = model.generate(
|
| 78 |
+
**model_inputs,
|
| 79 |
+
max_new_tokens=32768
|
| 80 |
+
)
|
| 81 |
+
generated_ids = [
|
| 82 |
+
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
| 83 |
+
]
|
| 84 |
+
|
| 85 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 86 |
+
print(response)
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
For deployment, you can use `sglang>=0.4.6.post1` or `vllm>=0.8.5` or to create an OpenAI-compatible API endpoint:
|
| 90 |
+
- SGLang:
|
| 91 |
+
```shell
|
| 92 |
+
python -m sglang.launch_server --model-path star-lab/STAR-1b7 --reasoning-parser qwen3
|
| 93 |
+
```
|
| 94 |
+
- vLLM:
|
| 95 |
+
```shell
|
| 96 |
+
vllm serve star-lab/STAR-1b7 --enable-reasoning --reasoning-parser deepseek_r1
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported STAR-1b7.
|
| 100 |
+
|
| 101 |
+
## Evaluation & Performance
|
| 102 |
+
|
| 103 |
+
STAR-1b7 has achieved outstanding performance for models of its size on renowned function calling benchmarks.
|
| 104 |
+
|
| 105 |
+
- BFCLv3: Achieved 56.05% overall accuracy.
|
| 106 |
+
- ACEBench: Achieved 60.90% summary score, demonstrating superior generalization and robustness.
|