sunweiwei commited on
Commit
b842c20
·
verified ·
1 Parent(s): 97ac334

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -6,3 +6,20 @@ A Seed-OSS-36B-Instruct (https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Inst
6
  **Paper:** Training Proactive and Personalized LLM Agents (https://arxiv.org/pdf/2511.02208)
7
 
8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  **Paper:** Training Proactive and Personalized LLM Agents (https://arxiv.org/pdf/2511.02208)
7
 
8
 
9
+ ## Serving with vLLM
10
+
11
+ This model includes bias terms in attention output projections. To serve correctly:
12
+
13
+ ```bash
14
+ wget https://huggingface.co/sunweiwei/PPP-36B/resolve/main/patch_seed_oss.py
15
+ python -c "import patch_seed_oss" && vllm serve sunweiwei/PPP-36B
16
+ ```
17
+
18
+ Or in Python:
19
+ ```python
20
+ import patch_seed_oss # Load once after downloading
21
+ from vllm import LLM
22
+
23
+ llm = LLM("sunweiwei/PPP-36B")
24
+ ```
25
+ The patch reads the attention_out_bias config parameter to properly initialize the model.