anm2211 commited on
Commit
5eac52c
·
verified ·
1 Parent(s): 7d1aa92

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -31
README.md CHANGED
@@ -96,37 +96,6 @@ humming-kernels`). See **Serving with vLLM** below.
96
 
97
  ## Serving with vLLM
98
 
99
- > **Temporary vLLM compatibility note:** the upstream vLLM Humming MoE
100
- > implementation currently has a bug that prevents this checkpoint from being
101
- > served correctly. Until the fix is available upstream, use one of the
102
- > following workarounds:
103
- >
104
- > 1. Build vLLM from our fork:
105
- >
106
- > ```bash
107
- > git clone https://github.com/adotdad/vllm.git
108
- > cd vllm
109
- > pip install -e .
110
- > ```
111
- >
112
- > 2. Or patch your existing vLLM installation by replacing:
113
- >
114
- > ```text
115
- > vllm/model_executor/layers/quantization/humming.py
116
- > ```
117
- >
118
- > with the version from our fork:
119
- >
120
- > ```text
121
- > https://github.com/adotdad/vllm
122
- > ```
123
-
124
- Install the Humming kernels (required for vLLM to load this checkpoint):
125
-
126
- ```bash
127
- pip install humming-kernels
128
- ```
129
-
130
  Hopper (sm_90) or Ampere (sm ≥ 80) GPUs required for serving. On 8× H100/H200,
131
  valid TP sizes are `1, 2, 4, 8` (Marlin MoE constraint with group size 128).
132
 
 
96
 
97
  ## Serving with vLLM
98
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
  Hopper (sm_90) or Ampere (sm ≥ 80) GPUs required for serving. On 8× H100/H200,
100
  valid TP sizes are `1, 2, 4, 8` (Marlin MoE constraint with group size 128).
101