Update README.md
Browse files
README.md
CHANGED
|
@@ -30,7 +30,8 @@ vllm bench serve --backend vllm --model INC4AI/Step-3.5-Flash-int4-AutoRound
|
|
| 30 |
## Generate the Model
|
| 31 |
|
| 32 |
```bash
|
| 33 |
-
|
|
|
|
| 34 |
```
|
| 35 |
|
| 36 |
## Ethical Considerations and Limitations
|
|
|
|
| 30 |
## Generate the Model
|
| 31 |
|
| 32 |
```bash
|
| 33 |
+
hf download stepfun-ai/Step-3.5-Flash --local-dir Step-3.5-Flash
|
| 34 |
+
auto_round ./Step-3.5-Flash --scheme W4A16 --iters 0 --disable_opt_rtn --ignore_layers eh_proj,shared_head,layers.45 --layer_config "{mlp:{bits:8,data_type:int},self_attn:{bits:8,data_type:int},layers.46:{bits:8,data_type:int},layers.47:{bits:8,data_type:int}}"
|
| 35 |
```
|
| 36 |
|
| 37 |
## Ethical Considerations and Limitations
|