Update README
Browse files
README.md
CHANGED
|
@@ -104,10 +104,10 @@ pip install -e .
|
|
| 104 |
The recommended way to serve FrogMini-14B-2510 is with vLLM.
|
| 105 |
|
| 106 |
```bash
|
| 107 |
-
vllm serve microsoft/FrogMini-14B-2510 --tensor-parallel-size 4
|
| 108 |
-
--enable-prefix-caching
|
| 109 |
-
--gpu-memory-utilization 0.9
|
| 110 |
-
--max-model-len 65536
|
| 111 |
--hf-overrides '{"max_position_embeddings": 65536}'
|
| 112 |
```
|
| 113 |
|
|
|
|
| 104 |
The recommended way to serve FrogMini-14B-2510 is with vLLM.
|
| 105 |
|
| 106 |
```bash
|
| 107 |
+
vllm serve microsoft/FrogMini-14B-2510 --tensor-parallel-size 4 \
|
| 108 |
+
--enable-prefix-caching \
|
| 109 |
+
--gpu-memory-utilization 0.9 \
|
| 110 |
+
--max-model-len 65536 \
|
| 111 |
--hf-overrides '{"max_position_embeddings": 65536}'
|
| 112 |
```
|
| 113 |
|