Card: the revision to serve
Browse files
README.md
CHANGED
|
@@ -129,7 +129,7 @@ print(json.dumps(result["answers"], indent=2))
|
|
| 129 |
[vllm-project/semantic-router#5066](https://github.com/vllm-project/semantic-router/pull/5066) adds FP8 and int4 Decision 3.0 packages to the [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) model runtime. Until it is merged and released, the model runtime from that pull request serves this checkpoint:
|
| 130 |
|
| 131 |
```bash
|
| 132 |
-
vllm-srun serve vllm-sr/d3-FP8 --revision
|
| 133 |
```
|
| 134 |
|
| 135 |
```bash
|
|
|
|
| 129 |
[vllm-project/semantic-router#5066](https://github.com/vllm-project/semantic-router/pull/5066) adds FP8 and int4 Decision 3.0 packages to the [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) model runtime. Until it is merged and released, the model runtime from that pull request serves this checkpoint:
|
| 130 |
|
| 131 |
```bash
|
| 132 |
+
vllm-srun serve vllm-sr/d3-FP8 --revision 94c520d306f0150df84c40e55071b315d9b4dec9 --device rocm:0 --port 8100
|
| 133 |
```
|
| 134 |
|
| 135 |
```bash
|