Spaces:
Running
Running
docs: update modal docs to serve q8 instead of q4
Browse files- modal_serve/README.md +4 -4
- modal_serve/serve.py +1 -1
modal_serve/README.md
CHANGED
|
@@ -19,18 +19,18 @@ All three scale to zero independently. The Demo URL is the user-facing entrypoin
|
|
| 19 |
- `gazet` (model checkpoints)
|
| 20 |
- `gazet-data` (normalized parquet datasets)
|
| 21 |
|
| 22 |
-
## 1. Upload the
|
| 23 |
|
| 24 |
```bash
|
| 25 |
modal volume put gazet \
|
| 26 |
-
finetune/models/ckpt-
|
| 27 |
-
checkpoints/qwen35-fientune-
|
| 28 |
```
|
| 29 |
|
| 30 |
Verify:
|
| 31 |
|
| 32 |
```bash
|
| 33 |
-
modal volume ls gazet checkpoints/qwen35-fientune-
|
| 34 |
```
|
| 35 |
|
| 36 |
## 2. Deploy
|
|
|
|
| 19 |
- `gazet` (model checkpoints)
|
| 20 |
- `gazet-data` (normalized parquet datasets)
|
| 21 |
|
| 22 |
+
## 1. Upload the Q8 GGUF model
|
| 23 |
|
| 24 |
```bash
|
| 25 |
modal volume put gazet \
|
| 26 |
+
finetune/models/ckpt-q8_0.gguf \
|
| 27 |
+
checkpoints/qwen35-fientune-v4/ckpt-q8_0.gguf
|
| 28 |
```
|
| 29 |
|
| 30 |
Verify:
|
| 31 |
|
| 32 |
```bash
|
| 33 |
+
modal volume ls gazet checkpoints/qwen35-fientune-v4 | grep ckpt-q8_0
|
| 34 |
```
|
| 35 |
|
| 36 |
## 2. Deploy
|
modal_serve/serve.py
CHANGED
|
@@ -15,7 +15,7 @@ import modal
|
|
| 15 |
|
| 16 |
APP_NAME = "gazet"
|
| 17 |
|
| 18 |
-
MODEL_PATH = "/models/checkpoints/qwen35-fientune-
|
| 19 |
DATA_PATH = "/data"
|
| 20 |
|
| 21 |
app = modal.App(APP_NAME)
|
|
|
|
| 15 |
|
| 16 |
APP_NAME = "gazet"
|
| 17 |
|
| 18 |
+
MODEL_PATH = "/models/checkpoints/qwen35-fientune-v4/ckpt-q8_0.gguf"
|
| 19 |
DATA_PATH = "/data"
|
| 20 |
|
| 21 |
app = modal.App(APP_NAME)
|