srmsoumya commited on
Commit
1cb291d
·
1 Parent(s): b4a6b6b

docs: update modal docs to serve q8 instead of q4

Browse files
Files changed (2) hide show
  1. modal_serve/README.md +4 -4
  2. modal_serve/serve.py +1 -1
modal_serve/README.md CHANGED
@@ -19,18 +19,18 @@ All three scale to zero independently. The Demo URL is the user-facing entrypoin
19
  - `gazet` (model checkpoints)
20
  - `gazet-data` (normalized parquet datasets)
21
 
22
- ## 1. Upload the Q4 GGUF model
23
 
24
  ```bash
25
  modal volume put gazet \
26
- finetune/models/ckpt-q4_k_m.gguf \
27
- checkpoints/qwen35-fientune-v3/ckpt-q4_k_m.gguf
28
  ```
29
 
30
  Verify:
31
 
32
  ```bash
33
- modal volume ls gazet checkpoints/qwen35-fientune-v3 | grep ckpt-q4_k_m
34
  ```
35
 
36
  ## 2. Deploy
 
19
  - `gazet` (model checkpoints)
20
  - `gazet-data` (normalized parquet datasets)
21
 
22
+ ## 1. Upload the Q8 GGUF model
23
 
24
  ```bash
25
  modal volume put gazet \
26
+ finetune/models/ckpt-q8_0.gguf \
27
+ checkpoints/qwen35-fientune-v4/ckpt-q8_0.gguf
28
  ```
29
 
30
  Verify:
31
 
32
  ```bash
33
+ modal volume ls gazet checkpoints/qwen35-fientune-v4 | grep ckpt-q8_0
34
  ```
35
 
36
  ## 2. Deploy
modal_serve/serve.py CHANGED
@@ -15,7 +15,7 @@ import modal
15
 
16
  APP_NAME = "gazet"
17
 
18
- MODEL_PATH = "/models/checkpoints/qwen35-fientune-v3/ckpt-q4_k_m.gguf"
19
  DATA_PATH = "/data"
20
 
21
  app = modal.App(APP_NAME)
 
15
 
16
  APP_NAME = "gazet"
17
 
18
+ MODEL_PATH = "/models/checkpoints/qwen35-fientune-v4/ckpt-q8_0.gguf"
19
  DATA_PATH = "/data"
20
 
21
  app = modal.App(APP_NAME)