Joshua-Abok commited on
Commit
ccc5600
·
1 Parent(s): b432939

Modal: switch L4->T4 (L4 pool constrained/Pending); T4+enforce-eager fits 4B

Browse files
Files changed (1) hide show
  1. modal_serve_qwen3b.py +3 -1
modal_serve_qwen3b.py CHANGED
@@ -23,7 +23,9 @@ import modal
23
  MODEL_NAME = "nvidia/Nemotron-Mini-4B-Instruct"
24
  SERVED_NAME = "nvidia/nemotron-mini-4b-instruct" # must match app.py ADVISORY_MODEL
25
 
26
- GPU_CONFIG = "L4" # 24GBample for a 4B model (~8.4GB FP16)
 
 
27
  VLLM_PORT = 8000
28
  MINUTES = 60
29
 
 
23
  MODEL_NAME = "nvidia/Nemotron-Mini-4B-Instruct"
24
  SERVED_NAME = "nvidia/nemotron-mini-4b-instruct" # must match app.py ADVISORY_MODEL
25
 
26
+ GPU_CONFIG = "T4" # 16GBfits the 4B (~8.4GB FP16); biggest/most-available pool.
27
+ # (L4 was stuck "Pending" — constrained. T4 + --enforce-eager
28
+ # below is the reliable combo for a 4B on Turing.)
29
  VLLM_PORT = 8000
30
  MINUTES = 60
31