agharsallah commited on
Commit
dfae9ee
·
1 Parent(s): ade9df5

feat: add Nemotron-Cascade-14B-Thinking model configuration to catalogue and README

Browse files
Files changed (2) hide show
  1. modal/README.md +2 -1
  2. modal/catalogue.py +17 -0
modal/README.md CHANGED
@@ -15,7 +15,7 @@ modal/
15
  service.py Reusable serving layer: image + vllm command, register_model()
16
  (provider-agnostic). Imports ModelConfig from catalogue.
17
  registry.py Back-compat re-export of the catalogue's model lists.
18
- app_nvidia.py App "nvidia-llms" — Nemotron 3 Nano 4B + 30B.
19
  app_openbmb.py App "openbmb-llms" — MiniCPM4.1-8B + MiniCPM-o 4.5.
20
  app_google.py App "google-llms" — Gemma 4 12B + 26B.
21
  client.py OpenAI-SDK smoke-test client for any endpoint.
@@ -36,6 +36,7 @@ Each running endpoint also self-documents at `/docs` (Swagger UI) and
36
  | Provider | App | Model | Endpoint name | GPU |
37
  | -------- | -------------- | --------------------------------------- | --------------------- | ------- |
38
  | NVIDIA | `nvidia-llms` | NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | `nemotron-3-nano-30b` | H200:1 |
 
39
  | NVIDIA | `nvidia-llms` | NVIDIA-Nemotron-3-Nano-4B-BF16 | `nemotron-3-nano-4b` | L4:1 |
40
  | OpenBMB | `openbmb-llms` | MiniCPM-o-4_5 (omni) | `minicpm-o-4-5` | L40S:1 |
41
  | OpenBMB | `openbmb-llms` | MiniCPM4.1-8B | `minicpm-4-1-8b` | L40S:1 |
 
15
  service.py Reusable serving layer: image + vllm command, register_model()
16
  (provider-agnostic). Imports ModelConfig from catalogue.
17
  registry.py Back-compat re-export of the catalogue's model lists.
18
+ app_nvidia.py App "nvidia-llms" — Nemotron 3 Nano 4B + 30B, Cascade 14B Thinking.
19
  app_openbmb.py App "openbmb-llms" — MiniCPM4.1-8B + MiniCPM-o 4.5.
20
  app_google.py App "google-llms" — Gemma 4 12B + 26B.
21
  client.py OpenAI-SDK smoke-test client for any endpoint.
 
36
  | Provider | App | Model | Endpoint name | GPU |
37
  | -------- | -------------- | --------------------------------------- | --------------------- | ------- |
38
  | NVIDIA | `nvidia-llms` | NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | `nemotron-3-nano-30b` | H200:1 |
39
+ | NVIDIA | `nvidia-llms` | Nemotron-Cascade-14B-Thinking | `nemotron-cascade-14b-thinking` | L40S:1 |
40
  | NVIDIA | `nvidia-llms` | NVIDIA-Nemotron-3-Nano-4B-BF16 | `nemotron-3-nano-4b` | L4:1 |
41
  | OpenBMB | `openbmb-llms` | MiniCPM-o-4_5 (omni) | `minicpm-o-4-5` | L40S:1 |
42
  | OpenBMB | `openbmb-llms` | MiniCPM4.1-8B | `minicpm-4-1-8b` | L40S:1 |
modal/catalogue.py CHANGED
@@ -139,6 +139,23 @@ NVIDIA_MODELS: tuple[ModelConfig, ...] = (
139
  gated=True,
140
  max_concurrent_inputs=64,
141
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
142
  )
143
 
144
  # --- OpenBMB (MiniCPM) ---------------------------------------------------------
 
139
  gated=True,
140
  max_concurrent_inputs=64,
141
  ),
142
+ ModelConfig(
143
+ name="nvidia/Nemotron-Cascade-14B-Thinking",
144
+ endpoint_name="nemotron-cascade-14b-thinking",
145
+ # Dense 14B reasoning model built on Qwen3-14B Base; thinking-only. BF16
146
+ # weights (~28GB) plus KV cache fit a single 48GB L40S. A specialist
147
+ # model — left unbound so it can be cast explicitly at a reasoning-heavy
148
+ # agent (e.g. the Judge) without displacing a tier default.
149
+ params_b=14,
150
+ gpu="L40S:1",
151
+ max_model_len=32768,
152
+ # Qwen3-native in vLLM (no custom code); ChatML template with a thinking
153
+ # block parsed by the Qwen3 reasoning parser.
154
+ reasoning_parser="qwen3",
155
+ tool_call_parser="hermes",
156
+ enable_auto_tool_choice=True,
157
+ max_concurrent_inputs=48,
158
+ ),
159
  )
160
 
161
  # --- OpenBMB (MiniCPM) ---------------------------------------------------------