arturo-fredes commited on
Commit
6cf34e1
·
verified ·
1 Parent(s): f28684b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +91 -84
README.md CHANGED
@@ -3,15 +3,10 @@ license: apache-2.0
3
  language:
4
  - en
5
  base_model:
6
- - MultiverseComputingCAI/Pulsar-16B
7
  - nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
8
- tags:
9
- - fp8
10
- - quantized
11
- - compactifai
12
  ---
13
 
14
- # Pulsar 16B — FP8
15
  <div align="center">
16
 
17
  [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
@@ -20,7 +15,7 @@ tags:
20
 
21
  Powered by CompactifAI
22
 
23
- **Optimized for Fast and Efficient Inference** · **Reduced Memory Footprint** · **FP8 Quantization**
24
 
25
  </div>
26
 
@@ -30,7 +25,6 @@ Powered by CompactifAI
30
 
31
  - [Model Overview](#model-overview)
32
  - [Key Characteristics](#key-characteristics)
33
- - [Quantization](#quantization)
34
  - [Quick Start](#quick-start)
35
  - [Reasoning Control](#thinking-reasoning-control)
36
  - [Tool Calling](#tool-calling)
@@ -45,10 +39,14 @@ Powered by CompactifAI
45
 
46
  ## Model Overview
47
 
48
- **Pulsar 16B FP8** is the **FP8-quantized release** of [Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B), a model based on [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16), developed by **Multiverse Computing**. The original model is a **~31.6B parameter**, part of the Nemotron model family. It supports **long-context inference up to 1M tokens** and is designed for general-purpose language modeling tasks.
49
 
50
- The BF16 release applies **model compression techniques** to significantly reduce parameter count and deployment requirements while maintaining compatibility with the Nemotron Hybrid Mamba2-Transformer with MoE architecture. The resulting model achieves **50% compression**, reducing the parameter count to **16.15B parameters** and lowering memory requirements. **This checkpoint additionally applies FP8 quantization** on top of the compressed model.
51
 
 
 
 
 
52
  ---
53
 
54
  ## Key Characteristics
@@ -56,22 +54,12 @@ The BF16 release applies **model compression techniques** to significantly reduc
56
  | Characteristic | Description |
57
  |-----------------------|-------------|
58
  | Base model | [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). **31.6B** total parameters, **3.6B** activated per forward pass (11.34% activation ratio). [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
59
- | Pulsar (BF16 release) | [MultiverseComputingCAI/Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B). **16.15B** total parameters, **3.1B** activated per forward pass (19.28% activation ratio) after CompactifAI compression. |
60
- | Pulsar (this model) | **16.15B** parameters with weights quantized to **FP8**. |
61
  | 📐 **Architecture** | Hybrid Mamba2-Transformer with MoE (same family as the base checkpoint). |
62
  | 🛠️ **Tool calling** | Yes. Same tool-call structure and format as [Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). See [Tool Calling](#tool-calling). |
63
  | 🗜️ **Compression** | CompactifAI (proprietary compression technology) |
64
- | 🔢 **Quantization** | FP8 (weights). |
65
  | Primary language | English |
66
-
67
- ---
68
-
69
- ## Quantization
70
-
71
- This checkpoint is a quantized version of [Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B): its weights have been quantized to **FP8**.
72
-
73
  ---
74
-
75
  ## Quick Start
76
  This model can be loaded with the **Transformers** API. Use `trust_remote_code=True`. Recommended approach: `AutoModelForCausalLM` with `apply_chat_template`. This configuration has been tested with Transformers 4.57.6.
77
 
@@ -79,7 +67,7 @@ This model can be loaded with the **Transformers** API. Use `trust_remote_code=T
79
  import torch
80
  from transformers import AutoTokenizer, AutoModelForCausalLM
81
 
82
- model_id = "MultiverseComputingCAI/Pulsar-16B-FP8"
83
 
84
  tokenizer = AutoTokenizer.from_pretrained(
85
  model_id,
@@ -89,6 +77,7 @@ tokenizer = AutoTokenizer.from_pretrained(
89
  model = AutoModelForCausalLM.from_pretrained(
90
  model_id,
91
  device_map="cuda" if torch.cuda.is_available() else "auto",
 
92
  trust_remote_code=True,
93
  )
94
  messages = [
@@ -123,7 +112,7 @@ pip install -U "vllm>=0.12.0"
123
 
124
  #### Reasoning parser (NVIDIA)
125
 
126
- Pulsar uses the same Nemotron v3 reasoning tags as the base model. NVIDIA provides the vLLM plugin as [`nano_v3_reasoning_parser.py`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/blob/main/nano_v3_reasoning_parser.py) on the base Hugging Face repo (not specific to Pulsar). Direct download:
127
 
128
  ```bash
129
  wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py
@@ -134,7 +123,7 @@ You can keep any local filename; the `vllm serve` flags below assume the file is
134
  #### Serve
135
 
136
  ```bash
137
- vllm serve MultiverseComputingCAI/Pulsar-16B-FP8 \
138
  --served-model-name model \
139
  --max-num-seqs 8 \
140
  --tensor-parallel-size 1 \
@@ -152,9 +141,9 @@ vllm serve MultiverseComputingCAI/Pulsar-16B-FP8 \
152
 
153
  ## Thinking (Reasoning) Control
154
 
155
- Pulsar supports a **hybrid reasoning mode**: the model can either think step-by-step before answering (reasoning mode) or reply directly (non-reasoning mode). The behaviour is controlled via the `enable_thinking` flag in the chat template.
156
 
157
- > This section provides a brief overview of reasoning control in Pulsar. For comprehensive details please see the official Nemotron-3 Nano-30B model card at: https://build.nvidia.com/nvidia/nemotron-3-nano-30b-a3b/modelcard
158
 
159
 
160
  ---
@@ -210,7 +199,7 @@ Set the default for **all requests** at startup with `--default-chat-template-kw
210
 
211
  **Thinking OFF for all requests**
212
  ```bash
213
- vllm serve MultiverseComputingCAI/Pulsar-16B-FP8 \
214
  --served-model-name model \
215
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
216
  --reasoning-parser nano_v3 \
@@ -221,7 +210,7 @@ vllm serve MultiverseComputingCAI/Pulsar-16B-FP8 \
221
 
222
  **Thinking ON for all requests (default if flag is omitted)**
223
  ```bash
224
- vllm serve MultiverseComputingCAI/Pulsar-16B-FP8 \
225
  --served-model-name model \
226
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
227
  --reasoning-parser nano_v3 \
@@ -256,7 +245,7 @@ response = requests.post("http://localhost:8000/v1/chat/completions", json={
256
 
257
  ## Tool Calling
258
 
259
- Pulsar emits tool calls in the following format:
260
 
261
  ```
262
  <tool_call>
@@ -287,61 +276,87 @@ The base model [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.
287
 
288
  CompactifAI was applied to produce a smaller, efficient model (16B parameters) while aiming to preserve reasoning and tool-use capabilities. Supervised Fine Tuning was applied for improving cabapilities.
289
 
290
- ### FP8 Quantization
291
-
292
- After compression and fine-tuning, the BF16 checkpoint was quantized to FP8.
293
-
294
  ---
295
 
296
  ## Evaluation & Benchmarks
297
 
298
  ![Combined benchmark chart](assets/benchmarks.png)
299
 
300
- | Benchmark | Pulsar-16B (BF16) | Pulsar-16B (FP8) | Pulsar-16B (NVFP4) |
301
- |----------------|-------------------:|------------------:|--------------------:|
302
- | SciCode | 25.84 | 23.08 | 23.28 |
303
- | AA-LCR | 29.33 | 21.33 | 25.67 |
304
- | IFBench | 70.79 | 69.60 | 69.90 |
305
- | GPQA-D | 71.41 | 70.61 | 71.11 |
306
- | AIME 2025 | 87.22 | 86.67 | 82.00 |
307
- | MMLU-Pro | 74.78 | 74.76 | 74.19 |
308
- | LiveCodeBench | 68.04 | 68.68 | 65.60 |
309
- | Tau2-bench | 37.43 | 31.55 | 35.92 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
310
 
311
  ### Performance
312
  ![Performance results](assets/performance.png)
313
  - **Framework:** [guidellm](https://github.com/vllm-project/guidellm)
314
  - **Inference:** vLLM 0.18.0
315
- - **Decode:** temperature=1.0, top_p=1.0
316
- - **Phase duration:** Each phase lasts 3 minutes (excluding ramp-up and cool-down periods).
317
- - **Workload shape:** Sweep across different levels of concurrency. 180s measure windows with ramp-up and cooldown excluded. 8k/16k workload as in the original model's card.
318
- - **Estimated Peak Memory Usage:** KV cache utilization is monitored during the phase and we estimate memory usage as follows: `model_ weights_{gb} + kv_ cache_{usage_pct} × (nvml_used_{gb} − model_ weights_{gb})`
319
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
320
  ### Evaluation Methodology
321
 
322
  Benchmark scores were obtained with the following setups. Methodology varies by benchmark family.
323
 
324
- #### MMLU-Pro, AIME25, GPQA-D, LiveCodeBench, IFBench, AA-LCR, SciCode
325
-
326
- - **Framework**: [Nemo-Skills](https://github.com/NVIDIA-NeMo/Skills)
327
- - **Inference**: vLLM 0.18.0
328
- - **Decode**: temp 1.0, top_p 1.0
329
-
330
- #### Tau2-bench (Telecom)
331
-
332
- - **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
333
- - **Inference**: vLLM 0.18.0
334
- - **Decode (agent)**: temp 0.6, top_p 0.95
335
- - **Decode (judge / user sim)**: temp 0.7, top_p 0.8, top_k 20, min_p 0
336
- - **Repro**: telecom subset; max steps 100; 3 runs; judge parser hermes
337
-
338
- #### AA-LCR
339
-
340
- - **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
341
- - **Dataset**: `evalscope/AA-LCR` from ModelScope, default test split, 300 examples, 3 repeats.
342
- - **Inference**: vLLM 0.18.0
343
- - **Decode**: temperature 1.0, top_p 1.0
344
- - **Decode (judge)**: judge top_p 0.8, top_k 20, min_p 0.0; pattern scoring.
345
 
346
  ---
347
 
@@ -360,7 +375,7 @@ Trained mainly on English with added Spanish. No systematic evaluation for langu
360
 
361
  - English-centric training data (inherited from base model).
362
  - Tool calling depends on correct schema and tool design; exact parity with the original model is not guaranteed.
363
- - Compression and quantization may affect some behaviors; evaluate for your use case.
364
 
365
  ### Recommendations
366
 
@@ -374,9 +389,8 @@ Trained mainly on English with added Spanish. No systematic evaluation for langu
374
 
375
  | Field | Value |
376
  |--------------|--------------------- |
377
- | Model name | Pulsar 16B FP8 |
378
- | Based on | [MultiverseComputingCAI/Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B) (originally [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)) |
379
- | Precision | FP8 weights |
380
  | Version | v1.5.0 |
381
  | Release date | TBD |
382
  | Developed by | Multiverse Computing |
@@ -387,7 +401,7 @@ Trained mainly on English with added Spanish. No systematic evaluation for langu
387
 
388
  ## Citation
389
 
390
- If you use this model, please cite the base model and pulsar:
391
 
392
  ```bibtex
393
  @misc{nemotron3nanoTR,
@@ -396,20 +410,13 @@ If you use this model, please cite the base model and pulsar:
396
  year = {2025},
397
  url = {https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf}
398
  }
399
- @misc{pulsar16b,
400
- title = {Pulsar 16B: Model developed from NVIDIA Nemotron-3-Nano-30B-A3B},
401
  author = {Multiverse Computing},
402
  year = {2026},
403
- url = {https://huggingface.co/MultiverseComputingCAI/Pulsar-16B},
404
  note = {Model developed based on nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 using CompactifAI technology}
405
  }
406
- @misc{pulsar16bfp8,
407
- title = {Pulsar 16B FP8: FP8-quantized release of Pulsar-16B},
408
- author = {Multiverse Computing},
409
- year = {2026},
410
- url = {https://huggingface.co/MultiverseComputingCAI/Pulsar-16B-FP8},
411
- note = {FP8 weight quantization of Pulsar-16B}
412
- }
413
  ```
414
 
415
  **Built by [Multiverse Computing](https://www.multiversecomputing.com)** · [Report an issue](TODO_PULSAR_HF_URL/discussions) · [Discord](https://discord.gg/cGas9uStqp)
 
3
  language:
4
  - en
5
  base_model:
 
6
  - nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
 
 
 
 
7
  ---
8
 
9
+ # Nemotron 3 Nano Slim 16B
10
  <div align="center">
11
 
12
  [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 
15
 
16
  Powered by CompactifAI
17
 
18
+ **Optimized for Fast and Efficient Inference** · **Reduced Memory Footprint**
19
 
20
  </div>
21
 
 
25
 
26
  - [Model Overview](#model-overview)
27
  - [Key Characteristics](#key-characteristics)
 
28
  - [Quick Start](#quick-start)
29
  - [Reasoning Control](#thinking-reasoning-control)
30
  - [Tool Calling](#tool-calling)
 
39
 
40
  ## Model Overview
41
 
42
+ **Nemotron 3 Nano Slim 16B** is a **model based on [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)**, developed by **Multiverse Computing**. The original model is a **~31.6B parameter**, part of the Nemotron model family. It supports **long-context inference up to 1M tokens** and is designed for general-purpose language modeling tasks.
43
 
44
+ This version applies **model compression techniques** to significantly reduce parameter count and deployment requirements while maintaining compatibility with the Nemotron Hybrid Mamba2-Transformer with MoE architecture. The resulting model achieves **50% compression**, reducing the parameter count to **16.15B parameters** and lowering memory requirements.
45
 
46
+ - [BF16](https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16)
47
+ - [FP8](https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-FP8)
48
+ - [NVFP4](https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-NVFP4)
49
+
50
  ---
51
 
52
  ## Key Characteristics
 
54
  | Characteristic | Description |
55
  |-----------------------|-------------|
56
  | Base model | [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). **31.6B** total parameters, **3.6B** activated per forward pass (11.34% activation ratio). [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
57
+ | Nemotron-3-Nano-Slim-16B-BF16 (this model) | **16.15B** total parameters, **3.1B** activated per forward pass (19.28% activation ratio) after CompactifAI compression. |
 
58
  | 📐 **Architecture** | Hybrid Mamba2-Transformer with MoE (same family as the base checkpoint). |
59
  | 🛠️ **Tool calling** | Yes. Same tool-call structure and format as [Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). See [Tool Calling](#tool-calling). |
60
  | 🗜️ **Compression** | CompactifAI (proprietary compression technology) |
 
61
  | Primary language | English |
 
 
 
 
 
 
 
62
  ---
 
63
  ## Quick Start
64
  This model can be loaded with the **Transformers** API. Use `trust_remote_code=True`. Recommended approach: `AutoModelForCausalLM` with `apply_chat_template`. This configuration has been tested with Transformers 4.57.6.
65
 
 
67
  import torch
68
  from transformers import AutoTokenizer, AutoModelForCausalLM
69
 
70
+ model_id = "MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16"
71
 
72
  tokenizer = AutoTokenizer.from_pretrained(
73
  model_id,
 
77
  model = AutoModelForCausalLM.from_pretrained(
78
  model_id,
79
  device_map="cuda" if torch.cuda.is_available() else "auto",
80
+ torch_dtype=torch.bfloat16,
81
  trust_remote_code=True,
82
  )
83
  messages = [
 
112
 
113
  #### Reasoning parser (NVIDIA)
114
 
115
+ Nemotron 3 Nano Slim 16B uses the same Nemotron v3 reasoning tags as the base model. NVIDIA provides the vLLM plugin as [`nano_v3_reasoning_parser.py`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/blob/main/nano_v3_reasoning_parser.py) on the base Hugging Face repo (not specific to Pulsar). Direct download:
116
 
117
  ```bash
118
  wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py
 
123
  #### Serve
124
 
125
  ```bash
126
+ vllm serve MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16 \
127
  --served-model-name model \
128
  --max-num-seqs 8 \
129
  --tensor-parallel-size 1 \
 
141
 
142
  ## Thinking (Reasoning) Control
143
 
144
+ Nemotron 3 Nano Slim 16B supports a **hybrid reasoning mode**: the model can either think step-by-step before answering (reasoning mode) or reply directly (non-reasoning mode). The behaviour is controlled via the `enable_thinking` flag in the chat template.
145
 
146
+ > This section provides a brief overview of reasoning control in Nemotron 3 Nano Slim 16B. For comprehensive details please see the official Nemotron-3 Nano-30B model card at: https://build.nvidia.com/nvidia/nemotron-3-nano-30b-a3b/modelcard
147
 
148
 
149
  ---
 
199
 
200
  **Thinking OFF for all requests**
201
  ```bash
202
+ vllm serve MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16 \
203
  --served-model-name model \
204
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
205
  --reasoning-parser nano_v3 \
 
210
 
211
  **Thinking ON for all requests (default if flag is omitted)**
212
  ```bash
213
+ vllm serve MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16 \
214
  --served-model-name model \
215
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
216
  --reasoning-parser nano_v3 \
 
245
 
246
  ## Tool Calling
247
 
248
+ Nemotron 3 Nano Slim 16B emits tool calls in the following format:
249
 
250
  ```
251
  <tool_call>
 
276
 
277
  CompactifAI was applied to produce a smaller, efficient model (16B parameters) while aiming to preserve reasoning and tool-use capabilities. Supervised Fine Tuning was applied for improving cabapilities.
278
 
 
 
 
 
279
  ---
280
 
281
  ## Evaluation & Benchmarks
282
 
283
  ![Combined benchmark chart](assets/benchmarks.png)
284
 
285
+ | Benchmark | Nemotron 3 Nano 30B A3B | Nemotron 3 Nano Slim 16B | gpt-oss-20b | Qwen3-14B | Ministral-3-14B-Instruct-2512 |
286
+ | --- | ---: | ---: | ---: | ---: | ---: |
287
+ | AIME | 87.66 | 87.22 | 87.66 | 76.00 | 33.00 |
288
+ | GPQA | 74.04 | 71.41 | 68.99 | 63.63 | 56.45 |
289
+ | IFBench | 72.31 | 70.79 | 68.46 | 39.20 | 32.80 |
290
+ | MMLU-Pro | 78.90 | 74.78 | 76.65 | 85.01 | 70.09 |
291
+ | LiveCodeBench | 71.11 | 68.04 | 64.65 | 66.35 | 29.84 |
292
+
293
+ ### Quantizations
294
+
295
+ - [BF16](https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16)
296
+ - [FP8](https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-FP8)
297
+ - [NVFP4](https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-NVFP4)
298
+
299
+ ![Quantization results](assets/quantization_comparisons.png)
300
+
301
+ | Benchmark | Nemotron 3 Nano 30B A3B | Nemotron 3 Nano Slim 16B (BF16) | Nemotron 3 Nano Slim 16B (fp8) | Nemotron 3 Nano Slim 16B (nvfp4) |
302
+ | --- | ---: | ---: | ---: | ---: |
303
+ | AIME | 87.66 | 87.22 | 86.67 | 82.00 |
304
+ | GPQA | 74.04 | 71.41 | 70.61 | 71.11 |
305
+ | IFBench | 72.31 | 70.79 | 69.60 | 69.90 |
306
+ | MMLU-Pro | 78.90 | 74.78 | 74.76 | 74.19 |
307
+ | LiveCodeBench | 71.11 | 68.04 | 68.68 | 65.60 |
308
+
309
 
310
  ### Performance
311
  ![Performance results](assets/performance.png)
312
  - **Framework:** [guidellm](https://github.com/vllm-project/guidellm)
313
  - **Inference:** vLLM 0.18.0
314
+ - **GPU:** NVIDIA B200
315
+ - **Decode:** `temperature: 0.0`, `top_p: 1.0`
316
+ - **Measure Window:** Each phase lasts 3 minutes (excluding ramp-up and cool-down periods).
317
+ - **Workload shape:** 8k/16k workload as in the original model's card.
318
+
319
+ | Model | System throughput (tok/s, p50) | TTFT (s, p50) | Model weights (GB) |
320
+ | :--- | ---: | ---: | ---: |
321
+ | Nemotron 3 Nano 30B A3B | 3,363 | 2.18 | 59 |
322
+ | Nemotron 3 Nano Slim 16B (BF16) | 3,760 | 1.80 | 30 |
323
+ | Nemotron 3 Nano Slim 16B (fp8) | 4,808 | 1.24 | 16 |
324
+ | Nemotron 3 Nano Slim 16B (nvfp4) | 4,735 | 1.25 | 10 |
325
+
326
+ ### Long Context
327
+ Nemotron 3 Nano Slim 16B preserves strong long-context behavior after compression, tracking the Nemotron-3-Nano-30B-A3B baseline closely across retrieval-heavy and full-suite long-context evaluations. Results are reported for LongBench v1, AA-LCR, NIAH, and RULER groupings up to 256k context.
328
+
329
+ ![Long-context benchmark results](assets/long_context_comparison.png)
330
+
331
+ | Benchmark | Nemotron 3 Nano 30B A3B | Nemotron 3 Nano Slim 16B |
332
+ | :--- | ---: | ---: |
333
+ | Longbench | 31.84 | 29.84 |
334
+ | AA-LCR | 33.67 | 29.33 |
335
+ | NIAH (@100K) | 100.00 | 100.00 |
336
+ | RULER (@128K) | 95.02 | 94.20 |
337
+ | RULER (@256K) | 92.02 | 87.74 |
338
  ### Evaluation Methodology
339
 
340
  Benchmark scores were obtained with the following setups. Methodology varies by benchmark family.
341
 
342
+ ### Inference:
343
+ - **Backend:** VLLM 0.18.0
344
+ - **Nemotron models:** `temp 1.0`, `top_p 1.0`
345
+ - **GPT-OSS-20B:** `temp: 1.0`, `top_p: 1.0`, `reasoning_effort: high`
346
+ - **Qwen3-14B:** `temp: 0.6`, `top_p: 0.95`, `top_k: 20`, `min_p: 0.0`
347
+ - **Ministral-3-14B-Instruct-2512:** `temp: 0.15`
348
+
349
+ | Benchmark | Framework | Repeats | Other |
350
+ |-----------|-----------|--------:|-------|
351
+ | MMLU-Pro | [NeMo-Skills](https://github.com/NVIDIA-NeMo/Skills) | 1 | |
352
+ | AIME25 | [NeMo-Skills](https://github.com/NVIDIA-NeMo/Skills) | 10 | |
353
+ | GPQA:d | [NeMo-Skills](https://github.com/NVIDIA-NeMo/Skills) | 5 | |
354
+ | LiveCodeBench | [NeMo-Skills](https://github.com/NVIDIA-NeMo/Skills) | 3 | |
355
+ | IFBench | [NeMo-Skills](https://github.com/NVIDIA-NeMo/Skills) | 5 | |
356
+ | LongBench v1 | [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) | 1 | |
357
+ | AA-LCR | [EvalScope](https://github.com/modelscope/evalscope) 1.4.1 | 3 | Judge: **`Qwen/Qwen3-235B-A22B-Instruct-2507`**. **`judge_score_type`:** `pattern`. **`judge_args` → `generation_config`:** `top_p` 0.8, `top_k` 20, `min_p` 0.0, `temperature` 0.7. |
358
+ | NIAH | [EvalScope](https://github.com/modelscope/evalscope) 1.4.1 | 1 | Judge: **`qwen/qwen3-235b-a22b-2507`** . **`judge_model_args`:** `{}` (no extra judge settings in YAML). |
359
+ | RULER | [NeMo-Skills](https://github.com/NVIDIA-NeMo/Skills) (+ [RULER](https://github.com/NVIDIA/RULER)) | 1 | |
 
 
 
360
 
361
  ---
362
 
 
375
 
376
  - English-centric training data (inherited from base model).
377
  - Tool calling depends on correct schema and tool design; exact parity with the original model is not guaranteed.
378
+ - Compression may affect some behaviors; evaluate for your use case.
379
 
380
  ### Recommendations
381
 
 
389
 
390
  | Field | Value |
391
  |--------------|--------------------- |
392
+ | Model name | Nemotron 3 Nano Slim 16B |
393
+ | Based on | [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) |
 
394
  | Version | v1.5.0 |
395
  | Release date | TBD |
396
  | Developed by | Multiverse Computing |
 
401
 
402
  ## Citation
403
 
404
+ If you use this model, please cite the base model and Nemotron 3 Nano Slim 16B:
405
 
406
  ```bibtex
407
  @misc{nemotron3nanoTR,
 
410
  year = {2025},
411
  url = {https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf}
412
  }
413
+ @misc{nemotron3nanoslim16b,
414
+ title = {Nemotron 3 Nano Slim 16B: Model developed from NVIDIA Nemotron-3-Nano-30B-A3B},
415
  author = {Multiverse Computing},
416
  year = {2026},
417
+ url = {https://huggingface.co/MultiverseComputingCAI/Nemotron-3-Nano-Slim-16B-BF16},
418
  note = {Model developed based on nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 using CompactifAI technology}
419
  }
 
 
 
 
 
 
 
420
  ```
421
 
422
  **Built by [Multiverse Computing](https://www.multiversecomputing.com)** · [Report an issue](TODO_PULSAR_HF_URL/discussions) · [Discord](https://discord.gg/cGas9uStqp)