daniicruzz commited on
Commit
ecfecfc
·
verified ·
1 Parent(s): b12313d

Simplify quantization section

Browse files
Files changed (1) hide show
  1. README.md +11 -19
README.md CHANGED
@@ -8,7 +8,6 @@ base_model:
8
  tags:
9
  - fp8
10
  - quantized
11
- - modelopt
12
  - compactifai
13
  ---
14
 
@@ -19,9 +18,9 @@ tags:
19
  [![HuggingFace](https://img.shields.io/badge/🤗-Model_Hub-yellow.svg)](TODO_PULSAR_HF_URL)
20
  [![Discord](https://img.shields.io/badge/Discord-Community-5865F2?logo=discord&logoColor=white)](https://discord.gg/cGas9uStqp)
21
 
22
- Powered by CompactifAI · Quantized with NVIDIA TensorRT Model Optimizer
23
 
24
- **Optimized for Fast and Efficient Inference** · **Reduced Memory Footprint** · **FP8 Post-Training Quantization**
25
 
26
  </div>
27
 
@@ -48,7 +47,7 @@ Powered by CompactifAI · Quantized with NVIDIA TensorRT Model Optimizer
48
 
49
  **Pulsar 16B FP8** is the **FP8-quantized release** of [Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B), a model based on [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16), developed by **Multiverse Computing**. The original model is a **~31.6B parameter**, part of the Nemotron model family. It supports **long-context inference up to 1M tokens** and is designed for general-purpose language modeling tasks.
50
 
51
- The BF16 release applies **model compression techniques** to significantly reduce parameter count and deployment requirements while maintaining compatibility with the Nemotron Hybrid Mamba2-Transformer with MoE architecture. The resulting model achieves **50% compression**, reducing the parameter count to **16.15B parameters** and lowering memory requirements. **This checkpoint additionally applies FP8 post-training quantization** on top of the compressed model.
52
 
53
  ---
54
 
@@ -58,24 +57,18 @@ The BF16 release applies **model compression techniques** to significantly reduc
58
  |-----------------------|-------------|
59
  | Base model | [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). **31.6B** total parameters, **3.6B** activated per forward pass (11.34% activation ratio). [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
60
  | Pulsar (BF16 release) | [MultiverseComputingCAI/Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B). **16.15B** total parameters, **3.1B** activated per forward pass (19.28% activation ratio) after CompactifAI compression. |
61
- | Pulsar (this model) | **16.15B** parameters quantized to **FP8** (weights & activations). |
62
  | 📐 **Architecture** | Hybrid Mamba2-Transformer with MoE (same family as the base checkpoint). |
63
  | 🛠️ **Tool calling** | Yes. Same tool-call structure and format as [Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). See [Tool Calling](#tool-calling). |
64
  | 🗜️ **Compression** | CompactifAI (proprietary compression technology) |
65
- | 🔢 **Quantization** | FP8 post-training quantization via [NVIDIA TensorRT Model Optimizer](https://github.com/NVIDIA/TensorRT-Model-Optimizer). |
66
  | Primary language | English |
67
 
68
  ---
69
 
70
  ## Quantization
71
 
72
- This checkpoint is a **post-training-quantized (PTQ)** version of [Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B) produced with [NVIDIA TensorRT Model Optimizer](https://github.com/NVIDIA/TensorRT-Model-Optimizer) (`modelopt` v0.42.0).
73
-
74
- | Field | Value |
75
- |---------------------|-------|
76
- | Quantization scheme | **FP8** — weights & activations |
77
- | Producer | `modelopt` 0.42.0 |
78
- | Excluded modules | `lm_head`, all Mamba2 `conv1d` modules, and the attention/Mamba projections (`q/k/v/o_proj`, `in_proj`, `out_proj`) of selected layers (kept in higher precision). |
79
 
80
  ---
81
 
@@ -294,9 +287,9 @@ The base model [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.
294
 
295
  CompactifAI was applied to produce a smaller, efficient model (16B parameters) while aiming to preserve reasoning and tool-use capabilities. Supervised Fine Tuning was applied for improving cabapilities.
296
 
297
- ### FP8 Post-Training Quantization
298
 
299
- After compression and fine-tuning, the BF16 checkpoint was quantized to FP8 using [NVIDIA TensorRT Model Optimizer](https://github.com/NVIDIA/TensorRT-Model-Optimizer) (`modelopt` 0.42.0). A few selected modules (`lm_head`, all Mamba2 `conv1d` layers and the attention/Mamba projections of selected layers) were kept in higher precision.
300
 
301
  ---
302
 
@@ -417,8 +410,7 @@ Trained mainly on English with added Spanish. No systematic evaluation for langu
417
  |--------------|--------------------- |
418
  | Model name | Pulsar 16B FP8 |
419
  | Based on | [MultiverseComputingCAI/Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B) (originally [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)) |
420
- | Precision | FP8 weights & activations |
421
- | Quantization | NVIDIA TensorRT Model Optimizer (`modelopt` 0.42.0) |
422
  | Version | v1.5.0 |
423
  | Release date | TBD |
424
  | Developed by | Multiverse Computing |
@@ -446,11 +438,11 @@ If you use this model, please cite the base model and pulsar:
446
  note = {Model developed based on nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 using CompactifAI technology}
447
  }
448
  @misc{pulsar16bfp8,
449
- title = {Pulsar 16B FP8: FP8 post-training-quantized release of Pulsar-16B},
450
  author = {Multiverse Computing},
451
  year = {2026},
452
  url = {https://huggingface.co/MultiverseComputingCAI/Pulsar-16B-FP8},
453
- note = {FP8 quantization produced with NVIDIA TensorRT Model Optimizer}
454
  }
455
  ```
456
 
 
8
  tags:
9
  - fp8
10
  - quantized
 
11
  - compactifai
12
  ---
13
 
 
18
  [![HuggingFace](https://img.shields.io/badge/🤗-Model_Hub-yellow.svg)](TODO_PULSAR_HF_URL)
19
  [![Discord](https://img.shields.io/badge/Discord-Community-5865F2?logo=discord&logoColor=white)](https://discord.gg/cGas9uStqp)
20
 
21
+ Powered by CompactifAI
22
 
23
+ **Optimized for Fast and Efficient Inference** · **Reduced Memory Footprint** · **FP8 Quantization**
24
 
25
  </div>
26
 
 
47
 
48
  **Pulsar 16B FP8** is the **FP8-quantized release** of [Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B), a model based on [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16), developed by **Multiverse Computing**. The original model is a **~31.6B parameter**, part of the Nemotron model family. It supports **long-context inference up to 1M tokens** and is designed for general-purpose language modeling tasks.
49
 
50
+ The BF16 release applies **model compression techniques** to significantly reduce parameter count and deployment requirements while maintaining compatibility with the Nemotron Hybrid Mamba2-Transformer with MoE architecture. The resulting model achieves **50% compression**, reducing the parameter count to **16.15B parameters** and lowering memory requirements. **This checkpoint additionally applies FP8 quantization** on top of the compressed model.
51
 
52
  ---
53
 
 
57
  |-----------------------|-------------|
58
  | Base model | [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). **31.6B** total parameters, **3.6B** activated per forward pass (11.34% activation ratio). [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
59
  | Pulsar (BF16 release) | [MultiverseComputingCAI/Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B). **16.15B** total parameters, **3.1B** activated per forward pass (19.28% activation ratio) after CompactifAI compression. |
60
+ | Pulsar (this model) | **16.15B** parameters with weights quantized to **FP8**. |
61
  | 📐 **Architecture** | Hybrid Mamba2-Transformer with MoE (same family as the base checkpoint). |
62
  | 🛠️ **Tool calling** | Yes. Same tool-call structure and format as [Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). See [Tool Calling](#tool-calling). |
63
  | 🗜️ **Compression** | CompactifAI (proprietary compression technology) |
64
+ | 🔢 **Quantization** | FP8 (weights). |
65
  | Primary language | English |
66
 
67
  ---
68
 
69
  ## Quantization
70
 
71
+ This checkpoint is a quantized version of [Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B): its weights have been quantized to **FP8**.
 
 
 
 
 
 
72
 
73
  ---
74
 
 
287
 
288
  CompactifAI was applied to produce a smaller, efficient model (16B parameters) while aiming to preserve reasoning and tool-use capabilities. Supervised Fine Tuning was applied for improving cabapilities.
289
 
290
+ ### FP8 Quantization
291
 
292
+ After compression and fine-tuning, the BF16 checkpoint was quantized to FP8.
293
 
294
  ---
295
 
 
410
  |--------------|--------------------- |
411
  | Model name | Pulsar 16B FP8 |
412
  | Based on | [MultiverseComputingCAI/Pulsar-16B](https://huggingface.co/MultiverseComputingCAI/Pulsar-16B) (originally [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)) |
413
+ | Precision | FP8 weights |
 
414
  | Version | v1.5.0 |
415
  | Release date | TBD |
416
  | Developed by | Multiverse Computing |
 
438
  note = {Model developed based on nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 using CompactifAI technology}
439
  }
440
  @misc{pulsar16bfp8,
441
+ title = {Pulsar 16B FP8: FP8-quantized release of Pulsar-16B},
442
  author = {Multiverse Computing},
443
  year = {2026},
444
  url = {https://huggingface.co/MultiverseComputingCAI/Pulsar-16B-FP8},
445
+ note = {FP8 weight quantization of Pulsar-16B}
446
  }
447
  ```
448