Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -82,6 +82,23 @@ transformers-5 loading (llmcompressor, finetuning); vLLM serving does not use th
|
|
| 82 |
| [NVFP4-FP8-LoopShield](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4-FP8-LoopShield) | 4-bit family: GSM8K parity with FP8, best JSON, −0.4 GB |
|
| 83 |
| [NVFP4A16](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4A16) | speed/summarization: 175/220 tok/s — NOT for reasoning (gates published) |
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
## Links & provenance
|
| 86 |
|
| 87 |
- Base model: [Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B)
|
|
|
|
| 82 |
| [NVFP4-FP8-LoopShield](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4-FP8-LoopShield) | 4-bit family: GSM8K parity with FP8, best JSON, −0.4 GB |
|
| 83 |
| [NVFP4A16](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4A16) | speed/summarization: 175/220 tok/s — NOT for reasoning (gates published) |
|
| 84 |
|
| 85 |
+
|
| 86 |
+
## Full matrix vs the bf16 original
|
| 87 |
+
|
| 88 |
+
| metric | bf16 original | FP8 (this repo) |
|
| 89 |
+
|---|---|---|
|
| 90 |
+
| freeform tok/s (spec-off) / summarize (+ngram) | 96 / 136 | **154 / 206** |
|
| 91 |
+
| rewrite taxonomy (/20, non-think) | 18 | 18 |
|
| 92 |
+
| blind-judge summarization (same judging pass) | 4.61 | 4.55 (noise-level delta) |
|
| 93 |
+
| JSON parse rate (/48) | 44 | 42 |
|
| 94 |
+
| GSM8K, IFEval, MMLU-Pro, BBH, MultiHop-RAG | not measured on bf16 | measured (tables above) |
|
| 95 |
+
|
| 96 |
+
Where bf16 was measured, FP8 shows no regression; that is why FP8 serves as the
|
| 97 |
+
reference for the rest of the family instead of re-running every gate on bf16.
|
| 98 |
+
The vendor's published numbers are a different suite entirely (agentic/frontier
|
| 99 |
+
benches, thinking mode at a 131k token budget) and don't map onto these rows —
|
| 100 |
+
see Benchmark provenance below.
|
| 101 |
+
|
| 102 |
## Links & provenance
|
| 103 |
|
| 104 |
- Base model: [Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B)
|