NullSense commited on
Commit
58fb2ad
·
verified ·
1 Parent(s): f3f36f8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -82,6 +82,23 @@ transformers-5 loading (llmcompressor, finetuning); vLLM serving does not use th
82
  | [NVFP4-FP8-LoopShield](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4-FP8-LoopShield) | 4-bit family: GSM8K parity with FP8, best JSON, −0.4 GB |
83
  | [NVFP4A16](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4A16) | speed/summarization: 175/220 tok/s — NOT for reasoning (gates published) |
84
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
  ## Links & provenance
86
 
87
  - Base model: [Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B)
 
82
  | [NVFP4-FP8-LoopShield](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4-FP8-LoopShield) | 4-bit family: GSM8K parity with FP8, best JSON, −0.4 GB |
83
  | [NVFP4A16](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4A16) | speed/summarization: 175/220 tok/s — NOT for reasoning (gates published) |
84
 
85
+
86
+ ## Full matrix vs the bf16 original
87
+
88
+ | metric | bf16 original | FP8 (this repo) |
89
+ |---|---|---|
90
+ | freeform tok/s (spec-off) / summarize (+ngram) | 96 / 136 | **154 / 206** |
91
+ | rewrite taxonomy (/20, non-think) | 18 | 18 |
92
+ | blind-judge summarization (same judging pass) | 4.61 | 4.55 (noise-level delta) |
93
+ | JSON parse rate (/48) | 44 | 42 |
94
+ | GSM8K, IFEval, MMLU-Pro, BBH, MultiHop-RAG | not measured on bf16 | measured (tables above) |
95
+
96
+ Where bf16 was measured, FP8 shows no regression; that is why FP8 serves as the
97
+ reference for the rest of the family instead of re-running every gate on bf16.
98
+ The vendor's published numbers are a different suite entirely (agentic/frontier
99
+ benches, thinking mode at a 131k token budget) and don't map onto these rows —
100
+ see Benchmark provenance below.
101
+
102
  ## Links & provenance
103
 
104
  - Base model: [Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B)