Sera 14B request
Hi, thanks for creating this quantization!
Are you planning to quantize the 14B version of the SERA family?
Also, can you share the recipe (or point me in the right direction) for the quantization process?
Thanks again!
Yeah I can run Sera 14B. I'm not sure why I didn't do it along with the 8B and the 32B back when I made those.
As for the recipe, I have shared it over on this comment thread here:
https://huggingface.co/Firworks/MiroThinker-v1.0-30B-nvfp4/discussions/1#69269c6d40ce1d3b1a6ca1cc
The only thing I've updated since then is excluding out some patterns for vision elements to support multimodal models:
ignore=[
"lm_head",
"re:visual.*", # everything under the `visual` tower
"re:.*vision_tower.*", # common naming in other VL models
"re:.*video_tower.*", # in case the checkpoint has video tower
"re:.*audio_tower.*", # if there’s an audio tower
"re:.*multi_modal_projector.*" # bridge projector, if present
],
It often needs some specific tweaks per model which one of the frontier LLMs should be able to help with. NVFP4 quantization is a bit clunky.