MarshallHD/LFM2.5-230M-NVFP4

LFM2.5-230M NVFP4 Model Summary

The LFM2.5-230M NVFP4 model is a quantized version of Liquid AI's LFM2.5-230M, a compact hybrid language model designed for edge and on-device inference. This checkpoint was produced with NVIDIA Model Optimizer from GitHub main and exported in NVFP4 4-bit floating point format.

This release uses a conservative ModelOpt recipe: selected inner feed-forward linear operators are quantized, while attention, convolutional blocks, embeddings, normalization layers, the language-model head, early layers, and final layers remain in higher precision.

🚀 Key Specifications

• Architecture: Lfm2ForCausalLM, 14-layer hybrid architecture with LIV convolution and GQA blocks.
• Parameters: 230M.
• Base Model: LiquidAI/LFM2.5-230M.
• Precision: NVFP4 for selected feed-forward linear weights and activations.
• Quantization Scope: feed-forward linear operators in inner layers 3-12.
• Protected Modules: embeddings, lm_head, all self_attn* paths, all conv* paths, normalization layers, layers 0-2, and layer 13.
• Group Size: 16.
• Context Length: 32,768 tokens.
• KV Cache Quantization: none.
• License: inherits the base model's lfm1.0 license.
• Release Date: June 27, 2026.

🛠️ Software & Hardware Integration

• Quantization Tooling: NVIDIA Model Optimizer from GitHub main.
• ModelOpt Version: 0.0.1.dev1+g4b04e732e.
• ModelOpt Commit: 4b04e732ed9d84b26e23d1c9330771de2175a830.
• Torch: 2.11.0+cu130.
• Validated Runtime: vLLM with --quantization modelopt_fp4.
• Validated FP4 Kernel: FlashInferCutlassNvFp4LinearKernel.
• Validated Hardware: NVIDIA GB10.
• Operating System: Linux.

Deployment Example, vLLM

pip install -U "transformers>=5.3.0" vllm

MODEL_ID="MarshallHD/LFM2.5-230M-NVFP4"

vllm serve "$MODEL_ID" \
    --quantization modelopt_fp4 \
    --trust-remote-code \
    --max-model-len 32768

Python Example, vLLM

from vllm import LLM, SamplingParams

model_id = "MarshallHD/LFM2.5-230M-NVFP4"

llm = LLM(
    model=model_id,
    quantization="modelopt_fp4",
    trust_remote_code=True,
    max_model_len=32768,
)

outputs = llm.generate(
    ["Explain NVFP4 quantization in one sentence."],
    SamplingParams(max_tokens=64, temperature=0.1),
)
print(outputs[0].outputs[0].text)

This checkpoint is intended for runtimes with ModelOpt FP4 support, especially vLLM or SGLang using modelopt_fp4.

📊 Performance & Evaluation

The model was evaluated on 24,063 multiple-choice examples using prompt log-probability scoring through vLLM.

Metric	BF16 Reference	NVFP4 PTQ	Δ vs BF16
Weighted MCQA	28.284	27.731	-0.553
MMLU-Pro	20.188	19.955	-0.233
ARC-Challenge	40.956	37.799	-3.157
HellaSwag	36.537	35.780	-0.757
TruthfulQA MC1	27.907	28.886	+0.979

All values are accuracies reported as percentages. The weighted MCQA score is weighted by the number of examples in each benchmark.

📚 Datasets

Calibration Data

The calibration set is a mixed public text corpus rendered through the LFM2.5 tokenizer. It includes instruction-following, science, coding, math, software-engineering, agentic, and multilingual examples.

• Requested calibration rows: 8,192.
• Usable rendered calibration samples: 6,737.
• Calibration tokens: 3,414,508.
• Maximum calibration sequence length: 512.

Evaluation Benchmarks

• MMLU-Pro: broad multiple-choice knowledge and reasoning.
• ARC-Challenge: science multiple-choice questions.
• HellaSwag: commonsense continuation selection.
• TruthfulQA MC1: truthfulness multiple-choice evaluation.

🔬 Quantization Details

The exported checkpoint includes hf_quant_config.json and config.json metadata for ModelOpt runtime detection.

{
  "quant_algo": "NVFP4",
  "group_size": 16,
  "kv_cache_quant_algo": null,
  "exclude_modules": [
    "lm_head",
    "model.embed_tokens",
    "model.layers.0*",
    "model.layers.1.*",
    "model.layers.2*",
    "model.layers.3.conv*",
    "model.layers.4.self_attn*",
    "model.layers.5.conv*",
    "model.layers.6.self_attn*",
    "model.layers.7.conv*",
    "model.layers.8.self_attn*",
    "model.layers.9.conv*",
    "model.layers.10.self_attn*",
    "model.layers.11.conv*",
    "model.layers.12.self_attn*",
    "model.layers.13*"
  ]
}

Artifact Files

File	Size
model.safetensors	346,361,200 bytes
config.json	3,073 bytes
hf_quant_config.json	830 bytes
tokenizer.json	4,733,389 bytes
tokenizer_config.json	490 bytes
generation_config.json	303 bytes
chat_template.jinja	4,621 bytes

Citation

If you use this checkpoint, cite the base model and NVIDIA Model Optimizer:

@misc{liquidai_lfm25_230m,
  title = {LFM2.5-230M},
  author = {Liquid AI},
  year = {2026},
  url = {https://huggingface.co/LiquidAI/LFM2.5-230M}
}

@software{nvidia_modelopt,
  title = {NVIDIA Model Optimizer},
  author = {NVIDIA},
  url = {https://github.com/NVIDIA/Model-Optimizer}
}