Buckets:
external-sharing-models
| license: other | |
| license_name: lfm1.0 | |
| license_link: LICENSE | |
| base_model: LiquidAI/LFM3-Omni-2B | |
| tags: | |
| - gguf | |
| - lfm3 | |
| - lfm3-omni | |
| - liquid | |
| - omni | |
| - vision | |
| - audio | |
| - speech | |
| - quantized | |
| - mixed-precision | |
| - uncensored | |
| - abliterated | |
| - llama.cpp | |
| - xolotl | |
| language: | |
| - en | |
| pipeline_tag: any-to-any | |
| # LFM3-Omni-2B · XQ, abliterated | |
| Liquid AI's [LFM3-Omni-2B](https://huggingface.co/LiquidAI/LFM3-Omni-2B) (text, images, video and audio in; text | |
| and speech out) with the refusal direction edited out of the weights, then quantised with Indra's Mirror's | |
| mixed-precision **XQ** pipeline at five sizes. | |
| **XQ-6.0 (1.29 GiB) agrees with the abliterated 8-bit model on 95.3% of next tokens** (KL 0.021) and decodes at | |
| **432 tok/s** on an RTX 4090. XQ-4.5 is the same size as Q4_K_M with about a third less KL divergence. | |
| --- | |
| ## Files | |
| | file | size | what | | |
| |---|---|---| | |
| | `LFM3-Omni-2B-XQ-6.0-Uncensored.gguf` | 1.29 GiB | **recommended**: best quality per GB, still small enough to sit beside a big model | | |
| | `LFM3-Omni-2B-XQ-5.0-Uncensored.gguf` | 1.12 GiB | | | |
| | `LFM3-Omni-2B-XQ-4.5-Uncensored.gguf` | 1.04 GiB | same size as Q4_K_M, ~34% lower KL | | |
| | `LFM3-Omni-2B-XQ-4.0-Uncensored.gguf` | 0.95 GiB | | | |
| | `LFM3-Omni-2B-XQ-3.5-Uncensored.gguf` | 0.87 GiB | phones and small GPUs | | |
| | `LFM3-Omni-2B-Uncensored-Q8_0.gguf` | 1.72 GiB | the abliterated Q8_0 every XQ file is measured against | | |
| | `mmproj-LFM3-Omni-2B-Q8_0.gguf` | 0.47 GiB | Liquid AI's official vision/audio projector, unchanged; needed for images, video and audio | | |
| The text model is the only thing changed. The projector is Liquid AI's own file, copied as-is so one download | |
| has everything. | |
| --- | |
| ## How it was made | |
| 1. **Abliteration** on the full-precision weights: a rank-2 refusal-direction edit, fitted on Liquid AI's BF16 | |
| checkpoint. Checked on 100 held-out harmful prompts and 100 harmless ones it never saw: | |
| | | harmful prompts refused | harmless prompts refused | | |
| |---|---|---| | |
| | original LFM3-Omni-2B | 92 / 100 | 5 / 100 | | |
| | **abliterated** | **10 / 100** | **2 / 100** | | |
| 2. **Quantisation** with XQ: every tensor gets its own format (IQ3_S, IQ4_XS, IQ4_NL, Q3_K to Q6_K, Q8_0, BF16), | |
| picked by measured sensitivity on a coding/agent/OCR/long-context calibration set for a given bits-per-weight | |
| budget. The XQ pipeline itself is private; the files are standard GGUF and need no special loader. | |
| LFM3 is a non-reasoning model: the chat template opens every answer with an empty `<think></think>`. | |
| --- | |
| ## Quality | |
| Teacher-forced on a held-out text set (~256 KB, never used for calibration), against the **abliterated Q8_0** | |
| (perplexity 6.379). KL = mean KL divergence of the next-token distribution; top-1 = same most likely token. | |
| `llama-perplexity --kl-divergence`, llama.cpp f3586e6. | |
| | file | size | perplexity | KL (lower is better) | 99th pct KL | same top-1 | | |
| |---|---|---|---|---|---| | |
| | **XQ-6.0** | 1.29 GiB | 6.466 | **0.0207** | 0.225 | **95.29%** | | |
| | XQ-5.0 | 1.11 GiB | 6.442 | 0.0428 | | 93.18% | | |
| | XQ-4.5 | 1.03 GiB | 6.447 | 0.0543 | | 92.32% | | |
| | XQ-4.0 | 967 MiB | 6.549 | 0.0773 | | 90.97% | | |
| | XQ-3.5 | 879 MiB | 6.828 | 0.1311 | | 88.31% | | |
| | official Q4_K_M (not abliterated) | 1.02 GiB | 6.346 | 0.0822 | | 91.17% | | |
| | official Q8_0 (not abliterated) | 1.71 GiB | 6.464 | 0.0059 | 0.068 | 97.44% | | |
| The official files are not abliterated, so their KL includes the abliteration difference as well as their | |
| quantisation (Q8_0's 0.0059 is roughly that difference alone). Past 6.0 the curve flattens: going from XQ-6.0 to | |
| the Q8_0 costs another 0.42 GiB for 0.021 KL. | |
| **Multimodal and tool smoke test** (XQ-6.0 + the Q8_0 projector, llama-server, 32K context): tool call correct, | |
| image described correctly (shapes, colours and the number), a speech clip transcribed exactly. Speech *output* | |
| (XQ-6.0, through Xolotl below): the model's own voice transcribed back by the model at WER 0.00 / 0.00 / 0.11 on | |
| three sentences (the one miss is the word "Xolotl"). | |
| --- | |
| ## Speed | |
| RTX 4090 24 GB, Ryzen 7 5700X, Linux. `llama-bench` tg128, all layers on the GPU: | |
| | file | tg128 | | |
| |---|---| | |
| | XQ-3.5 | 514 tok/s | | |
| | XQ-4.0 | 501 tok/s | | |
| | XQ-4.5 | 481 tok/s | | |
| | XQ-5.0 | 466 tok/s | | |
| | **XQ-6.0** | **432 tok/s** | | |
| | official Q4_K_M | 497 tok/s | | |
| | official Q8_0 | 355 tok/s | | |
| Served (llama-server, XQ-6.0 + projector, 32K context): **3.5 GB of VRAM**, 389 tok/s decode on the GPU, 23 tok/s | |
| decode CPU-only (8 threads). | |
| --- | |
| ## Quick start | |
| Needs a llama.cpp with LFM3 support (upstream from 31 Oct 2026, f3586e6 or later). | |
| ```sh | |
| hf download IndrasMirror/LFM3-Omni-2B-XQ-Uncensored \ | |
| LFM3-Omni-2B-XQ-6.0-Uncensored.gguf mmproj-LFM3-Omni-2B-Q8_0.gguf --local-dir lfm3 | |
| llama-server -m lfm3/LFM3-Omni-2B-XQ-6.0-Uncensored.gguf --mmproj lfm3/mmproj-LFM3-Omni-2B-Q8_0.gguf \ | |
| -ngl 99 -c 32768 --jinja -fa auto | |
| ``` | |
| Images, audio (`input_audio`) and video (`input_video`, with its soundtrack) go through the usual | |
| `/v1/chat/completions`; speech comes out of `/tts` and the `/v1/realtime` websocket (see llama.cpp's server | |
| README). Liquid AI's recommended sampling and system prompts are on the | |
| [original model card](https://huggingface.co/LiquidAI/LFM3-Omni-2B). | |
| --- | |
| ## Serve with Xolotl | |
| [Xolotl](https://github.com/Indras-Mirror/Xolotl) runs these files natively in its own LFM3 engine (`xolotl-lfm3`, | |
| [docs/LFM3.md](https://github.com/Indras-Mirror/Xolotl/blob/main/docs/LFM3.md)), behind its OpenAI / Anthropic | |
| server: chat, tool calls, pictures, audio, video, `/v1/audio/speech` and `/v1/audio/transcriptions` from the model | |
| itself, and park/unpark so it can share a card with a big model. Measured on the RTX 4090 with XQ-6.0, 32K context: | |
| | | | | |
| |---|---| | |
| | prefill (fresh prompt) | 32,796 tok/s at 5.3K · 26,049 tok/s at 14K | | |
| | decode | ~300 tok/s | | |
| | follow-up turn on a 7K conversation | 0.03 s | | |
| | HumanEval (21) / tool calls (10) / agentic loops (3) | 13 / 9 / 0 - a 2B model, thinking off | | |
| | speech out | 12x real time, first audio 0.02 s once loaded | | |
| | park / unpark | 51 ms / ~75 ms, ~1.4 GB of VRAM handed back | | |
| Its logits match llama.cpp's on the same file (mean KL 1.3e-7 with the same memory layout). | |
| ```sh | |
| git clone https://github.com/Indras-Mirror/Xolotl && cd Xolotl # build as in its README, then: | |
| ninja -C build-xolotl xolotl-lfm3 | |
| tools/lfm3/build-helpers.sh ~/llama.cpp # the picture/audio and speech helpers, on llama.cpp f3586e6+ | |
| cp tools/lfm3/xol-lfm3.example.json xol-lfm3.json # point it at the GGUF, the mmproj and the tokenizer | |
| .venv/bin/python -m serve.server --engine strata --config xol-lfm3.json --port 8132 | |
| ``` | |
| --- | |
| ## Licence | |
| [LFM Open License v1.0](LICENSE), the same as the original model. These files are **modified** versions of | |
| LiquidAI/LFM3-Omni-2B (abliterated and quantised by Indra's Mirror); the projector is unmodified. Commercial use is | |
| licensed only below Liquid AI's revenue threshold; read the licence. | |
| ## Credits | |
| * **Liquid AI** for [LFM3-Omni-2B](https://huggingface.co/LiquidAI/LFM3-Omni-2B) and its official GGUF projector. | |
| * **ggml / llama.cpp** for LFM3 support and the evaluation tools. | |
| * Abliteration, XQ quantisation and measurements: Indra's Mirror. | |
| ## Support | |
| If this has been useful and you want to help keep it going, you can | |
| [buy me a coffee](https://ko-fi.com/indrasmirror). | |
Xet Storage Details
- Size:
- 7.31 kB
- Xet hash:
- bb433b22cc0c618b1b64914321a1d136ced63a84177402b814e680d3effa07c8
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.