--- tags: - uzu - mirai - 4-bit library_name: uzu license: other license_name: lfm-open-license license_link: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE pipeline_tag: text-generation base_model: - LiquidAI/LFM2.5-1.2B-Instruct base_model_relation: quantized ---

Mirai's LFM2.5-1.2B-Instruct Medium Quantization

A medium-sized, high-quality quantization of LFM2.5-1.2B-Instruct prepared for efficient local inference on Apple silicon.

Mirai Labs
![KL divergence versus language-model checkpoint size](assets/kl-vs-size.svg) Mirai-M is on the size-KL Pareto frontier: we found no checkpoint that is smaller while also having lower KL divergence. *Evaluation data mixture: 45% public agentic, 30% public SFT/long-context, 25% private chat data.* ### Quickstart If you are on macOS, the easiest way is to install the `mirai` Homebrew package and then run the CLI: ```sh brew install mirai mirai --model trymirai/LFM2.5-1.2B-Instruct-M ``` Currently only Apple silicon inference is supported. If you want to build things from source, read this [overview](https://github.com/trymirai/uzu/blob/how-to/docs/how-to-run-uzu.md).
Learn more
--- ### Method Mirai Medium uses 4-bit asymmetric integer quantization with 4-bit zero points, bfloat16 scales, and group size 64. Block-diagonal Random Hadamard Transforms are used to reduce activation and weight outliers. The checkpoint was prepared with post-training quantization followed by quantization-aware distillation. ### Citation If you find our work helpful, feel free to give us a cite. ```bibtex @misc{mirai-quant, title = {{Mirai Quantization}: Redefining the speed-quality frontier for local LLMs on Apple silicon}, author = {Artur Chakhvadze and Ryan Mathieu and Roman Knyazhitskiy and Nikolai Voinilenko and Chen-Chen Yeh and Artur Mullakhmetov and Eugene Bokhan and others}, note = {In collaboration with others at Mirai Labs}, month = {June}, year = {2026}, url = {https://trymirai.com/blog/quantization} } ``` ### Original model This is a quantized version of [LiquidAI/LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct). For architecture details, intended use, evaluations, and limitations, see the [original model card](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/README.md).