| | --- |
| | license: mit |
| | tags: |
| | - bitnet |
| | - 1-bit |
| | - gguf |
| | - ios |
| | - apple-silicon |
| | - arm64-neon |
| | --- |
| | |
| | # BitNet iOS Models |
| |
|
| | Pre-converted GGUF models for use with [BitNet-iOS](https://github.com/dgavriloff/BitNet-iOS) — native 1-bit LLM inference on Apple Silicon using ARM64 NEON TL1 kernels. |
| |
|
| | These GGUFs were quantized using the BitNet.cpp `i2_s` format with locally-built `llama-quantize` from the [microsoft/BitNet](https://github.com/microsoft/BitNet) repo. **Using GGUFs from other sources may produce incorrect output** due to differences in `i2_s` packing between llama-quantize versions. |
| |
|
| | ## Models |
| |
|
| | | File | Original Model | Type | Size | License | |
| | |------|---------------|------|------|---------| |
| | | `Falcon3-1B-Instruct-i2s.gguf` | [tiiuae/Falcon3-1B-Instruct-1.58bit](https://huggingface.co/tiiuae/Falcon3-1B-Instruct-1.58bit) | Instruct (chat) | 1.36 GB | [TII Falcon License 2.0](https://falconllm.tii.ae/falcon-terms-and-conditions.html) | |
| | | `bitnet-b1.58-large-i2s.gguf` | [microsoft/bitnet_b1_58-large](https://huggingface.co/1bitLLM/bitnet_b1_58-large) | Base (completion) | 270 MB | MIT | |
| |
|
| | ## Usage |
| |
|
| | These models are designed for the BitNet-iOS demo app, which downloads them automatically from this repo. They can also be used with the BitNet-iOS CLI: |
| |
|
| | ```bash |
| | # Instruct model (chat) |
| | .build/debug/BitNetCLI /path/to/Falcon3-1B-Instruct-i2s.gguf --chat |
| | |
| | # Base model (completion) |
| | .build/debug/BitNetCLI /path/to/bitnet-b1.58-large-i2s.gguf "Once upon a time" |
| | ``` |
| |
|
| | ## Why self-hosted GGUFs? |
| |
|
| | The BitNet TL1 kernels are sensitive to the exact `i2_s` quantization format. GGUFs from the original model repos (e.g., tiiuae's Falcon3 GGUF) were quantized with a different version of `llama-quantize` and differ by ~224 bytes in header metadata. This causes the ARM64 NEON kernels to silently produce garbage output. These GGUFs were converted with the same toolchain used to build the BitNet-iOS XCFramework, ensuring compatibility. |
| |
|
| | ## Attribution |
| |
|
| | - **Falcon3-1B-Instruct** by [Technology Innovation Institute (TII)](https://www.tii.ae/) — [TII Falcon License 2.0](https://falconllm.tii.ae/falcon-terms-and-conditions.html) |
| | - **BitNet b1.58 Large** by [Microsoft Research](https://github.com/microsoft/BitNet) — MIT License |
| | - Quantization via [microsoft/BitNet](https://github.com/microsoft/BitNet) (MIT License) |
| |
|
