File size: 2,333 Bytes
26fdb70 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
license: mit
tags:
- bitnet
- 1-bit
- gguf
- ios
- apple-silicon
- arm64-neon
---
# BitNet iOS Models
Pre-converted GGUF models for use with [BitNet-iOS](https://github.com/dgavriloff/BitNet-iOS) — native 1-bit LLM inference on Apple Silicon using ARM64 NEON TL1 kernels.
These GGUFs were quantized using the BitNet.cpp `i2_s` format with locally-built `llama-quantize` from the [microsoft/BitNet](https://github.com/microsoft/BitNet) repo. **Using GGUFs from other sources may produce incorrect output** due to differences in `i2_s` packing between llama-quantize versions.
## Models
| File | Original Model | Type | Size | License |
|------|---------------|------|------|---------|
| `Falcon3-1B-Instruct-i2s.gguf` | [tiiuae/Falcon3-1B-Instruct-1.58bit](https://huggingface.co/tiiuae/Falcon3-1B-Instruct-1.58bit) | Instruct (chat) | 1.36 GB | [TII Falcon License 2.0](https://falconllm.tii.ae/falcon-terms-and-conditions.html) |
| `bitnet-b1.58-large-i2s.gguf` | [microsoft/bitnet_b1_58-large](https://huggingface.co/1bitLLM/bitnet_b1_58-large) | Base (completion) | 270 MB | MIT |
## Usage
These models are designed for the BitNet-iOS demo app, which downloads them automatically from this repo. They can also be used with the BitNet-iOS CLI:
```bash
# Instruct model (chat)
.build/debug/BitNetCLI /path/to/Falcon3-1B-Instruct-i2s.gguf --chat
# Base model (completion)
.build/debug/BitNetCLI /path/to/bitnet-b1.58-large-i2s.gguf "Once upon a time"
```
## Why self-hosted GGUFs?
The BitNet TL1 kernels are sensitive to the exact `i2_s` quantization format. GGUFs from the original model repos (e.g., tiiuae's Falcon3 GGUF) were quantized with a different version of `llama-quantize` and differ by ~224 bytes in header metadata. This causes the ARM64 NEON kernels to silently produce garbage output. These GGUFs were converted with the same toolchain used to build the BitNet-iOS XCFramework, ensuring compatibility.
## Attribution
- **Falcon3-1B-Instruct** by [Technology Innovation Institute (TII)](https://www.tii.ae/) — [TII Falcon License 2.0](https://falconllm.tii.ae/falcon-terms-and-conditions.html)
- **BitNet b1.58 Large** by [Microsoft Research](https://github.com/microsoft/BitNet) — MIT License
- Quantization via [microsoft/BitNet](https://github.com/microsoft/BitNet) (MIT License)
|