| --- |
| license: mit |
| tags: |
| - bitnet |
| - lora |
| - ternary |
| - trillim |
| - cpu-inference |
| base_model: microsoft/bitnet-b1.58-2B-4T-bf16 |
| --- |
| |
| # BitNet-Search-LoRA-TRNQ |
|
|
| Ternary-quantized LoRA adapter for [Trillim/BitNet-TRNQ](https://huggingface.co/Trillim/BitNet-TRNQ) that teaches the model to search. Generates search queries in `<search>...</search>` XML tags. Need to use Trillim (https://trillim.com) with a search harness to respond to the model's search calls and improve results. |
|
|
| This adapter runs entirely on CPU — no GPU required. |
|
|
| ## Adapter Details |
|
|
| | | | |
| |---|---| |
| | **Type** | LoRA adapter | |
| | **Style** | Search tool call | |
| | **Architecture** | BitNet (BitNetForCausalLM) | |
| | **Quantization** | Ternary ({-1, 0, 1}) | |
| | **Platforms** | x86_64, aarch64 | |
| | **Base model** | [Trillim/BitNet-TRNQ](https://huggingface.co/Trillim/BitNet-TRNQ) | |
| | **Source model** | [microsoft/bitnet-b1.58-2B-4T-bf16](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16) | |
| | **License** | MIT | |
| |
| ## Usage |
| |
| ```bash |
| pip install trillim |
| trillim pull Trillim/BitNet-TRNQ |
| trillim pull Trillim/BitNet-Search-LoRA-TRNQ |
| trillim chat Trillim/BitNet-TRNQ Trillim/BitNet-Search-LoRA-TRNQ |
| ``` |
| |
| This starts an interactive CLI chat. |
| |
| ## What's in this repo |
| |
| | File | Description | |
| |---|---| |
| | `qmodel.lora` | Ternary-quantized LoRA weights in Trillim format | |
| | `tokenizer.json` | Tokenizer | |
| | `tokenizer_config.json` | Tokenizer configuration | |
| | `chat_template.jinja` | Chat template | |
| | `trillim_config.json` | Trillim metadata | |
|
|
| ## License |
|
|
| This adapter is released under the [MIT License](https://opensource.org/licenses/MIT), following the license of the source model. |
|
|
|
|
