Feature Extraction
Safetensors
GGUF
English
llama.cpp
embeddings
sentence-similarity
retrieval
medical
biomedical
bitnet
1.58-bit
ternary
llm2vec
Instructions to use Rabe3/1-bit-embedding-general with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Rabe3/1-bit-embedding-general with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf Rabe3/1-bit-embedding-general --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf Rabe3/1-bit-embedding-general --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf Rabe3/1-bit-embedding-general --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use Rabe3/1-bit-embedding-general with Ollama:
ollama run hf.co/Rabe3/1-bit-embedding-general
- Unsloth Desktop
- Docker Model Runner
How to use Rabe3/1-bit-embedding-general with Docker Model Runner:
docker model run hf.co/Rabe3/1-bit-embedding-general
- Lemonade
How to use Rabe3/1-bit-embedding-general with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Rabe3/1-bit-embedding-general
Run and chat with the model
lemonade run user.1-bit-embedding-general-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -147,3 +147,24 @@ MIT (following the base model). Built on `microsoft/bitnet-b1.58-2B-4T-bf16`.
|
|
| 147 |
|
| 148 |
## Acknowledgements
|
| 149 |
LLM2Vec (BehnamGhader et al., 2024) · BitNet b1.58 (Microsoft, 2025) · E5 · MedCPT · `llama.cpp` (ggml-org).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
## Acknowledgements
|
| 149 |
LLM2Vec (BehnamGhader et al., 2024) · BitNet b1.58 (Microsoft, 2025) · E5 · MedCPT · `llama.cpp` (ggml-org).
|
| 150 |
+
|
| 151 |
+
---
|
| 152 |
+
|
| 153 |
+
## Full-precision (bf16) weights
|
| 154 |
+
|
| 155 |
+
The bf16 **master weights** are also provided (in the `bf16/` subfolder) for GPU inference with 🤗 Transformers
|
| 156 |
+
or further fine-tuning. Note: BitNet keeps a *ternary* forward pass even from these bf16 weights (online
|
| 157 |
+
quantization) — bf16 is the storage/master-weight format used for training.
|
| 158 |
+
|
| 159 |
+
```python
|
| 160 |
+
import torch
|
| 161 |
+
from transformers import AutoModel, AutoTokenizer
|
| 162 |
+
|
| 163 |
+
tok = AutoTokenizer.from_pretrained("Rabe3/1-bit-embedding-general", subfolder="bf16")
|
| 164 |
+
model = AutoModel.from_pretrained("Rabe3/1-bit-embedding-general", subfolder="bf16",
|
| 165 |
+
torch_dtype=torch.bfloat16).cuda().eval()
|
| 166 |
+
# NOTE: this is a decoder patched to BIDIRECTIONAL attention for embeddings; use mean pooling
|
| 167 |
+
# over the last hidden state and L2-normalize. See the repo scripts for the exact embedder.
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
**Files:** `bf16/model.safetensors` (~4.8 GB, bf16), `bf16/config.json` (BitNet, ternary-online), tokenizer.
|