How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf cstr/bert-base-NER-GGUF:# Run inference directly in the terminal:
llama cli -hf cstr/bert-base-NER-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf cstr/bert-base-NER-GGUF:# Run inference directly in the terminal:
./llama-cli -hf cstr/bert-base-NER-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf cstr/bert-base-NER-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf cstr/bert-base-NER-GGUF:Use Docker
docker model run hf.co/cstr/bert-base-NER-GGUF:Quick Links
BERT Base NER — GGUF
GGUF conversion of dslim/bert-base-NER for use with CrispEmbed.
Fixed-label Named Entity Recognition on English text. BERT-base-cased (110M params) fine-tuned on CoNLL-03 with 9 IOB labels.
Labels
| ID | Label | Description |
|---|---|---|
| 0 | O | Outside any entity |
| 1 | B-MISC | Beginning of miscellaneous entity |
| 2 | I-MISC | Inside miscellaneous entity |
| 3 | B-PER | Beginning of person name |
| 4 | I-PER | Inside person name |
| 5 | B-ORG | Beginning of organization |
| 6 | I-ORG | Inside organization |
| 7 | B-LOC | Beginning of location |
| 8 | I-LOC | Inside location |
Available Formats
| File | Format | Size |
|---|---|---|
bert-base-ner-f32.gguf |
Float32 | 412 MB |
bert-base-ner-q8_0.gguf |
Q8_0 | 111 MB |
bert-base-ner-q4_k.gguf |
Q4_K | 70 MB |
Usage
crispembed -m bert-base-ner-q8_0.gguf --ner "Barack Obama was born in Hawaii"
from crispembed import CrispNER
ner = CrispNER("bert-base-ner-q8_0.gguf")
entities = ner.extract("Barack Obama was born in Hawaii")
# [{"text": "Barack Obama", "label": "PER", "start": 0, "end": 12, "score": 0.999},
# {"text": "Hawaii", "label": "LOC", "start": 25, "end": 31, "score": 1.000}]
Auto-detected as BERT NER (vs GLiNER zero-shot) from ner.classifier.weight in GGUF.
Parity
Encoder output: cos_min=0.999971 vs HuggingFace transformers (F32).
Provenance and EU AI Act Art. 53 note
- Upstream model: dslim/bert-base-NER — published by
dslim. - Upstream licence:
mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not. - What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
- Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
- Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.
- Downloads last month
- 639
Hardware compatibility
Log In to add your hardware
Model tree for cstr/bert-base-NER-GGUF
Base model
dslim/bert-base-NER
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf cstr/bert-base-NER-GGUF:# Run inference directly in the terminal: llama cli -hf cstr/bert-base-NER-GGUF: