File size: 3,260 Bytes
29cd9ec | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | ---
library_name: gliner2-onnx
base_model: fastino/gliner2-multi-v1
tags:
- onnx
- gliner
- gliner2
- ner
- named-entity-recognition
- zero-shot
- classification
license: mit
---
> **Experimental ONNX build** - Unofficial ONNX export of [fastino/gliner2-multi-v1](https://huggingface.co/fastino/gliner2-multi-v1).
# gliner2-onnx
GLiNER2 ONNX runtime for Python. Runs GLiNER2 models without PyTorch.
This library is experimental. The API may change between versions.
## Features
- Zero-shot NER and text classification
- Runs with ONNX Runtime (no PyTorch dependency)
- FP32 and FP16 precision support
- GPU acceleration via CUDA
All other GLiNER2 features such as JSON export are not supported.
## Installation
```bash
pip install gliner2-onnx
```
## NER
```python
from gliner2_onnx import GLiNER2ONNXRuntime
runtime = GLiNER2ONNXRuntime.from_pretrained("lmo3/gliner2-large-v1-onnx")
entities = runtime.extract_entities(
"John works at Google in Seattle",
["person", "organization", "location"]
)
# [
# Entity(text='John', label='person', start=0, end=4, score=0.98),
# Entity(text='Google', label='organization', start=14, end=20, score=0.97),
# Entity(text='Seattle', label='location', start=24, end=31, score=0.96)
# ]
```
## Classification
```python
from gliner2_onnx import GLiNER2ONNXRuntime
runtime = GLiNER2ONNXRuntime.from_pretrained("lmo3/gliner2-large-v1-onnx")
# Single-label classification
result = runtime.classify(
"Buy milk from the store",
["shopping", "work", "entertainment"]
)
# {'shopping': 0.95}
# Multi-label classification
result = runtime.classify(
"Buy milk and finish the report",
["shopping", "work", "entertainment"],
threshold=0.3,
multi_label=True
)
# {'shopping': 0.85, 'work': 0.72}
```
## CUDA
To use CUDA for GPU acceleration:
```python
runtime = GLiNER2ONNXRuntime.from_pretrained(
"lmo3/gliner2-large-v1-onnx",
providers=["CUDAExecutionProvider", "CPUExecutionProvider"]
)
```
## Precision
Both FP32 and FP16 models are supported. Only the requested precision is downloaded.
```python
runtime = GLiNER2ONNXRuntime.from_pretrained(
"lmo3/gliner2-large-v1-onnx",
precision="fp16"
)
```
## Models
Pre-exported ONNX models:
| Model | HuggingFace |
|-------|-------------|
| gliner2-large-v1 | [lmo3/gliner2-large-v1-onnx](https://huggingface.co/lmo3/gliner2-large-v1-onnx) |
| gliner2-multi-v1 | [lmo3/gliner2-multi-v1-onnx](https://huggingface.co/lmo3/gliner2-multi-v1-onnx) |
Note: `gliner2-base-v1` is not supported (uses a different architecture).
## Exporting Models
To export your own models, clone the repository and use make:
```bash
git clone https://github.com/lmoe/gliner2-onnx
cd gliner2-onnx
# FP32 only
make onnx-export MODEL=fastino/gliner2-large-v1
# FP32 + FP16
make onnx-export MODEL=fastino/gliner2-large-v1 QUANTIZE=fp16
```
Output is saved to `model_out/<model-name>/`.
## JavaScript/TypeScript
For Node.js, see [@lmoe/gliner-onnx.js](https://github.com/lmoe/gliner-onnx.js).
## Credits
- [fastino-ai/GLiNER2](https://github.com/fastino-ai/GLiNER2) - Original GLiNER2 implementation
- [fastino/gliner2-large-v1](https://huggingface.co/fastino/gliner2-large-v1) - Pre-trained models
## License
MIT
|