Instructions to use soyuj/deeper-impact with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use soyuj/deeper-impact with Transformers:
# Load model directly from transformers import AutoTokenizer, DeepImpact tokenizer = AutoTokenizer.from_pretrained("soyuj/deeper-impact") model = DeepImpact.from_pretrained("soyuj/deeper-impact") - Notebooks
- Google Colab
- Kaggle
Add ONNX export (onnx/model.onnx) and model-card section
#1
by AlJohri - opened
- README.md +13 -0
- onnx/model.onnx +3 -0
README.md
CHANGED
|
@@ -31,3 +31,16 @@ python -m src.deep_impact.index \
|
|
| 31 |
```
|
| 32 |
|
| 33 |
It distributes the inference across multiple GPUs in the machine. To manually set the GPUs, use `CUDA_VISIBLE_DEVICES` environment variable.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
```
|
| 32 |
|
| 33 |
It distributes the inference across multiple GPUs in the machine. To manually set the GPUs, use `CUDA_VISIBLE_DEVICES` environment variable.
|
| 34 |
+
|
| 35 |
+
## ONNX
|
| 36 |
+
|
| 37 |
+
An ONNX export is available at [`onnx/model.onnx`](onnx/model.onnx) for inference with [ONNX Runtime](https://onnxruntime.ai/) — e.g. from Rust/C++/JS, or from Python without PyTorch.
|
| 38 |
+
|
| 39 |
+
| | names | dtype | shape |
|
| 40 |
+
|--------|------------------------------------------------|---------|----------------|
|
| 41 |
+
| inputs | `input_ids`, `attention_mask`, `token_type_ids` | int64 | `[batch, seq]` |
|
| 42 |
+
| output | `impact_scores` | float32 | `[batch, seq]` |
|
| 43 |
+
|
| 44 |
+
`impact_scores` is a per-subword-token score. A term's impact is the score at its **first** subword token — the same indexing as `DeepImpact.compute_term_impacts` (`##` continuation subwords are skipped; punctuation and terms past the 512-token window are dropped). Batch and sequence axes are dynamic.
|
| 45 |
+
|
| 46 |
+
The file was exported with [`src/deep_impact/scripts/export_onnx.py`](https://github.com/basnetsoyuj/DeeperImpact/blob/master/src/deep_impact/scripts/export_onnx.py) and matches the PyTorch model within `max |diff| ~ 6e-6`.
|
onnx/model.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:38cb884a1b7bb3f2dbf969c22219cb5b4eba64b824f0291ea8249baa616fc8a5
|
| 3 |
+
size 435818763
|