Translation
Transformers.js
ONNX
Basque
marian
text2text-generation
basque
euskara
capitalization
punctuation
restoration
txukun
quantized
int8
Instructions to use itzune/txukun-cap-punct-eu with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use itzune/txukun-cap-punct-eu with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('translation', 'itzune/txukun-cap-punct-eu');
docs: add Python/optimum usage example, list source.spm and vocab.json in files table
Browse files
README.md
CHANGED
|
@@ -41,12 +41,16 @@ MarianMT model trained on 9.78M Basque sentences that restores capitalization an
|
|
| 41 |
|------|------|-------------|
|
| 42 |
| `encoder_model_quantized.onnx` | 34 MB | Encoder (IR 8, int8 quantized) |
|
| 43 |
| `decoder_model_merged_quantized.onnx` | 41 MB | Decoder with KV-cache (IR 8, int8 quantized) |
|
| 44 |
-
| `
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
| `config.json` | 979 B | Model configuration |
|
| 46 |
| `tokenizer_config.json` | 864 B | Tokenizer metadata |
|
| 47 |
| `generation_config.json` | 288 B | Generation defaults |
|
| 48 |
|
| 49 |
-
## Usage with Transformers.js
|
| 50 |
|
| 51 |
```javascript
|
| 52 |
import { pipeline } from '@huggingface/transformers';
|
|
@@ -62,6 +66,46 @@ console.log(result[0].translation_text);
|
|
| 62 |
// → "Kaixo, zer moduz zaude?"
|
| 63 |
```
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
## Quantization details
|
| 66 |
|
| 67 |
Dynamically quantized with `onnxruntime.quantization.quantize_dynamic(QuantType.QInt8, extra_options={"EnableSubgraph": True})`. The `EnableSubgraph` flag traverses into the `If`-node subgraphs of the merged decoder, quantizing `MatMul` operations in both branches. Results:
|
|
|
|
| 41 |
|------|------|-------------|
|
| 42 |
| `encoder_model_quantized.onnx` | 34 MB | Encoder (IR 8, int8 quantized) |
|
| 43 |
| `decoder_model_merged_quantized.onnx` | 41 MB | Decoder with KV-cache (IR 8, int8 quantized) |
|
| 44 |
+
| `encoder_model.onnx` | 136 MB | Encoder (fp32, for reference / non-WASM use) |
|
| 45 |
+
| `decoder_model_merged.onnx` | 160 MB | Decoder with KV-cache (fp32, for reference / non-WASM use) |
|
| 46 |
+
| `tokenizer.json` | 2.1 MB | Custom Unigram + Metaspace pre-tokenizer (for Transformers.js) |
|
| 47 |
+
| `source.spm` | 842 KB | SentencePiece model (for Python / HF MarianTokenizer) |
|
| 48 |
+
| `vocab.json` | 2.1 MB | Vocab mapping (for Python / HF MarianTokenizer) |
|
| 49 |
| `config.json` | 979 B | Model configuration |
|
| 50 |
| `tokenizer_config.json` | 864 B | Tokenizer metadata |
|
| 51 |
| `generation_config.json` | 288 B | Generation defaults |
|
| 52 |
|
| 53 |
+
## Usage with Transformers.js (browser)
|
| 54 |
|
| 55 |
```javascript
|
| 56 |
import { pipeline } from '@huggingface/transformers';
|
|
|
|
| 66 |
// → "Kaixo, zer moduz zaude?"
|
| 67 |
```
|
| 68 |
|
| 69 |
+
## Usage with Python (ONNX Runtime via optimum)
|
| 70 |
+
|
| 71 |
+
Install dependencies:
|
| 72 |
+
|
| 73 |
+
```bash
|
| 74 |
+
pip install optimum[onnxruntime] sentencepiece
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
Basic inference:
|
| 78 |
+
|
| 79 |
+
```python
|
| 80 |
+
from optimum.onnxruntime import ORTModelForSeq2SeqLM
|
| 81 |
+
from transformers import AutoTokenizer, pipeline
|
| 82 |
+
|
| 83 |
+
model_id = "itzune/txukun-cap-punct-eu"
|
| 84 |
+
|
| 85 |
+
# Load int8 quantized ONNX model
|
| 86 |
+
model = ORTModelForSeq2SeqLM.from_pretrained(
|
| 87 |
+
model_id,
|
| 88 |
+
encoder_file_name="encoder_model_quantized.onnx",
|
| 89 |
+
decoder_file_name="decoder_model_merged_quantized.onnx",
|
| 90 |
+
decoder_with_past_file_name="decoder_model_merged_quantized.onnx",
|
| 91 |
+
provider="CPUExecutionProvider",
|
| 92 |
+
use_cache=True,
|
| 93 |
+
)
|
| 94 |
+
|
| 95 |
+
# Tokenizer: load from our repo or HiTZ
|
| 96 |
+
tokenizer = AutoTokenizer.from_pretrained("HiTZ/cap-punct-eu")
|
| 97 |
+
|
| 98 |
+
# Create pipeline
|
| 99 |
+
corrector = pipeline("translation", model=model, tokenizer=tokenizer, max_length=512)
|
| 100 |
+
|
| 101 |
+
# Correct text
|
| 102 |
+
result = corrector("euskal herrian euskaraz bizi nahi dugu")
|
| 103 |
+
print(result[0]["translation_text"])
|
| 104 |
+
# → "Euskal Herrian euskaraz bizi nahi dugu."
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
For a complete CLI tool using this model, see [txukun-cli](https://github.com/itzune/txukun-cli).
|
| 108 |
+
|
| 109 |
## Quantization details
|
| 110 |
|
| 111 |
Dynamically quantized with `onnxruntime.quantization.quantize_dynamic(QuantType.QInt8, extra_options={"EnableSubgraph": True})`. The `EnableSubgraph` flag traverses into the `If`-node subgraphs of the merged decoder, quantizing `MatMul` operations in both branches. Results:
|