xezpeleta commited on
Commit
2e20dc7
·
verified ·
1 Parent(s): bfc3ef9

docs: add Python/optimum usage example, list source.spm and vocab.json in files table

Browse files
Files changed (1) hide show
  1. README.md +46 -2
README.md CHANGED
@@ -41,12 +41,16 @@ MarianMT model trained on 9.78M Basque sentences that restores capitalization an
41
  |------|------|-------------|
42
  | `encoder_model_quantized.onnx` | 34 MB | Encoder (IR 8, int8 quantized) |
43
  | `decoder_model_merged_quantized.onnx` | 41 MB | Decoder with KV-cache (IR 8, int8 quantized) |
44
- | `tokenizer.json` | 2.1 MB | Custom Unigram + Metaspace pre-tokenizer |
 
 
 
 
45
  | `config.json` | 979 B | Model configuration |
46
  | `tokenizer_config.json` | 864 B | Tokenizer metadata |
47
  | `generation_config.json` | 288 B | Generation defaults |
48
 
49
- ## Usage with Transformers.js
50
 
51
  ```javascript
52
  import { pipeline } from '@huggingface/transformers';
@@ -62,6 +66,46 @@ console.log(result[0].translation_text);
62
  // → "Kaixo, zer moduz zaude?"
63
  ```
64
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
  ## Quantization details
66
 
67
  Dynamically quantized with `onnxruntime.quantization.quantize_dynamic(QuantType.QInt8, extra_options={"EnableSubgraph": True})`. The `EnableSubgraph` flag traverses into the `If`-node subgraphs of the merged decoder, quantizing `MatMul` operations in both branches. Results:
 
41
  |------|------|-------------|
42
  | `encoder_model_quantized.onnx` | 34 MB | Encoder (IR 8, int8 quantized) |
43
  | `decoder_model_merged_quantized.onnx` | 41 MB | Decoder with KV-cache (IR 8, int8 quantized) |
44
+ | `encoder_model.onnx` | 136 MB | Encoder (fp32, for reference / non-WASM use) |
45
+ | `decoder_model_merged.onnx` | 160 MB | Decoder with KV-cache (fp32, for reference / non-WASM use) |
46
+ | `tokenizer.json` | 2.1 MB | Custom Unigram + Metaspace pre-tokenizer (for Transformers.js) |
47
+ | `source.spm` | 842 KB | SentencePiece model (for Python / HF MarianTokenizer) |
48
+ | `vocab.json` | 2.1 MB | Vocab mapping (for Python / HF MarianTokenizer) |
49
  | `config.json` | 979 B | Model configuration |
50
  | `tokenizer_config.json` | 864 B | Tokenizer metadata |
51
  | `generation_config.json` | 288 B | Generation defaults |
52
 
53
+ ## Usage with Transformers.js (browser)
54
 
55
  ```javascript
56
  import { pipeline } from '@huggingface/transformers';
 
66
  // → "Kaixo, zer moduz zaude?"
67
  ```
68
 
69
+ ## Usage with Python (ONNX Runtime via optimum)
70
+
71
+ Install dependencies:
72
+
73
+ ```bash
74
+ pip install optimum[onnxruntime] sentencepiece
75
+ ```
76
+
77
+ Basic inference:
78
+
79
+ ```python
80
+ from optimum.onnxruntime import ORTModelForSeq2SeqLM
81
+ from transformers import AutoTokenizer, pipeline
82
+
83
+ model_id = "itzune/txukun-cap-punct-eu"
84
+
85
+ # Load int8 quantized ONNX model
86
+ model = ORTModelForSeq2SeqLM.from_pretrained(
87
+ model_id,
88
+ encoder_file_name="encoder_model_quantized.onnx",
89
+ decoder_file_name="decoder_model_merged_quantized.onnx",
90
+ decoder_with_past_file_name="decoder_model_merged_quantized.onnx",
91
+ provider="CPUExecutionProvider",
92
+ use_cache=True,
93
+ )
94
+
95
+ # Tokenizer: load from our repo or HiTZ
96
+ tokenizer = AutoTokenizer.from_pretrained("HiTZ/cap-punct-eu")
97
+
98
+ # Create pipeline
99
+ corrector = pipeline("translation", model=model, tokenizer=tokenizer, max_length=512)
100
+
101
+ # Correct text
102
+ result = corrector("euskal herrian euskaraz bizi nahi dugu")
103
+ print(result[0]["translation_text"])
104
+ # → "Euskal Herrian euskaraz bizi nahi dugu."
105
+ ```
106
+
107
+ For a complete CLI tool using this model, see [txukun-cli](https://github.com/itzune/txukun-cli).
108
+
109
  ## Quantization details
110
 
111
  Dynamically quantized with `onnxruntime.quantization.quantize_dynamic(QuantType.QInt8, extra_options={"EnableSubgraph": True})`. The `EnableSubgraph` flag traverses into the `If`-node subgraphs of the merged decoder, quantizing `MatMul` operations in both branches. Results: