Upload folder using huggingface_hub

README.md

@@ -0,0 +1,435 @@
+---
+language:
+  - de
+  - en
+  - fr
+  - it
+  - pt
+  - es
+  - tr
+  - sv
+license: cc-by-nc-4.0
+tags:
+  - text-normalization
+  - tts
+  - text-to-speech
+  - byt5
+  - multilingual
+  - seq2seq
+  - speech-processing
+datasets:
+  - custom
+metrics:
+  - accuracy
+pipeline_tag: text2text-generation
+model-index:
+  - name: saytext
+    results:
+      - task:
+          type: text2text-generation
+          name: Text Normalization
+        metrics:
+          - type: accuracy
+            value: 94.2
+            name: Sentence Accuracy (test set)
+          - type: accuracy
+            value: 22.6
+            name: Sentence Accuracy (PolyNorm benchmark)
+---
+
+# SayText — Multilingual Text Normalization for TTS
+
+**A multilingual neural text normalization model for TTS (text-to-speech) pipelines.** Converts written text to spoken form across 8 European languages using a fine-tuned ByT5-Base (580M parameters).
+
+> `"Das kostet 12,50 €."` → `"Das kostet zwölf Euro fünfzig."`
+
+## Key Features
+
+- **8 languages**: German, English, French, Italian, Portuguese, Spanish, Turkish, Swedish
+- **24 semiotic classes**: cardinals, money, dates, time, phone numbers, percentages, units, passthrough, and more
+- **Passthrough-aware**: learns when NOT to normalize (plain text, already-spoken forms, technical identifiers)
+- **Voice-agent optimized**: designed for LLM → TN → TTS pipelines where input is always well-formatted
+- **Byte-level**: ByT5 processes raw UTF-8 bytes — no tokenizer vocabulary limitations, handles `€`, `₺`, `°C` natively
+
+## Quick Start
+
+```python
+from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
+import torch
+
+model = AutoModelForSeq2SeqLM.from_pretrained("smaoai/saytext")
+tokenizer = AutoTokenizer.from_pretrained("google/byt5-base")
+model.eval()
+
+def normalize(text: str, language: str) -> str:
+    """Normalize text for TTS. Language: de, en, fr, it, pt, es, tr, sv"""
+    input_text = f"<{language}> {text}"
+    inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True)
+    with torch.no_grad():
+        output = model.generate(**inputs, max_new_tokens=512, num_beams=1)
+    return tokenizer.decode(output[0], skip_special_tokens=True)
+
+# Examples
+print(normalize("Das kostet 12,50 €.", "de"))
+# → Das kostet zwölf Euro fünfzig.
+
+print(normalize("The flight departs at 8:45 AM.", "en"))
+# → The flight departs at eight forty-five AM.
+
+print(normalize("Le prix est de 327,67 €.", "fr"))
+# → Le prix est de trois cent vingt-sept euros et soixante-sept centimes.
+
+print(normalize("Ich helfe Ihnen gerne weiter.", "de"))
+# → Ich helfe Ihnen gerne weiter.  (passthrough — no normalization needed)
+
+print(normalize("Wir verwenden Python 3.10 in unserem Projekt.", "de"))
+# → Wir verwenden Python 3.10 in unserem Projekt.  (technical identifier preserved)
+```
+
+## Available Formats
+
+This repo includes the model in three formats:
+
+| Format | Path | Size | Use case |
+|--------|------|------|----------|
+| **PyTorch** (default) | `model.safetensors` | 2.2 GB | Development, fine-tuning, HuggingFace `pipeline()` |
+| **CTranslate2 FP16** | `ct2_float16/` | 1.1 GB | GPU production inference (T4, A10G, A100) |
+| **CTranslate2 INT8** | `ct2_int8/` | 556 MB | CPU production inference, edge deployment |
+
+```
+repo/
+  model.safetensors          # PyTorch weights
+  config.json                # Model architecture
+  tokenizer_config.json      # Tokenizer config
+  added_tokens.json          # Additional tokens
+  generation_config.json     # Default generation parameters
+  handler.py                 # HuggingFace Inference API handler
+  ct2_float16/               # CTranslate2 FP16 (GPU)
+    model.bin
+    config.json
+    shared_vocabulary.json
+  ct2_int8/                  # CTranslate2 INT8 (CPU)
+    model.bin
+    config.json
+    shared_vocabulary.json
+```
+
+## Model Description
+
+### Why ByT5?
+
+Text normalization deals with symbols, digits, and special characters (`€12,50`, `+49(0)30`, `info@web.de`) that subword tokenizers fragment unpredictably. ByT5 operates on raw UTF-8 bytes — every character is processed individually with no tokenizer artifacts. This is critical for:
+
+- **Locale-sensitive formats**: `1.500` means "one thousand five hundred" in German but "one point five" in English
+- **Special symbols**: `€`, `£`, `₺`, `%`, `°C` are single bytes, not fragmented subwords
+- **Phone numbers**: `+49 (0)30 12345678` stays intact byte-by-byte
+
+### Architecture
+
+| Component | Details |
+|-----------|---------|
+| Base model | [google/byt5-base](https://huggingface.co/google/byt5-base) |
+| Parameters | 580M |
+| Type | Encoder-decoder (seq2seq) |
+| Tokenization | Byte-level UTF-8 (no SentencePiece) |
+| Max input length | 512 bytes (~250 characters) |
+| Max output length | 512 bytes |
+| Language conditioning | Prefix token: `<de>`, `<en>`, `<fr>`, etc. |
+
+### How It Works
+
+The model takes text with a language prefix and outputs the spoken form:
+
+```
+Input:  <de> Am 03.04.2026 um 14:30 kostet der Flug 249,99 €.
+Output: Am dritte April zweitausendsechsundzwanzig um vierzehn Uhr dreißig
+        kostet der Flug zweihundertneunundvierzig Euro neunundneunzig.
+```
+
+For text that doesn't need normalization, the model learns to pass it through unchanged:
+
+```
+Input:  <en> Sure, I can help you with that.
+Output: Sure, I can help you with that.
+```
+
+## Training Data
+
+The model was trained on **3.07M pairs** across 8 languages, generated through a two-layer pipeline:
+
+### Data Pipeline
+
+1. **Entity Sampler** (deterministic) — Generates verified (written, spoken) pairs per semiotic class using locale-aware libraries (Babel, num2words). Every pair is programmatically verified correct.
+
+2. **Sentence Generator** (LLM-powered) — 192,000 natural sentence templates with `{SLOT}` placeholders, generated via Nebius GLM-5. Templates are filled with entity sampler pairs at assembly time.
+
+3. **Real-world enrichment** — Leipzig news corpora + OPUS ECB financial text, auto-labeled by NVIDIA NeMo WFST grammars, validated and merged.
+
+### Data Composition
+
+| Category | Count | % | Description |
+|----------|-------|---|-------------|
+| MIXED (multi-entity) | 434k | 14.1% | Sentences with 2-3 different entity types |
+| Cardinal numbers | 373k | 12.1% | `1,523` → `eintausendfünfhundertdreiundzwanzig` |
+| Money | 313k | 10.2% | `12,50 €` → `zwölf Euro fünfzig` |
+| Written dates | 192k | 6.3% | `3. April 2026` → `dritter April...` |
+| **Plain (passthrough)** | 166k | 5.4% | Clean text → unchanged output |
+| Decimals | 161k | 5.3% | `3,14` → `drei Komma eins vier` |
+| Time | 160k | 5.2% | `14:30` → `vierzehn Uhr dreißig` |
+| Units | 145k | 4.7% | `3,5 kg` → `drei Komma fünf Kilogramm` |
+| **Already normalized** | 145k | 4.7% | Already-spoken forms → unchanged |
+| Percentages | 133k | 4.3% | `15,3%` → `fünfzehn Komma drei Prozent` |
+| Ordinals | 123k | 4.0% | `3.` → `dritte` |
+| Phone (local) | 108k | 3.5% | `030 12345678` → digit-by-digit |
+| Numeric dates | 106k | 3.5% | `03.04.2026` → `dritter April...` |
+| Years | 95k | 3.1% | `2026` → `zweitausendsechsundzwanzig` |
+| Ranges | 81k | 2.7% | `10–15` → `zehn bis fünfzehn` |
+| Phone (international) | 80k | 2.6% | `+49 30 12345678` → `plus vier neun...` |
+| Email | 54k | 1.8% | Spelled out |
+| Version | 53k | 1.7% | `v2.3.1` → `Version zwei Punkt drei Punkt eins` |
+| Fractions | 51k | 1.7% | `3/4` → `drei Viertel` |
+| URL | 27k | 0.9% | Spelled out |
+| Abbreviations | 27k | 0.9% | `z.B.` → `zum Beispiel` |
+| Postal codes | 27k | 0.9% | Digit-by-digit |
+| **Don't normalize** | 8k | 0.3% | `Python 3.10`, `HTTP 404` → pass through |
+
+### Per-Language Distribution
+
+| Language | Train pairs | Real-world data |
+|----------|------------|-----------------|
+| German (DE) | 373k | Leipzig + ECB (NeMo labeled) |
+| English (EN) | 379k | Leipzig + ECB + NeMo eval pairs |
+| Spanish (ES) | 364k | Leipzig + ECB (NeMo labeled) |
+| French (FR) | 359k | Leipzig + ECB (NeMo labeled) |
+| Italian (IT) | 359k | Leipzig + ECB (NeMo labeled) |
+| Portuguese (PT) | 308k | Synthetic only (NeMo doesn't support PT) |
+| Swedish (SV) | 477k | Leipzig (NeMo labeled), 1.5x oversampled |
+| Turkish (TR) | 447k | Synthetic only (NeMo doesn't support TR), 1.5x oversampled |
+
+Turkish and Swedish are oversampled 1.5x because ByT5's pretraining data has less coverage for these languages.
+
+## Evaluation Results
+
+### Internal Test Set (1,900 stratified sample)
+
+**Overall: 94.2% sentence-level exact match accuracy**
+
+| Language | Accuracy |
+|----------|----------|
+| Swedish | 96.7% |
+| French | 95.4% |
+| Turkish | 94.8% |
+| Spanish | 94.2% |
+| English | 93.8% |
+| Italian | 93.3% |
+| German | 92.9% |
+| Portuguese | 92.6% |
+
+#### Per-Class Accuracy (selected)
+
+| Class | Accuracy | Notes |
+|-------|----------|-------|
+| Cardinal | 97.5% | |
+| Money | 97.5% | |
+| Year | 97.5% | |
+| Time | 96.2% | |
+| Ordinal | 96.2% | |
+| Passthrough (plain) | 100% | Correctly leaves clean text unchanged |
+| Already normalized | 93.8% | |
+| Don't normalize | 97.5% | Correctly preserves technical identifiers |
+| Phone numbers | 90.0% | Some formatting differences |
+| Multi-entity (MIXED) | 51.2% | Complex sentences with 2-3 entities |
+
+### Apple PolyNorm Benchmark (1,620 pairs, 5 languages)
+
+**22.6% sentence accuracy** — lower due to:
+- 30% of PolyNorm tests classes we intentionally dropped (ACRONYM, HASHTAG, ROMAN, IBAN, SCORE)
+- Unseen date format variants (abbreviated months, slash dates, 2-digit years)
+- German case/declension differences
+
+This benchmark measures generalization to unseen formats, which is an area for improvement in future versions.
+
+## Inference
+
+### Input Format
+
+The model expects a **language prefix** followed by the text:
+
+```
+<de> Das kostet 12,50 €.
+<en> The price is $99.99.
+<fr> Le prix est de 327,67 €.
+```
+
+The language prefix is **required** — it tells the model which normalization rules to apply. The same written form can normalize differently per language (e.g., `1.500` → German "eintausendfünfhundert" vs English "one point five").
+
+### CPU Inference (PyTorch)
+
+| Metric | Value |
+|--------|-------|
+| Average latency | 600ms per sentence |
+| Passthrough (no entities) | 300ms |
+| Complex multi-entity | 800-1300ms |
+| Model size | 2.2 GB |
+| RAM usage | ~3 GB |
+
+**ByT5 is inherently slow on CPU** because it generates one byte at a time. A 30-character output requires 30 autoregressive decoder steps, each being a full forward pass through 580M parameters. This is the fundamental trade-off for byte-level accuracy.
+
+### CPU Inference (CTranslate2 INT8)
+
+| Metric | Value |
+|--------|-------|
+| Average latency (single) | 248ms |
+| Average latency (batch of 10) | 120ms per sentence |
+| Passthrough | 165ms |
+| Model size | 556 MB |
+| RAM usage | ~1 GB |
+
+CTranslate2 with INT8 quantization provides **~2.5x speedup** over PyTorch and **4x smaller** model. Quality is preserved with minor token mapping artifacts on special characters.
+
+```bash
+# Convert to CTranslate2 INT8
+pip install ctranslate2
+ct2-transformers-converter --model smaoai/saytext \
+    --output_dir ct2_int8 --quantization int8
+
+# Usage
+import ctranslate2
+from transformers import AutoTokenizer
+
+translator = ctranslate2.Translator("ct2_int8", device="cpu", intra_threads=4)
+tokenizer = AutoTokenizer.from_pretrained("google/byt5-base")
+
+text = "<de> Das kostet 12,50 €."
+tokens = tokenizer(text, return_tensors=None)["input_ids"]
+token_strs = [tokenizer.decode([t]) for t in tokens]
+
+result = translator.translate_batch([token_strs], beam_size=1, max_decoding_length=512)
+output_ids = tokenizer.convert_tokens_to_ids(result[0].hypotheses[0])
+print(tokenizer.decode(output_ids, skip_special_tokens=True))
+```
+
+### GPU Inference
+
+| Setup | Single sentence | Batch of 8 | Model size |
+|-------|----------------|------------|------------|
+| PyTorch FP32, A100 | ~280ms | ~63ms/sent | 2.2 GB |
+| CTranslate2 FP16, A100 | ~15-30ms | ~8ms/sent | 1.1 GB |
+| CTranslate2 FP16, T4 | ~30-50ms | ~15ms/sent | 1.1 GB |
+
+**For production TTS pipelines, a small GPU (T4, L4) is recommended.** With CTranslate2 FP16 on a T4 ($0.20/hr on cloud), you get <50ms latency — well within real-time TTS requirements.
+
+### Latency vs. Quality Summary
+
+```
+                    Latency          Quality    Cost
+                    ───────          ───────    ────
+PyTorch CPU         600ms/sent       Best       Free
+CT2 INT8 CPU        248ms/sent       Good*      Free
+CT2 FP16 T4 GPU     30-50ms/sent     Best       $0.20/hr
+CT2 FP16 A100 GPU   15-30ms/sent     Best       $1.50/hr
+
+* Minor token mapping artifacts on special characters (€, ₺)
+```
+
+## Supported Languages
+
+| Code | Language | Example Input | Example Output |
+|------|----------|---------------|----------------|
+| `de` | German | `Das kostet 12,50 €.` | `Das kostet zwölf Euro fünfzig.` |
+| `en` | English | `The flight departs at 8:45 AM.` | `The flight departs at eight forty-five AM.` |
+| `fr` | French | `Le prix est de 327,67 €.` | `Le prix est de trois cent vingt-sept euros et soixante-sept centimes.` |
+| `it` | Italian | `La riunione è alle 14:30.` | `La riunione è alle quattordici e trenta.` |
+| `pt` | Portuguese | `Em 2023, o crescimento foi notável.` | `Em dois mil e vinte e três, o crescimento foi notável.` |
+| `es` | Spanish | `La fecha es el 20 de enero de 2025.` | `La fecha es el veinte de enero de dos mil veinticinco.` |
+| `tr` | Turkish | `Toplam 250 kişi katıldı.` | `Toplam ikiyüzelli kişi katıldı.` |
+| `sv` | Swedish | `Det väger 2,5 kg.` | `Det väger två komma fem kilogram.` |
+
+## Limitations & Known Issues
+
+### Current Limitations
+
+1. **Multi-entity accuracy is 51%** — sentences with 2-3 different entity types (date + time + money) sometimes produce errors on one of the entities.
+
+2. **Date format variants** — the model handles standard formats (`03.04.2026`, `March 15, 2026`) well but struggles with abbreviated months (`5. Nov. 1990`), slash dates (`3/4/2024`), and 2-digit years (`'23`).
+
+3. **German declension** — sometimes produces nominative case (`fünfter`) instead of the correct accusative/dative (`fünften`). This is a systematic issue with the training data.
+
+4. **CPU latency** — 250-600ms per sentence on CPU. Not suitable for real-time applications without GPU. See inference section for alternatives.
+
+5. **Currency symbol verbalization** — the `€` symbol is sometimes passed through instead of being verbalized as "Euro". More consistent with spelled-out currency codes.
+
+6. **Training not complete** — the model was trained for 1.29 epochs out of 3 planned. While already performant, further training would improve accuracy, especially on edge cases.
+
+### Not Designed For
+
+- **Raw user input** — the model is optimized for well-formatted LLM output, not OCR text or social media posts with typos
+- **Acronym spelling** — `BMW → B M W` is intentionally not included (TTS engines handle this natively via SSML)
+- **Hashtags, IBANs, Roman numerals, sports scores** — dropped from training as low-priority for voice agent use cases
+
+### Recommended Post-Processing
+
+For production use, add a post-validation step:
+
+```python
+import re
+
+def post_validate(input_text: str, output_text: str) -> bool:
+    """Check if the model output is safe for TTS."""
+    # Flag if output still contains raw digits (possible normalization failure)
+    if re.search(r"\d", output_text):
+        return False  # Fall back to rule-based normalization
+    # Flag if output is suspiciously short or long
+    ratio = len(output_text) / max(len(input_text), 1)
+    if ratio > 5.0 or ratio < 0.3:
+        return False
+    return True
+```
+
+## Training Details
+
+| Parameter | Value |
+|-----------|-------|
+| Base model | google/byt5-base |
+| Training pairs | 3,067,205 |
+| Validation pairs | 170,317 |
+| Effective batch size | 128 (16 per device × 8 accumulation) |
+| Learning rate | 3e-4 (cosine schedule) |
+| Precision | bf16 |
+| Hardware | NVIDIA A100 80GB SXM |
+| Training time | ~13 hours (31,000 steps, 1.29 epochs) |
+| Best eval loss | 0.000897 |
+| Framework | HuggingFace Transformers 5.5 + PyTorch 2.6 |
+
+## Intended Use
+
+This model is designed for **text-to-speech preprocessing** in voice agent / conversational AI pipelines:
+
+```
+User speaks → ASR → LLM generates response → Text Normalizer → TTS speaks
+```
+
+The text normalizer converts written forms (numbers, dates, currencies, etc.) in the LLM's response to spoken forms that the TTS engine can pronounce naturally.
+
+## License
+
+This model is released under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — free for research and non-commercial use.
+
+**For commercial licensing**, contact us at [business@smao.ai](mailto:business@smao.ai).
+
+## About
+
+Built by [SMAO](https://smao.ai) — Michael Müller and team.
+
+For questions, issues, or commercial inquiries: [business@smao.ai](mailto:business@smao.ai)
+
+## Citation
+
+```bibtex
+@misc{smao-byt5-tn-2026,
+  title={SayText: Multilingual Text Normalization for TTS},
+  author={Michael Müller and SMAO AI},
+  year={2026},
+  url={https://huggingface.co/smaoai/saytext}
+}
+```

added_tokens.json

@@ -0,0 +1,127 @@
+{
+  "<extra_id_0>": 259,
+  "<extra_id_100>": 359,
+  "<extra_id_101>": 360,
+  "<extra_id_102>": 361,
+  "<extra_id_103>": 362,
+  "<extra_id_104>": 363,
+  "<extra_id_105>": 364,
+  "<extra_id_106>": 365,
+  "<extra_id_107>": 366,
+  "<extra_id_108>": 367,
+  "<extra_id_109>": 368,
+  "<extra_id_10>": 269,
+  "<extra_id_110>": 369,
+  "<extra_id_111>": 370,
+  "<extra_id_112>": 371,
+  "<extra_id_113>": 372,
+  "<extra_id_114>": 373,
+  "<extra_id_115>": 374,
+  "<extra_id_116>": 375,
+  "<extra_id_117>": 376,
+  "<extra_id_118>": 377,
+  "<extra_id_119>": 378,
+  "<extra_id_11>": 270,
+  "<extra_id_120>": 379,
+  "<extra_id_121>": 380,
+  "<extra_id_122>": 381,
+  "<extra_id_123>": 382,
+  "<extra_id_124>": 383,
+  "<extra_id_12>": 271,
+  "<extra_id_13>": 272,
+  "<extra_id_14>": 273,
+  "<extra_id_15>": 274,
+  "<extra_id_16>": 275,
+  "<extra_id_17>": 276,
+  "<extra_id_18>": 277,
+  "<extra_id_19>": 278,
+  "<extra_id_1>": 260,
+  "<extra_id_20>": 279,
+  "<extra_id_21>": 280,
+  "<extra_id_22>": 281,
+  "<extra_id_23>": 282,
+  "<extra_id_24>": 283,
+  "<extra_id_25>": 284,
+  "<extra_id_26>": 285,
+  "<extra_id_27>": 286,
+  "<extra_id_28>": 287,
+  "<extra_id_29>": 288,
+  "<extra_id_2>": 261,
+  "<extra_id_30>": 289,
+  "<extra_id_31>": 290,
+  "<extra_id_32>": 291,
+  "<extra_id_33>": 292,
+  "<extra_id_34>": 293,
+  "<extra_id_35>": 294,
+  "<extra_id_36>": 295,
+  "<extra_id_37>": 296,
+  "<extra_id_38>": 297,
+  "<extra_id_39>": 298,
+  "<extra_id_3>": 262,
+  "<extra_id_40>": 299,
+  "<extra_id_41>": 300,
+  "<extra_id_42>": 301,
+  "<extra_id_43>": 302,
+  "<extra_id_44>": 303,
+  "<extra_id_45>": 304,
+  "<extra_id_46>": 305,
+  "<extra_id_47>": 306,
+  "<extra_id_48>": 307,
+  "<extra_id_49>": 308,
+  "<extra_id_4>": 263,
+  "<extra_id_50>": 309,
+  "<extra_id_51>": 310,
+  "<extra_id_52>": 311,
+  "<extra_id_53>": 312,
+  "<extra_id_54>": 313,
+  "<extra_id_55>": 314,
+  "<extra_id_56>": 315,
+  "<extra_id_57>": 316,
+  "<extra_id_58>": 317,
+  "<extra_id_59>": 318,
+  "<extra_id_5>": 264,
+  "<extra_id_60>": 319,
+  "<extra_id_61>": 320,
+  "<extra_id_62>": 321,
+  "<extra_id_63>": 322,
+  "<extra_id_64>": 323,
+  "<extra_id_65>": 324,
+  "<extra_id_66>": 325,
+  "<extra_id_67>": 326,
+  "<extra_id_68>": 327,
+  "<extra_id_69>": 328,
+  "<extra_id_6>": 265,
+  "<extra_id_70>": 329,
+  "<extra_id_71>": 330,
+  "<extra_id_72>": 331,
+  "<extra_id_73>": 332,
+  "<extra_id_74>": 333,
+  "<extra_id_75>": 334,
+  "<extra_id_76>": 335,
+  "<extra_id_77>": 336,
+  "<extra_id_78>": 337,
+  "<extra_id_79>": 338,
+  "<extra_id_7>": 266,
+  "<extra_id_80>": 339,
+  "<extra_id_81>": 340,
+  "<extra_id_82>": 341,
+  "<extra_id_83>": 342,
+  "<extra_id_84>": 343,
+  "<extra_id_85>": 344,
+  "<extra_id_86>": 345,
+  "<extra_id_87>": 346,
+  "<extra_id_88>": 347,
+  "<extra_id_89>": 348,
+  "<extra_id_8>": 267,
+  "<extra_id_90>": 349,
+  "<extra_id_91>": 350,
+  "<extra_id_92>": 351,
+  "<extra_id_93>": 352,
+  "<extra_id_94>": 353,
+  "<extra_id_95>": 354,
+  "<extra_id_96>": 355,
+  "<extra_id_97>": 356,
+  "<extra_id_98>": 357,
+  "<extra_id_99>": 358,
+  "<extra_id_9>": 268
+}

config.json

@@ -0,0 +1,35 @@
+{
+  "architectures": [
+    "T5ForConditionalGeneration"
+  ],
+  "classifier_dropout": 0.0,
+  "d_ff": 3968,
+  "d_kv": 64,
+  "d_model": 1536,
+  "decoder_start_token_id": 0,
+  "dense_act_fn": "gelu_new",
+  "dropout_rate": 0.1,
+  "dtype": "float32",
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "gradient_checkpointing": false,
+  "initializer_factor": 1.0,
+  "is_decoder": false,
+  "is_encoder_decoder": true,
+  "is_gated_act": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "num_decoder_layers": 6,
+  "num_heads": 12,
+  "num_layers": 18,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_max_distance": 128,
+  "relative_attention_num_buckets": 32,
+  "scale_decoder_outputs": false,
+  "tie_word_embeddings": true,
+  "tokenizer_class": "ByT5Tokenizer",
+  "transformers_version": "5.5.0",
+  "use_cache": false,
+  "vocab_size": 384
+}

ct2_float16/config.json

@@ -0,0 +1,10 @@
+{
+  "add_source_bos": false,
+  "add_source_eos": false,
+  "bos_token": "<pad>",
+  "decoder_start_token": "<pad>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "multi_query_attention": false,
+  "unk_token": "<unk>"
+}

ct2_float16/model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16ce8b9fa5b53f6cc041b18c0801bfa6e5c065fb1a0a0c467395a647406b3d13
+size 1163324406

ct2_float16/shared_vocabulary.json

@@ -0,0 +1,386 @@
+[
+  "<pad>",
+  "</s>",
+  "<unk>",
+  "\u0000",
+  "\u0001",
+  "\u0002",
+  "\u0003",
+  "\u0004",
+  "\u0005",
+  "\u0006",
+  "\u0007",
+  "\b",
+  "\t",
+  "\n",
+  "\u000b",
+  "\f",
+  "\r",
+  "\u000e",
+  "\u000f",
+  "\u0010",
+  "\u0011",
+  "\u0012",
+  "\u0013",
+  "\u0014",
+  "\u0015",
+  "\u0016",
+  "\u0017",
+  "\u0018",
+  "\u0019",
+  "\u001a",
+  "\u001b",
+  "\u001c",
+  "\u001d",
+  "\u001e",
+  "\u001f",
+  " ",
+  "!",
+  "\"",
+  "#",
+  "$",
+  "%",
+  "&",
+  "'",
+  "(",
+  ")",
+  "*",
+  "+",
+  ",",
+  "-",
+  ".",
+  "/",
+  "0",
+  "1",
+  "2",
+  "3",
+  "4",
+  "5",
+  "6",
+  "7",
+  "8",
+  "9",
+  ":",
+  ";",
+  "<",
+  "=",
+  ">",
+  "?",
+  "@",
+  "A",
+  "B",
+  "C",
+  "D",
+  "E",
+  "F",
+  "G",
+  "H",
+  "I",
+  "J",
+  "K",
+  "L",
+  "M",
+  "N",
+  "O",
+  "P",
+  "Q",
+  "R",
+  "S",
+  "T",
+  "U",
+  "V",
+  "W",
+  "X",
+  "Y",
+  "Z",
+  "[",
+  "\\",
+  "]",
+  "^",
+  "_",
+  "`",
+  "a",
+  "b",
+  "c",
+  "d",
+  "e",
+  "f",
+  "g",
+  "h",
+  "i",
+  "j",
+  "k",
+  "l",
+  "m",
+  "n",
+  "o",
+  "p",
+  "q",
+  "r",
+  "s",
+  "t",
+  "u",
+  "v",
+  "w",
+  "x",
+  "y",
+  "z",
+  "{",
+  "|",
+  "}",
+  "~",
+  "\u007f",
+  "\u0080",
+  "\u0081",
+  "\u0082",
+  "\u0083",
+  "\u0084",
+  "\u0085",
+  "\u0086",
+  "\u0087",
+  "\u0088",
+  "\u0089",
+  "\u008a",
+  "\u008b",
+  "\u008c",
+  "\u008d",
+  "\u008e",
+  "\u008f",
+  "\u0090",
+  "\u0091",
+  "\u0092",
+  "\u0093",
+  "\u0094",
+  "\u0095",
+  "\u0096",
+  "\u0097",
+  "\u0098",
+  "\u0099",
+  "\u009a",
+  "\u009b",
+  "\u009c",
+  "\u009d",
+  "\u009e",
+  "\u009f",
+  "\u00a0",
+  "\u00a1",
+  "\u00a2",
+  "\u00a3",
+  "\u00a4",
+  "\u00a5",
+  "\u00a6",
+  "\u00a7",
+  "\u00a8",
+  "\u00a9",
+  "\u00aa",
+  "\u00ab",
+  "\u00ac",
+  "\u00ad",
+  "\u00ae",
+  "\u00af",
+  "\u00b0",
+  "\u00b1",
+  "\u00b2",
+  "\u00b3",
+  "\u00b4",
+  "\u00b5",
+  "\u00b6",
+  "\u00b7",
+  "\u00b8",
+  "\u00b9",
+  "\u00ba",
+  "\u00bb",
+  "\u00bc",
+  "\u00bd",
+  "\u00be",
+  "\u00bf",
+  "\u00c0",
+  "\u00c1",
+  "\u00c2",
+  "\u00c3",
+  "\u00c4",
+  "\u00c5",
+  "\u00c6",
+  "\u00c7",
+  "\u00c8",
+  "\u00c9",
+  "\u00ca",
+  "\u00cb",
+  "\u00cc",
+  "\u00cd",
+  "\u00ce",
+  "\u00cf",
+  "\u00d0",
+  "\u00d1",
+  "\u00d2",
+  "\u00d3",
+  "\u00d4",
+  "\u00d5",
+  "\u00d6",
+  "\u00d7",
+  "\u00d8",
+  "\u00d9",
+  "\u00da",
+  "\u00db",
+  "\u00dc",
+  "\u00dd",
+  "\u00de",
+  "\u00df",
+  "\u00e0",
+  "\u00e1",
+  "\u00e2",
+  "\u00e3",
+  "\u00e4",
+  "\u00e5",
+  "\u00e6",
+  "\u00e7",
+  "\u00e8",
+  "\u00e9",
+  "\u00ea",
+  "\u00eb",
+  "\u00ec",
+  "\u00ed",
+  "\u00ee",
+  "\u00ef",
+  "\u00f0",
+  "\u00f1",
+  "\u00f2",
+  "\u00f3",
+  "\u00f4",
+  "\u00f5",
+  "\u00f6",
+  "\u00f7",
+  "\u00f8",
+  "\u00f9",
+  "\u00fa",
+  "\u00fb",
+  "\u00fc",
+  "\u00fd",
+  "\u00fe",
+  "\u00ff",
+  "<extra_id_0>",
+  "<extra_id_1>",
+  "<extra_id_2>",
+  "<extra_id_3>",
+  "<extra_id_4>",
+  "<extra_id_5>",
+  "<extra_id_6>",
+  "<extra_id_7>",
+  "<extra_id_8>",
+  "<extra_id_9>",
+  "<extra_id_10>",
+  "<extra_id_11>",
+  "<extra_id_12>",
+  "<extra_id_13>",
+  "<extra_id_14>",
+  "<extra_id_15>",
+  "<extra_id_16>",
+  "<extra_id_17>",
+  "<extra_id_18>",
+  "<extra_id_19>",
+  "<extra_id_20>",
+  "<extra_id_21>",
+  "<extra_id_22>",
+  "<extra_id_23>",
+  "<extra_id_24>",
+  "<extra_id_25>",
+  "<extra_id_26>",
+  "<extra_id_27>",
+  "<extra_id_28>",
+  "<extra_id_29>",
+  "<extra_id_30>",
+  "<extra_id_31>",
+  "<extra_id_32>",
+  "<extra_id_33>",
+  "<extra_id_34>",
+  "<extra_id_35>",
+  "<extra_id_36>",
+  "<extra_id_37>",
+  "<extra_id_38>",
+  "<extra_id_39>",
+  "<extra_id_40>",
+  "<extra_id_41>",
+  "<extra_id_42>",
+  "<extra_id_43>",
+  "<extra_id_44>",
+  "<extra_id_45>",
+  "<extra_id_46>",
+  "<extra_id_47>",
+  "<extra_id_48>",
+  "<extra_id_49>",
+  "<extra_id_50>",
+  "<extra_id_51>",
+  "<extra_id_52>",
+  "<extra_id_53>",
+  "<extra_id_54>",
+  "<extra_id_55>",
+  "<extra_id_56>",
+  "<extra_id_57>",
+  "<extra_id_58>",
+  "<extra_id_59>",
+  "<extra_id_60>",
+  "<extra_id_61>",
+  "<extra_id_62>",
+  "<extra_id_63>",
+  "<extra_id_64>",
+  "<extra_id_65>",
+  "<extra_id_66>",
+  "<extra_id_67>",
+  "<extra_id_68>",
+  "<extra_id_69>",
+  "<extra_id_70>",
+  "<extra_id_71>",
+  "<extra_id_72>",
+  "<extra_id_73>",
+  "<extra_id_74>",
+  "<extra_id_75>",
+  "<extra_id_76>",
+  "<extra_id_77>",
+  "<extra_id_78>",
+  "<extra_id_79>",
+  "<extra_id_80>",
+  "<extra_id_81>",
+  "<extra_id_82>",
+  "<extra_id_83>",
+  "<extra_id_84>",
+  "<extra_id_85>",
+  "<extra_id_86>",
+  "<extra_id_87>",
+  "<extra_id_88>",
+  "<extra_id_89>",
+  "<extra_id_90>",
+  "<extra_id_91>",
+  "<extra_id_92>",
+  "<extra_id_93>",
+  "<extra_id_94>",
+  "<extra_id_95>",
+  "<extra_id_96>",
+  "<extra_id_97>",
+  "<extra_id_98>",
+  "<extra_id_99>",
+  "<extra_id_100>",
+  "<extra_id_101>",
+  "<extra_id_102>",
+  "<extra_id_103>",
+  "<extra_id_104>",
+  "<extra_id_105>",
+  "<extra_id_106>",
+  "<extra_id_107>",
+  "<extra_id_108>",
+  "<extra_id_109>",
+  "<extra_id_110>",
+  "<extra_id_111>",
+  "<extra_id_112>",
+  "<extra_id_113>",
+  "<extra_id_114>",
+  "<extra_id_115>",
+  "<extra_id_116>",
+  "<extra_id_117>",
+  "<extra_id_118>",
+  "<extra_id_119>",
+  "<extra_id_120>",
+  "<extra_id_121>",
+  "<extra_id_122>",
+  "<extra_id_123>",
+  "<extra_id_124>"
+]

ct2_int8/config.json

@@ -0,0 +1,10 @@
+{
+  "add_source_bos": false,
+  "add_source_eos": false,
+  "bos_token": "<pad>",
+  "decoder_start_token": "<pad>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "multi_query_attention": false,
+  "unk_token": "<unk>"
+}

ct2_int8/model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9b288eabb289e368558bba02334d7d6c200609388cdc36fc19becb6d7a61fed
+size 583313054

ct2_int8/shared_vocabulary.json

@@ -0,0 +1,386 @@
+[
+  "<pad>",
+  "</s>",
+  "<unk>",
+  "\u0000",
+  "\u0001",
+  "\u0002",
+  "\u0003",
+  "\u0004",
+  "\u0005",
+  "\u0006",
+  "\u0007",
+  "\b",
+  "\t",
+  "\n",
+  "\u000b",
+  "\f",
+  "\r",
+  "\u000e",
+  "\u000f",
+  "\u0010",
+  "\u0011",
+  "\u0012",
+  "\u0013",
+  "\u0014",
+  "\u0015",
+  "\u0016",
+  "\u0017",
+  "\u0018",
+  "\u0019",
+  "\u001a",
+  "\u001b",
+  "\u001c",
+  "\u001d",
+  "\u001e",
+  "\u001f",
+  " ",
+  "!",
+  "\"",
+  "#",
+  "$",
+  "%",
+  "&",
+  "'",
+  "(",
+  ")",
+  "*",
+  "+",
+  ",",
+  "-",
+  ".",
+  "/",
+  "0",
+  "1",
+  "2",
+  "3",
+  "4",
+  "5",
+  "6",
+  "7",
+  "8",
+  "9",
+  ":",
+  ";",
+  "<",
+  "=",
+  ">",
+  "?",
+  "@",
+  "A",
+  "B",
+  "C",
+  "D",
+  "E",
+  "F",
+  "G",
+  "H",
+  "I",
+  "J",
+  "K",
+  "L",
+  "M",
+  "N",
+  "O",
+  "P",
+  "Q",
+  "R",
+  "S",
+  "T",
+  "U",
+  "V",
+  "W",
+  "X",
+  "Y",
+  "Z",
+  "[",
+  "\\",
+  "]",
+  "^",
+  "_",
+  "`",
+  "a",
+  "b",
+  "c",
+  "d",
+  "e",
+  "f",
+  "g",
+  "h",
+  "i",
+  "j",
+  "k",
+  "l",
+  "m",
+  "n",
+  "o",
+  "p",
+  "q",
+  "r",
+  "s",
+  "t",
+  "u",
+  "v",
+  "w",
+  "x",
+  "y",
+  "z",
+  "{",
+  "|",
+  "}",
+  "~",
+  "\u007f",
+  "\u0080",
+  "\u0081",
+  "\u0082",
+  "\u0083",
+  "\u0084",
+  "\u0085",
+  "\u0086",
+  "\u0087",
+  "\u0088",
+  "\u0089",
+  "\u008a",
+  "\u008b",
+  "\u008c",
+  "\u008d",
+  "\u008e",
+  "\u008f",
+  "\u0090",
+  "\u0091",
+  "\u0092",
+  "\u0093",
+  "\u0094",
+  "\u0095",
+  "\u0096",
+  "\u0097",
+  "\u0098",
+  "\u0099",
+  "\u009a",
+  "\u009b",
+  "\u009c",
+  "\u009d",
+  "\u009e",
+  "\u009f",
+  "\u00a0",
+  "\u00a1",
+  "\u00a2",
+  "\u00a3",
+  "\u00a4",
+  "\u00a5",
+  "\u00a6",
+  "\u00a7",
+  "\u00a8",
+  "\u00a9",
+  "\u00aa",
+  "\u00ab",
+  "\u00ac",
+  "\u00ad",
+  "\u00ae",
+  "\u00af",
+  "\u00b0",
+  "\u00b1",
+  "\u00b2",
+  "\u00b3",
+  "\u00b4",
+  "\u00b5",
+  "\u00b6",
+  "\u00b7",
+  "\u00b8",
+  "\u00b9",
+  "\u00ba",
+  "\u00bb",
+  "\u00bc",
+  "\u00bd",
+  "\u00be",
+  "\u00bf",
+  "\u00c0",
+  "\u00c1",
+  "\u00c2",
+  "\u00c3",
+  "\u00c4",
+  "\u00c5",
+  "\u00c6",
+  "\u00c7",
+  "\u00c8",
+  "\u00c9",
+  "\u00ca",
+  "\u00cb",
+  "\u00cc",
+  "\u00cd",
+  "\u00ce",
+  "\u00cf",
+  "\u00d0",
+  "\u00d1",
+  "\u00d2",
+  "\u00d3",
+  "\u00d4",
+  "\u00d5",
+  "\u00d6",
+  "\u00d7",
+  "\u00d8",
+  "\u00d9",
+  "\u00da",
+  "\u00db",
+  "\u00dc",
+  "\u00dd",
+  "\u00de",
+  "\u00df",
+  "\u00e0",
+  "\u00e1",
+  "\u00e2",
+  "\u00e3",
+  "\u00e4",
+  "\u00e5",
+  "\u00e6",
+  "\u00e7",
+  "\u00e8",
+  "\u00e9",
+  "\u00ea",
+  "\u00eb",
+  "\u00ec",
+  "\u00ed",
+  "\u00ee",
+  "\u00ef",
+  "\u00f0",
+  "\u00f1",
+  "\u00f2",
+  "\u00f3",
+  "\u00f4",
+  "\u00f5",
+  "\u00f6",
+  "\u00f7",
+  "\u00f8",
+  "\u00f9",
+  "\u00fa",
+  "\u00fb",
+  "\u00fc",
+  "\u00fd",
+  "\u00fe",
+  "\u00ff",
+  "<extra_id_0>",
+  "<extra_id_1>",
+  "<extra_id_2>",
+  "<extra_id_3>",
+  "<extra_id_4>",
+  "<extra_id_5>",
+  "<extra_id_6>",
+  "<extra_id_7>",
+  "<extra_id_8>",
+  "<extra_id_9>",
+  "<extra_id_10>",
+  "<extra_id_11>",
+  "<extra_id_12>",
+  "<extra_id_13>",
+  "<extra_id_14>",
+  "<extra_id_15>",
+  "<extra_id_16>",
+  "<extra_id_17>",
+  "<extra_id_18>",
+  "<extra_id_19>",
+  "<extra_id_20>",
+  "<extra_id_21>",
+  "<extra_id_22>",
+  "<extra_id_23>",
+  "<extra_id_24>",
+  "<extra_id_25>",
+  "<extra_id_26>",
+  "<extra_id_27>",
+  "<extra_id_28>",
+  "<extra_id_29>",
+  "<extra_id_30>",
+  "<extra_id_31>",
+  "<extra_id_32>",
+  "<extra_id_33>",
+  "<extra_id_34>",
+  "<extra_id_35>",
+  "<extra_id_36>",
+  "<extra_id_37>",
+  "<extra_id_38>",
+  "<extra_id_39>",
+  "<extra_id_40>",
+  "<extra_id_41>",
+  "<extra_id_42>",
+  "<extra_id_43>",
+  "<extra_id_44>",
+  "<extra_id_45>",
+  "<extra_id_46>",
+  "<extra_id_47>",
+  "<extra_id_48>",
+  "<extra_id_49>",
+  "<extra_id_50>",
+  "<extra_id_51>",
+  "<extra_id_52>",
+  "<extra_id_53>",
+  "<extra_id_54>",
+  "<extra_id_55>",
+  "<extra_id_56>",
+  "<extra_id_57>",
+  "<extra_id_58>",
+  "<extra_id_59>",
+  "<extra_id_60>",
+  "<extra_id_61>",
+  "<extra_id_62>",
+  "<extra_id_63>",
+  "<extra_id_64>",
+  "<extra_id_65>",
+  "<extra_id_66>",
+  "<extra_id_67>",
+  "<extra_id_68>",
+  "<extra_id_69>",
+  "<extra_id_70>",
+  "<extra_id_71>",
+  "<extra_id_72>",
+  "<extra_id_73>",
+  "<extra_id_74>",
+  "<extra_id_75>",
+  "<extra_id_76>",
+  "<extra_id_77>",
+  "<extra_id_78>",
+  "<extra_id_79>",
+  "<extra_id_80>",
+  "<extra_id_81>",
+  "<extra_id_82>",
+  "<extra_id_83>",
+  "<extra_id_84>",
+  "<extra_id_85>",
+  "<extra_id_86>",
+  "<extra_id_87>",
+  "<extra_id_88>",
+  "<extra_id_89>",
+  "<extra_id_90>",
+  "<extra_id_91>",
+  "<extra_id_92>",
+  "<extra_id_93>",
+  "<extra_id_94>",
+  "<extra_id_95>",
+  "<extra_id_96>",
+  "<extra_id_97>",
+  "<extra_id_98>",
+  "<extra_id_99>",
+  "<extra_id_100>",
+  "<extra_id_101>",
+  "<extra_id_102>",
+  "<extra_id_103>",
+  "<extra_id_104>",
+  "<extra_id_105>",
+  "<extra_id_106>",
+  "<extra_id_107>",
+  "<extra_id_108>",
+  "<extra_id_109>",
+  "<extra_id_110>",
+  "<extra_id_111>",
+  "<extra_id_112>",
+  "<extra_id_113>",
+  "<extra_id_114>",
+  "<extra_id_115>",
+  "<extra_id_116>",
+  "<extra_id_117>",
+  "<extra_id_118>",
+  "<extra_id_119>",
+  "<extra_id_120>",
+  "<extra_id_121>",
+  "<extra_id_122>",
+  "<extra_id_123>",
+  "<extra_id_124>"
+]

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "decoder_start_token_id": 0,
+  "eos_token_id": [
+    1
+  ],
+  "pad_token_id": 0,
+  "transformers_version": "5.5.0"
+}

handler.py

@@ -0,0 +1,45 @@
+"""HuggingFace Inference API handler for text normalization.
+
+This enables the model to work with the HuggingFace Inference API
+and the `text2text-generation` pipeline.
+"""
+
+from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline
+
+
+class EndpointHandler:
+    def __init__(self, path: str = ""):
+        self.model = AutoModelForSeq2SeqLM.from_pretrained(path)
+        self.tokenizer = AutoTokenizer.from_pretrained("google/byt5-base")
+        self.model.eval()
+
+    def __call__(self, data):
+        """Handle inference request.
+
+        Expected input format:
+            {"inputs": "<de> Das kostet 12,50 €."}
+        or:
+            {"inputs": "Das kostet 12,50 €.", "parameters": {"language": "de"}}
+        """
+        inputs = data.get("inputs", "")
+        params = data.get("parameters", {})
+
+        # If language is passed separately, add the prefix
+        if not inputs.startswith("<") and "language" in params:
+            inputs = f"<{params['language']}> {inputs}"
+
+        tokenized = self.tokenizer(
+            inputs, return_tensors="pt", max_length=512, truncation=True
+        )
+
+        import torch
+
+        with torch.no_grad():
+            output = self.model.generate(
+                **tokenized,
+                max_new_tokens=params.get("max_new_tokens", 512),
+                num_beams=params.get("num_beams", 1),
+            )
+
+        result = self.tokenizer.decode(output[0], skip_special_tokens=True)
+        return [{"generated_text": result}]

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d134b0cf7cd66a6b5d8bf0a9bc678ddc2179c2d453ea2a00bad27ce7591029a
+size 2326643632

tokenizer_config.json

@@ -0,0 +1,1290 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259": {
+      "content": "<extra_id_0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "260": {
+      "content": "<extra_id_1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "261": {
+      "content": "<extra_id_2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "262": {
+      "content": "<extra_id_3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "263": {
+      "content": "<extra_id_4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "264": {
+      "content": "<extra_id_5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "265": {
+      "content": "<extra_id_6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "266": {
+      "content": "<extra_id_7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "267": {
+      "content": "<extra_id_8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "268": {
+      "content": "<extra_id_9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "269": {
+      "content": "<extra_id_10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "270": {
+      "content": "<extra_id_11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "271": {
+      "content": "<extra_id_12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "272": {
+      "content": "<extra_id_13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "273": {
+      "content": "<extra_id_14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "274": {
+      "content": "<extra_id_15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "275": {
+      "content": "<extra_id_16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "276": {
+      "content": "<extra_id_17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "277": {
+      "content": "<extra_id_18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "278": {
+      "content": "<extra_id_19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "279": {
+      "content": "<extra_id_20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "280": {
+      "content": "<extra_id_21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "281": {
+      "content": "<extra_id_22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "282": {
+      "content": "<extra_id_23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "283": {
+      "content": "<extra_id_24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "284": {
+      "content": "<extra_id_25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "285": {
+      "content": "<extra_id_26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "286": {
+      "content": "<extra_id_27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "287": {
+      "content": "<extra_id_28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "288": {
+      "content": "<extra_id_29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "289": {
+      "content": "<extra_id_30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "290": {
+      "content": "<extra_id_31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "291": {
+      "content": "<extra_id_32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "292": {
+      "content": "<extra_id_33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "293": {
+      "content": "<extra_id_34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "294": {
+      "content": "<extra_id_35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "295": {
+      "content": "<extra_id_36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "296": {
+      "content": "<extra_id_37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "297": {
+      "content": "<extra_id_38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "298": {
+      "content": "<extra_id_39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "299": {
+      "content": "<extra_id_40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "300": {
+      "content": "<extra_id_41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "301": {
+      "content": "<extra_id_42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "302": {
+      "content": "<extra_id_43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "303": {
+      "content": "<extra_id_44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "304": {
+      "content": "<extra_id_45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "305": {
+      "content": "<extra_id_46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "306": {
+      "content": "<extra_id_47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "307": {
+      "content": "<extra_id_48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "308": {
+      "content": "<extra_id_49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "309": {
+      "content": "<extra_id_50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "310": {
+      "content": "<extra_id_51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "311": {
+      "content": "<extra_id_52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "312": {
+      "content": "<extra_id_53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "313": {
+      "content": "<extra_id_54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "314": {
+      "content": "<extra_id_55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "315": {
+      "content": "<extra_id_56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "316": {
+      "content": "<extra_id_57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "317": {
+      "content": "<extra_id_58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "318": {
+      "content": "<extra_id_59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "319": {
+      "content": "<extra_id_60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "320": {
+      "content": "<extra_id_61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "321": {
+      "content": "<extra_id_62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "322": {
+      "content": "<extra_id_63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "323": {
+      "content": "<extra_id_64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "324": {
+      "content": "<extra_id_65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "325": {
+      "content": "<extra_id_66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "326": {
+      "content": "<extra_id_67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "327": {
+      "content": "<extra_id_68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "328": {
+      "content": "<extra_id_69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "329": {
+      "content": "<extra_id_70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "330": {
+      "content": "<extra_id_71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "331": {
+      "content": "<extra_id_72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "332": {
+      "content": "<extra_id_73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "333": {
+      "content": "<extra_id_74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "334": {
+      "content": "<extra_id_75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "335": {
+      "content": "<extra_id_76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "336": {
+      "content": "<extra_id_77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "337": {
+      "content": "<extra_id_78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "338": {
+      "content": "<extra_id_79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "339": {
+      "content": "<extra_id_80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "340": {
+      "content": "<extra_id_81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "341": {
+      "content": "<extra_id_82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "342": {
+      "content": "<extra_id_83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "343": {
+      "content": "<extra_id_84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "344": {
+      "content": "<extra_id_85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "345": {
+      "content": "<extra_id_86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "346": {
+      "content": "<extra_id_87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "347": {
+      "content": "<extra_id_88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "348": {
+      "content": "<extra_id_89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "349": {
+      "content": "<extra_id_90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "350": {
+      "content": "<extra_id_91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "351": {
+      "content": "<extra_id_92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "352": {
+      "content": "<extra_id_93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "353": {
+      "content": "<extra_id_94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "354": {
+      "content": "<extra_id_95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "355": {
+      "content": "<extra_id_96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "356": {
+      "content": "<extra_id_97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "357": {
+      "content": "<extra_id_98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "358": {
+      "content": "<extra_id_99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "359": {
+      "content": "<extra_id_100>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "360": {
+      "content": "<extra_id_101>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "361": {
+      "content": "<extra_id_102>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "362": {
+      "content": "<extra_id_103>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "363": {
+      "content": "<extra_id_104>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "364": {
+      "content": "<extra_id_105>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "365": {
+      "content": "<extra_id_106>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "366": {
+      "content": "<extra_id_107>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "367": {
+      "content": "<extra_id_108>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "368": {
+      "content": "<extra_id_109>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "369": {
+      "content": "<extra_id_110>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "370": {
+      "content": "<extra_id_111>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "371": {
+      "content": "<extra_id_112>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "372": {
+      "content": "<extra_id_113>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "373": {
+      "content": "<extra_id_114>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "374": {
+      "content": "<extra_id_115>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "375": {
+      "content": "<extra_id_116>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "376": {
+      "content": "<extra_id_117>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "377": {
+      "content": "<extra_id_118>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "378": {
+      "content": "<extra_id_119>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "379": {
+      "content": "<extra_id_120>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "380": {
+      "content": "<extra_id_121>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "381": {
+      "content": "<extra_id_122>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "382": {
+      "content": "<extra_id_123>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "383": {
+      "content": "<extra_id_124>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<extra_id_0>",
+    "<extra_id_1>",
+    "<extra_id_2>",
+    "<extra_id_3>",
+    "<extra_id_4>",
+    "<extra_id_5>",
+    "<extra_id_6>",
+    "<extra_id_7>",
+    "<extra_id_8>",
+    "<extra_id_9>",
+    "<extra_id_10>",
+    "<extra_id_11>",
+    "<extra_id_12>",
+    "<extra_id_13>",
+    "<extra_id_14>",
+    "<extra_id_15>",
+    "<extra_id_16>",
+    "<extra_id_17>",
+    "<extra_id_18>",
+    "<extra_id_19>",
+    "<extra_id_20>",
+    "<extra_id_21>",
+    "<extra_id_22>",
+    "<extra_id_23>",
+    "<extra_id_24>",
+    "<extra_id_25>",
+    "<extra_id_26>",
+    "<extra_id_27>",
+    "<extra_id_28>",
+    "<extra_id_29>",
+    "<extra_id_30>",
+    "<extra_id_31>",
+    "<extra_id_32>",
+    "<extra_id_33>",
+    "<extra_id_34>",
+    "<extra_id_35>",
+    "<extra_id_36>",
+    "<extra_id_37>",
+    "<extra_id_38>",
+    "<extra_id_39>",
+    "<extra_id_40>",
+    "<extra_id_41>",
+    "<extra_id_42>",
+    "<extra_id_43>",
+    "<extra_id_44>",
+    "<extra_id_45>",
+    "<extra_id_46>",
+    "<extra_id_47>",
+    "<extra_id_48>",
+    "<extra_id_49>",
+    "<extra_id_50>",
+    "<extra_id_51>",
+    "<extra_id_52>",
+    "<extra_id_53>",
+    "<extra_id_54>",
+    "<extra_id_55>",
+    "<extra_id_56>",
+    "<extra_id_57>",
+    "<extra_id_58>",
+    "<extra_id_59>",
+    "<extra_id_60>",
+    "<extra_id_61>",
+    "<extra_id_62>",
+    "<extra_id_63>",
+    "<extra_id_64>",
+    "<extra_id_65>",
+    "<extra_id_66>",
+    "<extra_id_67>",
+    "<extra_id_68>",
+    "<extra_id_69>",
+    "<extra_id_70>",
+    "<extra_id_71>",
+    "<extra_id_72>",
+    "<extra_id_73>",
+    "<extra_id_74>",
+    "<extra_id_75>",
+    "<extra_id_76>",
+    "<extra_id_77>",
+    "<extra_id_78>",
+    "<extra_id_79>",
+    "<extra_id_80>",
+    "<extra_id_81>",
+    "<extra_id_82>",
+    "<extra_id_83>",
+    "<extra_id_84>",
+    "<extra_id_85>",
+    "<extra_id_86>",
+    "<extra_id_87>",
+    "<extra_id_88>",
+    "<extra_id_89>",
+    "<extra_id_90>",
+    "<extra_id_91>",
+    "<extra_id_92>",
+    "<extra_id_93>",
+    "<extra_id_94>",
+    "<extra_id_95>",
+    "<extra_id_96>",
+    "<extra_id_97>",
+    "<extra_id_98>",
+    "<extra_id_99>",
+    "<extra_id_100>",
+    "<extra_id_101>",
+    "<extra_id_102>",
+    "<extra_id_103>",
+    "<extra_id_104>",
+    "<extra_id_105>",
+    "<extra_id_106>",
+    "<extra_id_107>",
+    "<extra_id_108>",
+    "<extra_id_109>",
+    "<extra_id_110>",
+    "<extra_id_111>",
+    "<extra_id_112>",
+    "<extra_id_113>",
+    "<extra_id_114>",
+    "<extra_id_115>",
+    "<extra_id_116>",
+    "<extra_id_117>",
+    "<extra_id_118>",
+    "<extra_id_119>",
+    "<extra_id_120>",
+    "<extra_id_121>",
+    "<extra_id_122>",
+    "<extra_id_123>",
+    "<extra_id_124>"
+  ],
+  "backend": "custom",
+  "eos_token": "</s>",
+  "extra_ids": 0,
+  "extra_special_tokens": [
+    "<extra_id_0>",
+    "<extra_id_1>",
+    "<extra_id_2>",
+    "<extra_id_3>",
+    "<extra_id_4>",
+    "<extra_id_5>",
+    "<extra_id_6>",
+    "<extra_id_7>",
+    "<extra_id_8>",
+    "<extra_id_9>",
+    "<extra_id_10>",
+    "<extra_id_11>",
+    "<extra_id_12>",
+    "<extra_id_13>",
+    "<extra_id_14>",
+    "<extra_id_15>",
+    "<extra_id_16>",
+    "<extra_id_17>",
+    "<extra_id_18>",
+    "<extra_id_19>",
+    "<extra_id_20>",
+    "<extra_id_21>",
+    "<extra_id_22>",
+    "<extra_id_23>",
+    "<extra_id_24>",
+    "<extra_id_25>",
+    "<extra_id_26>",
+    "<extra_id_27>",
+    "<extra_id_28>",
+    "<extra_id_29>",
+    "<extra_id_30>",
+    "<extra_id_31>",
+    "<extra_id_32>",
+    "<extra_id_33>",
+    "<extra_id_34>",
+    "<extra_id_35>",
+    "<extra_id_36>",
+    "<extra_id_37>",
+    "<extra_id_38>",
+    "<extra_id_39>",
+    "<extra_id_40>",
+    "<extra_id_41>",
+    "<extra_id_42>",
+    "<extra_id_43>",
+    "<extra_id_44>",
+    "<extra_id_45>",
+    "<extra_id_46>",
+    "<extra_id_47>",
+    "<extra_id_48>",
+    "<extra_id_49>",
+    "<extra_id_50>",
+    "<extra_id_51>",
+    "<extra_id_52>",
+    "<extra_id_53>",
+    "<extra_id_54>",
+    "<extra_id_55>",
+    "<extra_id_56>",
+    "<extra_id_57>",
+    "<extra_id_58>",
+    "<extra_id_59>",
+    "<extra_id_60>",
+    "<extra_id_61>",
+    "<extra_id_62>",
+    "<extra_id_63>",
+    "<extra_id_64>",
+    "<extra_id_65>",
+    "<extra_id_66>",
+    "<extra_id_67>",
+    "<extra_id_68>",
+    "<extra_id_69>",
+    "<extra_id_70>",
+    "<extra_id_71>",
+    "<extra_id_72>",
+    "<extra_id_73>",
+    "<extra_id_74>",
+    "<extra_id_75>",
+    "<extra_id_76>",
+    "<extra_id_77>",
+    "<extra_id_78>",
+    "<extra_id_79>",
+    "<extra_id_80>",
+    "<extra_id_81>",
+    "<extra_id_82>",
+    "<extra_id_83>",
+    "<extra_id_84>",
+    "<extra_id_85>",
+    "<extra_id_86>",
+    "<extra_id_87>",
+    "<extra_id_88>",
+    "<extra_id_89>",
+    "<extra_id_90>",
+    "<extra_id_91>",
+    "<extra_id_92>",
+    "<extra_id_93>",
+    "<extra_id_94>",
+    "<extra_id_95>",
+    "<extra_id_96>",
+    "<extra_id_97>",
+    "<extra_id_98>",
+    "<extra_id_99>",
+    "<extra_id_100>",
+    "<extra_id_101>",
+    "<extra_id_102>",
+    "<extra_id_103>",
+    "<extra_id_104>",
+    "<extra_id_105>",
+    "<extra_id_106>",
+    "<extra_id_107>",
+    "<extra_id_108>",
+    "<extra_id_109>",
+    "<extra_id_110>",
+    "<extra_id_111>",
+    "<extra_id_112>",
+    "<extra_id_113>",
+    "<extra_id_114>",
+    "<extra_id_115>",
+    "<extra_id_116>",
+    "<extra_id_117>",
+    "<extra_id_118>",
+    "<extra_id_119>",
+    "<extra_id_120>",
+    "<extra_id_121>",
+    "<extra_id_122>",
+    "<extra_id_123>",
+    "<extra_id_124>"
+  ],
+  "is_local": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "tokenizer_class": "ByT5Tokenizer",
+  "unk_token": "<unk>"
+}