Initial commit

.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct/base/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct/dp8/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+qwen3-4b-instruct/dp3/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,246 @@
+---
+language:
+- code
+license: apache-2.0
+tags:
+- differential-privacy
+- code-generation
+- continued-pretraining
+- lora
+- dp-sgd
+- opacus
+- privacy
+datasets:
+- codedp-ase26/codedp-cpt
+base_model:
+- ibm-granite/granite-4.0-h-tiny
+- bigcode/starcoder2-7b
+- Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+---
+
+# CodeDP-CPT: Differentially Private Continued Pre-Training for Code Models
+
+This repository contains LoRA adapters for code language models trained with **Continued Pre-Training (CPT)** under **Differential Privacy (DP-SGD)**. The models demonstrate that formal privacy guarantees can be applied to code generation models while preserving utility.
+
+## Models
+
+Nine adapter checkpoints are provided — three base models × three privacy configurations:
+
+| Base Model | Variant | DP | Target ε | Achieved ε | Adapter Path |
+|---|---|---|---|---|---|
+| [ibm-granite/granite-4.0-h-tiny](https://huggingface.co/ibm-granite/granite-4.0-h-tiny) | base | No | — | — | `granite-4.0-h-tiny/base/adapter/` |
+| [ibm-granite/granite-4.0-h-tiny](https://huggingface.co/ibm-granite/granite-4.0-h-tiny) | dp3 | Yes | 3.0 | 2.99 | `granite-4.0-h-tiny/dp3/adapter/` |
+| [ibm-granite/granite-4.0-h-tiny](https://huggingface.co/ibm-granite/granite-4.0-h-tiny) | dp8 | Yes | 8.0 | 8.00 | `granite-4.0-h-tiny/dp8/adapter/` |
+| [bigcode/starcoder2-7b](https://huggingface.co/bigcode/starcoder2-7b) | base | No | — | — | `starcoder2-7b/base/adapter/` |
+| [bigcode/starcoder2-7b](https://huggingface.co/bigcode/starcoder2-7b) | dp3 | Yes | 3.0 | 3.00 | `starcoder2-7b/dp3/adapter/` |
+| [bigcode/starcoder2-7b](https://huggingface.co/bigcode/starcoder2-7b) | dp8 | Yes | 8.0 | 8.00 | `starcoder2-7b/dp8/adapter/` |
+| [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) | base | No | — | — | `qwen3-4b-instruct/base/adapter/` |
+| [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) | dp3 | Yes | 3.0 | 2.99 | `qwen3-4b-instruct/dp3/adapter/` |
+| [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) | dp8 | Yes | 8.0 | 8.00 | `qwen3-4b-instruct/dp8/adapter/` |
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base_model_name = "ibm-granite/granite-4.0-h-tiny"
+adapter_path = "codedp-ase26/codedp-cpt-models"
+subfolder = "granite-4.0-h-tiny/dp8/adapter"
+
+tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(base_model_name, trust_remote_code=True)
+model = PeftModel.from_pretrained(model, adapter_path, subfolder=subfolder)
+```
+
+## Training Details
+
+### Dataset
+
+- **Dataset:** [codedp-ase26/codedp-cpt](https://huggingface.co/datasets/codedp-ase26/codedp-cpt) — code mined from GitHub repositories with quality filtering and decontamination (file-level, Type-1, and Type-2 clone detection against evaluation benchmarks)
+- **Mode:** Causal language modeling (continued pre-training)
+- **Validation split:** 5% held out
+
+### LoRA Configuration
+
+| Parameter | Value |
+|---|---|
+| Rank (r) | 16 |
+| Alpha (α) | 32 |
+| Dropout | 0.05 |
+| Target modules | q_proj, k_proj, v_proj, o_proj |
+| Modules to save | lm_head |
+
+### Training Hyperparameters
+
+| Parameter | No-DP (base) | DP variants |
+|---|---|---|
+| Epochs | 2 | 2 |
+| Micro-batch size (per GPU) | 8 | 8 |
+| Learning rate | 1e-4 | 2e-4 |
+| Optimizer | AdamW | AdamW |
+| LR scheduler | Cosine | Cosine |
+| Warmup ratio | 5% | 5% |
+| Max gradient norm | 1.0 | 1.0 |
+| Sequence length | 1024 | 1024 |
+| Precision | bfloat16 | bfloat16 |
+| Seed | 42 | 42 |
+
+**Effective batch sizes** (micro-batch × gradient accumulation steps × GPUs):
+
+| Model | GPUs | No-DP | DP ε=3 / ε=8 |
+|---|---|---|---|
+| Granite-4.0-H-Tiny | 4 | 256 (8×8×4) | 512 (8×16×4) |
+| StarCoder2-7B | 4 | 256 (8×8×4) | 512 (8×16×4) |
+| Qwen3-4B-Instruct | 8 | 256 (8×4×8) | 512 (8×8×8) |
+
+### Differential Privacy
+
+| Parameter | Value |
+|---|---|
+| Engine | Opacus PrivacyEngine |
+| Mechanism | Gaussian (DP-SGD) |
+| Per-sample gradients | Hook-based |
+| Clipping | Flat (global) |
+| Target δ | 1e-5 |
+| Target ε | 3.0 or 8.0 |
+| Privacy accounting | RDP (Rényi Differential Privacy) |
+
+### Infrastructure
+
+- **GPUs:** NVIDIA H200 (140 GB VRAM each) — 4 GPUs for Granite and StarCoder2, 8 GPUs for Qwen
+- **CUDA:** 13.0
+- **Distributed strategy:** DDP (Distributed Data Parallel) with NCCL backend
+
+## Evaluation Results
+
+### Functional Correctness — CodeDP-FC (Granite-4.0-H-Tiny)
+
+103 code generation tasks, 10 samples per task, temperature 0.8.
+
+| Variant | pass@1 | pass@5 | pass@10 |
+|---|---|---|---|
+| No fine-tuning | 13.5% | 18.4% | 20.4% |
+| CPT (no DP) | 10.1% | 16.6% | 18.4% |
+| CPT + DP (ε=3) | 13.7% | 19.1% | 21.4% |
+| CPT + DP (ε=8) | **14.5%** | **21.1%** | **23.3%** |
+
+### Training Loss (Eval Set)
+
+| Model | No-DP | DP ε=3 | DP ε=8 |
+|---|---|---|---|
+| Granite-4.0-H-Tiny | 0.946 | 1.044 | 1.038 |
+| StarCoder2-7B | 0.745 | 0.843 | 0.841 |
+| Qwen3-4B-Instruct | 0.808 | 0.941 | 0.925 |
+
+### Privacy Audit
+
+New-token canary audit (500 members, 500 non-members, 49-token random prefixes). Higher AUC = more memorization; lower = better privacy.
+
+| Model | Variant | Loss AUC | Embedding AUC | Empirical ε (p=0.01) |
+|---|---|---|---|---|
+| Granite-4.0-H-Tiny | base | 1.000 | 1.000 | 3.02 |
+| Granite-4.0-H-Tiny | dp3 | 0.543 | 0.513 | 0.00 |
+| Granite-4.0-H-Tiny | dp8 | 0.564 | 0.508 | 0.16 |
+| StarCoder2-7B | base | 1.000 | 0.916 | 3.02 |
+| StarCoder2-7B | dp3 | 0.526 | 0.521 | 0.00 |
+| StarCoder2-7B | dp8 | 0.520 | 0.523 | 0.00 |
+| Qwen3-4B-Instruct | base | 0.969 | 0.884 | 3.02 |
+| Qwen3-4B-Instruct | dp3 | 0.505 | 0.515 | 0.00 |
+| Qwen3-4B-Instruct | dp8 | 0.515 | 0.516 | 0.00 |
+
+**Key finding:** DP training reduces canary audit AUC to near-random (0.5), with empirical ε dropping to 0 in most cases — confirming that the formal privacy guarantees hold in practice.
+
+### MIA Benchmark Validation — BoW Distribution Shift
+
+The canary MIA benchmark ([codedp-ase26/codedp-bench-canary-mia](https://huggingface.co/datasets/codedp-ase26/codedp-bench-canary-mia)) uses a targeted design where member and non-member samples share the same code prefix and differ only in the PII secret. A bag-of-words Random Forest classifier (5-fold CV) confirms no distribution shift:
+
+| PII Type | BoW AUC | ± std | n |
+|---|---|---|---|
+| Overall | 0.099 | 0.018 | 400 |
+| api_key | 0.033 | 0.047 | 80 |
+| db_url | 0.311 | 0.105 | 80 |
+| email | 0.078 | 0.099 | 80 |
+| internal_ip | 0.028 | 0.021 | 80 |
+| password | 0.055 | 0.048 | 80 |
+
+All BoW AUC values are well below 0.5, confirming that MIA signal must come from the model's knowledge of the secret, not surface-level text features.
+
+<details>
+<summary>BoW shift test code</summary>
+
+```python
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.feature_extraction.text import CountVectorizer
+from sklearn.model_selection import StratifiedKFold
+from sklearn.metrics import roc_auc_score
+import numpy as np, json
+from datasets import load_dataset
+
+ds = load_dataset("codedp-ase26/codedp-bench-canary-mia", split="train")
+records = list(ds)
+
+def bow_shift(texts, labels, n_folds=5):
+    X = CountVectorizer(max_features=5000, stop_words="english").fit_transform(texts)
+    y = np.array(labels)
+    aucs = []
+    for tr, te in StratifiedKFold(n_folds, shuffle=True, random_state=42).split(X, y):
+        clf = RandomForestClassifier(100, random_state=42, n_jobs=-1)
+        clf.fit(X[tr], y[tr])
+        aucs.append(roc_auc_score(y[te], clf.predict_proba(X[te])[:, 1]))
+    return np.mean(aucs), np.std(aucs)
+
+# Overall
+texts = [r["input"] for r in records]
+labels = [r["label"] for r in records]
+print("Overall:", bow_shift(texts, labels))
+
+# Per PII category
+for pii_type in sorted(set(r["pii_type"] for r in records)):
+    cat = [r for r in records if r["pii_type"] == pii_type]
+    print(f"{pii_type}:", bow_shift([r["input"] for r in cat], [r["label"] for r in cat]))
+```
+</details>
+
+## Repository Structure
+
+```
+├── granite-4.0-h-tiny/
+│   ├── base/                    # No-DP baseline
+│   ├── dp3/                     # DP ε=3
+│   └── dp8/                     # DP ε=8
+├── starcoder2-7b/
+│   ├── base/
+│   ├── dp3/
+│   └── dp8/
+└── qwen3-4b-instruct/
+    ├── base/
+    ├── dp3/
+    └── dp8/
+```
+
+Each variant directory contains:
+- `adapter/` — LoRA adapter weights (PEFT-compatible)
+- `tokenizer/` — Tokenizer with any added audit tokens
+- `resolved_config.yaml` — Full training configuration
+- `summary.json` — Training and audit metrics
+- `audit_results.json`, `audit_scores.npz` — Privacy audit artifacts
+- `metrics.jsonl`, `scalars.csv` — Training logs
+- `tensorboard/` — TensorBoard events
+- `codecarbon.csv` — Carbon emissions tracking
+- `epochs/` — Per-epoch checkpoints and audit results
+
+## Limitations
+
+- These are **LoRA adapters**, not standalone models. They require the corresponding base model for inference.
+- The adapters include additional tokenizer tokens added during the privacy audit process (canary tokens). These do not affect normal generation.
+- Evaluation results are on the CodeDP-FC benchmark; performance may vary on other code generation tasks.
+- DP training with tight privacy budgets (ε=3) incurs a utility cost, particularly visible in validation loss.
+
+## Related Resources
+
+- **Training dataset:** [codedp-ase26/codedp-cpt](https://huggingface.co/datasets/codedp-ase26/codedp-cpt)
+- **MIA benchmark (general):** [codedp-ase26/codedp-bench-mia-cpt](https://huggingface.co/datasets/codedp-ase26/codedp-bench-mia-cpt)
+- **MIA benchmark (canary):** [codedp-ase26/codedp-bench-canary-mia](https://huggingface.co/datasets/codedp-ase26/codedp-bench-canary-mia)

granite-4.0-h-tiny/base/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: ibm-granite/granite-4.0-h-tiny
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:ibm-granite/granite-4.0-h-tiny
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

granite-4.0-h-tiny/base/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

granite-4.0-h-tiny/base/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70251fc485b545d89488f5ee78643e9b93f01dad47b16545f7544fc4a6148427
+size 1861530512

granite-4.0-h-tiny/base/adapter/added_tokens.json

@@ -0,0 +1,502 @@
+{
+  "114848338811": 100360,
+  "116427191984": 100714,
+  "116639746413": 100686,
+  "117298239631": 100576,
+  "117657531962": 100633,
+  "117783274348": 100650,
+  "119342152414": 100723,
+  "121557291859": 100582,
+  "123741461297": 100491,
+  "124589118494": 100487,
+  "125866399464": 100596,
+  "126574388585": 100538,
+  "129693547287": 100472,
+  "133373659361": 100667,
+  "135441121756": 100355,
+  "136931728327": 100457,
+  "138154781148": 100767,
+  "138218735276": 100651,
+  "141499982867": 100745,
+  "142526557681": 100845,
+  "142799718159": 100709,
+  "143182978129": 100671,
+  "143777992227": 100366,
+  "144373897488": 100777,
+  "145669731981": 100581,
+  "147444452794": 100830,
+  "147541537162": 100775,
+  "151859598873": 100547,
+  "152676556624": 100682,
+  "154962668224": 100696,
+  "158142752582": 100577,
+  "158929292238": 100587,
+  "159337222655": 100422,
+  "161238482259": 100790,
+  "161588262349": 100437,
+  "163165456396": 100788,
+  "163497196769": 100820,
+  "165429152138": 100717,
+  "172599315861": 100371,
+  "173781362446": 100423,
+  "175469284871": 100515,
+  "175528632226": 100833,
+  "178478368922": 100762,
+  "179386156569": 100804,
+  "179564995814": 100697,
+  "181233596124": 100782,
+  "181439694557": 100398,
+  "182345464792": 100442,
+  "182782912224": 100624,
+  "183368827562": 100551,
+  "185438575221": 100537,
+  "186198784585": 100598,
+  "186872457894": 100835,
+  "191887635211": 100721,
+  "195388849133": 100792,
+  "195792996468": 100546,
+  "196119929397": 100611,
+  "196735786156": 100493,
+  "198462689911": 100428,
+  "211411365397": 100377,
+  "211662639865": 100706,
+  "212985874612": 100720,
+  "214193645583": 100802,
+  "215671314196": 100602,
+  "215826764872": 100756,
+  "216667245274": 100484,
+  "217756494954": 100648,
+  "218391991184": 100679,
+  "218784516525": 100707,
+  "221762325616": 100678,
+  "225164373623": 100439,
+  "228358422847": 100669,
+  "228565148417": 100726,
+  "229775525672": 100793,
+  "231853711778": 100516,
+  "232776999554": 100443,
+  "232787996724": 100601,
+  "232896526252": 100489,
+  "233795848681": 100507,
+  "234316622259": 100625,
+  "242851284913": 100435,
+  "243165586174": 100784,
+  "244371196654": 100615,
+  "244885399387": 100533,
+  "245416776926": 100583,
+  "251789649997": 100613,
+  "252867443568": 100578,
+  "256497188281": 100735,
+  "259533577263": 100458,
+  "259666844669": 100434,
+  "262296912232": 100481,
+  "262437452884": 100794,
+  "264259917554": 100413,
+  "265847332563": 100818,
+  "267514771964": 100574,
+  "267954879268": 100646,
+  "268433315926": 100822,
+  "268921994538": 100715,
+  "272362973463": 100406,
+  "275597777299": 100565,
+  "281673864127": 100408,
+  "282322838685": 100373,
+  "284445431422": 100664,
+  "284776712475": 100780,
+  "286173241916": 100603,
+  "291762694874": 100670,
+  "293847485454": 100760,
+  "295118729589": 100609,
+  "295448593321": 100490,
+  "296953381169": 100479,
+  "298464134954": 100483,
+  "311632789559": 100391,
+  "313145133463": 100773,
+  "313313831273": 100815,
+  "314697386682": 100449,
+  "315534574435": 100754,
+  "319584985416": 100570,
+  "322589379462": 100680,
+  "322683334381": 100668,
+  "325196781219": 100844,
+  "327158398268": 100673,
+  "327491773134": 100387,
+  "328552864359": 100705,
+  "331369926711": 100731,
+  "332234613346": 100843,
+  "332522971941": 100683,
+  "334513529294": 100662,
+  "336813343298": 100747,
+  "337326533813": 100444,
+  "339882449689": 100597,
+  "342221452223": 100742,
+  "345271433112": 100657,
+  "349249721685": 100751,
+  "351885857332": 100426,
+  "353755529131": 100753,
+  "355977139358": 100379,
+  "356722692178": 100846,
+  "357299947518": 100832,
+  "357621613582": 100738,
+  "363214487524": 100532,
+  "364196983365": 100545,
+  "364492359246": 100695,
+  "364947325679": 100618,
+  "367459894796": 100356,
+  "367968398551": 100757,
+  "368763327262": 100799,
+  "371496153876": 100816,
+  "374327157578": 100390,
+  "374439448821": 100468,
+  "374598961236": 100512,
+  "375429755114": 100781,
+  "377273562477": 100480,
+  "377338713663": 100494,
+  "378749551722": 100789,
+  "379557332953": 100451,
+  "379571934299": 100612,
+  "383837596997": 100381,
+  "384598329253": 100616,
+  "387763951128": 100730,
+  "389715484387": 100759,
+  "393741199486": 100704,
+  "396521188476": 100665,
+  "396847976144": 100785,
+  "398549545767": 100502,
+  "399223169721": 100548,
+  "399749397872": 100471,
+  "412843856172": 100803,
+  "415956914763": 100629,
+  "417668946583": 100737,
+  "418412875665": 100698,
+  "421369185549": 100626,
+  "421435255895": 100666,
+  "425949483533": 100368,
+  "429781429464": 100453,
+  "431149748713": 100594,
+  "431751228368": 100534,
+  "433581868192": 100535,
+  "434632855397": 100627,
+  "438124184139": 100812,
+  "438435178228": 100739,
+  "438745114173": 100690,
+  "441373944925": 100463,
+  "441377892146": 100795,
+  "445925149649": 100364,
+  "447445629421": 100396,
+  "449318681694": 100847,
+  "449549215429": 100819,
+  "451161679736": 100676,
+  "451885565366": 100796,
+  "454164859837": 100656,
+  "454244839926": 100503,
+  "455423278865": 100703,
+  "457919547633": 100358,
+  "462294118988": 100658,
+  "467893531755": 100465,
+  "472129283475": 100749,
+  "473822473819": 100397,
+  "475794684356": 100556,
+  "476969195485": 100772,
+  "479227349997": 100456,
+  "479676316326": 100409,
+  "481884157827": 100416,
+  "482829231639": 100828,
+  "484489193824": 100529,
+  "484773717614": 100497,
+  "485177821815": 100354,
+  "485692541169": 100740,
+  "486391569221": 100536,
+  "488496781997": 100826,
+  "488538864842": 100459,
+  "489562189822": 100530,
+  "489991411364": 100461,
+  "492373714791": 100590,
+  "492516914298": 100769,
+  "497294727353": 100660,
+  "499148455254": 100462,
+  "499156317423": 100654,
+  "499444466686": 100448,
+  "511785788222": 100659,
+  "511923256573": 100672,
+  "512659818733": 100591,
+  "513349618943": 100685,
+  "513361312532": 100520,
+  "514354144759": 100750,
+  "514532995959": 100436,
+  "515856611931": 100655,
+  "516968424863": 100476,
+  "517379856925": 100450,
+  "517778845323": 100431,
+  "522515164754": 100778,
+  "523723475786": 100755,
+  "523918658846": 100392,
+  "523937669294": 100716,
+  "524444658383": 100649,
+  "527117488925": 100466,
+  "527874729936": 100728,
+  "529455466433": 100610,
+  "529679678956": 100420,
+  "529854679937": 100540,
+  "532833541879": 100642,
+  "533417736566": 100552,
+  "535787224917": 100425,
+  "538631466654": 100677,
+  "545512344383": 100447,
+  "546276298359": 100752,
+  "547149832394": 100482,
+  "547563892268": 100813,
+  "549946382131": 100663,
+  "552738847828": 100836,
+  "556814553861": 100827,
+  "556976448182": 100605,
+  "557163366873": 100776,
+  "557533689478": 100527,
+  "558837194455": 100787,
+  "562389434565": 100734,
+  "562716493614": 100732,
+  "563117729724": 100510,
+  "563365646341": 100571,
+  "564395222747": 100675,
+  "564544131355": 100614,
+  "569765693871": 100353,
+  "569853653352": 100758,
+  "571317457151": 100712,
+  "572154333646": 100486,
+  "578827344666": 100427,
+  "579184949249": 100418,
+  "581493616981": 100763,
+  "581976444672": 100588,
+  "582471228164": 100525,
+  "583527834464": 100825,
+  "585544633356": 100524,
+  "586855179568": 100496,
+  "587541149322": 100817,
+  "587787496812": 100417,
+  "589439538674": 100764,
+  "589547936288": 100531,
+  "591464355756": 100719,
+  "592538279337": 100399,
+  "594688742928": 100748,
+  "598916996413": 100630,
+  "599961512792": 100606,
+  "612128769512": 100389,
+  "612788593128": 100580,
+  "614583493135": 100841,
+  "614797267726": 100452,
+  "616248973917": 100542,
+  "618423867817": 100725,
+  "618878266619": 100829,
+  "619454958782": 100593,
+  "622553389126": 100693,
+  "624221476921": 100595,
+  "624266421831": 100701,
+  "625334487352": 100807,
+  "626748382482": 100837,
+  "632962475133": 100528,
+  "635127952696": 100713,
+  "635185846287": 100687,
+  "636324136426": 100801,
+  "637551427887": 100647,
+  "643489267333": 100372,
+  "643692393748": 100401,
+  "644234165531": 100623,
+  "645527857972": 100834,
+  "646979124677": 100410,
+  "649291669397": 100362,
+  "649516938598": 100810,
+  "652154162978": 100557,
+  "653573457168": 100492,
+  "655517954651": 100619,
+  "658481324922": 100415,
+  "661436365453": 100433,
+  "662426712668": 100517,
+  "664847713559": 100637,
+  "665661658354": 100554,
+  "668134915514": 100400,
+  "668514843129": 100559,
+  "669183782449": 100643,
+  "669689115557": 100386,
+  "673948786567": 100620,
+  "676539973863": 100440,
+  "676676697292": 100445,
+  "677896358599": 100499,
+  "678666359123": 100526,
+  "682564697312": 100380,
+  "687284547244": 100848,
+  "689285841825": 100473,
+  "689362171782": 100382,
+  "691315125346": 100635,
+  "691587454538": 100365,
+  "693281533643": 100555,
+  "694235616268": 100407,
+  "696651276628": 100402,
+  "697747991249": 100743,
+  "698218415444": 100378,
+  "699446724178": 100700,
+  "711412347158": 100518,
+  "711641973413": 100653,
+  "712992815957": 100521,
+  "714738141818": 100761,
+  "716763689965": 100744,
+  "717628619782": 100811,
+  "718726519731": 100370,
+  "718951851413": 100699,
+  "721737645236": 100455,
+  "722339992679": 100766,
+  "722618266242": 100429,
+  "724492262421": 100692,
+  "724883568652": 100475,
+  "724964923582": 100779,
+  "729116422489": 100727,
+  "729825168745": 100634,
+  "731681792655": 100566,
+  "733737988257": 100477,
+  "739246183345": 100718,
+  "739394546916": 100645,
+  "739667844957": 100733,
+  "739784153482": 100729,
+  "741757479784": 100539,
+  "742765629356": 100438,
+  "745977673725": 100604,
+  "746285987371": 100361,
+  "749665846687": 100432,
+  "749966767285": 100383,
+  "752644929761": 100563,
+  "753159165157": 100384,
+  "753233719644": 100558,
+  "753466987363": 100519,
+  "754978243758": 100791,
+  "756916173536": 100805,
+  "757422791351": 100800,
+  "757785319772": 100768,
+  "758963111966": 100774,
+  "762486672996": 100708,
+  "762495822823": 100798,
+  "763531783491": 100543,
+  "765474393376": 100359,
+  "765737843639": 100514,
+  "765981581453": 100393,
+  "766294538337": 100607,
+  "767858144912": 100711,
+  "768461952319": 100636,
+  "771619544974": 100474,
+  "773671349113": 100631,
+  "773687856563": 100424,
+  "774679513717": 100823,
+  "778466723723": 100404,
+  "781653545886": 100374,
+  "786391632558": 100553,
+  "794825141891": 100394,
+  "795693824762": 100385,
+  "795838284624": 100421,
+  "796415361892": 100375,
+  "797238311233": 100851,
+  "799364566435": 100504,
+  "799417897197": 100584,
+  "811371928234": 100572,
+  "813981174797": 100689,
+  "814611949491": 100652,
+  "815458725125": 100702,
+  "816115936673": 100622,
+  "817378986438": 100506,
+  "818479319499": 100746,
+  "819162623892": 100560,
+  "824387273757": 100508,
+  "824488329416": 100640,
+  "825815739116": 100741,
+  "826214551218": 100441,
+  "829235161526": 100523,
+  "836746273134": 100414,
+  "837131396371": 100573,
+  "837873762491": 100641,
+  "839269684865": 100579,
+  "841824384614": 100488,
+  "841991688488": 100376,
+  "843998562287": 100485,
+  "846657726193": 100765,
+  "848845611563": 100500,
+  "851254219729": 100824,
+  "851852651677": 100501,
+  "852347289392": 100478,
+  "853859348234": 100403,
+  "856317483891": 100814,
+  "857231548835": 100621,
+  "858146415154": 100710,
+  "859289776479": 100840,
+  "859927364588": 100736,
+  "861342291298": 100821,
+  "864718285734": 100568,
+  "865331112869": 100352,
+  "866519986723": 100661,
+  "867678158885": 100470,
+  "873898736873": 100395,
+  "876537814263": 100681,
+  "877385381798": 100562,
+  "877482678543": 100357,
+  "879425227932": 100469,
+  "879764613759": 100674,
+  "881112998642": 100592,
+  "881136466196": 100564,
+  "883685933161": 100412,
+  "884354318946": 100684,
+  "886464195129": 100639,
+  "887417525236": 100809,
+  "887753195844": 100617,
+  "889965243135": 100694,
+  "892336777248": 100586,
+  "892697125149": 100806,
+  "893974619684": 100850,
+  "894885246797": 100498,
+  "897784492783": 100589,
+  "899151413682": 100464,
+  "912193598348": 100419,
+  "916198946615": 100509,
+  "919181521716": 100770,
+  "921889371893": 100628,
+  "921894985642": 100838,
+  "922327493433": 100411,
+  "922466849763": 100454,
+  "922867396683": 100513,
+  "927914615679": 100363,
+  "928335588653": 100467,
+  "929185622831": 100446,
+  "929826356991": 100405,
+  "933775286797": 100842,
+  "934912591213": 100608,
+  "937495658852": 100460,
+  "938488813964": 100549,
+  "941169431859": 100561,
+  "941861857715": 100808,
+  "941976537588": 100644,
+  "943193294691": 100599,
+  "943878645871": 100839,
+  "944513428457": 100575,
+  "946444632795": 100724,
+  "947262995132": 100849,
+  "948585673431": 100783,
+  "949235426629": 100831,
+  "951794811935": 100511,
+  "952465217972": 100632,
+  "952872416512": 100430,
+  "953267689786": 100638,
+  "955456118295": 100544,
+  "955668961269": 100600,
+  "961133895172": 100567,
+  "961981959227": 100550,
+  "963852916563": 100569,
+  "964342468552": 100495,
+  "967114116556": 100505,
+  "967214577653": 100722,
+  "971621888444": 100522,
+  "981277526855": 100797,
+  "982993456429": 100369,
+  "983493951323": 100691,
+  "983569279932": 100388,
+  "985781138935": 100771,
+  "993832498489": 100688,
+  "996116119839": 100541,
+  "997724962668": 100786,
+  "997958836435": 100585,
+  "997981281989": 100367
+}

granite-4.0-h-tiny/base/adapter/chat_template.jinja

@@ -0,0 +1,118 @@
+{%- set tools_system_message_prefix = 'You are a helpful assistant with access to the following tools. You may call one or more tools to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>'  %}
+{%- set tools_system_message_suffix = '\n</tools>\n\nFor each tool call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.' %}
+{%- set documents_system_message_prefix = 'You are a helpful assistant with access to the following documents. You may use one or more documents to assist with the user query.\n\nYou are given a list of documents within <documents></documents> XML tags:\n<documents>' %}
+{%- set documents_system_message_suffix = '\n</documents>\n\nWrite the response to the user\'s input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.' %}
+{%- set g4_default_system_message = 'You are a helpful assistant. Please ensure responses are professional, accurate, and safe.' %}
+{%- if available_tools is defined and available_tools %}
+    {%- set tools = available_tools %}
+{%- endif %}
+{%- set ns = namespace(tools_system_message=tools_system_message_prefix,
+                       documents_system_message=documents_system_message_prefix,
+                       default_system_message=g4_default_system_message,
+                       system_message=''
+                       ) %}
+{%- if tools %}
+    {%- for tool in tools %}
+        {%- set ns.tools_system_message = ns.tools_system_message + '\n' + (tool | tojson) %}
+    {%- endfor %}
+    {%- set ns.tools_system_message = ns.tools_system_message + tools_system_message_suffix %}
+{%- else %}
+    {%- set ns.tools_system_message = '' %}
+{%- endif %}
+{%- if documents %}
+    {%- for document in documents %}
+        {%- set ns.documents_system_message = ns.documents_system_message + '\n' + (document | tojson) %}
+    {%- endfor %}
+    {%- set ns.documents_system_message = ns.documents_system_message + documents_system_message_suffix %}
+{%- else %}
+    {%- set ns.documents_system_message = '' %}
+{%- endif %}
+{%- if messages[0].role == 'system' %}
+    {%- if messages[0].content is string %}
+        {%- set ns.system_message = messages[0].content %}
+    {%- elif messages[0].content is iterable %}
+        {%- for entry in messages[0].content %}
+            {%- if entry.type== 'text' %}
+                {%- if ns.system_message != '' %}
+                    {%- set ns.system_message = ns.system_message + '\n' %}
+                {%- endif %}
+                {%- set ns.system_message = ns.system_message + entry.text %}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+    {%- if tools and documents %}
+        {%- set ns.system_message = ns.system_message + '\n\n' +  ns.tools_system_message + '\n\n' + ns.documents_system_message %}
+    {%- elif tools %}
+        {%- set ns.system_message = ns.system_message + '\n\n' + ns.tools_system_message %}
+    {%- elif documents %}
+        {%- set ns.system_message = ns.system_message + '\n\n' + ns.documents_system_message %}
+    {%- endif %}
+{%- else %}
+    {%- if tools and documents %}
+        {%- set ns.system_message = ns.tools_system_message + '\n\n' + ns.documents_system_message  %}
+    {%- elif tools %}
+        {%- set ns.system_message = ns.tools_system_message %}
+    {%- elif documents %}
+        {%- set ns.system_message = ns.documents_system_message %}
+    {%- endif %}
+{%- endif %}
+{%- if ns.system_message %}
+    {{- '<|start_of_role|>system<|end_of_role|>' + ns.system_message + '<|end_of_text|>\n' }}
+{%- else %}
+    {{- '<|start_of_role|>system<|end_of_role|>' + ns.default_system_message + '<|end_of_text|>\n' }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = namespace(val='') %}
+    {%- if message.content is string %}
+        {%- set content.val = message.content %}
+    {%- else %}
+        {%- if message.content is iterable %}
+            {%- for entry in message.content %}
+                {%- if entry.type== 'text' %}
+                    {%- if content.val != '' %}
+                        {%- set content.val = content.val + '\n' %}
+                    {%- endif %}
+                    {%- set content.val = content.val + entry.text %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- endif %}
+    {%- if (message.role == 'user') or (message.role == 'system' and not loop.first) %}
+        {{- '<|start_of_role|>' + message.role + '<|end_of_role|>' + content.val + '<|end_of_text|>\n' }}
+    {%- elif message.role == 'assistant' %}
+        {{- '<|start_of_role|>' + message.role + '<|end_of_role|>' + content.val }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content.val) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|end_of_text|>\n' }}
+    {%- elif message.role == 'tool' %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != 'tool') %}
+            {{- '<|start_of_role|>user<|end_of_role|>' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content.val }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != 'tool') %}
+            {{- '<|end_of_text|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_of_role|>assistant<|end_of_role|>' }}
+{%- endif %}

granite-4.0-h-tiny/base/adapter/special_tokens_map.json

@@ -0,0 +1,3532 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "865331112869",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "569765693871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "485177821815",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "135441121756",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "367459894796",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "877482678543",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "457919547633",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765474393376",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "114848338811",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "746285987371",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "649291669397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "927914615679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "445925149649",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "691587454538",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "143777992227",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997981281989",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "425949483533",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "982993456429",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "718726519731",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "172599315861",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "643489267333",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "282322838685",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "781653545886",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "796415361892",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "841991688488",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "211411365397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "698218415444",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "355977139358",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "682564697312",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "383837596997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "689362171782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "749966767285",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753159165157",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "795693824762",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "669689115557",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "327491773134",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "983569279932",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "612128769512",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374327157578",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "311632789559",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523918658846",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765981581453",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "794825141891",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "873898736873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "447445629421",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "473822473819",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "181439694557",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "592538279337",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "668134915514",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "643692393748",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "696651276628",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "853859348234",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "778466723723",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "929826356991",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "272362973463",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "694235616268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "281673864127",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "479676316326",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "646979124677",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922327493433",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "883685933161",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "264259917554",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "836746273134",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "658481324922",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "481884157827",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "587787496812",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "579184949249",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "912193598348",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529679678956",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "795838284624",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "159337222655",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "173781362446",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "773687856563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "535787224917",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "351885857332",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "578827344666",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "198462689911",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "722618266242",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "952872416512",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "517778845323",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "749665846687",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "661436365453",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "259666844669",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "242851284913",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "514532995959",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "161588262349",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "742765629356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "225164373623",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "676539973863",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "826214551218",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "182345464792",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232776999554",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "337326533813",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "676676697292",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "929185622831",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "545512344383",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499444466686",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "314697386682",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "517379856925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "379557332953",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "614797267726",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "429781429464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922466849763",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "721737645236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "479227349997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "136931728327",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "259533577263",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "488538864842",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "937495658852",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "489991411364",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499148455254",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "441373944925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "899151413682",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "467893531755",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "527117488925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "928335588653",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374439448821",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "879425227932",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "867678158885",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "399749397872",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "129693547287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "689285841825",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "771619544974",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724883568652",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "516968424863",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "733737988257",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "852347289392",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "296953381169",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "377273562477",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "262296912232",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "547149832394",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "298464134954",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "216667245274",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "843998562287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "572154333646",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "124589118494",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "841824384614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232896526252",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "295448593321",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "123741461297",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "653573457168",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "196735786156",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "377338713663",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "964342468552",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "586855179568",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "484773717614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "894885246797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "677896358599",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "848845611563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "851852651677",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "398549545767",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "454244839926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "799364566435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "967114116556",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "817378986438",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "233795848681",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "824387273757",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "916198946615",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "563117729724",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "951794811935",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374598961236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922867396683",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765737843639",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "175469284871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "231853711778",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "662426712668",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "711412347158",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753466987363",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "513361312532",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "712992815957",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "971621888444",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "829235161526",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "585544633356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "582471228164",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "678666359123",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "557533689478",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "632962475133",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "484489193824",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "489562189822",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "589547936288",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "363214487524",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "244885399387",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "431751228368",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "433581868192",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "486391569221",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "185438575221",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "126574388585",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "741757479784",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529854679937",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "996116119839",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "616248973917",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "763531783491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "955456118295",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364196983365",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "195792996468",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "151859598873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "399223169721",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "938488813964",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "961981959227",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "183368827562",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "533417736566",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "786391632558",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "665661658354",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "693281533643",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "475794684356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "652154162978",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753233719644",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "668514843129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "819162623892",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941169431859",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "877385381798",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "752644929761",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "881136466196",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "275597777299",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "731681792655",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "961133895172",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "864718285734",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "963852916563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "319584985416",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "563365646341",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "811371928234",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "837131396371",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "267514771964",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "944513428457",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117298239631",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "158142752582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "252867443568",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "839269684865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "612788593128",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "145669731981",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "121557291859",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "245416776926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "799417897197",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997958836435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "892336777248",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "158929292238",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "581976444672",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "897784492783",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "492373714791",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "512659818733",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "881112998642",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "619454958782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "431149748713",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "624221476921",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "125866399464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "339882449689",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "186198784585",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "943193294691",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "955668961269",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232787996724",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "215671314196",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "286173241916",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "745977673725",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "556976448182",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "599961512792",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "766294538337",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "934912591213",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "295118729589",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529455466433",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "196119929397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "379571934299",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "251789649997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "564544131355",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "244371196654",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "384598329253",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "887753195844",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364947325679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "655517954651",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "673948786567",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "857231548835",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "816115936673",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "644234165531",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "182782912224",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "234316622259",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "421369185549",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "434632855397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "921889371893",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "415956914763",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "598916996413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "773671349113",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "952465217972",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117657531962",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "729825168745",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "691315125346",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "768461952319",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "664847713559",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "953267689786",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "886464195129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "824488329416",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "837873762491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "532833541879",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "669183782449",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941976537588",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739394546916",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "267954879268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "637551427887",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "217756494954",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "524444658383",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117783274348",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "138218735276",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "814611949491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "711641973413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499156317423",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "515856611931",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "454164859837",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "345271433112",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "462294118988",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "511785788222",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "497294727353",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "866519986723",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "334513529294",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "549946382131",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "284445431422",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "396521188476",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "421435255895",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "133373659361",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "322683334381",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "228358422847",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "291762694874",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "143182978129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "511923256573",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "327158398268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "879764613759",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "564395222747",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "451161679736",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "538631466654",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "221762325616",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "218391991184",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "322589379462",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "876537814263",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "152676556624",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "332522971941",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "884354318946",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "513349618943",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "116639746413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "635185846287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "993832498489",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "813981174797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438745114173",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "983493951323",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724492262421",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "622553389126",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "889965243135",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364492359246",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "154962668224",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "179564995814",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "418412875665",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "718951851413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "699446724178",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "624266421831",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "815458725125",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "455423278865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "393741199486",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "328552864359",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "211662639865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "218784516525",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "762486672996",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "142799718159",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "858146415154",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "767858144912",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "571317457151",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "635127952696",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "116427191984",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "268921994538",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523937669294",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "165429152138",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739246183345",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "591464355756",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "212985874612",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "191887635211",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "967214577653",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "119342152414",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "946444632795",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "618423867817",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "228565148417",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "729116422489",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "527874729936",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739784153482",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "387763951128",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "331369926711",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "562716493614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739667844957",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "562389434565",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "256497188281",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "859927364588",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "417668946583",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "357621613582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438435178228",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "485692541169",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "825815739116",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "342221452223",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "697747991249",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "716763689965",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "141499982867",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "818479319499",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "336813343298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "594688742928",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "472129283475",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "514354144759",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "349249721685",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "546276298359",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "353755529131",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "315534574435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523723475786",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "215826764872",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "367968398551",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "569853653352",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "389715484387",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "293847485454",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "714738141818",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "178478368922",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "581493616981",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "589439538674",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "846657726193",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "722339992679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "138154781148",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "757785319772",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "492516914298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "919181521716",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "985781138935",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "476969195485",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "313145133463",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "758963111966",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "147541537162",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "557163366873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "144373897488",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "522515164754",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724964923582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "284776712475",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "375429755114",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "181233596124",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "948585673431",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "243165586174",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "396847976144",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997724962668",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "558837194455",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "163165456396",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "378749551722",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "161238482259",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "754978243758",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "195388849133",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "229775525672",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "262437452884",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "441377892146",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "451885565366",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "981277526855",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "762495822823",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "368763327262",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "757422791351",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "636324136426",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "214193645583",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "412843856172",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "179386156569",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "756916173536",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "892697125149",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "625334487352",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941861857715",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "887417525236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "649516938598",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "717628619782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438124184139",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "547563892268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "856317483891",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "313313831273",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "371496153876",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "587541149322",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "265847332563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "449549215429",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "163497196769",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "861342291298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "268433315926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "774679513717",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "851254219729",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "583527834464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "488496781997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "556814553861",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "482829231639",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "618878266619",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "147444452794",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "949235426629",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "357299947518",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "175528632226",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "645527857972",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "186872457894",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "552738847828",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "626748382482",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "921894985642",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "943878645871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "859289776479",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "614583493135",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "933775286797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "332234613346",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "325196781219",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "142526557681",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "356722692178",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "449318681694",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "687284547244",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "947262995132",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "893974619684",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "797238311233",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|unk|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

granite-4.0-h-tiny/base/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.999984,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 1.0,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

granite-4.0-h-tiny/base/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34da75c0edfb72be6a5fcae54d3eea5887f056455b8b18f5b8b3ffd5370be9d3
+size 12784

granite-4.0-h-tiny/base/codecarbon.csv

@@ -0,0 +1,2 @@
+timestamp,project_name,run_id,experiment_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,water_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,cpu_utilization_percent,gpu_utilization_percent,ram_utilization_percent,ram_used_gb,on_cloud,pue,wue
+2026-03-18T16:42:28,codedp-granite-4.0-h-tiny-cpt-base,0d617dc4-165b-4ebe-9a8a-529efa68766a,5b0fa12a-3dd7-45bb-9766-cc326314d9f1,3257.8297634990886,0.046521798865538776,1.427999688221026e-05,84.78240267963766,1356.0935826800528,38.0,0.0740143041223195,1.2279951440620138,0.033174056839308075,1.3351835050236422,0.0,Switzerland,CHE,aargau,,,Linux-6.8.0-100-generic-x86_64-with-glibc2.39,3.11.0,3.2.3,8,AMD EPYC 9565 72-Core Processor,5,5 x NVIDIA H200,8.212,47.4843,128,machine,12.756927339901509,63.19950738916256,10.355603448275856,208.74299130651164,N,1.0,0.0

granite-4.0-h-tiny/base/epochs/epoch_001/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: ibm-granite/granite-4.0-h-tiny
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:ibm-granite/granite-4.0-h-tiny
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

granite-4.0-h-tiny/base/epochs/epoch_001/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

granite-4.0-h-tiny/base/epochs/epoch_001/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf3c3c3250a97032490c58c74dad731c73aa83114a196a9a86544b376e84579e
+size 1861530512

granite-4.0-h-tiny/base/epochs/epoch_001/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 0.996728,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 0.996,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

granite-4.0-h-tiny/base/epochs/epoch_001/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5a9ad870becca301d5c8edf6d81b8cf4473aee0fd423b8a65e0b5a3714d6889
+size 12784

granite-4.0-h-tiny/base/epochs/epoch_002/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: ibm-granite/granite-4.0-h-tiny
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:ibm-granite/granite-4.0-h-tiny
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

granite-4.0-h-tiny/base/epochs/epoch_002/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

granite-4.0-h-tiny/base/epochs/epoch_002/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70251fc485b545d89488f5ee78643e9b93f01dad47b16545f7544fc4a6148427
+size 1861530512

granite-4.0-h-tiny/base/epochs/epoch_002/audit_results.json

@@ -0,0 +1,137 @@
+{
+  "delta": 1e-05,
+  "num_canaries": 500,
+  "num_members": 250,
+  "paper_guess_fraction": 0.2,
+  "paper_guess_steps": 20,
+  "loss": {
+    "auc": 1.0,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  },
+  "embedding": {
+    "auc": 1.0,
+    "empirical_epsilon": {
+      "0.05": 3.4791953936219215,
+      "0.01": 3.023197554051876
+    },
+    "empirical_epsilon_details": {
+      "0.05": {
+        "epsilon": 3.4791953936219215,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      },
+      "0.01": {
+        "epsilon": 3.023197554051876,
+        "num_guesses": 100,
+        "correct_guesses": 100,
+        "candidate_num_guesses": [
+          5,
+          10,
+          15,
+          20,
+          25,
+          30,
+          35,
+          40,
+          45,
+          50,
+          55,
+          60,
+          65,
+          70,
+          75,
+          80,
+          85,
+          90,
+          95,
+          100
+        ],
+        "direction": "lower"
+      }
+    }
+  }
+}

granite-4.0-h-tiny/base/epochs/epoch_002/audit_scores.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa46d4a1ea859ebce2d9d9bc90ddd974f72104077f53bd0d9802a30f63b26722
+size 12784

granite-4.0-h-tiny/base/metrics.jsonl

@@ -0,0 +1,49 @@
+{"timestamp": 1773845500.3890991, "event": "train_step", "step": 10, "epoch": 1, "metrics": {"train/step_loss": 1.5491975618131233, "train/step_real_loss": 1.1542506143450737, "train/lr": 5.2631578947368424e-05, "train/step_canary_loss": 14.1875, "perf/step_duration_sec": 7.20499980985187, "perf/samples_per_sec": 9.160305585262314, "perf/tokens_per_sec": 6590.423491069632, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 47484.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773845576.5618713, "event": "train_step", "step": 20, "epoch": 1, "metrics": {"train/step_loss": 1.749317905796108, "train/step_real_loss": 1.195575013756752, "train/lr": 9.999795083071328e-05, "train/step_canary_loss": 13.5625, "perf/step_duration_sec": 7.156173000810668, "perf/samples_per_sec": 9.362546153147791, "perf/tokens_per_sec": 6812.859330605484, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 48754.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 17.044722080230713, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773845647.392718, "event": "train_step", "step": 30, "epoch": 1, "metrics": {"train/step_loss": 1.0699390769004822, "train/step_real_loss": 1.0699390769004822, "train/lr": 9.975225368753412e-05, "perf/step_duration_sec": 6.728532065171748, "perf/samples_per_sec": 9.511732927792234, "perf/tokens_per_sec": 7888.793496987684, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 53080.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773845720.600243, "event": "train_step", "step": 40, "epoch": 1, "metrics": {"train/step_loss": 1.095807023346424, "train/step_real_loss": 1.095807023346424, "train/lr": 9.909902907826884e-05, "perf/step_duration_sec": 6.983004813082516, "perf/samples_per_sec": 9.165108962848961, "perf/tokens_per_sec": 7604.176342613747, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 53100.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773845792.8609428, "event": "train_step", "step": 50, "epoch": 1, "metrics": {"train/step_loss": 1.0978227560336773, "train/step_real_loss": 1.0427106022834778, "train/lr": 9.804362765439688e-05, "train/step_canary_loss": 4.625, "perf/step_duration_sec": 7.137449341127649, "perf/samples_per_sec": 9.106894759372206, "perf/tokens_per_sec": 6859.453238832368, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 48959.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773845810.9490905, "event": "eval_step", "step": 50, "epoch": 1, "metrics": {"eval/loss": 0.9920933229195606, "eval/duration_sec": 18.071402312023565}}
+{"timestamp": 1773845883.288467, "event": "train_step", "step": 60, "epoch": 1, "metrics": {"train/step_loss": 1.3474635359118967, "train/step_real_loss": 1.0449612587690353, "train/lr": 9.659469435229992e-05, "train/step_canary_loss": 6.1875, "perf/step_duration_sec": 7.404544448945671, "perf/samples_per_sec": 9.183549436276593, "perf/tokens_per_sec": 7116.845656521583, "perf/logical_batch_size": 68.0, "perf/logical_token_count": 52697.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773845955.9986777, "event": "train_step", "step": 70, "epoch": 1, "metrics": {"train/step_loss": 1.2259749336676165, "train/step_real_loss": 0.9742475748062134, "train/lr": 9.476409758141405e-05, "train/step_canary_loss": 9.28125, "perf/step_duration_sec": 7.936319092987105, "perf/samples_per_sec": 8.316197877970987, "perf/tokens_per_sec": 6486.50834181922, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 51479.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 17.044722080230713, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846033.6489208, "event": "train_step", "step": 80, "epoch": 1, "metrics": {"train/step_loss": 1.075090453028679, "train/step_real_loss": 1.075090453028679, "train/lr": 9.256683200847638e-05, "perf/step_duration_sec": 8.235574607970193, "perf/samples_per_sec": 7.771163889167165, "perf/tokens_per_sec": 6101.2136240466025, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 50247.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846114.691344, "event": "train_step", "step": 90, "epoch": 1, "metrics": {"train/step_loss": 1.0333750322461128, "train/step_real_loss": 1.0333750322461128, "train/lr": 9.002089573417356e-05, "perf/step_duration_sec": 8.228110728086904, "perf/samples_per_sec": 7.778213263651652, "perf/tokens_per_sec": 6766.3163318834695, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 55674.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846196.510411, "event": "train_step", "step": 100, "epoch": 1, "metrics": {"train/step_loss": 1.3118896195382783, "train/step_real_loss": 1.0975150763988495, "train/lr": 8.714714286825512e-05, "train/step_canary_loss": 8.171875, "perf/step_duration_sec": 8.322382139973342, "perf/samples_per_sec": 7.930421709788419, "perf/tokens_per_sec": 5789.32800605024, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 48181.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846218.1695988, "event": "eval_step", "step": 100, "epoch": 1, "metrics": {"eval/loss": 0.9644145566921729, "eval/duration_sec": 21.646080798935145}}
+{"timestamp": 1773846300.9108367, "event": "train_step", "step": 110, "epoch": 1, "metrics": {"train/step_loss": 1.1254502712790646, "train/step_real_loss": 0.9916823133826256, "train/lr": 8.39691127106884e-05, "train/step_canary_loss": 3.9791666666666665, "perf/step_duration_sec": 8.406522131990641, "perf/samples_per_sec": 7.970002213523535, "perf/tokens_per_sec": 6148.7972301049485, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 51690.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846382.1696744, "event": "train_step", "step": 120, "epoch": 1, "metrics": {"train/step_loss": 0.9898425564169884, "train/step_real_loss": 0.9898425564169884, "train/lr": 8.051283693805624e-05, "perf/step_duration_sec": 7.923252637963742, "perf/samples_per_sec": 8.077490763496323, "perf/tokens_per_sec": 6447.73079116776, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 51087.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846465.942783, "event": "train_step", "step": 130, "epoch": 1, "metrics": {"train/step_loss": 1.0856322862885215, "train/step_real_loss": 1.0018824934959412, "train/lr": 7.680662637455889e-05, "train/step_canary_loss": 3.765625, "perf/step_duration_sec": 8.345965349115431, "perf/samples_per_sec": 7.908012703047609, "perf/tokens_per_sec": 6140.811500665045, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 51251.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846547.3686187, "event": "train_step", "step": 140, "epoch": 1, "metrics": {"train/step_loss": 1.11777756430886, "train/step_real_loss": 0.9652081280946732, "train/lr": 7.288083909420865e-05, "train/step_canary_loss": 6.0, "perf/step_duration_sec": 7.923310180194676, "perf/samples_per_sec": 8.329851854717921, "perf/tokens_per_sec": 6493.371940505792, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 51449.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846630.8803382, "event": "train_step", "step": 150, "epoch": 1, "metrics": {"train/step_loss": 1.0294065218705397, "train/step_real_loss": 1.005940206348896, "train/lr": 6.876763175372305e-05, "train/step_canary_loss": 2.53125, "perf/step_duration_sec": 8.249663470080122, "perf/samples_per_sec": 7.879109279516914, "perf/tokens_per_sec": 6361.229181084441, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 52478.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846652.1329367, "event": "eval_step", "step": 150, "epoch": 1, "metrics": {"eval/loss": 0.9538889107379046, "eval/duration_sec": 21.243608912918717}}
+{"timestamp": 1773846734.8427546, "event": "train_step", "step": 160, "epoch": 1, "metrics": {"train/step_loss": 1.02191624045372, "train/step_real_loss": 1.02191624045372, "train/lr": 6.450069619298299e-05, "perf/step_duration_sec": 8.291792104020715, "perf/samples_per_sec": 7.718476198766031, "perf/tokens_per_sec": 6133.052947063246, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 50854.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846816.398206, "event": "train_step", "step": 170, "epoch": 1, "metrics": {"train/step_loss": 0.9282315596938133, "train/step_real_loss": 0.9282315596938133, "train/lr": 6.011498346059712e-05, "perf/step_duration_sec": 7.786464124917984, "perf/samples_per_sec": 8.219391879709473, "perf/tokens_per_sec": 6340.875551201497, "perf/logical_batch_size": 64.0, "perf/logical_token_count": 49373.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846898.7244656, "event": "train_step", "step": 180, "epoch": 1, "metrics": {"train/step_loss": 1.1359728827620998, "train/step_real_loss": 1.0057005435228348, "train/lr": 5.564641752511637e-05, "train/step_canary_loss": 5.3046875, "perf/step_duration_sec": 8.632995563792065, "perf/samples_per_sec": 7.645086750283158, "perf/tokens_per_sec": 5906.756191775592, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 50993.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 64.87937498092651}}
+{"timestamp": 1773846945.409436, "event": "train_epoch", "step": 183, "epoch": 1, "metrics": {"train/epoch_loss": 1.1745113145391142, "train/epoch_real_loss": 1.045788711171007, "train/epoch_canary_loss": 7.361844605070935, "perf/epoch_duration_sec": 1529.87840606994, "perf/epoch_samples_per_sec": 31.26849807814928, "perf/epoch_tokens_per_sec": 24337.648568848304, "perf/epoch_samples": 47837.0, "perf/epoch_tokens": 37233643.0, "system/cuda_epoch_peak_memory_gb": 64.87937498092651, "eval/loss": 0.9498279011481768, "eval/duration_sec": 21.582291743019596}}
+{"timestamp": 1773846954.5938265, "event": "audit_epoch", "step": 183, "epoch": 1, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.996728, "audit/loss/empirical_epsilon/0.05": 3.4791953936219215, "audit/loss/empirical_epsilon/0.01": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0, "audit/embedding/auc": 0.996, "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215, "audit/embedding/empirical_epsilon/0.01": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0, "perf/audit_duration_sec": 7.048272565938532}}
+{"timestamp": 1773847012.740762, "event": "train_step", "step": 190, "epoch": 2, "metrics": {"train/step_loss": 1.0674721995989482, "train/step_real_loss": 1.0078131332993507, "train/lr": 5.113160101692939e-05, "train/step_canary_loss": 2.9765625, "perf/step_duration_sec": 8.316238466184586, "perf/samples_per_sec": 7.9362803590070925, "perf/tokens_per_sec": 5525.09408993421, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 45948.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 56.812254428863525}}
+{"timestamp": 1773847096.9887106, "event": "train_step", "step": 200, "epoch": 2, "metrics": {"train/step_loss": 1.1259089543269232, "train/step_real_loss": 1.0414504930377007, "train/lr": 4.660751541114641e-05, "train/step_canary_loss": 6.53125, "perf/step_duration_sec": 8.248281789012253, "perf/samples_per_sec": 7.880429119988136, "perf/tokens_per_sec": 5337.475261653503, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 44025.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 56.81271266937256}}
+{"timestamp": 1773847117.9581919, "event": "eval_step", "step": 200, "epoch": 2, "metrics": {"eval/loss": 0.9487124373773477, "eval/duration_sec": 20.96032243501395}}
+{"timestamp": 1773847200.0371358, "event": "train_step", "step": 210, "epoch": 2, "metrics": {"train/step_loss": 1.1445510784784954, "train/step_real_loss": 1.0787557885050774, "train/lr": 4.2111218107314846e-05, "train/step_canary_loss": 3.25, "perf/step_duration_sec": 8.08854101691395, "perf/samples_per_sec": 8.159691576266644, "perf/tokens_per_sec": 6864.773249451238, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 55526.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 17.044722080230713, "system/cuda_max_memory_allocated_gb": 56.81271266937256}}
+{"timestamp": 1773847279.7295897, "event": "train_step", "step": 220, "epoch": 2, "metrics": {"train/step_loss": 1.0096144080162048, "train/step_real_loss": 0.9559597969055176, "train/lr": 3.7679538887227244e-05, "train/step_canary_loss": 2.7265625, "perf/step_duration_sec": 8.052449005888775, "perf/samples_per_sec": 8.196264260939007, "perf/tokens_per_sec": 6205.4413462857765, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 49969.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84816789627075}}
+{"timestamp": 1773847361.656875, "event": "train_step", "step": 230, "epoch": 2, "metrics": {"train/step_loss": 0.9799742924444603, "train/step_real_loss": 0.9675686806440353, "train/lr": 3.334877823717737e-05, "train/step_canary_loss": 1.376953125, "perf/step_duration_sec": 8.545294485986233, "perf/samples_per_sec": 7.7235489201964915, "perf/tokens_per_sec": 6140.923532367137, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 52476.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84816789627075}}
+{"timestamp": 1773847448.8259733, "event": "train_step", "step": 240, "epoch": 2, "metrics": {"train/step_loss": 1.0699740214781328, "train/step_real_loss": 0.9959888234734535, "train/lr": 2.9154410005747583e-05, "train/step_canary_loss": 3.4375, "perf/step_duration_sec": 8.351536879083142, "perf/samples_per_sec": 7.902737059726149, "perf/tokens_per_sec": 6316.322464206273, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 52751.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84816789627075}}
+{"timestamp": 1773847527.7394445, "event": "train_step", "step": 250, "epoch": 2, "metrics": {"train/step_loss": 1.0023332100648146, "train/step_real_loss": 0.9728286564350128, "train/lr": 2.5130790832697737e-05, "train/step_canary_loss": 2.890625, "perf/step_duration_sec": 8.24364663194865, "perf/samples_per_sec": 7.884860050658815, "perf/tokens_per_sec": 5500.114454721866, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 45341.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84816789627075}}
+{"timestamp": 1773847548.8306634, "event": "eval_step", "step": 250, "epoch": 2, "metrics": {"eval/loss": 0.9464517607317342, "eval/duration_sec": 21.085849778959528}}
+{"timestamp": 1773847631.4214828, "event": "train_step", "step": 260, "epoch": 2, "metrics": {"train/step_loss": 1.0758904056115584, "train/step_real_loss": 0.9906154796481133, "train/lr": 2.1310878729063642e-05, "train/step_canary_loss": 3.8046875, "perf/step_duration_sec": 8.149904164019972, "perf/samples_per_sec": 8.098254736709105, "perf/tokens_per_sec": 6327.313666786037, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 51567.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 17.044722080230713, "system/cuda_max_memory_allocated_gb": 60.84816789627075}}
+{"timestamp": 1773847712.8631833, "event": "train_step", "step": 270, "epoch": 2, "metrics": {"train/step_loss": 1.1142344745722683, "train/step_real_loss": 0.9749820455908775, "train/lr": 1.7725963113612998e-05, "train/step_canary_loss": 5.5703125, "perf/step_duration_sec": 7.97826447407715, "perf/samples_per_sec": 8.272475826596892, "perf/tokens_per_sec": 6396.37858155898, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 51032.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84816789627075}}
+{"timestamp": 1773847793.1259449, "event": "train_step", "step": 280, "epoch": 2, "metrics": {"train/step_loss": 1.0694914211088151, "train/step_real_loss": 1.0146433636546135, "train/lr": 1.4405408516967328e-05, "train/step_canary_loss": 2.2395833333333335, "perf/step_duration_sec": 8.121065079933032, "perf/samples_per_sec": 8.25014937579499, "perf/tokens_per_sec": 6357.909891349593, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 51633.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773847869.4765995, "event": "train_step", "step": 290, "epoch": 2, "metrics": {"train/step_loss": 0.9731774203705065, "train/step_real_loss": 0.9380374550819397, "train/lr": 1.1376414052744056e-05, "train/step_canary_loss": 2.09765625, "perf/step_duration_sec": 7.720702049089596, "perf/samples_per_sec": 8.54844541083962, "perf/tokens_per_sec": 6774.642988090398, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 52305.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773847946.6124372, "event": "train_step", "step": 300, "epoch": 2, "metrics": {"train/step_loss": 1.0307691061135493, "train/step_real_loss": 1.0040206909179688, "train/lr": 8.66379062592345e-06, "train/step_canary_loss": 1.88671875, "perf/step_duration_sec": 7.759747894015163, "perf/samples_per_sec": 8.505430962635218, "perf/tokens_per_sec": 6224.171282323573, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 48298.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 17.044722080230713, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773847966.612822, "event": "eval_step", "step": 300, "epoch": 2, "metrics": {"eval/loss": 0.9457971912312817, "eval/duration_sec": 19.98325987579301}}
+{"timestamp": 1773848043.95005, "event": "train_step", "step": 310, "epoch": 2, "metrics": {"train/step_loss": 1.0016437012757828, "train/step_real_loss": 0.9807856976985931, "train/lr": 6.2897577033565965e-06, "train/step_canary_loss": 1.4466145833333333, "perf/step_duration_sec": 7.719462159788236, "perf/samples_per_sec": 8.679361154072678, "perf/tokens_per_sec": 6271.6804613921595, "perf/logical_batch_size": 67.0, "perf/logical_token_count": 48414.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773848122.1587205, "event": "train_step", "step": 320, "epoch": 2, "metrics": {"train/step_loss": 0.9575878876906175, "train/step_real_loss": 0.9505775347352028, "train/lr": 4.2737613110951925e-06, "train/step_canary_loss": 1.40625, "perf/step_duration_sec": 7.7270688829012215, "perf/samples_per_sec": 8.411986612910194, "perf/tokens_per_sec": 6291.130665027544, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 48612.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773848200.6989753, "event": "train_step", "step": 330, "epoch": 2, "metrics": {"train/step_loss": 1.0191495253489569, "train/step_real_loss": 0.9818510636687279, "train/lr": 2.6323147493514833e-06, "train/step_canary_loss": 3.40625, "perf/step_duration_sec": 7.586550263920799, "perf/samples_per_sec": 8.567794022155125, "perf/tokens_per_sec": 6317.495875290012, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 47928.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773848281.642083, "event": "train_step", "step": 340, "epoch": 2, "metrics": {"train/step_loss": 1.0784487796552253, "train/step_real_loss": 1.0184003114700317, "train/lr": 1.3788633298137288e-06, "train/step_canary_loss": 3.0, "perf/step_duration_sec": 8.599153365008533, "perf/samples_per_sec": 7.675174194306803, "perf/tokens_per_sec": 5509.263294777043, "perf/logical_batch_size": 66.0, "perf/logical_token_count": 47375.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773848363.9098017, "event": "train_step", "step": 350, "epoch": 2, "metrics": {"train/step_loss": 1.1414880389752595, "train/step_real_loss": 0.950454406440258, "train/lr": 5.236742432724262e-07, "train/step_canary_loss": 3.58671875, "perf/step_duration_sec": 8.310993053019047, "perf/samples_per_sec": 8.302256969753463, "perf/tokens_per_sec": 6413.433347852161, "perf/logical_batch_size": 69.0, "perf/logical_token_count": 53302.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773848384.8834276, "event": "eval_step", "step": 350, "epoch": 2, "metrics": {"eval/loss": 0.9456373546030614, "eval/duration_sec": 20.951530331978574}}
+{"timestamp": 1773848463.9385724, "event": "train_step", "step": 360, "epoch": 2, "metrics": {"train/step_loss": 1.0201530621601984, "train/step_real_loss": 0.9811613112688065, "train/lr": 7.375245966623756e-08, "train/step_canary_loss": 3.515625, "perf/step_duration_sec": 7.831112121930346, "perf/samples_per_sec": 8.300225943384616, "perf/tokens_per_sec": 6854.454279830759, "perf/logical_batch_size": 65.0, "perf/logical_token_count": 53678.0, "perf/gradient_accumulation_steps": 8.0, "system/cuda_memory_allocated_gb": 16.85233783721924, "system/cuda_max_memory_allocated_gb": 60.84869050979614}}
+{"timestamp": 1773848532.3133028, "event": "train_epoch", "step": 366, "epoch": 2, "metrics": {"train/epoch_loss": 1.0387111434896155, "train/epoch_real_loss": 0.9917015731131118, "train/epoch_canary_loss": 3.26868664640822, "perf/epoch_duration_sec": 1556.666436379077, "perf/epoch_samples_per_sec": 30.74068977250494, "perf/epoch_tokens_per_sec": 23919.13266056493, "perf/epoch_samples": 47853.0, "perf/epoch_tokens": 37234111.0, "system/cuda_epoch_peak_memory_gb": 60.84869050979614, "eval/loss": 0.9456853411801449, "eval/duration_sec": 21.018074851948768}}
+{"timestamp": 1773848539.6229737, "event": "audit_epoch", "step": 366, "epoch": 2, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 1.0, "audit/loss/empirical_epsilon/0.05": 3.4791953936219215, "audit/loss/empirical_epsilon/0.01": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0, "audit/embedding/auc": 1.0, "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215, "audit/embedding/empirical_epsilon/0.01": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0, "perf/audit_duration_sec": 5.547323588980362}}
+{"timestamp": 1773848547.60095, "event": "audit_final", "step": 366, "epoch": 2, "metrics": {"audit/delta": 1e-05, "audit/num_canaries": 500.0, "audit/num_members": 250.0, "audit/paper_guess_fraction": 0.2, "audit/paper_guess_steps": 20.0, "audit/loss/auc": 0.999984, "audit/loss/empirical_epsilon/0.05": 3.4791953936219215, "audit/loss/empirical_epsilon/0.01": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0, "audit/embedding/auc": 1.0, "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215, "audit/embedding/empirical_epsilon/0.01": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215, "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876, "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0, "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0}}
+{"timestamp": 1773848548.142728, "event": "energy_final", "step": 366, "epoch": null, "metrics": {"energy/codecarbon/duration": 3257.8297634990886, "energy/codecarbon/emissions": 0.046521798865538776, "energy/codecarbon/emissions_rate": 1.427999688221026e-05, "energy/codecarbon/cpu_power": 84.78240267963766, "energy/codecarbon/gpu_power": 1356.0935826800528, "energy/codecarbon/ram_power": 38.0, "energy/codecarbon/cpu_energy": 0.0740143041223195, "energy/codecarbon/gpu_energy": 1.2279951440620138, "energy/codecarbon/ram_energy": 0.033174056839308075, "energy/codecarbon/energy_consumed": 1.3351835050236422, "energy/codecarbon/water_consumed": 0.0, "energy/codecarbon/cpu_count": 8.0, "energy/codecarbon/gpu_count": 5.0, "energy/codecarbon/longitude": 8.212, "energy/codecarbon/latitude": 47.4843, "energy/codecarbon/ram_total_size": 128.0, "energy/codecarbon/cpu_utilization_percent": 12.756927339901509, "energy/codecarbon/gpu_utilization_percent": 63.19950738916256, "energy/codecarbon/ram_utilization_percent": 10.355603448275856, "energy/codecarbon/ram_used_gb": 208.74299130651164, "energy/codecarbon/pue": 1.0, "energy/codecarbon/wue": 0.0}}

granite-4.0-h-tiny/base/pretrain_lm_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23d4c2cffb51b108b4c377ffa44775b5e654a4b01b09c46ca4e90494d144fd82
+size 309818984

granite-4.0-h-tiny/base/resolved_config.yaml

@@ -0,0 +1,110 @@
+model:
+  name: ibm-granite/granite-4.0-h-tiny
+  tokenizer_name: ibm-granite/granite-4.0-h-tiny
+  max_length: 1024
+  dtype: bfloat16
+  trust_remote_code: true
+  use_fast_tokenizer: true
+  cache_dir: null
+  local_files_only: false
+  low_cpu_mem_usage: true
+  tie_word_embeddings: true
+  gradient_checkpointing: false
+  use_chat_template: false
+dataset:
+  name: codedp-ase26/codedp-cpt
+  split: train
+  mode: cpt
+  text_column: text
+  validation_ratio: 0.05
+  max_samples: -1
+lora:
+  enabled: true
+  r: 16
+  alpha: 32
+  dropout: 0.05
+  target_modules:
+  - q_proj
+  - k_proj
+  - v_proj
+  - o_proj
+  modules_to_save:
+  - lm_head
+  bias: none
+training:
+  seed: 42
+  epochs: 2
+  warmup_steps: null
+  warmup_ratio: 0.05
+  mixed_precision: false
+  mixed_precision_dtype: bfloat16
+  batch_size: 8
+  eval_batch_size: 8
+  eval_every_steps: 50
+  eval_every_epochs: 1
+  learning_rate: 0.0001
+  optimizer: adamw
+  lr_scheduler: cosine
+  adam_beta1: 0.9
+  adam_beta2: 0.999
+  adam_epsilon: 1.0e-08
+  sgd_momentum: 0.9
+  weight_decay: 0.01
+  max_grad_norm: 1.0
+  log_every: 10
+  gradient_accumulation_steps: 8
+  num_workers: 4
+  output_dir: runs/cpt/granite-4.0-h-tiny/base
+distributed:
+  strategy: dpddp
+  backend: nccl
+  devices: null
+dp:
+  module_validator: auto
+  target_delta: 1.0e-05
+  noise_multiplier: null
+  max_grad_norm: 1.0
+  grad_sample_mode: hooks
+  clipping: flat
+  secure_mode: false
+  enabled: false
+  target_epsilon: 8.0
+audit:
+  enabled: true
+  run_every_epoch: true
+  epoch_device: cuda
+  q_canary: auto
+  num_canaries: 500
+  prefix_length: 49
+  num_digits: 12
+  batch_size: 32
+  delta: 1.0e-05
+  p_values:
+  - 0.05
+  - 0.01
+  paper_guess_fraction: 0.2
+  paper_guess_steps: 20
+  enable_holdout_empirical_epsilon: false
+  holdout_seed: 42
+  tie_seed: 42
+tracking:
+  enabled: true
+  tensorboard: true
+  wandb: false
+  wandb_project: codedp-finetune-h200-audit
+  wandb_run_name: granite-4.0-h-tiny-cpt-base
+  wandb_mode: online
+  codecarbon: true
+  codecarbon_output_file: codecarbon.csv
+  codecarbon_measure_power_secs: 15
+  codecarbon_country_iso_code: null
+  codecarbon_project_name: codedp-granite-4.0-h-tiny-cpt-base
+moe:
+  output_router_logits: false
+  router_aux_loss_coef: 0.0
+  freeze_router: true
+  profile:
+    enabled: false
+    num_batches: 8
+    top_experts: 8
+    output_file: moe_expert_profile.json

granite-4.0-h-tiny/base/scalars.csv

@@ -0,0 +1,555 @@
+timestamp,event,step,epoch,key,value
+1773845500.3890991,train_step,10,1,train/step_loss,1.5491975618131233
+1773845500.3890991,train_step,10,1,train/step_real_loss,1.1542506143450737
+1773845500.3890991,train_step,10,1,train/lr,5.2631578947368424e-05
+1773845500.3890991,train_step,10,1,train/step_canary_loss,14.1875
+1773845500.3890991,train_step,10,1,perf/step_duration_sec,7.20499980985187
+1773845500.3890991,train_step,10,1,perf/samples_per_sec,9.160305585262314
+1773845500.3890991,train_step,10,1,perf/tokens_per_sec,6590.423491069632
+1773845500.3890991,train_step,10,1,perf/logical_batch_size,66.0
+1773845500.3890991,train_step,10,1,perf/logical_token_count,47484.0
+1773845500.3890991,train_step,10,1,perf/gradient_accumulation_steps,8.0
+1773845500.3890991,train_step,10,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773845500.3890991,train_step,10,1,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773845576.5618713,train_step,20,1,train/step_loss,1.749317905796108
+1773845576.5618713,train_step,20,1,train/step_real_loss,1.195575013756752
+1773845576.5618713,train_step,20,1,train/lr,9.999795083071328e-05
+1773845576.5618713,train_step,20,1,train/step_canary_loss,13.5625
+1773845576.5618713,train_step,20,1,perf/step_duration_sec,7.156173000810668
+1773845576.5618713,train_step,20,1,perf/samples_per_sec,9.362546153147791
+1773845576.5618713,train_step,20,1,perf/tokens_per_sec,6812.859330605484
+1773845576.5618713,train_step,20,1,perf/logical_batch_size,67.0
+1773845576.5618713,train_step,20,1,perf/logical_token_count,48754.0
+1773845576.5618713,train_step,20,1,perf/gradient_accumulation_steps,8.0
+1773845576.5618713,train_step,20,1,system/cuda_memory_allocated_gb,17.044722080230713
+1773845576.5618713,train_step,20,1,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773845647.392718,train_step,30,1,train/step_loss,1.0699390769004822
+1773845647.392718,train_step,30,1,train/step_real_loss,1.0699390769004822
+1773845647.392718,train_step,30,1,train/lr,9.975225368753412e-05
+1773845647.392718,train_step,30,1,perf/step_duration_sec,6.728532065171748
+1773845647.392718,train_step,30,1,perf/samples_per_sec,9.511732927792234
+1773845647.392718,train_step,30,1,perf/tokens_per_sec,7888.793496987684
+1773845647.392718,train_step,30,1,perf/logical_batch_size,64.0
+1773845647.392718,train_step,30,1,perf/logical_token_count,53080.0
+1773845647.392718,train_step,30,1,perf/gradient_accumulation_steps,8.0
+1773845647.392718,train_step,30,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773845647.392718,train_step,30,1,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773845720.600243,train_step,40,1,train/step_loss,1.095807023346424
+1773845720.600243,train_step,40,1,train/step_real_loss,1.095807023346424
+1773845720.600243,train_step,40,1,train/lr,9.909902907826884e-05
+1773845720.600243,train_step,40,1,perf/step_duration_sec,6.983004813082516
+1773845720.600243,train_step,40,1,perf/samples_per_sec,9.165108962848961
+1773845720.600243,train_step,40,1,perf/tokens_per_sec,7604.176342613747
+1773845720.600243,train_step,40,1,perf/logical_batch_size,64.0
+1773845720.600243,train_step,40,1,perf/logical_token_count,53100.0
+1773845720.600243,train_step,40,1,perf/gradient_accumulation_steps,8.0
+1773845720.600243,train_step,40,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773845720.600243,train_step,40,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773845792.8609428,train_step,50,1,train/step_loss,1.0978227560336773
+1773845792.8609428,train_step,50,1,train/step_real_loss,1.0427106022834778
+1773845792.8609428,train_step,50,1,train/lr,9.804362765439688e-05
+1773845792.8609428,train_step,50,1,train/step_canary_loss,4.625
+1773845792.8609428,train_step,50,1,perf/step_duration_sec,7.137449341127649
+1773845792.8609428,train_step,50,1,perf/samples_per_sec,9.106894759372206
+1773845792.8609428,train_step,50,1,perf/tokens_per_sec,6859.453238832368
+1773845792.8609428,train_step,50,1,perf/logical_batch_size,65.0
+1773845792.8609428,train_step,50,1,perf/logical_token_count,48959.0
+1773845792.8609428,train_step,50,1,perf/gradient_accumulation_steps,8.0
+1773845792.8609428,train_step,50,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773845792.8609428,train_step,50,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773845810.9490905,eval_step,50,1,eval/loss,0.9920933229195606
+1773845810.9490905,eval_step,50,1,eval/duration_sec,18.071402312023565
+1773845883.288467,train_step,60,1,train/step_loss,1.3474635359118967
+1773845883.288467,train_step,60,1,train/step_real_loss,1.0449612587690353
+1773845883.288467,train_step,60,1,train/lr,9.659469435229992e-05
+1773845883.288467,train_step,60,1,train/step_canary_loss,6.1875
+1773845883.288467,train_step,60,1,perf/step_duration_sec,7.404544448945671
+1773845883.288467,train_step,60,1,perf/samples_per_sec,9.183549436276593
+1773845883.288467,train_step,60,1,perf/tokens_per_sec,7116.845656521583
+1773845883.288467,train_step,60,1,perf/logical_batch_size,68.0
+1773845883.288467,train_step,60,1,perf/logical_token_count,52697.0
+1773845883.288467,train_step,60,1,perf/gradient_accumulation_steps,8.0
+1773845883.288467,train_step,60,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773845883.288467,train_step,60,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773845955.9986777,train_step,70,1,train/step_loss,1.2259749336676165
+1773845955.9986777,train_step,70,1,train/step_real_loss,0.9742475748062134
+1773845955.9986777,train_step,70,1,train/lr,9.476409758141405e-05
+1773845955.9986777,train_step,70,1,train/step_canary_loss,9.28125
+1773845955.9986777,train_step,70,1,perf/step_duration_sec,7.936319092987105
+1773845955.9986777,train_step,70,1,perf/samples_per_sec,8.316197877970987
+1773845955.9986777,train_step,70,1,perf/tokens_per_sec,6486.50834181922
+1773845955.9986777,train_step,70,1,perf/logical_batch_size,66.0
+1773845955.9986777,train_step,70,1,perf/logical_token_count,51479.0
+1773845955.9986777,train_step,70,1,perf/gradient_accumulation_steps,8.0
+1773845955.9986777,train_step,70,1,system/cuda_memory_allocated_gb,17.044722080230713
+1773845955.9986777,train_step,70,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846033.6489208,train_step,80,1,train/step_loss,1.075090453028679
+1773846033.6489208,train_step,80,1,train/step_real_loss,1.075090453028679
+1773846033.6489208,train_step,80,1,train/lr,9.256683200847638e-05
+1773846033.6489208,train_step,80,1,perf/step_duration_sec,8.235574607970193
+1773846033.6489208,train_step,80,1,perf/samples_per_sec,7.771163889167165
+1773846033.6489208,train_step,80,1,perf/tokens_per_sec,6101.2136240466025
+1773846033.6489208,train_step,80,1,perf/logical_batch_size,64.0
+1773846033.6489208,train_step,80,1,perf/logical_token_count,50247.0
+1773846033.6489208,train_step,80,1,perf/gradient_accumulation_steps,8.0
+1773846033.6489208,train_step,80,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846033.6489208,train_step,80,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846114.691344,train_step,90,1,train/step_loss,1.0333750322461128
+1773846114.691344,train_step,90,1,train/step_real_loss,1.0333750322461128
+1773846114.691344,train_step,90,1,train/lr,9.002089573417356e-05
+1773846114.691344,train_step,90,1,perf/step_duration_sec,8.228110728086904
+1773846114.691344,train_step,90,1,perf/samples_per_sec,7.778213263651652
+1773846114.691344,train_step,90,1,perf/tokens_per_sec,6766.3163318834695
+1773846114.691344,train_step,90,1,perf/logical_batch_size,64.0
+1773846114.691344,train_step,90,1,perf/logical_token_count,55674.0
+1773846114.691344,train_step,90,1,perf/gradient_accumulation_steps,8.0
+1773846114.691344,train_step,90,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846114.691344,train_step,90,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846196.510411,train_step,100,1,train/step_loss,1.3118896195382783
+1773846196.510411,train_step,100,1,train/step_real_loss,1.0975150763988495
+1773846196.510411,train_step,100,1,train/lr,8.714714286825512e-05
+1773846196.510411,train_step,100,1,train/step_canary_loss,8.171875
+1773846196.510411,train_step,100,1,perf/step_duration_sec,8.322382139973342
+1773846196.510411,train_step,100,1,perf/samples_per_sec,7.930421709788419
+1773846196.510411,train_step,100,1,perf/tokens_per_sec,5789.32800605024
+1773846196.510411,train_step,100,1,perf/logical_batch_size,66.0
+1773846196.510411,train_step,100,1,perf/logical_token_count,48181.0
+1773846196.510411,train_step,100,1,perf/gradient_accumulation_steps,8.0
+1773846196.510411,train_step,100,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846196.510411,train_step,100,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846218.1695988,eval_step,100,1,eval/loss,0.9644145566921729
+1773846218.1695988,eval_step,100,1,eval/duration_sec,21.646080798935145
+1773846300.9108367,train_step,110,1,train/step_loss,1.1254502712790646
+1773846300.9108367,train_step,110,1,train/step_real_loss,0.9916823133826256
+1773846300.9108367,train_step,110,1,train/lr,8.39691127106884e-05
+1773846300.9108367,train_step,110,1,train/step_canary_loss,3.9791666666666665
+1773846300.9108367,train_step,110,1,perf/step_duration_sec,8.406522131990641
+1773846300.9108367,train_step,110,1,perf/samples_per_sec,7.970002213523535
+1773846300.9108367,train_step,110,1,perf/tokens_per_sec,6148.7972301049485
+1773846300.9108367,train_step,110,1,perf/logical_batch_size,67.0
+1773846300.9108367,train_step,110,1,perf/logical_token_count,51690.0
+1773846300.9108367,train_step,110,1,perf/gradient_accumulation_steps,8.0
+1773846300.9108367,train_step,110,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846300.9108367,train_step,110,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846382.1696744,train_step,120,1,train/step_loss,0.9898425564169884
+1773846382.1696744,train_step,120,1,train/step_real_loss,0.9898425564169884
+1773846382.1696744,train_step,120,1,train/lr,8.051283693805624e-05
+1773846382.1696744,train_step,120,1,perf/step_duration_sec,7.923252637963742
+1773846382.1696744,train_step,120,1,perf/samples_per_sec,8.077490763496323
+1773846382.1696744,train_step,120,1,perf/tokens_per_sec,6447.73079116776
+1773846382.1696744,train_step,120,1,perf/logical_batch_size,64.0
+1773846382.1696744,train_step,120,1,perf/logical_token_count,51087.0
+1773846382.1696744,train_step,120,1,perf/gradient_accumulation_steps,8.0
+1773846382.1696744,train_step,120,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846382.1696744,train_step,120,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846465.942783,train_step,130,1,train/step_loss,1.0856322862885215
+1773846465.942783,train_step,130,1,train/step_real_loss,1.0018824934959412
+1773846465.942783,train_step,130,1,train/lr,7.680662637455889e-05
+1773846465.942783,train_step,130,1,train/step_canary_loss,3.765625
+1773846465.942783,train_step,130,1,perf/step_duration_sec,8.345965349115431
+1773846465.942783,train_step,130,1,perf/samples_per_sec,7.908012703047609
+1773846465.942783,train_step,130,1,perf/tokens_per_sec,6140.811500665045
+1773846465.942783,train_step,130,1,perf/logical_batch_size,66.0
+1773846465.942783,train_step,130,1,perf/logical_token_count,51251.0
+1773846465.942783,train_step,130,1,perf/gradient_accumulation_steps,8.0
+1773846465.942783,train_step,130,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846465.942783,train_step,130,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846547.3686187,train_step,140,1,train/step_loss,1.11777756430886
+1773846547.3686187,train_step,140,1,train/step_real_loss,0.9652081280946732
+1773846547.3686187,train_step,140,1,train/lr,7.288083909420865e-05
+1773846547.3686187,train_step,140,1,train/step_canary_loss,6.0
+1773846547.3686187,train_step,140,1,perf/step_duration_sec,7.923310180194676
+1773846547.3686187,train_step,140,1,perf/samples_per_sec,8.329851854717921
+1773846547.3686187,train_step,140,1,perf/tokens_per_sec,6493.371940505792
+1773846547.3686187,train_step,140,1,perf/logical_batch_size,66.0
+1773846547.3686187,train_step,140,1,perf/logical_token_count,51449.0
+1773846547.3686187,train_step,140,1,perf/gradient_accumulation_steps,8.0
+1773846547.3686187,train_step,140,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846547.3686187,train_step,140,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846630.8803382,train_step,150,1,train/step_loss,1.0294065218705397
+1773846630.8803382,train_step,150,1,train/step_real_loss,1.005940206348896
+1773846630.8803382,train_step,150,1,train/lr,6.876763175372305e-05
+1773846630.8803382,train_step,150,1,train/step_canary_loss,2.53125
+1773846630.8803382,train_step,150,1,perf/step_duration_sec,8.249663470080122
+1773846630.8803382,train_step,150,1,perf/samples_per_sec,7.879109279516914
+1773846630.8803382,train_step,150,1,perf/tokens_per_sec,6361.229181084441
+1773846630.8803382,train_step,150,1,perf/logical_batch_size,65.0
+1773846630.8803382,train_step,150,1,perf/logical_token_count,52478.0
+1773846630.8803382,train_step,150,1,perf/gradient_accumulation_steps,8.0
+1773846630.8803382,train_step,150,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846630.8803382,train_step,150,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846652.1329367,eval_step,150,1,eval/loss,0.9538889107379046
+1773846652.1329367,eval_step,150,1,eval/duration_sec,21.243608912918717
+1773846734.8427546,train_step,160,1,train/step_loss,1.02191624045372
+1773846734.8427546,train_step,160,1,train/step_real_loss,1.02191624045372
+1773846734.8427546,train_step,160,1,train/lr,6.450069619298299e-05
+1773846734.8427546,train_step,160,1,perf/step_duration_sec,8.291792104020715
+1773846734.8427546,train_step,160,1,perf/samples_per_sec,7.718476198766031
+1773846734.8427546,train_step,160,1,perf/tokens_per_sec,6133.052947063246
+1773846734.8427546,train_step,160,1,perf/logical_batch_size,64.0
+1773846734.8427546,train_step,160,1,perf/logical_token_count,50854.0
+1773846734.8427546,train_step,160,1,perf/gradient_accumulation_steps,8.0
+1773846734.8427546,train_step,160,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846734.8427546,train_step,160,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846816.398206,train_step,170,1,train/step_loss,0.9282315596938133
+1773846816.398206,train_step,170,1,train/step_real_loss,0.9282315596938133
+1773846816.398206,train_step,170,1,train/lr,6.011498346059712e-05
+1773846816.398206,train_step,170,1,perf/step_duration_sec,7.786464124917984
+1773846816.398206,train_step,170,1,perf/samples_per_sec,8.219391879709473
+1773846816.398206,train_step,170,1,perf/tokens_per_sec,6340.875551201497
+1773846816.398206,train_step,170,1,perf/logical_batch_size,64.0
+1773846816.398206,train_step,170,1,perf/logical_token_count,49373.0
+1773846816.398206,train_step,170,1,perf/gradient_accumulation_steps,8.0
+1773846816.398206,train_step,170,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846816.398206,train_step,170,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846898.7244656,train_step,180,1,train/step_loss,1.1359728827620998
+1773846898.7244656,train_step,180,1,train/step_real_loss,1.0057005435228348
+1773846898.7244656,train_step,180,1,train/lr,5.564641752511637e-05
+1773846898.7244656,train_step,180,1,train/step_canary_loss,5.3046875
+1773846898.7244656,train_step,180,1,perf/step_duration_sec,8.632995563792065
+1773846898.7244656,train_step,180,1,perf/samples_per_sec,7.645086750283158
+1773846898.7244656,train_step,180,1,perf/tokens_per_sec,5906.756191775592
+1773846898.7244656,train_step,180,1,perf/logical_batch_size,66.0
+1773846898.7244656,train_step,180,1,perf/logical_token_count,50993.0
+1773846898.7244656,train_step,180,1,perf/gradient_accumulation_steps,8.0
+1773846898.7244656,train_step,180,1,system/cuda_memory_allocated_gb,16.85233783721924
+1773846898.7244656,train_step,180,1,system/cuda_max_memory_allocated_gb,64.87937498092651
+1773846945.409436,train_epoch,183,1,train/epoch_loss,1.1745113145391142
+1773846945.409436,train_epoch,183,1,train/epoch_real_loss,1.045788711171007
+1773846945.409436,train_epoch,183,1,train/epoch_canary_loss,7.361844605070935
+1773846945.409436,train_epoch,183,1,perf/epoch_duration_sec,1529.87840606994
+1773846945.409436,train_epoch,183,1,perf/epoch_samples_per_sec,31.26849807814928
+1773846945.409436,train_epoch,183,1,perf/epoch_tokens_per_sec,24337.648568848304
+1773846945.409436,train_epoch,183,1,perf/epoch_samples,47837.0
+1773846945.409436,train_epoch,183,1,perf/epoch_tokens,37233643.0
+1773846945.409436,train_epoch,183,1,system/cuda_epoch_peak_memory_gb,64.87937498092651
+1773846945.409436,train_epoch,183,1,eval/loss,0.9498279011481768
+1773846945.409436,train_epoch,183,1,eval/duration_sec,21.582291743019596
+1773846954.5938265,audit_epoch,183,1,audit/delta,1e-05
+1773846954.5938265,audit_epoch,183,1,audit/num_canaries,500.0
+1773846954.5938265,audit_epoch,183,1,audit/num_members,250.0
+1773846954.5938265,audit_epoch,183,1,audit/paper_guess_fraction,0.2
+1773846954.5938265,audit_epoch,183,1,audit/paper_guess_steps,20.0
+1773846954.5938265,audit_epoch,183,1,audit/loss/auc,0.996728
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon/0.05,3.4791953936219215
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon/0.01,3.023197554051876
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon_details/0.05/num_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon_details/0.01/num_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/loss/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/embedding/auc,0.996
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon/0.05,3.4791953936219215
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon/0.01,3.023197554051876
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon_details/0.05/num_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon_details/0.01/num_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773846954.5938265,audit_epoch,183,1,perf/audit_duration_sec,7.048272565938532
+1773847012.740762,train_step,190,2,train/step_loss,1.0674721995989482
+1773847012.740762,train_step,190,2,train/step_real_loss,1.0078131332993507
+1773847012.740762,train_step,190,2,train/lr,5.113160101692939e-05
+1773847012.740762,train_step,190,2,train/step_canary_loss,2.9765625
+1773847012.740762,train_step,190,2,perf/step_duration_sec,8.316238466184586
+1773847012.740762,train_step,190,2,perf/samples_per_sec,7.9362803590070925
+1773847012.740762,train_step,190,2,perf/tokens_per_sec,5525.09408993421
+1773847012.740762,train_step,190,2,perf/logical_batch_size,66.0
+1773847012.740762,train_step,190,2,perf/logical_token_count,45948.0
+1773847012.740762,train_step,190,2,perf/gradient_accumulation_steps,8.0
+1773847012.740762,train_step,190,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847012.740762,train_step,190,2,system/cuda_max_memory_allocated_gb,56.812254428863525
+1773847096.9887106,train_step,200,2,train/step_loss,1.1259089543269232
+1773847096.9887106,train_step,200,2,train/step_real_loss,1.0414504930377007
+1773847096.9887106,train_step,200,2,train/lr,4.660751541114641e-05
+1773847096.9887106,train_step,200,2,train/step_canary_loss,6.53125
+1773847096.9887106,train_step,200,2,perf/step_duration_sec,8.248281789012253
+1773847096.9887106,train_step,200,2,perf/samples_per_sec,7.880429119988136
+1773847096.9887106,train_step,200,2,perf/tokens_per_sec,5337.475261653503
+1773847096.9887106,train_step,200,2,perf/logical_batch_size,65.0
+1773847096.9887106,train_step,200,2,perf/logical_token_count,44025.0
+1773847096.9887106,train_step,200,2,perf/gradient_accumulation_steps,8.0
+1773847096.9887106,train_step,200,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847096.9887106,train_step,200,2,system/cuda_max_memory_allocated_gb,56.81271266937256
+1773847117.9581919,eval_step,200,2,eval/loss,0.9487124373773477
+1773847117.9581919,eval_step,200,2,eval/duration_sec,20.96032243501395
+1773847200.0371358,train_step,210,2,train/step_loss,1.1445510784784954
+1773847200.0371358,train_step,210,2,train/step_real_loss,1.0787557885050774
+1773847200.0371358,train_step,210,2,train/lr,4.2111218107314846e-05
+1773847200.0371358,train_step,210,2,train/step_canary_loss,3.25
+1773847200.0371358,train_step,210,2,perf/step_duration_sec,8.08854101691395
+1773847200.0371358,train_step,210,2,perf/samples_per_sec,8.159691576266644
+1773847200.0371358,train_step,210,2,perf/tokens_per_sec,6864.773249451238
+1773847200.0371358,train_step,210,2,perf/logical_batch_size,66.0
+1773847200.0371358,train_step,210,2,perf/logical_token_count,55526.0
+1773847200.0371358,train_step,210,2,perf/gradient_accumulation_steps,8.0
+1773847200.0371358,train_step,210,2,system/cuda_memory_allocated_gb,17.044722080230713
+1773847200.0371358,train_step,210,2,system/cuda_max_memory_allocated_gb,56.81271266937256
+1773847279.7295897,train_step,220,2,train/step_loss,1.0096144080162048
+1773847279.7295897,train_step,220,2,train/step_real_loss,0.9559597969055176
+1773847279.7295897,train_step,220,2,train/lr,3.7679538887227244e-05
+1773847279.7295897,train_step,220,2,train/step_canary_loss,2.7265625
+1773847279.7295897,train_step,220,2,perf/step_duration_sec,8.052449005888775
+1773847279.7295897,train_step,220,2,perf/samples_per_sec,8.196264260939007
+1773847279.7295897,train_step,220,2,perf/tokens_per_sec,6205.4413462857765
+1773847279.7295897,train_step,220,2,perf/logical_batch_size,66.0
+1773847279.7295897,train_step,220,2,perf/logical_token_count,49969.0
+1773847279.7295897,train_step,220,2,perf/gradient_accumulation_steps,8.0
+1773847279.7295897,train_step,220,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847279.7295897,train_step,220,2,system/cuda_max_memory_allocated_gb,60.84816789627075
+1773847361.656875,train_step,230,2,train/step_loss,0.9799742924444603
+1773847361.656875,train_step,230,2,train/step_real_loss,0.9675686806440353
+1773847361.656875,train_step,230,2,train/lr,3.334877823717737e-05
+1773847361.656875,train_step,230,2,train/step_canary_loss,1.376953125
+1773847361.656875,train_step,230,2,perf/step_duration_sec,8.545294485986233
+1773847361.656875,train_step,230,2,perf/samples_per_sec,7.7235489201964915
+1773847361.656875,train_step,230,2,perf/tokens_per_sec,6140.923532367137
+1773847361.656875,train_step,230,2,perf/logical_batch_size,66.0
+1773847361.656875,train_step,230,2,perf/logical_token_count,52476.0
+1773847361.656875,train_step,230,2,perf/gradient_accumulation_steps,8.0
+1773847361.656875,train_step,230,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847361.656875,train_step,230,2,system/cuda_max_memory_allocated_gb,60.84816789627075
+1773847448.8259733,train_step,240,2,train/step_loss,1.0699740214781328
+1773847448.8259733,train_step,240,2,train/step_real_loss,0.9959888234734535
+1773847448.8259733,train_step,240,2,train/lr,2.9154410005747583e-05
+1773847448.8259733,train_step,240,2,train/step_canary_loss,3.4375
+1773847448.8259733,train_step,240,2,perf/step_duration_sec,8.351536879083142
+1773847448.8259733,train_step,240,2,perf/samples_per_sec,7.902737059726149
+1773847448.8259733,train_step,240,2,perf/tokens_per_sec,6316.322464206273
+1773847448.8259733,train_step,240,2,perf/logical_batch_size,66.0
+1773847448.8259733,train_step,240,2,perf/logical_token_count,52751.0
+1773847448.8259733,train_step,240,2,perf/gradient_accumulation_steps,8.0
+1773847448.8259733,train_step,240,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847448.8259733,train_step,240,2,system/cuda_max_memory_allocated_gb,60.84816789627075
+1773847527.7394445,train_step,250,2,train/step_loss,1.0023332100648146
+1773847527.7394445,train_step,250,2,train/step_real_loss,0.9728286564350128
+1773847527.7394445,train_step,250,2,train/lr,2.5130790832697737e-05
+1773847527.7394445,train_step,250,2,train/step_canary_loss,2.890625
+1773847527.7394445,train_step,250,2,perf/step_duration_sec,8.24364663194865
+1773847527.7394445,train_step,250,2,perf/samples_per_sec,7.884860050658815
+1773847527.7394445,train_step,250,2,perf/tokens_per_sec,5500.114454721866
+1773847527.7394445,train_step,250,2,perf/logical_batch_size,65.0
+1773847527.7394445,train_step,250,2,perf/logical_token_count,45341.0
+1773847527.7394445,train_step,250,2,perf/gradient_accumulation_steps,8.0
+1773847527.7394445,train_step,250,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847527.7394445,train_step,250,2,system/cuda_max_memory_allocated_gb,60.84816789627075
+1773847548.8306634,eval_step,250,2,eval/loss,0.9464517607317342
+1773847548.8306634,eval_step,250,2,eval/duration_sec,21.085849778959528
+1773847631.4214828,train_step,260,2,train/step_loss,1.0758904056115584
+1773847631.4214828,train_step,260,2,train/step_real_loss,0.9906154796481133
+1773847631.4214828,train_step,260,2,train/lr,2.1310878729063642e-05
+1773847631.4214828,train_step,260,2,train/step_canary_loss,3.8046875
+1773847631.4214828,train_step,260,2,perf/step_duration_sec,8.149904164019972
+1773847631.4214828,train_step,260,2,perf/samples_per_sec,8.098254736709105
+1773847631.4214828,train_step,260,2,perf/tokens_per_sec,6327.313666786037
+1773847631.4214828,train_step,260,2,perf/logical_batch_size,66.0
+1773847631.4214828,train_step,260,2,perf/logical_token_count,51567.0
+1773847631.4214828,train_step,260,2,perf/gradient_accumulation_steps,8.0
+1773847631.4214828,train_step,260,2,system/cuda_memory_allocated_gb,17.044722080230713
+1773847631.4214828,train_step,260,2,system/cuda_max_memory_allocated_gb,60.84816789627075
+1773847712.8631833,train_step,270,2,train/step_loss,1.1142344745722683
+1773847712.8631833,train_step,270,2,train/step_real_loss,0.9749820455908775
+1773847712.8631833,train_step,270,2,train/lr,1.7725963113612998e-05
+1773847712.8631833,train_step,270,2,train/step_canary_loss,5.5703125
+1773847712.8631833,train_step,270,2,perf/step_duration_sec,7.97826447407715
+1773847712.8631833,train_step,270,2,perf/samples_per_sec,8.272475826596892
+1773847712.8631833,train_step,270,2,perf/tokens_per_sec,6396.37858155898
+1773847712.8631833,train_step,270,2,perf/logical_batch_size,66.0
+1773847712.8631833,train_step,270,2,perf/logical_token_count,51032.0
+1773847712.8631833,train_step,270,2,perf/gradient_accumulation_steps,8.0
+1773847712.8631833,train_step,270,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847712.8631833,train_step,270,2,system/cuda_max_memory_allocated_gb,60.84816789627075
+1773847793.1259449,train_step,280,2,train/step_loss,1.0694914211088151
+1773847793.1259449,train_step,280,2,train/step_real_loss,1.0146433636546135
+1773847793.1259449,train_step,280,2,train/lr,1.4405408516967328e-05
+1773847793.1259449,train_step,280,2,train/step_canary_loss,2.2395833333333335
+1773847793.1259449,train_step,280,2,perf/step_duration_sec,8.121065079933032
+1773847793.1259449,train_step,280,2,perf/samples_per_sec,8.25014937579499
+1773847793.1259449,train_step,280,2,perf/tokens_per_sec,6357.909891349593
+1773847793.1259449,train_step,280,2,perf/logical_batch_size,67.0
+1773847793.1259449,train_step,280,2,perf/logical_token_count,51633.0
+1773847793.1259449,train_step,280,2,perf/gradient_accumulation_steps,8.0
+1773847793.1259449,train_step,280,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847793.1259449,train_step,280,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773847869.4765995,train_step,290,2,train/step_loss,0.9731774203705065
+1773847869.4765995,train_step,290,2,train/step_real_loss,0.9380374550819397
+1773847869.4765995,train_step,290,2,train/lr,1.1376414052744056e-05
+1773847869.4765995,train_step,290,2,train/step_canary_loss,2.09765625
+1773847869.4765995,train_step,290,2,perf/step_duration_sec,7.720702049089596
+1773847869.4765995,train_step,290,2,perf/samples_per_sec,8.54844541083962
+1773847869.4765995,train_step,290,2,perf/tokens_per_sec,6774.642988090398
+1773847869.4765995,train_step,290,2,perf/logical_batch_size,66.0
+1773847869.4765995,train_step,290,2,perf/logical_token_count,52305.0
+1773847869.4765995,train_step,290,2,perf/gradient_accumulation_steps,8.0
+1773847869.4765995,train_step,290,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773847869.4765995,train_step,290,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773847946.6124372,train_step,300,2,train/step_loss,1.0307691061135493
+1773847946.6124372,train_step,300,2,train/step_real_loss,1.0040206909179688
+1773847946.6124372,train_step,300,2,train/lr,8.66379062592345e-06
+1773847946.6124372,train_step,300,2,train/step_canary_loss,1.88671875
+1773847946.6124372,train_step,300,2,perf/step_duration_sec,7.759747894015163
+1773847946.6124372,train_step,300,2,perf/samples_per_sec,8.505430962635218
+1773847946.6124372,train_step,300,2,perf/tokens_per_sec,6224.171282323573
+1773847946.6124372,train_step,300,2,perf/logical_batch_size,66.0
+1773847946.6124372,train_step,300,2,perf/logical_token_count,48298.0
+1773847946.6124372,train_step,300,2,perf/gradient_accumulation_steps,8.0
+1773847946.6124372,train_step,300,2,system/cuda_memory_allocated_gb,17.044722080230713
+1773847946.6124372,train_step,300,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773847966.612822,eval_step,300,2,eval/loss,0.9457971912312817
+1773847966.612822,eval_step,300,2,eval/duration_sec,19.98325987579301
+1773848043.95005,train_step,310,2,train/step_loss,1.0016437012757828
+1773848043.95005,train_step,310,2,train/step_real_loss,0.9807856976985931
+1773848043.95005,train_step,310,2,train/lr,6.2897577033565965e-06
+1773848043.95005,train_step,310,2,train/step_canary_loss,1.4466145833333333
+1773848043.95005,train_step,310,2,perf/step_duration_sec,7.719462159788236
+1773848043.95005,train_step,310,2,perf/samples_per_sec,8.679361154072678
+1773848043.95005,train_step,310,2,perf/tokens_per_sec,6271.6804613921595
+1773848043.95005,train_step,310,2,perf/logical_batch_size,67.0
+1773848043.95005,train_step,310,2,perf/logical_token_count,48414.0
+1773848043.95005,train_step,310,2,perf/gradient_accumulation_steps,8.0
+1773848043.95005,train_step,310,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773848043.95005,train_step,310,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773848122.1587205,train_step,320,2,train/step_loss,0.9575878876906175
+1773848122.1587205,train_step,320,2,train/step_real_loss,0.9505775347352028
+1773848122.1587205,train_step,320,2,train/lr,4.2737613110951925e-06
+1773848122.1587205,train_step,320,2,train/step_canary_loss,1.40625
+1773848122.1587205,train_step,320,2,perf/step_duration_sec,7.7270688829012215
+1773848122.1587205,train_step,320,2,perf/samples_per_sec,8.411986612910194
+1773848122.1587205,train_step,320,2,perf/tokens_per_sec,6291.130665027544
+1773848122.1587205,train_step,320,2,perf/logical_batch_size,65.0
+1773848122.1587205,train_step,320,2,perf/logical_token_count,48612.0
+1773848122.1587205,train_step,320,2,perf/gradient_accumulation_steps,8.0
+1773848122.1587205,train_step,320,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773848122.1587205,train_step,320,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773848200.6989753,train_step,330,2,train/step_loss,1.0191495253489569
+1773848200.6989753,train_step,330,2,train/step_real_loss,0.9818510636687279
+1773848200.6989753,train_step,330,2,train/lr,2.6323147493514833e-06
+1773848200.6989753,train_step,330,2,train/step_canary_loss,3.40625
+1773848200.6989753,train_step,330,2,perf/step_duration_sec,7.586550263920799
+1773848200.6989753,train_step,330,2,perf/samples_per_sec,8.567794022155125
+1773848200.6989753,train_step,330,2,perf/tokens_per_sec,6317.495875290012
+1773848200.6989753,train_step,330,2,perf/logical_batch_size,65.0
+1773848200.6989753,train_step,330,2,perf/logical_token_count,47928.0
+1773848200.6989753,train_step,330,2,perf/gradient_accumulation_steps,8.0
+1773848200.6989753,train_step,330,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773848200.6989753,train_step,330,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773848281.642083,train_step,340,2,train/step_loss,1.0784487796552253
+1773848281.642083,train_step,340,2,train/step_real_loss,1.0184003114700317
+1773848281.642083,train_step,340,2,train/lr,1.3788633298137288e-06
+1773848281.642083,train_step,340,2,train/step_canary_loss,3.0
+1773848281.642083,train_step,340,2,perf/step_duration_sec,8.599153365008533
+1773848281.642083,train_step,340,2,perf/samples_per_sec,7.675174194306803
+1773848281.642083,train_step,340,2,perf/tokens_per_sec,5509.263294777043
+1773848281.642083,train_step,340,2,perf/logical_batch_size,66.0
+1773848281.642083,train_step,340,2,perf/logical_token_count,47375.0
+1773848281.642083,train_step,340,2,perf/gradient_accumulation_steps,8.0
+1773848281.642083,train_step,340,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773848281.642083,train_step,340,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773848363.9098017,train_step,350,2,train/step_loss,1.1414880389752595
+1773848363.9098017,train_step,350,2,train/step_real_loss,0.950454406440258
+1773848363.9098017,train_step,350,2,train/lr,5.236742432724262e-07
+1773848363.9098017,train_step,350,2,train/step_canary_loss,3.58671875
+1773848363.9098017,train_step,350,2,perf/step_duration_sec,8.310993053019047
+1773848363.9098017,train_step,350,2,perf/samples_per_sec,8.302256969753463
+1773848363.9098017,train_step,350,2,perf/tokens_per_sec,6413.433347852161
+1773848363.9098017,train_step,350,2,perf/logical_batch_size,69.0
+1773848363.9098017,train_step,350,2,perf/logical_token_count,53302.0
+1773848363.9098017,train_step,350,2,perf/gradient_accumulation_steps,8.0
+1773848363.9098017,train_step,350,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773848363.9098017,train_step,350,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773848384.8834276,eval_step,350,2,eval/loss,0.9456373546030614
+1773848384.8834276,eval_step,350,2,eval/duration_sec,20.951530331978574
+1773848463.9385724,train_step,360,2,train/step_loss,1.0201530621601984
+1773848463.9385724,train_step,360,2,train/step_real_loss,0.9811613112688065
+1773848463.9385724,train_step,360,2,train/lr,7.375245966623756e-08
+1773848463.9385724,train_step,360,2,train/step_canary_loss,3.515625
+1773848463.9385724,train_step,360,2,perf/step_duration_sec,7.831112121930346
+1773848463.9385724,train_step,360,2,perf/samples_per_sec,8.300225943384616
+1773848463.9385724,train_step,360,2,perf/tokens_per_sec,6854.454279830759
+1773848463.9385724,train_step,360,2,perf/logical_batch_size,65.0
+1773848463.9385724,train_step,360,2,perf/logical_token_count,53678.0
+1773848463.9385724,train_step,360,2,perf/gradient_accumulation_steps,8.0
+1773848463.9385724,train_step,360,2,system/cuda_memory_allocated_gb,16.85233783721924
+1773848463.9385724,train_step,360,2,system/cuda_max_memory_allocated_gb,60.84869050979614
+1773848532.3133028,train_epoch,366,2,train/epoch_loss,1.0387111434896155
+1773848532.3133028,train_epoch,366,2,train/epoch_real_loss,0.9917015731131118
+1773848532.3133028,train_epoch,366,2,train/epoch_canary_loss,3.26868664640822
+1773848532.3133028,train_epoch,366,2,perf/epoch_duration_sec,1556.666436379077
+1773848532.3133028,train_epoch,366,2,perf/epoch_samples_per_sec,30.74068977250494
+1773848532.3133028,train_epoch,366,2,perf/epoch_tokens_per_sec,23919.13266056493
+1773848532.3133028,train_epoch,366,2,perf/epoch_samples,47853.0
+1773848532.3133028,train_epoch,366,2,perf/epoch_tokens,37234111.0
+1773848532.3133028,train_epoch,366,2,system/cuda_epoch_peak_memory_gb,60.84869050979614
+1773848532.3133028,train_epoch,366,2,eval/loss,0.9456853411801449
+1773848532.3133028,train_epoch,366,2,eval/duration_sec,21.018074851948768
+1773848539.6229737,audit_epoch,366,2,audit/delta,1e-05
+1773848539.6229737,audit_epoch,366,2,audit/num_canaries,500.0
+1773848539.6229737,audit_epoch,366,2,audit/num_members,250.0
+1773848539.6229737,audit_epoch,366,2,audit/paper_guess_fraction,0.2
+1773848539.6229737,audit_epoch,366,2,audit/paper_guess_steps,20.0
+1773848539.6229737,audit_epoch,366,2,audit/loss/auc,1.0
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon/0.05,3.4791953936219215
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon/0.01,3.023197554051876
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon_details/0.05/num_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon_details/0.01/num_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/loss/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/embedding/auc,1.0
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon/0.05,3.4791953936219215
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon/0.01,3.023197554051876
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon_details/0.05/num_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon_details/0.01/num_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773848539.6229737,audit_epoch,366,2,perf/audit_duration_sec,5.547323588980362
+1773848547.60095,audit_final,366,2,audit/delta,1e-05
+1773848547.60095,audit_final,366,2,audit/num_canaries,500.0
+1773848547.60095,audit_final,366,2,audit/num_members,250.0
+1773848547.60095,audit_final,366,2,audit/paper_guess_fraction,0.2
+1773848547.60095,audit_final,366,2,audit/paper_guess_steps,20.0
+1773848547.60095,audit_final,366,2,audit/loss/auc,0.999984
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon/0.05,3.4791953936219215
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon/0.01,3.023197554051876
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon_details/0.05/num_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon_details/0.01/num_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/loss/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/embedding/auc,1.0
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon/0.05,3.4791953936219215
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon/0.01,3.023197554051876
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon_details/0.05/epsilon,3.4791953936219215
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon_details/0.05/num_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon_details/0.05/correct_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon_details/0.01/epsilon,3.023197554051876
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon_details/0.01/num_guesses,100.0
+1773848547.60095,audit_final,366,2,audit/embedding/empirical_epsilon_details/0.01/correct_guesses,100.0
+1773848548.142728,energy_final,366,,energy/codecarbon/duration,3257.8297634990886
+1773848548.142728,energy_final,366,,energy/codecarbon/emissions,0.046521798865538776
+1773848548.142728,energy_final,366,,energy/codecarbon/emissions_rate,1.427999688221026e-05
+1773848548.142728,energy_final,366,,energy/codecarbon/cpu_power,84.78240267963766
+1773848548.142728,energy_final,366,,energy/codecarbon/gpu_power,1356.0935826800528
+1773848548.142728,energy_final,366,,energy/codecarbon/ram_power,38.0
+1773848548.142728,energy_final,366,,energy/codecarbon/cpu_energy,0.0740143041223195
+1773848548.142728,energy_final,366,,energy/codecarbon/gpu_energy,1.2279951440620138
+1773848548.142728,energy_final,366,,energy/codecarbon/ram_energy,0.033174056839308075
+1773848548.142728,energy_final,366,,energy/codecarbon/energy_consumed,1.3351835050236422
+1773848548.142728,energy_final,366,,energy/codecarbon/water_consumed,0.0
+1773848548.142728,energy_final,366,,energy/codecarbon/cpu_count,8.0
+1773848548.142728,energy_final,366,,energy/codecarbon/gpu_count,5.0
+1773848548.142728,energy_final,366,,energy/codecarbon/longitude,8.212
+1773848548.142728,energy_final,366,,energy/codecarbon/latitude,47.4843
+1773848548.142728,energy_final,366,,energy/codecarbon/ram_total_size,128.0
+1773848548.142728,energy_final,366,,energy/codecarbon/cpu_utilization_percent,12.756927339901509
+1773848548.142728,energy_final,366,,energy/codecarbon/gpu_utilization_percent,63.19950738916256
+1773848548.142728,energy_final,366,,energy/codecarbon/ram_utilization_percent,10.355603448275856
+1773848548.142728,energy_final,366,,energy/codecarbon/ram_used_gb,208.74299130651164
+1773848548.142728,energy_final,366,,energy/codecarbon/pue,1.0
+1773848548.142728,energy_final,366,,energy/codecarbon/wue,0.0

granite-4.0-h-tiny/base/summary.json

@@ -0,0 +1,71 @@
+{
+  "audit/delta": 1e-05,
+  "audit/embedding/auc": 1.0,
+  "audit/embedding/empirical_epsilon/0.01": 3.023197554051876,
+  "audit/embedding/empirical_epsilon/0.05": 3.4791953936219215,
+  "audit/embedding/empirical_epsilon_details/0.01/correct_guesses": 100.0,
+  "audit/embedding/empirical_epsilon_details/0.01/epsilon": 3.023197554051876,
+  "audit/embedding/empirical_epsilon_details/0.01/num_guesses": 100.0,
+  "audit/embedding/empirical_epsilon_details/0.05/correct_guesses": 100.0,
+  "audit/embedding/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215,
+  "audit/embedding/empirical_epsilon_details/0.05/num_guesses": 100.0,
+  "audit/loss/auc": 0.999984,
+  "audit/loss/empirical_epsilon/0.01": 3.023197554051876,
+  "audit/loss/empirical_epsilon/0.05": 3.4791953936219215,
+  "audit/loss/empirical_epsilon_details/0.01/correct_guesses": 100.0,
+  "audit/loss/empirical_epsilon_details/0.01/epsilon": 3.023197554051876,
+  "audit/loss/empirical_epsilon_details/0.01/num_guesses": 100.0,
+  "audit/loss/empirical_epsilon_details/0.05/correct_guesses": 100.0,
+  "audit/loss/empirical_epsilon_details/0.05/epsilon": 3.4791953936219215,
+  "audit/loss/empirical_epsilon_details/0.05/num_guesses": 100.0,
+  "audit/num_canaries": 500.0,
+  "audit/num_members": 250.0,
+  "audit/paper_guess_fraction": 0.2,
+  "audit/paper_guess_steps": 20.0,
+  "energy/codecarbon/cpu_count": 8.0,
+  "energy/codecarbon/cpu_energy": 0.0740143041223195,
+  "energy/codecarbon/cpu_power": 84.78240267963766,
+  "energy/codecarbon/cpu_utilization_percent": 12.756927339901509,
+  "energy/codecarbon/duration": 3257.8297634990886,
+  "energy/codecarbon/emissions": 0.046521798865538776,
+  "energy/codecarbon/emissions_rate": 1.427999688221026e-05,
+  "energy/codecarbon/energy_consumed": 1.3351835050236422,
+  "energy/codecarbon/gpu_count": 5.0,
+  "energy/codecarbon/gpu_energy": 1.2279951440620138,
+  "energy/codecarbon/gpu_power": 1356.0935826800528,
+  "energy/codecarbon/gpu_utilization_percent": 63.19950738916256,
+  "energy/codecarbon/latitude": 47.4843,
+  "energy/codecarbon/longitude": 8.212,
+  "energy/codecarbon/pue": 1.0,
+  "energy/codecarbon/ram_energy": 0.033174056839308075,
+  "energy/codecarbon/ram_power": 38.0,
+  "energy/codecarbon/ram_total_size": 128.0,
+  "energy/codecarbon/ram_used_gb": 208.74299130651164,
+  "energy/codecarbon/ram_utilization_percent": 10.355603448275856,
+  "energy/codecarbon/water_consumed": 0.0,
+  "energy/codecarbon/wue": 0.0,
+  "eval/duration_sec": 21.018074851948768,
+  "eval/loss": 0.9456853411801449,
+  "perf/audit_duration_sec": 5.547323588980362,
+  "perf/epoch_duration_sec": 1556.666436379077,
+  "perf/epoch_samples": 47853.0,
+  "perf/epoch_samples_per_sec": 30.74068977250494,
+  "perf/epoch_tokens": 37234111.0,
+  "perf/epoch_tokens_per_sec": 23919.13266056493,
+  "perf/gradient_accumulation_steps": 8.0,
+  "perf/logical_batch_size": 65.0,
+  "perf/logical_token_count": 53678.0,
+  "perf/samples_per_sec": 8.300225943384616,
+  "perf/step_duration_sec": 7.831112121930346,
+  "perf/tokens_per_sec": 6854.454279830759,
+  "system/cuda_epoch_peak_memory_gb": 60.84869050979614,
+  "system/cuda_max_memory_allocated_gb": 60.84869050979614,
+  "system/cuda_memory_allocated_gb": 16.85233783721924,
+  "train/epoch_canary_loss": 3.26868664640822,
+  "train/epoch_loss": 1.0387111434896155,
+  "train/epoch_real_loss": 0.9917015731131118,
+  "train/lr": 7.375245966623756e-08,
+  "train/step_canary_loss": 3.515625,
+  "train/step_loss": 1.0201530621601984,
+  "train/step_real_loss": 0.9811613112688065
+}

granite-4.0-h-tiny/base/tensorboard/events.out.tfevents.1773845289.u24-chaihm0-651.3823128.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:246cbae6b2374c38213340a0cfc06596e0bc5a6d062308c3d65092eb003b1bfb
+size 37695

granite-4.0-h-tiny/base/tokenizer/added_tokens.json

@@ -0,0 +1,502 @@
+{
+  "114848338811": 100360,
+  "116427191984": 100714,
+  "116639746413": 100686,
+  "117298239631": 100576,
+  "117657531962": 100633,
+  "117783274348": 100650,
+  "119342152414": 100723,
+  "121557291859": 100582,
+  "123741461297": 100491,
+  "124589118494": 100487,
+  "125866399464": 100596,
+  "126574388585": 100538,
+  "129693547287": 100472,
+  "133373659361": 100667,
+  "135441121756": 100355,
+  "136931728327": 100457,
+  "138154781148": 100767,
+  "138218735276": 100651,
+  "141499982867": 100745,
+  "142526557681": 100845,
+  "142799718159": 100709,
+  "143182978129": 100671,
+  "143777992227": 100366,
+  "144373897488": 100777,
+  "145669731981": 100581,
+  "147444452794": 100830,
+  "147541537162": 100775,
+  "151859598873": 100547,
+  "152676556624": 100682,
+  "154962668224": 100696,
+  "158142752582": 100577,
+  "158929292238": 100587,
+  "159337222655": 100422,
+  "161238482259": 100790,
+  "161588262349": 100437,
+  "163165456396": 100788,
+  "163497196769": 100820,
+  "165429152138": 100717,
+  "172599315861": 100371,
+  "173781362446": 100423,
+  "175469284871": 100515,
+  "175528632226": 100833,
+  "178478368922": 100762,
+  "179386156569": 100804,
+  "179564995814": 100697,
+  "181233596124": 100782,
+  "181439694557": 100398,
+  "182345464792": 100442,
+  "182782912224": 100624,
+  "183368827562": 100551,
+  "185438575221": 100537,
+  "186198784585": 100598,
+  "186872457894": 100835,
+  "191887635211": 100721,
+  "195388849133": 100792,
+  "195792996468": 100546,
+  "196119929397": 100611,
+  "196735786156": 100493,
+  "198462689911": 100428,
+  "211411365397": 100377,
+  "211662639865": 100706,
+  "212985874612": 100720,
+  "214193645583": 100802,
+  "215671314196": 100602,
+  "215826764872": 100756,
+  "216667245274": 100484,
+  "217756494954": 100648,
+  "218391991184": 100679,
+  "218784516525": 100707,
+  "221762325616": 100678,
+  "225164373623": 100439,
+  "228358422847": 100669,
+  "228565148417": 100726,
+  "229775525672": 100793,
+  "231853711778": 100516,
+  "232776999554": 100443,
+  "232787996724": 100601,
+  "232896526252": 100489,
+  "233795848681": 100507,
+  "234316622259": 100625,
+  "242851284913": 100435,
+  "243165586174": 100784,
+  "244371196654": 100615,
+  "244885399387": 100533,
+  "245416776926": 100583,
+  "251789649997": 100613,
+  "252867443568": 100578,
+  "256497188281": 100735,
+  "259533577263": 100458,
+  "259666844669": 100434,
+  "262296912232": 100481,
+  "262437452884": 100794,
+  "264259917554": 100413,
+  "265847332563": 100818,
+  "267514771964": 100574,
+  "267954879268": 100646,
+  "268433315926": 100822,
+  "268921994538": 100715,
+  "272362973463": 100406,
+  "275597777299": 100565,
+  "281673864127": 100408,
+  "282322838685": 100373,
+  "284445431422": 100664,
+  "284776712475": 100780,
+  "286173241916": 100603,
+  "291762694874": 100670,
+  "293847485454": 100760,
+  "295118729589": 100609,
+  "295448593321": 100490,
+  "296953381169": 100479,
+  "298464134954": 100483,
+  "311632789559": 100391,
+  "313145133463": 100773,
+  "313313831273": 100815,
+  "314697386682": 100449,
+  "315534574435": 100754,
+  "319584985416": 100570,
+  "322589379462": 100680,
+  "322683334381": 100668,
+  "325196781219": 100844,
+  "327158398268": 100673,
+  "327491773134": 100387,
+  "328552864359": 100705,
+  "331369926711": 100731,
+  "332234613346": 100843,
+  "332522971941": 100683,
+  "334513529294": 100662,
+  "336813343298": 100747,
+  "337326533813": 100444,
+  "339882449689": 100597,
+  "342221452223": 100742,
+  "345271433112": 100657,
+  "349249721685": 100751,
+  "351885857332": 100426,
+  "353755529131": 100753,
+  "355977139358": 100379,
+  "356722692178": 100846,
+  "357299947518": 100832,
+  "357621613582": 100738,
+  "363214487524": 100532,
+  "364196983365": 100545,
+  "364492359246": 100695,
+  "364947325679": 100618,
+  "367459894796": 100356,
+  "367968398551": 100757,
+  "368763327262": 100799,
+  "371496153876": 100816,
+  "374327157578": 100390,
+  "374439448821": 100468,
+  "374598961236": 100512,
+  "375429755114": 100781,
+  "377273562477": 100480,
+  "377338713663": 100494,
+  "378749551722": 100789,
+  "379557332953": 100451,
+  "379571934299": 100612,
+  "383837596997": 100381,
+  "384598329253": 100616,
+  "387763951128": 100730,
+  "389715484387": 100759,
+  "393741199486": 100704,
+  "396521188476": 100665,
+  "396847976144": 100785,
+  "398549545767": 100502,
+  "399223169721": 100548,
+  "399749397872": 100471,
+  "412843856172": 100803,
+  "415956914763": 100629,
+  "417668946583": 100737,
+  "418412875665": 100698,
+  "421369185549": 100626,
+  "421435255895": 100666,
+  "425949483533": 100368,
+  "429781429464": 100453,
+  "431149748713": 100594,
+  "431751228368": 100534,
+  "433581868192": 100535,
+  "434632855397": 100627,
+  "438124184139": 100812,
+  "438435178228": 100739,
+  "438745114173": 100690,
+  "441373944925": 100463,
+  "441377892146": 100795,
+  "445925149649": 100364,
+  "447445629421": 100396,
+  "449318681694": 100847,
+  "449549215429": 100819,
+  "451161679736": 100676,
+  "451885565366": 100796,
+  "454164859837": 100656,
+  "454244839926": 100503,
+  "455423278865": 100703,
+  "457919547633": 100358,
+  "462294118988": 100658,
+  "467893531755": 100465,
+  "472129283475": 100749,
+  "473822473819": 100397,
+  "475794684356": 100556,
+  "476969195485": 100772,
+  "479227349997": 100456,
+  "479676316326": 100409,
+  "481884157827": 100416,
+  "482829231639": 100828,
+  "484489193824": 100529,
+  "484773717614": 100497,
+  "485177821815": 100354,
+  "485692541169": 100740,
+  "486391569221": 100536,
+  "488496781997": 100826,
+  "488538864842": 100459,
+  "489562189822": 100530,
+  "489991411364": 100461,
+  "492373714791": 100590,
+  "492516914298": 100769,
+  "497294727353": 100660,
+  "499148455254": 100462,
+  "499156317423": 100654,
+  "499444466686": 100448,
+  "511785788222": 100659,
+  "511923256573": 100672,
+  "512659818733": 100591,
+  "513349618943": 100685,
+  "513361312532": 100520,
+  "514354144759": 100750,
+  "514532995959": 100436,
+  "515856611931": 100655,
+  "516968424863": 100476,
+  "517379856925": 100450,
+  "517778845323": 100431,
+  "522515164754": 100778,
+  "523723475786": 100755,
+  "523918658846": 100392,
+  "523937669294": 100716,
+  "524444658383": 100649,
+  "527117488925": 100466,
+  "527874729936": 100728,
+  "529455466433": 100610,
+  "529679678956": 100420,
+  "529854679937": 100540,
+  "532833541879": 100642,
+  "533417736566": 100552,
+  "535787224917": 100425,
+  "538631466654": 100677,
+  "545512344383": 100447,
+  "546276298359": 100752,
+  "547149832394": 100482,
+  "547563892268": 100813,
+  "549946382131": 100663,
+  "552738847828": 100836,
+  "556814553861": 100827,
+  "556976448182": 100605,
+  "557163366873": 100776,
+  "557533689478": 100527,
+  "558837194455": 100787,
+  "562389434565": 100734,
+  "562716493614": 100732,
+  "563117729724": 100510,
+  "563365646341": 100571,
+  "564395222747": 100675,
+  "564544131355": 100614,
+  "569765693871": 100353,
+  "569853653352": 100758,
+  "571317457151": 100712,
+  "572154333646": 100486,
+  "578827344666": 100427,
+  "579184949249": 100418,
+  "581493616981": 100763,
+  "581976444672": 100588,
+  "582471228164": 100525,
+  "583527834464": 100825,
+  "585544633356": 100524,
+  "586855179568": 100496,
+  "587541149322": 100817,
+  "587787496812": 100417,
+  "589439538674": 100764,
+  "589547936288": 100531,
+  "591464355756": 100719,
+  "592538279337": 100399,
+  "594688742928": 100748,
+  "598916996413": 100630,
+  "599961512792": 100606,
+  "612128769512": 100389,
+  "612788593128": 100580,
+  "614583493135": 100841,
+  "614797267726": 100452,
+  "616248973917": 100542,
+  "618423867817": 100725,
+  "618878266619": 100829,
+  "619454958782": 100593,
+  "622553389126": 100693,
+  "624221476921": 100595,
+  "624266421831": 100701,
+  "625334487352": 100807,
+  "626748382482": 100837,
+  "632962475133": 100528,
+  "635127952696": 100713,
+  "635185846287": 100687,
+  "636324136426": 100801,
+  "637551427887": 100647,
+  "643489267333": 100372,
+  "643692393748": 100401,
+  "644234165531": 100623,
+  "645527857972": 100834,
+  "646979124677": 100410,
+  "649291669397": 100362,
+  "649516938598": 100810,
+  "652154162978": 100557,
+  "653573457168": 100492,
+  "655517954651": 100619,
+  "658481324922": 100415,
+  "661436365453": 100433,
+  "662426712668": 100517,
+  "664847713559": 100637,
+  "665661658354": 100554,
+  "668134915514": 100400,
+  "668514843129": 100559,
+  "669183782449": 100643,
+  "669689115557": 100386,
+  "673948786567": 100620,
+  "676539973863": 100440,
+  "676676697292": 100445,
+  "677896358599": 100499,
+  "678666359123": 100526,
+  "682564697312": 100380,
+  "687284547244": 100848,
+  "689285841825": 100473,
+  "689362171782": 100382,
+  "691315125346": 100635,
+  "691587454538": 100365,
+  "693281533643": 100555,
+  "694235616268": 100407,
+  "696651276628": 100402,
+  "697747991249": 100743,
+  "698218415444": 100378,
+  "699446724178": 100700,
+  "711412347158": 100518,
+  "711641973413": 100653,
+  "712992815957": 100521,
+  "714738141818": 100761,
+  "716763689965": 100744,
+  "717628619782": 100811,
+  "718726519731": 100370,
+  "718951851413": 100699,
+  "721737645236": 100455,
+  "722339992679": 100766,
+  "722618266242": 100429,
+  "724492262421": 100692,
+  "724883568652": 100475,
+  "724964923582": 100779,
+  "729116422489": 100727,
+  "729825168745": 100634,
+  "731681792655": 100566,
+  "733737988257": 100477,
+  "739246183345": 100718,
+  "739394546916": 100645,
+  "739667844957": 100733,
+  "739784153482": 100729,
+  "741757479784": 100539,
+  "742765629356": 100438,
+  "745977673725": 100604,
+  "746285987371": 100361,
+  "749665846687": 100432,
+  "749966767285": 100383,
+  "752644929761": 100563,
+  "753159165157": 100384,
+  "753233719644": 100558,
+  "753466987363": 100519,
+  "754978243758": 100791,
+  "756916173536": 100805,
+  "757422791351": 100800,
+  "757785319772": 100768,
+  "758963111966": 100774,
+  "762486672996": 100708,
+  "762495822823": 100798,
+  "763531783491": 100543,
+  "765474393376": 100359,
+  "765737843639": 100514,
+  "765981581453": 100393,
+  "766294538337": 100607,
+  "767858144912": 100711,
+  "768461952319": 100636,
+  "771619544974": 100474,
+  "773671349113": 100631,
+  "773687856563": 100424,
+  "774679513717": 100823,
+  "778466723723": 100404,
+  "781653545886": 100374,
+  "786391632558": 100553,
+  "794825141891": 100394,
+  "795693824762": 100385,
+  "795838284624": 100421,
+  "796415361892": 100375,
+  "797238311233": 100851,
+  "799364566435": 100504,
+  "799417897197": 100584,
+  "811371928234": 100572,
+  "813981174797": 100689,
+  "814611949491": 100652,
+  "815458725125": 100702,
+  "816115936673": 100622,
+  "817378986438": 100506,
+  "818479319499": 100746,
+  "819162623892": 100560,
+  "824387273757": 100508,
+  "824488329416": 100640,
+  "825815739116": 100741,
+  "826214551218": 100441,
+  "829235161526": 100523,
+  "836746273134": 100414,
+  "837131396371": 100573,
+  "837873762491": 100641,
+  "839269684865": 100579,
+  "841824384614": 100488,
+  "841991688488": 100376,
+  "843998562287": 100485,
+  "846657726193": 100765,
+  "848845611563": 100500,
+  "851254219729": 100824,
+  "851852651677": 100501,
+  "852347289392": 100478,
+  "853859348234": 100403,
+  "856317483891": 100814,
+  "857231548835": 100621,
+  "858146415154": 100710,
+  "859289776479": 100840,
+  "859927364588": 100736,
+  "861342291298": 100821,
+  "864718285734": 100568,
+  "865331112869": 100352,
+  "866519986723": 100661,
+  "867678158885": 100470,
+  "873898736873": 100395,
+  "876537814263": 100681,
+  "877385381798": 100562,
+  "877482678543": 100357,
+  "879425227932": 100469,
+  "879764613759": 100674,
+  "881112998642": 100592,
+  "881136466196": 100564,
+  "883685933161": 100412,
+  "884354318946": 100684,
+  "886464195129": 100639,
+  "887417525236": 100809,
+  "887753195844": 100617,
+  "889965243135": 100694,
+  "892336777248": 100586,
+  "892697125149": 100806,
+  "893974619684": 100850,
+  "894885246797": 100498,
+  "897784492783": 100589,
+  "899151413682": 100464,
+  "912193598348": 100419,
+  "916198946615": 100509,
+  "919181521716": 100770,
+  "921889371893": 100628,
+  "921894985642": 100838,
+  "922327493433": 100411,
+  "922466849763": 100454,
+  "922867396683": 100513,
+  "927914615679": 100363,
+  "928335588653": 100467,
+  "929185622831": 100446,
+  "929826356991": 100405,
+  "933775286797": 100842,
+  "934912591213": 100608,
+  "937495658852": 100460,
+  "938488813964": 100549,
+  "941169431859": 100561,
+  "941861857715": 100808,
+  "941976537588": 100644,
+  "943193294691": 100599,
+  "943878645871": 100839,
+  "944513428457": 100575,
+  "946444632795": 100724,
+  "947262995132": 100849,
+  "948585673431": 100783,
+  "949235426629": 100831,
+  "951794811935": 100511,
+  "952465217972": 100632,
+  "952872416512": 100430,
+  "953267689786": 100638,
+  "955456118295": 100544,
+  "955668961269": 100600,
+  "961133895172": 100567,
+  "961981959227": 100550,
+  "963852916563": 100569,
+  "964342468552": 100495,
+  "967114116556": 100505,
+  "967214577653": 100722,
+  "971621888444": 100522,
+  "981277526855": 100797,
+  "982993456429": 100369,
+  "983493951323": 100691,
+  "983569279932": 100388,
+  "985781138935": 100771,
+  "993832498489": 100688,
+  "996116119839": 100541,
+  "997724962668": 100786,
+  "997958836435": 100585,
+  "997981281989": 100367
+}

granite-4.0-h-tiny/base/tokenizer/chat_template.jinja

@@ -0,0 +1,118 @@
+{%- set tools_system_message_prefix = 'You are a helpful assistant with access to the following tools. You may call one or more tools to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>'  %}
+{%- set tools_system_message_suffix = '\n</tools>\n\nFor each tool call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.' %}
+{%- set documents_system_message_prefix = 'You are a helpful assistant with access to the following documents. You may use one or more documents to assist with the user query.\n\nYou are given a list of documents within <documents></documents> XML tags:\n<documents>' %}
+{%- set documents_system_message_suffix = '\n</documents>\n\nWrite the response to the user\'s input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.' %}
+{%- set g4_default_system_message = 'You are a helpful assistant. Please ensure responses are professional, accurate, and safe.' %}
+{%- if available_tools is defined and available_tools %}
+    {%- set tools = available_tools %}
+{%- endif %}
+{%- set ns = namespace(tools_system_message=tools_system_message_prefix,
+                       documents_system_message=documents_system_message_prefix,
+                       default_system_message=g4_default_system_message,
+                       system_message=''
+                       ) %}
+{%- if tools %}
+    {%- for tool in tools %}
+        {%- set ns.tools_system_message = ns.tools_system_message + '\n' + (tool | tojson) %}
+    {%- endfor %}
+    {%- set ns.tools_system_message = ns.tools_system_message + tools_system_message_suffix %}
+{%- else %}
+    {%- set ns.tools_system_message = '' %}
+{%- endif %}
+{%- if documents %}
+    {%- for document in documents %}
+        {%- set ns.documents_system_message = ns.documents_system_message + '\n' + (document | tojson) %}
+    {%- endfor %}
+    {%- set ns.documents_system_message = ns.documents_system_message + documents_system_message_suffix %}
+{%- else %}
+    {%- set ns.documents_system_message = '' %}
+{%- endif %}
+{%- if messages[0].role == 'system' %}
+    {%- if messages[0].content is string %}
+        {%- set ns.system_message = messages[0].content %}
+    {%- elif messages[0].content is iterable %}
+        {%- for entry in messages[0].content %}
+            {%- if entry.type== 'text' %}
+                {%- if ns.system_message != '' %}
+                    {%- set ns.system_message = ns.system_message + '\n' %}
+                {%- endif %}
+                {%- set ns.system_message = ns.system_message + entry.text %}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+    {%- if tools and documents %}
+        {%- set ns.system_message = ns.system_message + '\n\n' +  ns.tools_system_message + '\n\n' + ns.documents_system_message %}
+    {%- elif tools %}
+        {%- set ns.system_message = ns.system_message + '\n\n' + ns.tools_system_message %}
+    {%- elif documents %}
+        {%- set ns.system_message = ns.system_message + '\n\n' + ns.documents_system_message %}
+    {%- endif %}
+{%- else %}
+    {%- if tools and documents %}
+        {%- set ns.system_message = ns.tools_system_message + '\n\n' + ns.documents_system_message  %}
+    {%- elif tools %}
+        {%- set ns.system_message = ns.tools_system_message %}
+    {%- elif documents %}
+        {%- set ns.system_message = ns.documents_system_message %}
+    {%- endif %}
+{%- endif %}
+{%- if ns.system_message %}
+    {{- '<|start_of_role|>system<|end_of_role|>' + ns.system_message + '<|end_of_text|>\n' }}
+{%- else %}
+    {{- '<|start_of_role|>system<|end_of_role|>' + ns.default_system_message + '<|end_of_text|>\n' }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = namespace(val='') %}
+    {%- if message.content is string %}
+        {%- set content.val = message.content %}
+    {%- else %}
+        {%- if message.content is iterable %}
+            {%- for entry in message.content %}
+                {%- if entry.type== 'text' %}
+                    {%- if content.val != '' %}
+                        {%- set content.val = content.val + '\n' %}
+                    {%- endif %}
+                    {%- set content.val = content.val + entry.text %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- endif %}
+    {%- if (message.role == 'user') or (message.role == 'system' and not loop.first) %}
+        {{- '<|start_of_role|>' + message.role + '<|end_of_role|>' + content.val + '<|end_of_text|>\n' }}
+    {%- elif message.role == 'assistant' %}
+        {{- '<|start_of_role|>' + message.role + '<|end_of_role|>' + content.val }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content.val) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|end_of_text|>\n' }}
+    {%- elif message.role == 'tool' %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != 'tool') %}
+            {{- '<|start_of_role|>user<|end_of_role|>' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content.val }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != 'tool') %}
+            {{- '<|end_of_text|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_of_role|>assistant<|end_of_role|>' }}
+{%- endif %}

granite-4.0-h-tiny/base/tokenizer/special_tokens_map.json

@@ -0,0 +1,3532 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "865331112869",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "569765693871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "485177821815",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "135441121756",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "367459894796",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "877482678543",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "457919547633",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765474393376",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "114848338811",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "746285987371",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "649291669397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "927914615679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "445925149649",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "691587454538",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "143777992227",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997981281989",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "425949483533",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "982993456429",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "718726519731",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "172599315861",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "643489267333",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "282322838685",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "781653545886",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "796415361892",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "841991688488",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "211411365397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "698218415444",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "355977139358",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "682564697312",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "383837596997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "689362171782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "749966767285",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753159165157",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "795693824762",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "669689115557",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "327491773134",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "983569279932",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "612128769512",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374327157578",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "311632789559",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523918658846",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765981581453",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "794825141891",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "873898736873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "447445629421",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "473822473819",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "181439694557",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "592538279337",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "668134915514",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "643692393748",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "696651276628",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "853859348234",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "778466723723",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "929826356991",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "272362973463",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "694235616268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "281673864127",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "479676316326",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "646979124677",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922327493433",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "883685933161",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "264259917554",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "836746273134",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "658481324922",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "481884157827",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "587787496812",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "579184949249",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "912193598348",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529679678956",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "795838284624",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "159337222655",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "173781362446",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "773687856563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "535787224917",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "351885857332",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "578827344666",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "198462689911",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "722618266242",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "952872416512",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "517778845323",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "749665846687",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "661436365453",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "259666844669",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "242851284913",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "514532995959",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "161588262349",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "742765629356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "225164373623",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "676539973863",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "826214551218",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "182345464792",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232776999554",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "337326533813",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "676676697292",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "929185622831",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "545512344383",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499444466686",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "314697386682",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "517379856925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "379557332953",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "614797267726",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "429781429464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922466849763",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "721737645236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "479227349997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "136931728327",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "259533577263",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "488538864842",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "937495658852",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "489991411364",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499148455254",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "441373944925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "899151413682",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "467893531755",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "527117488925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "928335588653",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374439448821",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "879425227932",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "867678158885",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "399749397872",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "129693547287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "689285841825",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "771619544974",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724883568652",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "516968424863",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "733737988257",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "852347289392",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "296953381169",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "377273562477",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "262296912232",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "547149832394",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "298464134954",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "216667245274",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "843998562287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "572154333646",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "124589118494",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "841824384614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232896526252",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "295448593321",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "123741461297",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "653573457168",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "196735786156",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "377338713663",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "964342468552",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "586855179568",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "484773717614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "894885246797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "677896358599",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "848845611563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "851852651677",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "398549545767",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "454244839926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "799364566435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "967114116556",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "817378986438",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "233795848681",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "824387273757",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "916198946615",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "563117729724",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "951794811935",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374598961236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922867396683",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765737843639",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "175469284871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "231853711778",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "662426712668",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "711412347158",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753466987363",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "513361312532",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "712992815957",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "971621888444",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "829235161526",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "585544633356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "582471228164",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "678666359123",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "557533689478",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "632962475133",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "484489193824",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "489562189822",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "589547936288",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "363214487524",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "244885399387",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "431751228368",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "433581868192",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "486391569221",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "185438575221",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "126574388585",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "741757479784",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529854679937",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "996116119839",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "616248973917",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "763531783491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "955456118295",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364196983365",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "195792996468",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "151859598873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "399223169721",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "938488813964",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "961981959227",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "183368827562",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "533417736566",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "786391632558",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "665661658354",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "693281533643",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "475794684356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "652154162978",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753233719644",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "668514843129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "819162623892",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941169431859",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "877385381798",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "752644929761",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "881136466196",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "275597777299",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "731681792655",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "961133895172",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "864718285734",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "963852916563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "319584985416",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "563365646341",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "811371928234",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "837131396371",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "267514771964",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "944513428457",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117298239631",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "158142752582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "252867443568",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "839269684865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "612788593128",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "145669731981",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "121557291859",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "245416776926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "799417897197",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997958836435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "892336777248",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "158929292238",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "581976444672",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "897784492783",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "492373714791",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "512659818733",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "881112998642",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "619454958782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "431149748713",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "624221476921",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "125866399464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "339882449689",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "186198784585",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "943193294691",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "955668961269",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232787996724",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "215671314196",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "286173241916",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "745977673725",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "556976448182",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "599961512792",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "766294538337",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "934912591213",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "295118729589",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529455466433",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "196119929397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "379571934299",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "251789649997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "564544131355",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "244371196654",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "384598329253",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "887753195844",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364947325679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "655517954651",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "673948786567",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "857231548835",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "816115936673",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "644234165531",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "182782912224",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "234316622259",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "421369185549",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "434632855397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "921889371893",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "415956914763",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "598916996413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "773671349113",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "952465217972",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117657531962",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "729825168745",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "691315125346",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "768461952319",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "664847713559",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "953267689786",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "886464195129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "824488329416",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "837873762491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "532833541879",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "669183782449",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941976537588",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739394546916",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "267954879268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "637551427887",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "217756494954",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "524444658383",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117783274348",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "138218735276",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "814611949491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "711641973413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499156317423",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "515856611931",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "454164859837",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "345271433112",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "462294118988",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "511785788222",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "497294727353",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "866519986723",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "334513529294",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "549946382131",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "284445431422",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "396521188476",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "421435255895",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "133373659361",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "322683334381",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "228358422847",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "291762694874",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "143182978129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "511923256573",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "327158398268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "879764613759",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "564395222747",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "451161679736",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "538631466654",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "221762325616",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "218391991184",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "322589379462",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "876537814263",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "152676556624",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "332522971941",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "884354318946",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "513349618943",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "116639746413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "635185846287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "993832498489",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "813981174797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438745114173",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "983493951323",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724492262421",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "622553389126",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "889965243135",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364492359246",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "154962668224",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "179564995814",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "418412875665",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "718951851413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "699446724178",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "624266421831",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "815458725125",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "455423278865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "393741199486",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "328552864359",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "211662639865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "218784516525",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "762486672996",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "142799718159",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "858146415154",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "767858144912",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "571317457151",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "635127952696",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "116427191984",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "268921994538",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523937669294",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "165429152138",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739246183345",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "591464355756",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "212985874612",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "191887635211",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "967214577653",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "119342152414",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "946444632795",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "618423867817",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "228565148417",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "729116422489",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "527874729936",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739784153482",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "387763951128",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "331369926711",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "562716493614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739667844957",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "562389434565",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "256497188281",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "859927364588",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "417668946583",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "357621613582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438435178228",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "485692541169",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "825815739116",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "342221452223",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "697747991249",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "716763689965",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "141499982867",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "818479319499",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "336813343298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "594688742928",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "472129283475",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "514354144759",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "349249721685",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "546276298359",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "353755529131",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "315534574435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523723475786",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "215826764872",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "367968398551",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "569853653352",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "389715484387",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "293847485454",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "714738141818",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "178478368922",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "581493616981",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "589439538674",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "846657726193",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "722339992679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "138154781148",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "757785319772",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "492516914298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "919181521716",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "985781138935",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "476969195485",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "313145133463",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "758963111966",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "147541537162",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "557163366873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "144373897488",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "522515164754",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724964923582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "284776712475",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "375429755114",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "181233596124",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "948585673431",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "243165586174",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "396847976144",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997724962668",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "558837194455",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "163165456396",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "378749551722",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "161238482259",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "754978243758",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "195388849133",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "229775525672",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "262437452884",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "441377892146",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "451885565366",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "981277526855",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "762495822823",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "368763327262",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "757422791351",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "636324136426",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "214193645583",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "412843856172",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "179386156569",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "756916173536",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "892697125149",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "625334487352",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941861857715",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "887417525236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "649516938598",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "717628619782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438124184139",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "547563892268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "856317483891",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "313313831273",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "371496153876",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "587541149322",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "265847332563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "449549215429",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "163497196769",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "861342291298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "268433315926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "774679513717",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "851254219729",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "583527834464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "488496781997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "556814553861",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "482829231639",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "618878266619",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "147444452794",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "949235426629",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "357299947518",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "175528632226",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "645527857972",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "186872457894",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "552738847828",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "626748382482",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "921894985642",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "943878645871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "859289776479",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "614583493135",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "933775286797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "332234613346",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "325196781219",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "142526557681",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "356722692178",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "449318681694",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "687284547244",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "947262995132",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "893974619684",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "797238311233",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|unk|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

granite-4.0-h-tiny/base/train.log

@@ -0,0 +1,43 @@
+2026-03-18 15:51:40,383 [INFO] new_opacus_codex.train_steps: epoch=1 step=10 loss=1.4540
+2026-03-18 15:52:56,561 [INFO] new_opacus_codex.train_steps: epoch=1 step=20 loss=1.4430
+2026-03-18 15:54:07,392 [INFO] new_opacus_codex.train_steps: epoch=1 step=30 loss=1.3321
+2026-03-18 15:55:20,599 [INFO] new_opacus_codex.train_steps: epoch=1 step=40 loss=1.3426
+2026-03-18 15:56:32,854 [INFO] new_opacus_codex.train_steps: epoch=1 step=50 loss=1.2148
+2026-03-18 15:56:50,948 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=50 eval_loss=0.9921 duration_sec=18.07
+2026-03-18 15:58:03,288 [INFO] new_opacus_codex.train_steps: epoch=1 step=60 loss=1.2121
+2026-03-18 15:59:15,998 [INFO] new_opacus_codex.train_steps: epoch=1 step=70 loss=1.1107
+2026-03-18 16:00:33,647 [INFO] new_opacus_codex.train_steps: epoch=1 step=80 loss=1.1375
+2026-03-18 16:01:54,690 [INFO] new_opacus_codex.train_steps: epoch=1 step=90 loss=1.1388
+2026-03-18 16:03:16,509 [INFO] new_opacus_codex.train_steps: epoch=1 step=100 loss=1.1609
+2026-03-18 16:03:38,169 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=100 eval_loss=0.9644 duration_sec=21.65
+2026-03-18 16:05:00,910 [INFO] new_opacus_codex.train_steps: epoch=1 step=110 loss=1.1657
+2026-03-18 16:06:22,160 [INFO] new_opacus_codex.train_steps: epoch=1 step=120 loss=1.0869
+2026-03-18 16:07:45,942 [INFO] new_opacus_codex.train_steps: epoch=1 step=130 loss=1.1091
+2026-03-18 16:09:07,362 [INFO] new_opacus_codex.train_steps: epoch=1 step=140 loss=1.1009
+2026-03-18 16:10:30,879 [INFO] new_opacus_codex.train_steps: epoch=1 step=150 loss=1.0421
+2026-03-18 16:10:52,132 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=1 step=150 eval_loss=0.9539 duration_sec=21.24
+2026-03-18 16:12:14,839 [INFO] new_opacus_codex.train_steps: epoch=1 step=160 loss=1.0549
+2026-03-18 16:13:36,397 [INFO] new_opacus_codex.train_steps: epoch=1 step=170 loss=1.0636
+2026-03-18 16:14:58,723 [INFO] new_opacus_codex.train_steps: epoch=1 step=180 loss=1.1121
+2026-03-18 16:16:52,739 [INFO] new_opacus_codex.train_steps: epoch=2 step=190 loss=1.0540
+2026-03-18 16:18:16,987 [INFO] new_opacus_codex.train_steps: epoch=2 step=200 loss=1.0693
+2026-03-18 16:18:37,958 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=200 eval_loss=0.9487 duration_sec=20.96
+2026-03-18 16:20:00,036 [INFO] new_opacus_codex.train_steps: epoch=2 step=210 loss=1.0738
+2026-03-18 16:21:19,728 [INFO] new_opacus_codex.train_steps: epoch=2 step=220 loss=1.0528
+2026-03-18 16:22:41,656 [INFO] new_opacus_codex.train_steps: epoch=2 step=230 loss=1.0376
+2026-03-18 16:24:08,819 [INFO] new_opacus_codex.train_steps: epoch=2 step=240 loss=1.0308
+2026-03-18 16:25:27,738 [INFO] new_opacus_codex.train_steps: epoch=2 step=250 loss=1.0333
+2026-03-18 16:25:48,830 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=250 eval_loss=0.9465 duration_sec=21.09
+2026-03-18 16:27:11,421 [INFO] new_opacus_codex.train_steps: epoch=2 step=260 loss=1.0529
+2026-03-18 16:28:32,862 [INFO] new_opacus_codex.train_steps: epoch=2 step=270 loss=1.0417
+2026-03-18 16:29:53,125 [INFO] new_opacus_codex.train_steps: epoch=2 step=280 loss=1.0822
+2026-03-18 16:31:09,476 [INFO] new_opacus_codex.train_steps: epoch=2 step=290 loss=1.0421
+2026-03-18 16:32:26,611 [INFO] new_opacus_codex.train_steps: epoch=2 step=300 loss=1.0475
+2026-03-18 16:32:46,612 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=300 eval_loss=0.9458 duration_sec=19.98
+2026-03-18 16:34:03,949 [INFO] new_opacus_codex.train_steps: epoch=2 step=310 loss=1.0151
+2026-03-18 16:35:22,158 [INFO] new_opacus_codex.train_steps: epoch=2 step=320 loss=1.0366
+2026-03-18 16:36:40,698 [INFO] new_opacus_codex.train_steps: epoch=2 step=330 loss=1.0433
+2026-03-18 16:38:01,641 [INFO] new_opacus_codex.train_steps: epoch=2 step=340 loss=1.0212
+2026-03-18 16:39:23,904 [INFO] new_opacus_codex.train_steps: epoch=2 step=350 loss=1.0581
+2026-03-18 16:39:44,883 [INFO] new_opacus_codex.train_steps: eval event=eval_step epoch=2 step=350 eval_loss=0.9456 duration_sec=20.95
+2026-03-18 16:41:03,938 [INFO] new_opacus_codex.train_steps: epoch=2 step=360 loss=1.0394

granite-4.0-h-tiny/dp3/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: ibm-granite/granite-4.0-h-tiny
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:ibm-granite/granite-4.0-h-tiny
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

granite-4.0-h-tiny/dp3/adapter/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": true,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head",
+    "embed_tokens"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

granite-4.0-h-tiny/dp3/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d79805fe2758c241625565c29cbb37723cab04fc94b1bdc961ba26c2801c079c
+size 1861530512

granite-4.0-h-tiny/dp3/adapter/added_tokens.json

@@ -0,0 +1,502 @@
+{
+  "114848338811": 100360,
+  "116427191984": 100714,
+  "116639746413": 100686,
+  "117298239631": 100576,
+  "117657531962": 100633,
+  "117783274348": 100650,
+  "119342152414": 100723,
+  "121557291859": 100582,
+  "123741461297": 100491,
+  "124589118494": 100487,
+  "125866399464": 100596,
+  "126574388585": 100538,
+  "129693547287": 100472,
+  "133373659361": 100667,
+  "135441121756": 100355,
+  "136931728327": 100457,
+  "138154781148": 100767,
+  "138218735276": 100651,
+  "141499982867": 100745,
+  "142526557681": 100845,
+  "142799718159": 100709,
+  "143182978129": 100671,
+  "143777992227": 100366,
+  "144373897488": 100777,
+  "145669731981": 100581,
+  "147444452794": 100830,
+  "147541537162": 100775,
+  "151859598873": 100547,
+  "152676556624": 100682,
+  "154962668224": 100696,
+  "158142752582": 100577,
+  "158929292238": 100587,
+  "159337222655": 100422,
+  "161238482259": 100790,
+  "161588262349": 100437,
+  "163165456396": 100788,
+  "163497196769": 100820,
+  "165429152138": 100717,
+  "172599315861": 100371,
+  "173781362446": 100423,
+  "175469284871": 100515,
+  "175528632226": 100833,
+  "178478368922": 100762,
+  "179386156569": 100804,
+  "179564995814": 100697,
+  "181233596124": 100782,
+  "181439694557": 100398,
+  "182345464792": 100442,
+  "182782912224": 100624,
+  "183368827562": 100551,
+  "185438575221": 100537,
+  "186198784585": 100598,
+  "186872457894": 100835,
+  "191887635211": 100721,
+  "195388849133": 100792,
+  "195792996468": 100546,
+  "196119929397": 100611,
+  "196735786156": 100493,
+  "198462689911": 100428,
+  "211411365397": 100377,
+  "211662639865": 100706,
+  "212985874612": 100720,
+  "214193645583": 100802,
+  "215671314196": 100602,
+  "215826764872": 100756,
+  "216667245274": 100484,
+  "217756494954": 100648,
+  "218391991184": 100679,
+  "218784516525": 100707,
+  "221762325616": 100678,
+  "225164373623": 100439,
+  "228358422847": 100669,
+  "228565148417": 100726,
+  "229775525672": 100793,
+  "231853711778": 100516,
+  "232776999554": 100443,
+  "232787996724": 100601,
+  "232896526252": 100489,
+  "233795848681": 100507,
+  "234316622259": 100625,
+  "242851284913": 100435,
+  "243165586174": 100784,
+  "244371196654": 100615,
+  "244885399387": 100533,
+  "245416776926": 100583,
+  "251789649997": 100613,
+  "252867443568": 100578,
+  "256497188281": 100735,
+  "259533577263": 100458,
+  "259666844669": 100434,
+  "262296912232": 100481,
+  "262437452884": 100794,
+  "264259917554": 100413,
+  "265847332563": 100818,
+  "267514771964": 100574,
+  "267954879268": 100646,
+  "268433315926": 100822,
+  "268921994538": 100715,
+  "272362973463": 100406,
+  "275597777299": 100565,
+  "281673864127": 100408,
+  "282322838685": 100373,
+  "284445431422": 100664,
+  "284776712475": 100780,
+  "286173241916": 100603,
+  "291762694874": 100670,
+  "293847485454": 100760,
+  "295118729589": 100609,
+  "295448593321": 100490,
+  "296953381169": 100479,
+  "298464134954": 100483,
+  "311632789559": 100391,
+  "313145133463": 100773,
+  "313313831273": 100815,
+  "314697386682": 100449,
+  "315534574435": 100754,
+  "319584985416": 100570,
+  "322589379462": 100680,
+  "322683334381": 100668,
+  "325196781219": 100844,
+  "327158398268": 100673,
+  "327491773134": 100387,
+  "328552864359": 100705,
+  "331369926711": 100731,
+  "332234613346": 100843,
+  "332522971941": 100683,
+  "334513529294": 100662,
+  "336813343298": 100747,
+  "337326533813": 100444,
+  "339882449689": 100597,
+  "342221452223": 100742,
+  "345271433112": 100657,
+  "349249721685": 100751,
+  "351885857332": 100426,
+  "353755529131": 100753,
+  "355977139358": 100379,
+  "356722692178": 100846,
+  "357299947518": 100832,
+  "357621613582": 100738,
+  "363214487524": 100532,
+  "364196983365": 100545,
+  "364492359246": 100695,
+  "364947325679": 100618,
+  "367459894796": 100356,
+  "367968398551": 100757,
+  "368763327262": 100799,
+  "371496153876": 100816,
+  "374327157578": 100390,
+  "374439448821": 100468,
+  "374598961236": 100512,
+  "375429755114": 100781,
+  "377273562477": 100480,
+  "377338713663": 100494,
+  "378749551722": 100789,
+  "379557332953": 100451,
+  "379571934299": 100612,
+  "383837596997": 100381,
+  "384598329253": 100616,
+  "387763951128": 100730,
+  "389715484387": 100759,
+  "393741199486": 100704,
+  "396521188476": 100665,
+  "396847976144": 100785,
+  "398549545767": 100502,
+  "399223169721": 100548,
+  "399749397872": 100471,
+  "412843856172": 100803,
+  "415956914763": 100629,
+  "417668946583": 100737,
+  "418412875665": 100698,
+  "421369185549": 100626,
+  "421435255895": 100666,
+  "425949483533": 100368,
+  "429781429464": 100453,
+  "431149748713": 100594,
+  "431751228368": 100534,
+  "433581868192": 100535,
+  "434632855397": 100627,
+  "438124184139": 100812,
+  "438435178228": 100739,
+  "438745114173": 100690,
+  "441373944925": 100463,
+  "441377892146": 100795,
+  "445925149649": 100364,
+  "447445629421": 100396,
+  "449318681694": 100847,
+  "449549215429": 100819,
+  "451161679736": 100676,
+  "451885565366": 100796,
+  "454164859837": 100656,
+  "454244839926": 100503,
+  "455423278865": 100703,
+  "457919547633": 100358,
+  "462294118988": 100658,
+  "467893531755": 100465,
+  "472129283475": 100749,
+  "473822473819": 100397,
+  "475794684356": 100556,
+  "476969195485": 100772,
+  "479227349997": 100456,
+  "479676316326": 100409,
+  "481884157827": 100416,
+  "482829231639": 100828,
+  "484489193824": 100529,
+  "484773717614": 100497,
+  "485177821815": 100354,
+  "485692541169": 100740,
+  "486391569221": 100536,
+  "488496781997": 100826,
+  "488538864842": 100459,
+  "489562189822": 100530,
+  "489991411364": 100461,
+  "492373714791": 100590,
+  "492516914298": 100769,
+  "497294727353": 100660,
+  "499148455254": 100462,
+  "499156317423": 100654,
+  "499444466686": 100448,
+  "511785788222": 100659,
+  "511923256573": 100672,
+  "512659818733": 100591,
+  "513349618943": 100685,
+  "513361312532": 100520,
+  "514354144759": 100750,
+  "514532995959": 100436,
+  "515856611931": 100655,
+  "516968424863": 100476,
+  "517379856925": 100450,
+  "517778845323": 100431,
+  "522515164754": 100778,
+  "523723475786": 100755,
+  "523918658846": 100392,
+  "523937669294": 100716,
+  "524444658383": 100649,
+  "527117488925": 100466,
+  "527874729936": 100728,
+  "529455466433": 100610,
+  "529679678956": 100420,
+  "529854679937": 100540,
+  "532833541879": 100642,
+  "533417736566": 100552,
+  "535787224917": 100425,
+  "538631466654": 100677,
+  "545512344383": 100447,
+  "546276298359": 100752,
+  "547149832394": 100482,
+  "547563892268": 100813,
+  "549946382131": 100663,
+  "552738847828": 100836,
+  "556814553861": 100827,
+  "556976448182": 100605,
+  "557163366873": 100776,
+  "557533689478": 100527,
+  "558837194455": 100787,
+  "562389434565": 100734,
+  "562716493614": 100732,
+  "563117729724": 100510,
+  "563365646341": 100571,
+  "564395222747": 100675,
+  "564544131355": 100614,
+  "569765693871": 100353,
+  "569853653352": 100758,
+  "571317457151": 100712,
+  "572154333646": 100486,
+  "578827344666": 100427,
+  "579184949249": 100418,
+  "581493616981": 100763,
+  "581976444672": 100588,
+  "582471228164": 100525,
+  "583527834464": 100825,
+  "585544633356": 100524,
+  "586855179568": 100496,
+  "587541149322": 100817,
+  "587787496812": 100417,
+  "589439538674": 100764,
+  "589547936288": 100531,
+  "591464355756": 100719,
+  "592538279337": 100399,
+  "594688742928": 100748,
+  "598916996413": 100630,
+  "599961512792": 100606,
+  "612128769512": 100389,
+  "612788593128": 100580,
+  "614583493135": 100841,
+  "614797267726": 100452,
+  "616248973917": 100542,
+  "618423867817": 100725,
+  "618878266619": 100829,
+  "619454958782": 100593,
+  "622553389126": 100693,
+  "624221476921": 100595,
+  "624266421831": 100701,
+  "625334487352": 100807,
+  "626748382482": 100837,
+  "632962475133": 100528,
+  "635127952696": 100713,
+  "635185846287": 100687,
+  "636324136426": 100801,
+  "637551427887": 100647,
+  "643489267333": 100372,
+  "643692393748": 100401,
+  "644234165531": 100623,
+  "645527857972": 100834,
+  "646979124677": 100410,
+  "649291669397": 100362,
+  "649516938598": 100810,
+  "652154162978": 100557,
+  "653573457168": 100492,
+  "655517954651": 100619,
+  "658481324922": 100415,
+  "661436365453": 100433,
+  "662426712668": 100517,
+  "664847713559": 100637,
+  "665661658354": 100554,
+  "668134915514": 100400,
+  "668514843129": 100559,
+  "669183782449": 100643,
+  "669689115557": 100386,
+  "673948786567": 100620,
+  "676539973863": 100440,
+  "676676697292": 100445,
+  "677896358599": 100499,
+  "678666359123": 100526,
+  "682564697312": 100380,
+  "687284547244": 100848,
+  "689285841825": 100473,
+  "689362171782": 100382,
+  "691315125346": 100635,
+  "691587454538": 100365,
+  "693281533643": 100555,
+  "694235616268": 100407,
+  "696651276628": 100402,
+  "697747991249": 100743,
+  "698218415444": 100378,
+  "699446724178": 100700,
+  "711412347158": 100518,
+  "711641973413": 100653,
+  "712992815957": 100521,
+  "714738141818": 100761,
+  "716763689965": 100744,
+  "717628619782": 100811,
+  "718726519731": 100370,
+  "718951851413": 100699,
+  "721737645236": 100455,
+  "722339992679": 100766,
+  "722618266242": 100429,
+  "724492262421": 100692,
+  "724883568652": 100475,
+  "724964923582": 100779,
+  "729116422489": 100727,
+  "729825168745": 100634,
+  "731681792655": 100566,
+  "733737988257": 100477,
+  "739246183345": 100718,
+  "739394546916": 100645,
+  "739667844957": 100733,
+  "739784153482": 100729,
+  "741757479784": 100539,
+  "742765629356": 100438,
+  "745977673725": 100604,
+  "746285987371": 100361,
+  "749665846687": 100432,
+  "749966767285": 100383,
+  "752644929761": 100563,
+  "753159165157": 100384,
+  "753233719644": 100558,
+  "753466987363": 100519,
+  "754978243758": 100791,
+  "756916173536": 100805,
+  "757422791351": 100800,
+  "757785319772": 100768,
+  "758963111966": 100774,
+  "762486672996": 100708,
+  "762495822823": 100798,
+  "763531783491": 100543,
+  "765474393376": 100359,
+  "765737843639": 100514,
+  "765981581453": 100393,
+  "766294538337": 100607,
+  "767858144912": 100711,
+  "768461952319": 100636,
+  "771619544974": 100474,
+  "773671349113": 100631,
+  "773687856563": 100424,
+  "774679513717": 100823,
+  "778466723723": 100404,
+  "781653545886": 100374,
+  "786391632558": 100553,
+  "794825141891": 100394,
+  "795693824762": 100385,
+  "795838284624": 100421,
+  "796415361892": 100375,
+  "797238311233": 100851,
+  "799364566435": 100504,
+  "799417897197": 100584,
+  "811371928234": 100572,
+  "813981174797": 100689,
+  "814611949491": 100652,
+  "815458725125": 100702,
+  "816115936673": 100622,
+  "817378986438": 100506,
+  "818479319499": 100746,
+  "819162623892": 100560,
+  "824387273757": 100508,
+  "824488329416": 100640,
+  "825815739116": 100741,
+  "826214551218": 100441,
+  "829235161526": 100523,
+  "836746273134": 100414,
+  "837131396371": 100573,
+  "837873762491": 100641,
+  "839269684865": 100579,
+  "841824384614": 100488,
+  "841991688488": 100376,
+  "843998562287": 100485,
+  "846657726193": 100765,
+  "848845611563": 100500,
+  "851254219729": 100824,
+  "851852651677": 100501,
+  "852347289392": 100478,
+  "853859348234": 100403,
+  "856317483891": 100814,
+  "857231548835": 100621,
+  "858146415154": 100710,
+  "859289776479": 100840,
+  "859927364588": 100736,
+  "861342291298": 100821,
+  "864718285734": 100568,
+  "865331112869": 100352,
+  "866519986723": 100661,
+  "867678158885": 100470,
+  "873898736873": 100395,
+  "876537814263": 100681,
+  "877385381798": 100562,
+  "877482678543": 100357,
+  "879425227932": 100469,
+  "879764613759": 100674,
+  "881112998642": 100592,
+  "881136466196": 100564,
+  "883685933161": 100412,
+  "884354318946": 100684,
+  "886464195129": 100639,
+  "887417525236": 100809,
+  "887753195844": 100617,
+  "889965243135": 100694,
+  "892336777248": 100586,
+  "892697125149": 100806,
+  "893974619684": 100850,
+  "894885246797": 100498,
+  "897784492783": 100589,
+  "899151413682": 100464,
+  "912193598348": 100419,
+  "916198946615": 100509,
+  "919181521716": 100770,
+  "921889371893": 100628,
+  "921894985642": 100838,
+  "922327493433": 100411,
+  "922466849763": 100454,
+  "922867396683": 100513,
+  "927914615679": 100363,
+  "928335588653": 100467,
+  "929185622831": 100446,
+  "929826356991": 100405,
+  "933775286797": 100842,
+  "934912591213": 100608,
+  "937495658852": 100460,
+  "938488813964": 100549,
+  "941169431859": 100561,
+  "941861857715": 100808,
+  "941976537588": 100644,
+  "943193294691": 100599,
+  "943878645871": 100839,
+  "944513428457": 100575,
+  "946444632795": 100724,
+  "947262995132": 100849,
+  "948585673431": 100783,
+  "949235426629": 100831,
+  "951794811935": 100511,
+  "952465217972": 100632,
+  "952872416512": 100430,
+  "953267689786": 100638,
+  "955456118295": 100544,
+  "955668961269": 100600,
+  "961133895172": 100567,
+  "961981959227": 100550,
+  "963852916563": 100569,
+  "964342468552": 100495,
+  "967114116556": 100505,
+  "967214577653": 100722,
+  "971621888444": 100522,
+  "981277526855": 100797,
+  "982993456429": 100369,
+  "983493951323": 100691,
+  "983569279932": 100388,
+  "985781138935": 100771,
+  "993832498489": 100688,
+  "996116119839": 100541,
+  "997724962668": 100786,
+  "997958836435": 100585,
+  "997981281989": 100367
+}

granite-4.0-h-tiny/dp3/adapter/chat_template.jinja

@@ -0,0 +1,118 @@
+{%- set tools_system_message_prefix = 'You are a helpful assistant with access to the following tools. You may call one or more tools to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>'  %}
+{%- set tools_system_message_suffix = '\n</tools>\n\nFor each tool call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.' %}
+{%- set documents_system_message_prefix = 'You are a helpful assistant with access to the following documents. You may use one or more documents to assist with the user query.\n\nYou are given a list of documents within <documents></documents> XML tags:\n<documents>' %}
+{%- set documents_system_message_suffix = '\n</documents>\n\nWrite the response to the user\'s input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.' %}
+{%- set g4_default_system_message = 'You are a helpful assistant. Please ensure responses are professional, accurate, and safe.' %}
+{%- if available_tools is defined and available_tools %}
+    {%- set tools = available_tools %}
+{%- endif %}
+{%- set ns = namespace(tools_system_message=tools_system_message_prefix,
+                       documents_system_message=documents_system_message_prefix,
+                       default_system_message=g4_default_system_message,
+                       system_message=''
+                       ) %}
+{%- if tools %}
+    {%- for tool in tools %}
+        {%- set ns.tools_system_message = ns.tools_system_message + '\n' + (tool | tojson) %}
+    {%- endfor %}
+    {%- set ns.tools_system_message = ns.tools_system_message + tools_system_message_suffix %}
+{%- else %}
+    {%- set ns.tools_system_message = '' %}
+{%- endif %}
+{%- if documents %}
+    {%- for document in documents %}
+        {%- set ns.documents_system_message = ns.documents_system_message + '\n' + (document | tojson) %}
+    {%- endfor %}
+    {%- set ns.documents_system_message = ns.documents_system_message + documents_system_message_suffix %}
+{%- else %}
+    {%- set ns.documents_system_message = '' %}
+{%- endif %}
+{%- if messages[0].role == 'system' %}
+    {%- if messages[0].content is string %}
+        {%- set ns.system_message = messages[0].content %}
+    {%- elif messages[0].content is iterable %}
+        {%- for entry in messages[0].content %}
+            {%- if entry.type== 'text' %}
+                {%- if ns.system_message != '' %}
+                    {%- set ns.system_message = ns.system_message + '\n' %}
+                {%- endif %}
+                {%- set ns.system_message = ns.system_message + entry.text %}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+    {%- if tools and documents %}
+        {%- set ns.system_message = ns.system_message + '\n\n' +  ns.tools_system_message + '\n\n' + ns.documents_system_message %}
+    {%- elif tools %}
+        {%- set ns.system_message = ns.system_message + '\n\n' + ns.tools_system_message %}
+    {%- elif documents %}
+        {%- set ns.system_message = ns.system_message + '\n\n' + ns.documents_system_message %}
+    {%- endif %}
+{%- else %}
+    {%- if tools and documents %}
+        {%- set ns.system_message = ns.tools_system_message + '\n\n' + ns.documents_system_message  %}
+    {%- elif tools %}
+        {%- set ns.system_message = ns.tools_system_message %}
+    {%- elif documents %}
+        {%- set ns.system_message = ns.documents_system_message %}
+    {%- endif %}
+{%- endif %}
+{%- if ns.system_message %}
+    {{- '<|start_of_role|>system<|end_of_role|>' + ns.system_message + '<|end_of_text|>\n' }}
+{%- else %}
+    {{- '<|start_of_role|>system<|end_of_role|>' + ns.default_system_message + '<|end_of_text|>\n' }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = namespace(val='') %}
+    {%- if message.content is string %}
+        {%- set content.val = message.content %}
+    {%- else %}
+        {%- if message.content is iterable %}
+            {%- for entry in message.content %}
+                {%- if entry.type== 'text' %}
+                    {%- if content.val != '' %}
+                        {%- set content.val = content.val + '\n' %}
+                    {%- endif %}
+                    {%- set content.val = content.val + entry.text %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- endif %}
+    {%- if (message.role == 'user') or (message.role == 'system' and not loop.first) %}
+        {{- '<|start_of_role|>' + message.role + '<|end_of_role|>' + content.val + '<|end_of_text|>\n' }}
+    {%- elif message.role == 'assistant' %}
+        {{- '<|start_of_role|>' + message.role + '<|end_of_role|>' + content.val }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content.val) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|end_of_text|>\n' }}
+    {%- elif message.role == 'tool' %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != 'tool') %}
+            {{- '<|start_of_role|>user<|end_of_role|>' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content.val }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != 'tool') %}
+            {{- '<|end_of_text|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_of_role|>assistant<|end_of_role|>' }}
+{%- endif %}

granite-4.0-h-tiny/dp3/adapter/special_tokens_map.json

@@ -0,0 +1,3532 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "865331112869",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "569765693871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "485177821815",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "135441121756",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "367459894796",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "877482678543",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "457919547633",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765474393376",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "114848338811",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "746285987371",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "649291669397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "927914615679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "445925149649",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "691587454538",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "143777992227",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997981281989",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "425949483533",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "982993456429",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "718726519731",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "172599315861",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "643489267333",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "282322838685",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "781653545886",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "796415361892",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "841991688488",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "211411365397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "698218415444",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "355977139358",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "682564697312",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "383837596997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "689362171782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "749966767285",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753159165157",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "795693824762",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "669689115557",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "327491773134",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "983569279932",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "612128769512",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374327157578",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "311632789559",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523918658846",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765981581453",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "794825141891",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "873898736873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "447445629421",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "473822473819",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "181439694557",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "592538279337",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "668134915514",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "643692393748",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "696651276628",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "853859348234",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "778466723723",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "929826356991",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "272362973463",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "694235616268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "281673864127",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "479676316326",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "646979124677",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922327493433",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "883685933161",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "264259917554",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "836746273134",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "658481324922",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "481884157827",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "587787496812",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "579184949249",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "912193598348",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529679678956",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "795838284624",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "159337222655",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "173781362446",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "773687856563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "535787224917",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "351885857332",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "578827344666",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "198462689911",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "722618266242",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "952872416512",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "517778845323",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "749665846687",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "661436365453",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "259666844669",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "242851284913",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "514532995959",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "161588262349",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "742765629356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "225164373623",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "676539973863",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "826214551218",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "182345464792",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232776999554",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "337326533813",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "676676697292",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "929185622831",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "545512344383",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499444466686",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "314697386682",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "517379856925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "379557332953",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "614797267726",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "429781429464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922466849763",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "721737645236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "479227349997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "136931728327",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "259533577263",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "488538864842",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "937495658852",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "489991411364",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499148455254",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "441373944925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "899151413682",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "467893531755",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "527117488925",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "928335588653",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374439448821",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "879425227932",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "867678158885",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "399749397872",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "129693547287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "689285841825",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "771619544974",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724883568652",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "516968424863",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "733737988257",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "852347289392",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "296953381169",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "377273562477",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "262296912232",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "547149832394",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "298464134954",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "216667245274",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "843998562287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "572154333646",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "124589118494",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "841824384614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232896526252",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "295448593321",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "123741461297",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "653573457168",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "196735786156",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "377338713663",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "964342468552",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "586855179568",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "484773717614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "894885246797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "677896358599",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "848845611563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "851852651677",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "398549545767",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "454244839926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "799364566435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "967114116556",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "817378986438",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "233795848681",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "824387273757",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "916198946615",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "563117729724",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "951794811935",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "374598961236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "922867396683",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "765737843639",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "175469284871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "231853711778",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "662426712668",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "711412347158",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753466987363",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "513361312532",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "712992815957",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "971621888444",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "829235161526",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "585544633356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "582471228164",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "678666359123",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "557533689478",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "632962475133",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "484489193824",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "489562189822",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "589547936288",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "363214487524",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "244885399387",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "431751228368",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "433581868192",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "486391569221",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "185438575221",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "126574388585",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "741757479784",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529854679937",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "996116119839",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "616248973917",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "763531783491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "955456118295",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364196983365",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "195792996468",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "151859598873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "399223169721",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "938488813964",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "961981959227",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "183368827562",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "533417736566",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "786391632558",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "665661658354",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "693281533643",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "475794684356",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "652154162978",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "753233719644",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "668514843129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "819162623892",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941169431859",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "877385381798",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "752644929761",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "881136466196",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "275597777299",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "731681792655",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "961133895172",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "864718285734",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "963852916563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "319584985416",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "563365646341",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "811371928234",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "837131396371",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "267514771964",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "944513428457",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117298239631",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "158142752582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "252867443568",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "839269684865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "612788593128",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "145669731981",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "121557291859",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "245416776926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "799417897197",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997958836435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "892336777248",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "158929292238",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "581976444672",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "897784492783",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "492373714791",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "512659818733",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "881112998642",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "619454958782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "431149748713",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "624221476921",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "125866399464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "339882449689",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "186198784585",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "943193294691",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "955668961269",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "232787996724",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "215671314196",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "286173241916",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "745977673725",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "556976448182",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "599961512792",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "766294538337",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "934912591213",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "295118729589",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "529455466433",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "196119929397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "379571934299",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "251789649997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "564544131355",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "244371196654",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "384598329253",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "887753195844",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364947325679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "655517954651",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "673948786567",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "857231548835",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "816115936673",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "644234165531",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "182782912224",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "234316622259",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "421369185549",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "434632855397",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "921889371893",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "415956914763",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "598916996413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "773671349113",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "952465217972",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117657531962",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "729825168745",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "691315125346",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "768461952319",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "664847713559",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "953267689786",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "886464195129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "824488329416",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "837873762491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "532833541879",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "669183782449",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941976537588",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739394546916",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "267954879268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "637551427887",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "217756494954",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "524444658383",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "117783274348",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "138218735276",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "814611949491",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "711641973413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "499156317423",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "515856611931",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "454164859837",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "345271433112",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "462294118988",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "511785788222",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "497294727353",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "866519986723",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "334513529294",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "549946382131",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "284445431422",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "396521188476",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "421435255895",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "133373659361",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "322683334381",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "228358422847",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "291762694874",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "143182978129",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "511923256573",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "327158398268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "879764613759",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "564395222747",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "451161679736",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "538631466654",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "221762325616",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "218391991184",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "322589379462",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "876537814263",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "152676556624",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "332522971941",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "884354318946",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "513349618943",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "116639746413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "635185846287",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "993832498489",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "813981174797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438745114173",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "983493951323",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724492262421",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "622553389126",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "889965243135",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "364492359246",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "154962668224",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "179564995814",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "418412875665",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "718951851413",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "699446724178",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "624266421831",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "815458725125",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "455423278865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "393741199486",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "328552864359",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "211662639865",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "218784516525",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "762486672996",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "142799718159",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "858146415154",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "767858144912",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "571317457151",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "635127952696",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "116427191984",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "268921994538",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523937669294",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "165429152138",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739246183345",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "591464355756",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "212985874612",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "191887635211",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "967214577653",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "119342152414",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "946444632795",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "618423867817",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "228565148417",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "729116422489",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "527874729936",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739784153482",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "387763951128",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "331369926711",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "562716493614",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "739667844957",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "562389434565",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "256497188281",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "859927364588",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "417668946583",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "357621613582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438435178228",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "485692541169",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "825815739116",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "342221452223",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "697747991249",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "716763689965",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "141499982867",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "818479319499",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "336813343298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "594688742928",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "472129283475",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "514354144759",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "349249721685",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "546276298359",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "353755529131",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "315534574435",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "523723475786",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "215826764872",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "367968398551",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "569853653352",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "389715484387",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "293847485454",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "714738141818",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "178478368922",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "581493616981",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "589439538674",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "846657726193",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "722339992679",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "138154781148",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "757785319772",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "492516914298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "919181521716",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "985781138935",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "476969195485",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "313145133463",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "758963111966",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "147541537162",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "557163366873",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "144373897488",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "522515164754",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "724964923582",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "284776712475",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "375429755114",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "181233596124",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "948585673431",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "243165586174",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "396847976144",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "997724962668",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "558837194455",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "163165456396",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "378749551722",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "161238482259",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "754978243758",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "195388849133",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "229775525672",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "262437452884",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "441377892146",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "451885565366",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "981277526855",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "762495822823",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "368763327262",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "757422791351",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "636324136426",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "214193645583",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "412843856172",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "179386156569",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "756916173536",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "892697125149",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "625334487352",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "941861857715",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "887417525236",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "649516938598",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "717628619782",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "438124184139",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "547563892268",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "856317483891",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "313313831273",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "371496153876",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "587541149322",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "265847332563",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "449549215429",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "163497196769",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "861342291298",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "268433315926",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "774679513717",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "851254219729",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "583527834464",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "488496781997",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "556814553861",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "482829231639",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "618878266619",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "147444452794",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "949235426629",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "357299947518",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "175528632226",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "645527857972",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "186872457894",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "552738847828",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "626748382482",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "921894985642",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "943878645871",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "859289776479",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "614583493135",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "933775286797",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "332234613346",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "325196781219",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "142526557681",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "356722692178",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "449318681694",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "687284547244",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "947262995132",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "893974619684",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "797238311233",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|unk|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}