Upload folder using huggingface_hub

.gitattributes

@@ -1,2 +1,3 @@
 *.gguf filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

AutoDevvLM-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56d98bb098bbb73144c80db8b6871ed9b9ad1b6c82451fa6f064635c9489463f
+size 986048160

README.md

@@ -1,83 +1,172 @@
----
-license: mit
-library_name: transformers
-tags:
-  - text-generation
-  - tiny
-  - gpt2
-  - decoder-only
-  - gguf
-pipeline_tag: text-generation
----
-
-# AutodevvLM v0.1
-
-AutodevvLM v0.1 is a **~10.06M parameter decoder-only Transformer** trained from scratch on CPU.
-It follows GPT-2 conventions (fused QKV attention, learned position embeddings, pre-LN blocks)
-and is small enough to run comfortably on a laptop.
-
-It ships with both a **PyTorch/Hugging Face** checkpoint (`model.safetensors`, load with
-`trust_remote_code=True`) and **GGUF** builds for [llama.cpp](https://github.com/ggerganov/llama.cpp).
-
-## Model details
-
-| Property | Value |
-|---|---|
-| Parameters | 10,064,896 |
-| Architecture | Decoder-only Transformer (GPT-2 style) |
-| Layers | 10 |
-| Model dim (`n_embd`) | 256 |
-| Attention heads | 8 |
-| FFN inner dim | 1024 |
-| Vocab size | 4096 |
-| Context (`block_size`) | 256 |
-| Params dtype | float32 (safetensors) |
-
-## Files
-
-| File | Description |
-|---|---|
-| `config.json` | Model config (+ `auto_map` for `trust_remote_code`) |
-| `configuration_autodevlm.py` | `AutoDevLMTinyGPTConfig` (remote code) |
-| `modeling_autodevlm.py` | `AutoDevLMTinyGPTForCausalLM` (remote code) |
-| `model.safetensors` | Weights (float32, ~38 MB) |
-| `tokenizer.json` | Byte-level BPE tokenizer |
-| `tokenizer_config.json` | Tokenizer special tokens |
-| `autodevv-f16.gguf` | GGUF, F16 (~19.3 MB) |
-| `autodevv-q4_k_m.gguf` | GGUF, Q4_K_M (~6.5 MB) |
-
-## Usage (PyTorch / transformers)
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-import torch
-
-model = AutoModelForCausalLM.from_pretrained("<repo-id>", trust_remote_code=True)
-tok = AutoTokenizer.from_pretrained("<repo-id>")
-
-ids = tok("the tiny transformer", return_tensors="pt").input_ids
-out = model.generate_text(ids, max_new_tokens=30, temperature=0.8, seed=0)
-print(tok.decode(out[0]))
-```
-
-## Usage (llama.cpp)
-
-```bash
-llama-cli -m autodevv-q4_k_m.gguf -p "the tiny transformer" -n 30 --temp 0.8 -s 0
-```
-
-## Training
-
-- Trained from scratch on CPU (`src/train.py`, AdamW).
-- 50 steps: train loss 1.74 → 0.05, val loss 2.37 → 1.24.
-- Byte-level tokenizer (`ByteLevel(use_regex=False)`), vocab padded to 4096.
-
-## Intended use & limitations
-
-This is a **tiny educational model** trained on a very small corpus. Outputs are simple and
-domain-limited; it is not a general-purpose assistant and should be used for learning and
-experimentation only.
-
-## License
-
-MIT.
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-1.5B-Instruct
+datasets:
+  - m-a-p/CodeFeedback-Filtered-Instruction
+language:
+  - en
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+  - code
+  - coding-assistant
+  - qwen2
+  - qlora
+  - sft
+  - gguf
+  - llama.cpp
+  - ollama
+---
+
+# AutoDevvLM
+
+AutoDevvLM is a small, code-focused language model fine-tuned from [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct). It was trained with QLoRA on general code-instruction data and is distributed in **GGUF** format for local inference with llama.cpp, Ollama, LM Studio and similar tools.
+
+> **Note:** AutoDevvLM is a fine-tune of Qwen2.5-1.5B-Instruct. It is not an independent model, and it may still identify itself as Qwen unless given a system prompt (see [Usage](#usage)).
+
+## Model Details
+
+| | |
+|---|---|
+| **Developed by** | `<your-name-or-org>` |
+| **Base model** | `Qwen/Qwen2.5-1.5B-Instruct` |
+| **Architecture** | Qwen2 (`Qwen2ForCausalLM`), ~1.5B parameters |
+| **Context length** | 32,768 tokens in the architecture; training used sequences up to 1,024 tokens |
+| **Fine-tuning method** | Supervised fine-tuning (SFT) with QLoRA |
+| **Language** | English (primary) |
+| **License** | Apache 2.0 |
+| **Format** | GGUF (quantized) |
+
+## Files
+
+| File | Description | Approx. size |
+|---|---|---|
+| `AutoDevvLM-Q4_K_M.gguf` | 4-bit quantized, recommended for most local use | ~1 GB |
+| `AutoDevvLM-f16.gguf` *(optional)* | Full FP16 precision | ~3 GB |
+
+## Intended Use
+
+AutoDevvLM is intended for lightweight, local coding assistance:
+
+- Writing short functions and scripts
+- Explaining code and simple debugging
+- Answering general programming questions
+- Experimentation and learning about fine-tuning small models
+
+## Out-of-Scope Use
+
+- Safety-critical or production-critical code generation without human review
+- Security-sensitive tasks (generated code may contain vulnerabilities)
+- Factual, legal, medical or financial advice
+- Use as a general-purpose assistant for tasks far outside programming
+
+## Usage
+
+The model uses the Qwen (ChatML) chat format. The template is embedded in the GGUF, so most runtimes apply it automatically.
+
+### llama.cpp
+
+```bash
+llama-cli -m AutoDevvLM-Q4_K_M.gguf -cnv \
+  -sys "You are AutoDevvLM, a helpful coding assistant."
+```
+
+### Ollama
+
+Create a `Modelfile`:
+
+```
+FROM ./AutoDevvLM-Q4_K_M.gguf
+SYSTEM """You are AutoDevvLM, a coding assistant. Help with writing, debugging and explaining code."""
+PARAMETER temperature 0.2
+```
+
+Then:
+
+```bash
+ollama create autodevvlm -f Modelfile
+ollama run autodevvlm
+```
+
+### Python (llama-cpp-python)
+
+```python
+from llama_cpp import Llama
+
+llm = Llama(model_path="AutoDevvLM-Q4_K_M.gguf", n_ctx=4096)
+
+out = llm.create_chat_completion(
+    messages=[
+        {"role": "system", "content": "You are AutoDevvLM, a helpful coding assistant."},
+        {"role": "user", "content": "Write a Python function that checks whether a string is a valid IPv4 address."},
+    ],
+    max_tokens=400,
+    temperature=0.2,
+    top_p=0.95,
+)
+print(out["choices"][0]["message"]["content"])
+```
+
+**Suggested sampling settings:** temperature 0.2 to 0.4, top_p 0.95 for code.
+
+## Training Details
+
+### Training data
+
+[`m-a-p/CodeFeedback-Filtered-Instruction`](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) (Apache 2.0). A random subset of **8,000** examples was used after filtering out examples longer than 1,024 tokens, with a 2% hold-out for evaluation loss. Each example was formatted with the Qwen chat template (user query, assistant answer).
+
+### Training procedure
+
+| Setting | Value |
+|---|---|
+| Method | QLoRA (4-bit NF4 with double quantization, via `bitsandbytes`) |
+| Compute precision | FP16 (BF16 not used; NVIDIA T4 lacks native BF16) |
+| LoRA rank / alpha / dropout | 16 / 32 / 0.05 |
+| LoRA target modules | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` |
+| Epochs | 1 |
+| Batch size | 2 per device x 8 gradient accumulation (effective 16) |
+| Learning rate | 2e-4, cosine schedule, 3% warmup |
+| Optimizer | `paged_adamw_8bit` |
+| Max grad norm | 0.3 |
+| Max sequence length | 1,024 |
+| Gradient checkpointing | Enabled |
+| Hardware | Google Colab, 1x NVIDIA Tesla T4 (16 GB) |
+| Frameworks | Transformers, PEFT, TRL, bitsandbytes |
+
+### Export
+
+The LoRA adapter was merged into the FP16 base model, converted to GGUF with llama.cpp's `convert_hf_to_gguf.py`, and quantized to Q4_K_M with `llama-quantize`.
+
+## Evaluation
+
+No formal benchmark evaluation has been run for this model yet. Results on HumanEval, MBPP or similar benchmarks have not been measured, and no claim is made that it outperforms the base model. Users are encouraged to compare outputs against `Qwen2.5-1.5B-Instruct` on their own tasks.
+
+## Limitations and Risks
+
+- **Small model:** at 1.5B parameters it can produce incorrect, incomplete or insecure code, and may hallucinate APIs or library functions. Always review and test its output.
+- **Light fine-tune:** training used 8,000 examples for a single epoch, so behavior is close to the base model, with modest changes in code style and explanation format.
+- **Identity:** the training data contained no identity examples, so the model may state that it is Qwen. Use a system prompt to set the assistant name.
+- **Quantization:** Q4_K_M trades some quality for size; use the F16 file if you need maximum fidelity.
+- **Inherited biases:** the model inherits the limitations and biases of the base model and the training dataset.
+- **Language coverage:** performance is strongest in English and in popular programming languages.
+
+## License
+
+Released under the **Apache 2.0** license, the same as the base model and the training dataset. Please retain the license notice and attribution when redistributing.
+
+## Acknowledgements
+
+- [Qwen Team](https://huggingface.co/Qwen) for the base model, Qwen2.5-1.5B-Instruct
+- [m-a-p](https://huggingface.co/m-a-p) for the CodeFeedback-Filtered-Instruction dataset
+- [llama.cpp](https://github.com/ggml-org/llama.cpp), [PEFT](https://github.com/huggingface/peft), [TRL](https://github.com/huggingface/trl) and [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes)
+
+## Citation
+
+```bibtex
+@misc{autodevvlm2026,
+  title  = {AutoDevvLM: A QLoRA fine-tune of Qwen2.5-1.5B-Instruct for code assistance},
+  author = {<your-name-or-org>},
+  year   = {2026},
+  howpublished = {\url{https://huggingface.co/<your-username>/AutoDevvLM}}
+}
+```

adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "gate_proj",
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "down_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c68312b8745dceccbb3eff4e4c895a7b6cd1c5e41d02a6dbd802d8ae6fa6808
+size 36981856

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

tokenizer.json

@@ -1,2118 +1,3 @@
-{
-  "version": "1.0",
-  "truncation": null,
-  "padding": null,
-  "added_tokens": [
-    {
-      "id": 0,
-      "content": "<pad>",
-      "single_word": false,
-      "lstrip": false,
-      "rstrip": false,
-      "normalized": false,
-      "special": true
-    },
-    {
-      "id": 1,
-      "content": "<unk>",
-      "single_word": false,
-      "lstrip": false,
-      "rstrip": false,
-      "normalized": false,
-      "special": true
-    },
-    {
-      "id": 2,
-      "content": "<bos>",
-      "single_word": false,
-      "lstrip": false,
-      "rstrip": false,
-      "normalized": false,
-      "special": true
-    },
-    {
-      "id": 3,
-      "content": "<eos>",
-      "single_word": false,
-      "lstrip": false,
-      "rstrip": false,
-      "normalized": false,
-      "special": true
-    }
-  ],
-  "normalizer": null,
-  "pre_tokenizer": {
-    "type": "ByteLevel",
-    "add_prefix_space": false,
-    "trim_offsets": true,
-    "use_regex": false
-  },
-  "post_processor": null,
-  "decoder": {
-    "type": "ByteLevel",
-    "add_prefix_space": true,
-    "trim_offsets": true,
-    "use_regex": true
-  },
-  "model": {
-    "type": "BPE",
-    "dropout": null,
-    "unk_token": "<unk>",
-    "continuing_subword_prefix": null,
-    "end_of_word_suffix": null,
-    "fuse_unk": false,
-    "byte_fallback": false,
-    "ignore_merges": false,
-    "vocab": {
-      "<pad>": 0,
-      "<unk>": 1,
-      "<bos>": 2,
-      "<eos>": 3,
-      "-": 4,
-      ".": 5,
-      "2": 6,
-      "5": 7,
-      "6": 8,
-      "A": 9,
-      "B": 10,
-      "C": 11,
-      "E": 12,
-      "G": 13,
-      "L": 14,
-      "R": 15,
-      "S": 16,
-      "T": 17,
-      "V": 18,
-      "W": 19,
-      "a": 20,
-      "b": 21,
-      "c": 22,
-      "d": 23,
-      "e": 24,
-      "f": 25,
-      "g": 26,
-      "h": 27,
-      "i": 28,
-      "k": 29,
-      "l": 30,
-      "m": 31,
-      "n": 32,
-      "o": 33,
-      "p": 34,
-      "r": 35,
-      "s": 36,
-      "t": 37,
-      "u": 38,
-      "v": 39,
-      "w": 40,
-      "x": 41,
-      "y": 42,
-      "z": 43,
-      "Ċ": 44,
-      "č": 45,
-      "Ġ": 46,
-      "sĠ": 47,
-      ".č": 48,
-      ".čĊ": 49,
-      "eĠ": 50,
-      "in": 51,
-      "en": 52,
-      "er": 53,
-      "or": 54,
-      "al": 55,
-      "at": 56,
-      "Ġt": 57,
-      "es": 58,
-      "on": 59,
-      "Ġs": 60,
-      "esĠ": 61,
-      "heĠ": 62,
-      "ok": 63,
-      "ing": 64,
-      "le": 65,
-      "ra": 66,
-      "tĠ": 67,
-      "erĠ": 68,
-      "oken": 69,
-      "ex": 70,
-      "iz": 71,
-      "ion": 72,
-      "me": 73,
-      "Ġr": 74,
-      "Ġsm": 75,
-      "ar": 76,
-      "ct": 77,
-      "ec": 78,
-      "ep": 79,
-      "hĠ": 80,
-      "it": 81,
-      "ro": 82,
-      "t.čĊ": 83,
-      "theĠ": 84,
-      "token": 85,
-      "yĠ": 86,
-      "Ġa": 87,
-      "sĠs": 88,
-      "eĠp": 89,
-      "all": 90,
-      "ation": 91,
-      "lear": 92,
-      "Ġsmall": 93,
-      "learn": 94,
-      "TheĠ": 95,
-      "am": 96,
-      "an": 97,
-      "ay": 98,
-      "bl": 99,
-      "di": 100,
-      "dĠ": 101,
-      "dsĠ": 102,
-      "el": 103,
-      "eep": 104,
-      "fĠ": 105,
-      "for": 106,
-      "ig": 107,
-      "iv": 108,
-      "keep": 109,
-      "lit": 110,
-      "mor": 111,
-      "usĠ": 112,
-      "wor": 113,
-      "Ġat": 114,
-      "int": 115,
-      "ent": 116,
-      "ener": 117,
-      "entĠ": 118,
-      "Ġtoken": 119,
-      "Ġtex": 120,
-      "es.čĊ": 121,
-      "esĠc": 122,
-      "ingĠ": 123,
-      "ingĠt": 124,
-      "ingĠr": 125,
-      "me.čĊ": 126,
-      "tokeniz": 127,
-      "ayerĠ": 128,
-      "usĠi": 129,
-      "Ġtext.čĊ": 130,
-      "-le": 131,
-      "-ent": 132,
-      "25": 133,
-      "6Ġtoken": 134,
-      "Ad": 135,
-      "AĠsmall": 136,
-      "By": 137,
-      "Ch": 138,
-      "Ea": 139,
-      "Gra": 140,
-      "Gener": 141,
-      "LayerĠ": 142,
-      "Rep": 143,
-      "Sh": 144,
-      "Tra": 145,
-      "Val": 146,
-      "WĠa": 147,
-      "ap": 148,
-      "au": 149,
-      "av": 150,
-      "aĠ": 151,
-      "ara": 152,
-      "act": 153,
-      "abl": 154,
-      "bu": 155,
-      "byĠ": 156,
-      "bwor": 157,
-      "ce": 158,
-      "ck": 159,
-      "cor": 160,
-      "cal": 161,
-      "chĠ": 162,
-      "can": 163,
-      "centĠ": 164,
-      "cau": 165,
-      "dat": 166,
-      "des": 167,
-      "dec": 168,
-      "dation": 169,
-      "dingĠr": 170,
-      "dap": 171,
-      "ev": 172,
-      "eat": 173,
-      "edi": 174,
-      "eig": 175,
-      "e-le": 176,
-      "ebu": 177,
-      "gr": 178,
-      "ghĠ": 179,
-      "giv": 180,
-      "gener": 181,
-      "hes": 182,
-      "htĠ": 183,
-      "il": 184,
-      "im": 185,
-      "ix": 186,
-      "iec": 187,
-      "ime.čĊ": 188,
-      "ice": 189,
-      "idation": 190,
-      "kp": 191,
-      "lo": 192,
-      "lsĠ": 193,
-      "lin": 194,
-      "lesĠ": 195,
-      "layerĠ": 196,
-      "mĠ": 197,
-      "m.čĊ": 198,
-      "min": 199,
-      "merĠ": 200,
-      "mix": 201,
-      "ns": 202,
-      "nsĠ": 203,
-      "nor": 204,
-      "nex": 205,
-      "ou": 206,
-      "oĠs": 207,
-      "oĠr": 208,
-      "ofĠ": 209,
-      "oint": 210,
-      "ock": 211,
-      "odingĠr": 212,
-      "pr": 213,
-      "pyĠ": 214,
-      "plit": 215,
-      "pusĠi": 216,
-      "para": 217,
-      "pdat": 218,
-      "piec": 219,
-      "plesĠ": 220,
-      "ss": 221,
-      "ssĠ": 222,
-      "sal": 223,
-      "tsĠ": 224,
-      "tin": 225,
-      "ter": 226,
-      "tes": 227,
-      "tle": 228,
-      "tex": 229,
-      "tel": 230,
-      "tent": 231,
-      "tĠtoken": 232,
-      "tabl": 233,
-      "te-le": 234,
-      "ume.čĊ": 235,
-      "ubwor": 236,
-      "unsĠ": 237,
-      "updat": 238,
-      "vel": 239,
-      "wit": 240,
-      "weig": 241,
-      "yĠt": 242,
-      "yĠsmall": 243,
-      "Ġme": 244,
-      "Ġcor": 245,
-      "Ġdec": 246,
-      "Ġlo": 247,
-      "sĠen": 248,
-      "sĠkeep": 249,
-      "sĠwor": 250,
-      "sĠact": 251,
-      "sĠbyĠ": 252,
-      "sĠofĠ": 253,
-      "eĠtoken": 254,
-      "eĠan": 255,
-      "eĠlin": 256,
-      "eĠtes": 257,
-      "inal": 258,
-      "iningĠ": 259,
-      "erĠs": 260,
-      "eryĠ": 261,
-      "ortĠ": 262,
-      "orig": 263,
-      "aliz": 264,
-      "ateĠp": 265,
-      "Ġtime.čĊ": 266,
-      "essĠs": 267,
-      "esume.čĊ": 268,
-      "onĠsmall": 269,
-      "ontex": 270,
-      "oneĠtoken": 271,
-      "Ġsam": 272,
-      "esĠtheĠ": 273,
-      "esĠev": 274,
-      "esĠgiv": 275,
-      "heĠnex": 276,
-      "okeĠtes": 277,
-      "ract": 278,
-      "rans": 279,
-      "tĠwit": 280,
-      "erĠmor": 281,
-      "erĠpara": 282,
-      "izesĠc": 283,
-      "ion.čĊ": 284,
-      "meter": 285,
-      "Ġresume.čĊ": 286,
-      "ĠsmokeĠtes": 287,
-      "ctingĠt": 288,
-      "eckp": 289,
-      "hĠcau": 290,
-      "rogr": 291,
-      "ropyĠ": 292,
-      "ross": 293,
-      "theĠlearn": 294,
-      "theĠorig": 295,
-      "ĠaĠtime.čĊ": 296,
-      "ĠaĠsmokeĠtes": 297,
-      "sĠsav": 298,
-      "sĠstabl": 299,
-      "eĠpract": 300,
-      "eĠprogr": 301,
-      "ationĠsam": 302,
-      "ationsĠstabl": 303,
-      "learningĠ": 304,
-      "learnsĠbyĠ": 305,
-      "TheĠtokeniz": 306,
-      "TheĠtin": 307,
-      "amWĠa": 308,
-      "block": 309,
-      "dientĠ": 310,
-      "dĠcal": 311,
-      "dĠpiec": 312,
-      "dsĠint": 313,
-      "dsĠtheĠorig": 314,
-      "fĠlearningĠ": 315,
-      "formerĠ": 316,
-      "forĠaĠsmokeĠtes": 317,
-      "ivationsĠstabl": 318,
-      "keepsĠact": 319,
-      "little": 320,
-      "moryĠsmall": 321,
-      "Ġattent": 322,
-      "ĠatĠaĠtime.čĊ": 323,
-      "esĠcontex": 324,
-      "ingĠthes": 325,
-      "ingĠrateĠp": 326,
-      "tokenizerĠmor": 327,
-      "usĠifĠlearningĠ": 328,
-      "-entropyĠ": 329,
-      "256Ġtoken": 330,
-      "AdamWĠa": 331,
-      "AĠsmallĠcor": 332,
-      "Byte-le": 333,
-      "Checkp": 334,
-      "EachĠ": 335,
-      "GradientĠ": 336,
-      "GenerationĠsam": 337,
-      "LayerĠnor": 338,
-      "Repeat": 339,
-      "ShortĠ": 340,
-      "TrainingĠ": 341,
-      "Validation": 342,
-      "aĠlittle": 343,
-      "canĠresume.čĊ": 344,
-      "centĠupdat": 345,
-      "descentĠupdat": 346,
-      "daptsĠ": 347,
-      "edictingĠt": 348,
-      "ebuil": 349,
-      "ghĠforĠaĠsmokeĠtes": 350,
-      "generaliz": 351,
-      "htĠaĠlittle": 352,
-      "imizesĠc": 353,
-      "ice.čĊ": 354,
-      "lsĠusĠifĠlearningĠ": 355,
-      "layerĠmix": 356,
-      "mĠkeepsĠact": 357,
-      "minimizesĠc": 358,
-      "oughĠforĠaĠsmokeĠtes": 359,
-      "oĠsubwor": 360,
-      "oĠrunsĠ": 361,
-      "ointsĠsav": 362,
-      "odingĠrebuil": 363,
-      "predictingĠt": 364,
-      "plitsĠwor": 365,
-      "pusĠisĠen": 366,
-      "plesĠoneĠtoken": 367,
-      "ssĠtel": 368,
-      "salĠattent": 369,
-      "tĠtoken.čĊ": 370,
-      "velĠdec": 371,
-      "weightĠaĠlittle": 372,
-      "yĠtrans": 373,
-      "ĠmemoryĠsmall": 374,
-      "ĠlossĠtel": 375,
-      "sĠkeepĠmemoryĠsmall": 376,
-      "sĠofĠ256Ġtoken": 377,
-      "eĠandĠcal": 378,
-      "eĠlinesĠgiv": 379,
-      "inalĠtext.čĊ": 380,
-      "erĠsplitsĠwor": 381,
-      "eryĠweightĠaĠlittle": 382,
-      "essĠsoĠrunsĠ": 383,
-      "onĠsmallĠtext.čĊ": 384,
-      "esĠtheĠtokenizerĠmor": 385,
-      "esĠeveryĠweightĠaĠlittle": 386,
-      "heĠnextĠtoken.čĊ": 387,
-      "tĠwithĠcau": 388,
-      "erĠparameter": 389,
-      "ross-entropyĠ": 390,
-      "theĠlearningĠrateĠp": 391,
-      "eĠpractice.čĊ": 392,
-      "eĠprogressĠsoĠrunsĠ": 393,
-      "learnsĠbyĠpredictingĠt": 394,
-      "TheĠtokenizerĠsplitsĠwor": 395,
-      "TheĠtinyĠtrans": 396,
-      "blocksĠofĠ256Ġtoken": 397,
-      "dĠpieces.čĊ": 398,
-      "dsĠintoĠsubwor": 399,
-      "dsĠtheĠoriginalĠtext.čĊ": 400,
-      "formerĠlearnsĠbyĠpredictingĠt": 401,
-      "ivationsĠstableĠandĠcal": 402,
-      "esĠcontextĠwithĠcau": 403,
-      "ingĠtheseĠlinesĠgiv": 404,
-      "AdamWĠadaptsĠ": 405,
-      "AĠsmallĠcorpusĠisĠen": 406,
-      "Byte-levelĠdec": 407,
-      "CheckpointsĠsav": 408,
-      "EachĠlayerĠmix": 409,
-      "GradientĠdescentĠupdat": 410,
-      "GenerationĠsamplesĠoneĠtoken": 411,
-      "LayerĠnormĠkeepsĠact": 412,
-      "RepeatingĠtheseĠlinesĠgiv": 413,
-      "ShortĠblocksĠofĠ256Ġtoken": 414,
-      "TrainingĠminimizesĠc": 415,
-      "ValidationĠlossĠtel": 416,
-      "generalizes.čĊ": 417,
-      "lsĠusĠifĠlearningĠgeneralizes.čĊ": 418,
-      "oughĠforĠaĠsmokeĠtest.čĊ": 419,
-      "odingĠrebuildsĠtheĠoriginalĠtext.čĊ": 420,
-      "salĠattention.čĊ": 421,
-      "sĠkeepĠmemoryĠsmall.čĊ": 422,
-      "esĠtheĠtokenizerĠmoreĠpractice.čĊ": 423,
-      "esĠeveryĠweightĠaĠlittle.čĊ": 424,
-      "erĠparameter.čĊ": 425,
-      "ross-entropyĠonĠsmallĠtext.čĊ": 426,
-      "theĠlearningĠrateĠperĠparameter.čĊ": 427,
-      "eĠprogressĠsoĠrunsĠcanĠresume.čĊ": 428,
-      "TheĠtokenizerĠsplitsĠwordsĠintoĠsubwor": 429,
-      "TheĠtinyĠtransformerĠlearnsĠbyĠpredictingĠt": 430,
-      "ivationsĠstableĠandĠcalm.čĊ": 431,
-      "esĠcontextĠwithĠcausalĠattention.čĊ": 432,
-      "AdamWĠadaptsĠtheĠlearningĠrateĠperĠparameter.čĊ": 433,
-      "AĠsmallĠcorpusĠisĠenoughĠforĠaĠsmokeĠtest.čĊ": 434,
-      "Byte-levelĠdecodingĠrebuildsĠtheĠoriginalĠtext.čĊ": 435,
-      "CheckpointsĠsaveĠprogressĠsoĠrunsĠcanĠresume.čĊ": 436,
-      "EachĠlayerĠmixesĠcontextĠwithĠcausalĠattention.čĊ": 437,
-      "GradientĠdescentĠupdatesĠeveryĠweightĠaĠlittle.čĊ": 438,
-      "GenerationĠsamplesĠoneĠtokenĠatĠaĠtime.čĊ": 439,
-      "LayerĠnormĠkeepsĠactivationsĠstableĠandĠcalm.čĊ": 440,
-      "RepeatingĠtheseĠlinesĠgivesĠtheĠtokenizerĠmoreĠpractice.čĊ": 441,
-      "ShortĠblocksĠofĠ256ĠtokensĠkeepĠmemoryĠsmall.čĊ": 442,
-      "TrainingĠminimizesĠcross-entropyĠonĠsmallĠtext.čĊ": 443,
-      "ValidationĠlossĠtellsĠusĠifĠlearningĠgeneralizes.čĊ": 444,
-      "TheĠtokenizerĠsplitsĠwordsĠintoĠsubwordĠpieces.čĊ": 445,
-      "TheĠtinyĠtransformerĠlearnsĠbyĠpredictingĠtheĠnextĠtoken.čĊ": 446
-    },
-    "merges": [
-      [
-        "s",
-        "Ġ"
-      ],
-      [
-        ".",
-        "č"
-      ],
-      [
-        ".č",
-        "Ċ"
-      ],
-      [
-        "e",
-        "Ġ"
-      ],
-      [
-        "i",
-        "n"
-      ],
-      [
-        "e",
-        "n"
-      ],
-      [
-        "e",
-        "r"
-      ],
-      [
-        "o",
-        "r"
-      ],
-      [
-        "a",
-        "l"
-      ],
-      [
-        "a",
-        "t"
-      ],
-      [
-        "Ġ",
-        "t"
-      ],
-      [
-        "e",
-        "s"
-      ],
-      [
-        "o",
-        "n"
-      ],
-      [
-        "Ġ",
-        "s"
-      ],
-      [
-        "e",
-        "sĠ"
-      ],
-      [
-        "h",
-        "eĠ"
-      ],
-      [
-        "o",
-        "k"
-      ],
-      [
-        "in",
-        "g"
-      ],
-      [
-        "l",
-        "e"
-      ],
-      [
-        "r",
-        "a"
-      ],
-      [
-        "t",
-        "Ġ"
-      ],
-      [
-        "er",
-        "Ġ"
-      ],
-      [
-        "ok",
-        "en"
-      ],
-      [
-        "e",
-        "x"
-      ],
-      [
-        "i",
-        "z"
-      ],
-      [
-        "i",
-        "on"
-      ],
-      [
-        "m",
-        "e"
-      ],
-      [
-        "Ġ",
-        "r"
-      ],
-      [
-        "Ġs",
-        "m"
-      ],
-      [
-        "a",
-        "r"
-      ],
-      [
-        "c",
-        "t"
-      ],
-      [
-        "e",
-        "c"
-      ],
-      [
-        "e",
-        "p"
-      ],
-      [
-        "h",
-        "Ġ"
-      ],
-      [
-        "i",
-        "t"
-      ],
-      [
-        "r",
-        "o"
-      ],
-      [
-        "t",
-        ".čĊ"
-      ],
-      [
-        "t",
-        "heĠ"
-      ],
-      [
-        "t",
-        "oken"
-      ],
-      [
-        "y",
-        "Ġ"
-      ],
-      [
-        "Ġ",
-        "a"
-      ],
-      [
-        "sĠ",
-        "s"
-      ],
-      [
-        "eĠ",
-        "p"
-      ],
-      [
-        "al",
-        "l"
-      ],
-      [
-        "at",
-        "ion"
-      ],
-      [
-        "le",
-        "ar"
-      ],
-      [
-        "Ġsm",
-        "all"
-      ],
-      [
-        "lear",
-        "n"
-      ],
-      [
-        "T",
-        "heĠ"
-      ],
-      [
-        "a",
-        "m"
-      ],
-      [
-        "a",
-        "n"
-      ],
-      [
-        "a",
-        "y"
-      ],
-      [
-        "b",
-        "l"
-      ],
-      [
-        "d",
-        "i"
-      ],
-      [
-        "d",
-        "Ġ"
-      ],
-      [
-        "d",
-        "sĠ"
-      ],
-      [
-        "e",
-        "l"
-      ],
-      [
-        "e",
-        "ep"
-      ],
-      [
-        "f",
-        "Ġ"
-      ],
-      [
-        "f",
-        "or"
-      ],
-      [
-        "i",
-        "g"
-      ],
-      [
-        "i",
-        "v"
-      ],
-      [
-        "k",
-        "eep"
-      ],
-      [
-        "l",
-        "it"
-      ],
-      [
-        "m",
-        "or"
-      ],
-      [
-        "u",
-        "sĠ"
-      ],
-      [
-        "w",
-        "or"
-      ],
-      [
-        "Ġ",
-        "at"
-      ],
-      [
-        "in",
-        "t"
-      ],
-      [
-        "en",
-        "t"
-      ],
-      [
-        "en",
-        "er"
-      ],
-      [
-        "en",
-        "tĠ"
-      ],
-      [
-        "Ġt",
-        "oken"
-      ],
-      [
-        "Ġt",
-        "ex"
-      ],
-      [
-        "es",
-        ".čĊ"
-      ],
-      [
-        "esĠ",
-        "c"
-      ],
-      [
-        "ing",
-        "Ġ"
-      ],
-      [
-        "ing",
-        "Ġt"
-      ],
-      [
-        "ing",
-        "Ġr"
-      ],
-      [
-        "me",
-        ".čĊ"
-      ],
-      [
-        "token",
-        "iz"
-      ],
-      [
-        "ay",
-        "erĠ"
-      ],
-      [
-        "usĠ",
-        "i"
-      ],
-      [
-        "Ġtex",
-        "t.čĊ"
-      ],
-      [
-        "-",
-        "le"
-      ],
-      [
-        "-",
-        "ent"
-      ],
-      [
-        "2",
-        "5"
-      ],
-      [
-        "6",
-        "Ġtoken"
-      ],
-      [
-        "A",
-        "d"
-      ],
-      [
-        "A",
-        "Ġsmall"
-      ],
-      [
-        "B",
-        "y"
-      ],
-      [
-        "C",
-        "h"
-      ],
-      [
-        "E",
-        "a"
-      ],
-      [
-        "G",
-        "ra"
-      ],
-      [
-        "G",
-        "ener"
-      ],
-      [
-        "L",
-        "ayerĠ"
-      ],
-      [
-        "R",
-        "ep"
-      ],
-      [
-        "S",
-        "h"
-      ],
-      [
-        "T",
-        "ra"
-      ],
-      [
-        "V",
-        "al"
-      ],
-      [
-        "W",
-        "Ġa"
-      ],
-      [
-        "a",
-        "p"
-      ],
-      [
-        "a",
-        "u"
-      ],
-      [
-        "a",
-        "v"
-      ],
-      [
-        "a",
-        "Ġ"
-      ],
-      [
-        "a",
-        "ra"
-      ],
-      [
-        "a",
-        "ct"
-      ],
-      [
-        "a",
-        "bl"
-      ],
-      [
-        "b",
-        "u"
-      ],
-      [
-        "b",
-        "yĠ"
-      ],
-      [
-        "b",
-        "wor"
-      ],
-      [
-        "c",
-        "e"
-      ],
-      [
-        "c",
-        "k"
-      ],
-      [
-        "c",
-        "or"
-      ],
-      [
-        "c",
-        "al"
-      ],
-      [
-        "c",
-        "hĠ"
-      ],
-      [
-        "c",
-        "an"
-      ],
-      [
-        "c",
-        "entĠ"
-      ],
-      [
-        "c",
-        "au"
-      ],
-      [
-        "d",
-        "at"
-      ],
-      [
-        "d",
-        "es"
-      ],
-      [
-        "d",
-        "ec"
-      ],
-      [
-        "d",
-        "ation"
-      ],
-      [
-        "d",
-        "ingĠr"
-      ],
-      [
-        "d",
-        "ap"
-      ],
-      [
-        "e",
-        "v"
-      ],
-      [
-        "e",
-        "at"
-      ],
-      [
-        "e",
-        "di"
-      ],
-      [
-        "e",
-        "ig"
-      ],
-      [
-        "e",
-        "-le"
-      ],
-      [
-        "e",
-        "bu"
-      ],
-      [
-        "g",
-        "r"
-      ],
-      [
-        "g",
-        "hĠ"
-      ],
-      [
-        "g",
-        "iv"
-      ],
-      [
-        "g",
-        "ener"
-      ],
-      [
-        "h",
-        "es"
-      ],
-      [
-        "h",
-        "tĠ"
-      ],
-      [
-        "i",
-        "l"
-      ],
-      [
-        "i",
-        "m"
-      ],
-      [
-        "i",
-        "x"
-      ],
-      [
-        "i",
-        "ec"
-      ],
-      [
-        "i",
-        "me.čĊ"
-      ],
-      [
-        "i",
-        "ce"
-      ],
-      [
-        "i",
-        "dation"
-      ],
-      [
-        "k",
-        "p"
-      ],
-      [
-        "l",
-        "o"
-      ],
-      [
-        "l",
-        "sĠ"
-      ],
-      [
-        "l",
-        "in"
-      ],
-      [
-        "l",
-        "esĠ"
-      ],
-      [
-        "l",
-        "ayerĠ"
-      ],
-      [
-        "m",
-        "Ġ"
-      ],
-      [
-        "m",
-        ".čĊ"
-      ],
-      [
-        "m",
-        "in"
-      ],
-      [
-        "m",
-        "erĠ"
-      ],
-      [
-        "m",
-        "ix"
-      ],
-      [
-        "n",
-        "s"
-      ],
-      [
-        "n",
-        "sĠ"
-      ],
-      [
-        "n",
-        "or"
-      ],
-      [
-        "n",
-        "ex"
-      ],
-      [
-        "o",
-        "u"
-      ],
-      [
-        "o",
-        "Ġs"
-      ],
-      [
-        "o",
-        "Ġr"
-      ],
-      [
-        "o",
-        "fĠ"
-      ],
-      [
-        "o",
-        "int"
-      ],
-      [
-        "o",
-        "ck"
-      ],
-      [
-        "o",
-        "dingĠr"
-      ],
-      [
-        "p",
-        "r"
-      ],
-      [
-        "p",
-        "yĠ"
-      ],
-      [
-        "p",
-        "lit"
-      ],
-      [
-        "p",
-        "usĠi"
-      ],
-      [
-        "p",
-        "ara"
-      ],
-      [
-        "p",
-        "dat"
-      ],
-      [
-        "p",
-        "iec"
-      ],
-      [
-        "p",
-        "lesĠ"
-      ],
-      [
-        "s",
-        "s"
-      ],
-      [
-        "s",
-        "sĠ"
-      ],
-      [
-        "s",
-        "al"
-      ],
-      [
-        "t",
-        "sĠ"
-      ],
-      [
-        "t",
-        "in"
-      ],
-      [
-        "t",
-        "er"
-      ],
-      [
-        "t",
-        "es"
-      ],
-      [
-        "t",
-        "le"
-      ],
-      [
-        "t",
-        "ex"
-      ],
-      [
-        "t",
-        "el"
-      ],
-      [
-        "t",
-        "ent"
-      ],
-      [
-        "t",
-        "Ġtoken"
-      ],
-      [
-        "t",
-        "abl"
-      ],
-      [
-        "t",
-        "e-le"
-      ],
-      [
-        "u",
-        "me.čĊ"
-      ],
-      [
-        "u",
-        "bwor"
-      ],
-      [
-        "u",
-        "nsĠ"
-      ],
-      [
-        "u",
-        "pdat"
-      ],
-      [
-        "v",
-        "el"
-      ],
-      [
-        "w",
-        "it"
-      ],
-      [
-        "w",
-        "eig"
-      ],
-      [
-        "y",
-        "Ġt"
-      ],
-      [
-        "y",
-        "Ġsmall"
-      ],
-      [
-        "Ġ",
-        "me"
-      ],
-      [
-        "Ġ",
-        "cor"
-      ],
-      [
-        "Ġ",
-        "dec"
-      ],
-      [
-        "Ġ",
-        "lo"
-      ],
-      [
-        "sĠ",
-        "en"
-      ],
-      [
-        "sĠ",
-        "keep"
-      ],
-      [
-        "sĠ",
-        "wor"
-      ],
-      [
-        "sĠ",
-        "act"
-      ],
-      [
-        "sĠ",
-        "byĠ"
-      ],
-      [
-        "sĠ",
-        "ofĠ"
-      ],
-      [
-        "eĠ",
-        "token"
-      ],
-      [
-        "eĠ",
-        "an"
-      ],
-      [
-        "eĠ",
-        "lin"
-      ],
-      [
-        "eĠ",
-        "tes"
-      ],
-      [
-        "in",
-        "al"
-      ],
-      [
-        "in",
-        "ingĠ"
-      ],
-      [
-        "er",
-        "Ġs"
-      ],
-      [
-        "er",
-        "yĠ"
-      ],
-      [
-        "or",
-        "tĠ"
-      ],
-      [
-        "or",
-        "ig"
-      ],
-      [
-        "al",
-        "iz"
-      ],
-      [
-        "at",
-        "eĠp"
-      ],
-      [
-        "Ġt",
-        "ime.čĊ"
-      ],
-      [
-        "es",
-        "sĠs"
-      ],
-      [
-        "es",
-        "ume.čĊ"
-      ],
-      [
-        "on",
-        "Ġsmall"
-      ],
-      [
-        "on",
-        "tex"
-      ],
-      [
-        "on",
-        "eĠtoken"
-      ],
-      [
-        "Ġs",
-        "am"
-      ],
-      [
-        "esĠ",
-        "theĠ"
-      ],
-      [
-        "esĠ",
-        "ev"
-      ],
-      [
-        "esĠ",
-        "giv"
-      ],
-      [
-        "heĠ",
-        "nex"
-      ],
-      [
-        "ok",
-        "eĠtes"
-      ],
-      [
-        "ra",
-        "ct"
-      ],
-      [
-        "ra",
-        "ns"
-      ],
-      [
-        "tĠ",
-        "wit"
-      ],
-      [
-        "erĠ",
-        "mor"
-      ],
-      [
-        "erĠ",
-        "para"
-      ],
-      [
-        "iz",
-        "esĠc"
-      ],
-      [
-        "ion",
-        ".čĊ"
-      ],
-      [
-        "me",
-        "ter"
-      ],
-      [
-        "Ġr",
-        "esume.čĊ"
-      ],
-      [
-        "Ġsm",
-        "okeĠtes"
-      ],
-      [
-        "ct",
-        "ingĠt"
-      ],
-      [
-        "ec",
-        "kp"
-      ],
-      [
-        "hĠ",
-        "cau"
-      ],
-      [
-        "ro",
-        "gr"
-      ],
-      [
-        "ro",
-        "pyĠ"
-      ],
-      [
-        "ro",
-        "ss"
-      ],
-      [
-        "theĠ",
-        "learn"
-      ],
-      [
-        "theĠ",
-        "orig"
-      ],
-      [
-        "Ġa",
-        "Ġtime.čĊ"
-      ],
-      [
-        "Ġa",
-        "ĠsmokeĠtes"
-      ],
-      [
-        "sĠs",
-        "av"
-      ],
-      [
-        "sĠs",
-        "tabl"
-      ],
-      [
-        "eĠp",
-        "ract"
-      ],
-      [
-        "eĠp",
-        "rogr"
-      ],
-      [
-        "ation",
-        "Ġsam"
-      ],
-      [
-        "ation",
-        "sĠstabl"
-      ],
-      [
-        "learn",
-        "ingĠ"
-      ],
-      [
-        "learn",
-        "sĠbyĠ"
-      ],
-      [
-        "TheĠ",
-        "tokeniz"
-      ],
-      [
-        "TheĠ",
-        "tin"
-      ],
-      [
-        "am",
-        "WĠa"
-      ],
-      [
-        "bl",
-        "ock"
-      ],
-      [
-        "di",
-        "entĠ"
-      ],
-      [
-        "dĠ",
-        "cal"
-      ],
-      [
-        "dĠ",
-        "piec"
-      ],
-      [
-        "dsĠ",
-        "int"
-      ],
-      [
-        "dsĠ",
-        "theĠorig"
-      ],
-      [
-        "fĠ",
-        "learningĠ"
-      ],
-      [
-        "for",
-        "merĠ"
-      ],
-      [
-        "for",
-        "ĠaĠsmokeĠtes"
-      ],
-      [
-        "iv",
-        "ationsĠstabl"
-      ],
-      [
-        "keep",
-        "sĠact"
-      ],
-      [
-        "lit",
-        "tle"
-      ],
-      [
-        "mor",
-        "yĠsmall"
-      ],
-      [
-        "Ġat",
-        "tent"
-      ],
-      [
-        "Ġat",
-        "ĠaĠtime.čĊ"
-      ],
-      [
-        "esĠc",
-        "ontex"
-      ],
-      [
-        "ingĠt",
-        "hes"
-      ],
-      [
-        "ingĠr",
-        "ateĠp"
-      ],
-      [
-        "tokeniz",
-        "erĠmor"
-      ],
-      [
-        "usĠi",
-        "fĠlearningĠ"
-      ],
-      [
-        "-ent",
-        "ropyĠ"
-      ],
-      [
-        "25",
-        "6Ġtoken"
-      ],
-      [
-        "Ad",
-        "amWĠa"
-      ],
-      [
-        "AĠsmall",
-        "Ġcor"
-      ],
-      [
-        "By",
-        "te-le"
-      ],
-      [
-        "Ch",
-        "eckp"
-      ],
-      [
-        "Ea",
-        "chĠ"
-      ],
-      [
-        "Gra",
-        "dientĠ"
-      ],
-      [
-        "Gener",
-        "ationĠsam"
-      ],
-      [
-        "LayerĠ",
-        "nor"
-      ],
-      [
-        "Rep",
-        "eat"
-      ],
-      [
-        "Sh",
-        "ortĠ"
-      ],
-      [
-        "Tra",
-        "iningĠ"
-      ],
-      [
-        "Val",
-        "idation"
-      ],
-      [
-        "aĠ",
-        "little"
-      ],
-      [
-        "can",
-        "Ġresume.čĊ"
-      ],
-      [
-        "centĠ",
-        "updat"
-      ],
-      [
-        "des",
-        "centĠupdat"
-      ],
-      [
-        "dap",
-        "tsĠ"
-      ],
-      [
-        "edi",
-        "ctingĠt"
-      ],
-      [
-        "ebu",
-        "il"
-      ],
-      [
-        "ghĠ",
-        "forĠaĠsmokeĠtes"
-      ],
-      [
-        "gener",
-        "aliz"
-      ],
-      [
-        "htĠ",
-        "aĠlittle"
-      ],
-      [
-        "im",
-        "izesĠc"
-      ],
-      [
-        "ice",
-        ".čĊ"
-      ],
-      [
-        "lsĠ",
-        "usĠifĠlearningĠ"
-      ],
-      [
-        "layerĠ",
-        "mix"
-      ],
-      [
-        "mĠ",
-        "keepsĠact"
-      ],
-      [
-        "min",
-        "imizesĠc"
-      ],
-      [
-        "ou",
-        "ghĠforĠaĠsmokeĠtes"
-      ],
-      [
-        "oĠs",
-        "ubwor"
-      ],
-      [
-        "oĠr",
-        "unsĠ"
-      ],
-      [
-        "oint",
-        "sĠsav"
-      ],
-      [
-        "odingĠr",
-        "ebuil"
-      ],
-      [
-        "pr",
-        "edictingĠt"
-      ],
-      [
-        "plit",
-        "sĠwor"
-      ],
-      [
-        "pusĠi",
-        "sĠen"
-      ],
-      [
-        "plesĠ",
-        "oneĠtoken"
-      ],
-      [
-        "ssĠ",
-        "tel"
-      ],
-      [
-        "sal",
-        "Ġattent"
-      ],
-      [
-        "tĠtoken",
-        ".čĊ"
-      ],
-      [
-        "vel",
-        "Ġdec"
-      ],
-      [
-        "weig",
-        "htĠaĠlittle"
-      ],
-      [
-        "yĠt",
-        "rans"
-      ],
-      [
-        "Ġme",
-        "moryĠsmall"
-      ],
-      [
-        "Ġlo",
-        "ssĠtel"
-      ],
-      [
-        "sĠkeep",
-        "ĠmemoryĠsmall"
-      ],
-      [
-        "sĠofĠ",
-        "256Ġtoken"
-      ],
-      [
-        "eĠan",
-        "dĠcal"
-      ],
-      [
-        "eĠlin",
-        "esĠgiv"
-      ],
-      [
-        "inal",
-        "Ġtext.čĊ"
-      ],
-      [
-        "erĠs",
-        "plitsĠwor"
-      ],
-      [
-        "eryĠ",
-        "weightĠaĠlittle"
-      ],
-      [
-        "essĠs",
-        "oĠrunsĠ"
-      ],
-      [
-        "onĠsmall",
-        "Ġtext.čĊ"
-      ],
-      [
-        "esĠtheĠ",
-        "tokenizerĠmor"
-      ],
-      [
-        "esĠev",
-        "eryĠweightĠaĠlittle"
-      ],
-      [
-        "heĠnex",
-        "tĠtoken.čĊ"
-      ],
-      [
-        "tĠwit",
-        "hĠcau"
-      ],
-      [
-        "erĠpara",
-        "meter"
-      ],
-      [
-        "ross",
-        "-entropyĠ"
-      ],
-      [
-        "theĠlearn",
-        "ingĠrateĠp"
-      ],
-      [
-        "eĠpract",
-        "ice.čĊ"
-      ],
-      [
-        "eĠprogr",
-        "essĠsoĠrunsĠ"
-      ],
-      [
-        "learnsĠbyĠ",
-        "predictingĠt"
-      ],
-      [
-        "TheĠtokeniz",
-        "erĠsplitsĠwor"
-      ],
-      [
-        "TheĠtin",
-        "yĠtrans"
-      ],
-      [
-        "block",
-        "sĠofĠ256Ġtoken"
-      ],
-      [
-        "dĠpiec",
-        "es.čĊ"
-      ],
-      [
-        "dsĠint",
-        "oĠsubwor"
-      ],
-      [
-        "dsĠtheĠorig",
-        "inalĠtext.čĊ"
-      ],
-      [
-        "formerĠ",
-        "learnsĠbyĠpredictingĠt"
-      ],
-      [
-        "ivationsĠstabl",
-        "eĠandĠcal"
-      ],
-      [
-        "esĠcontex",
-        "tĠwithĠcau"
-      ],
-      [
-        "ingĠthes",
-        "eĠlinesĠgiv"
-      ],
-      [
-        "AdamWĠa",
-        "daptsĠ"
-      ],
-      [
-        "AĠsmallĠcor",
-        "pusĠisĠen"
-      ],
-      [
-        "Byte-le",
-        "velĠdec"
-      ],
-      [
-        "Checkp",
-        "ointsĠsav"
-      ],
-      [
-        "EachĠ",
-        "layerĠmix"
-      ],
-      [
-        "GradientĠ",
-        "descentĠupdat"
-      ],
-      [
-        "GenerationĠsam",
-        "plesĠoneĠtoken"
-      ],
-      [
-        "LayerĠnor",
-        "mĠkeepsĠact"
-      ],
-      [
-        "Repeat",
-        "ingĠtheseĠlinesĠgiv"
-      ],
-      [
-        "ShortĠ",
-        "blocksĠofĠ256Ġtoken"
-      ],
-      [
-        "TrainingĠ",
-        "minimizesĠc"
-      ],
-      [
-        "Validation",
-        "ĠlossĠtel"
-      ],
-      [
-        "generaliz",
-        "es.čĊ"
-      ],
-      [
-        "lsĠusĠifĠlearningĠ",
-        "generalizes.čĊ"
-      ],
-      [
-        "oughĠforĠaĠsmokeĠtes",
-        "t.čĊ"
-      ],
-      [
-        "odingĠrebuil",
-        "dsĠtheĠoriginalĠtext.čĊ"
-      ],
-      [
-        "salĠattent",
-        "ion.čĊ"
-      ],
-      [
-        "sĠkeepĠmemoryĠsmall",
-        ".čĊ"
-      ],
-      [
-        "esĠtheĠtokenizerĠmor",
-        "eĠpractice.čĊ"
-      ],
-      [
-        "esĠeveryĠweightĠaĠlittle",
-        ".čĊ"
-      ],
-      [
-        "erĠparameter",
-        ".čĊ"
-      ],
-      [
-        "ross-entropyĠ",
-        "onĠsmallĠtext.čĊ"
-      ],
-      [
-        "theĠlearningĠrateĠp",
-        "erĠparameter.čĊ"
-      ],
-      [
-        "eĠprogressĠsoĠrunsĠ",
-        "canĠresume.čĊ"
-      ],
-      [
-        "TheĠtokenizerĠsplitsĠwor",
-        "dsĠintoĠsubwor"
-      ],
-      [
-        "TheĠtinyĠtrans",
-        "formerĠlearnsĠbyĠpredictingĠt"
-      ],
-      [
-        "ivationsĠstableĠandĠcal",
-        "m.čĊ"
-      ],
-      [
-        "esĠcontextĠwithĠcau",
-        "salĠattention.čĊ"
-      ],
-      [
-        "AdamWĠadaptsĠ",
-        "theĠlearningĠrateĠperĠparameter.čĊ"
-      ],
-      [
-        "AĠsmallĠcorpusĠisĠen",
-        "oughĠforĠaĠsmokeĠtest.čĊ"
-      ],
-      [
-        "Byte-levelĠdec",
-        "odingĠrebuildsĠtheĠoriginalĠtext.čĊ"
-      ],
-      [
-        "CheckpointsĠsav",
-        "eĠprogressĠsoĠrunsĠcanĠresume.čĊ"
-      ],
-      [
-        "EachĠlayerĠmix",
-        "esĠcontextĠwithĠcausalĠattention.čĊ"
-      ],
-      [
-        "GradientĠdescentĠupdat",
-        "esĠeveryĠweightĠaĠlittle.čĊ"
-      ],
-      [
-        "GenerationĠsamplesĠoneĠtoken",
-        "ĠatĠaĠtime.čĊ"
-      ],
-      [
-        "LayerĠnormĠkeepsĠact",
-        "ivationsĠstableĠandĠcalm.čĊ"
-      ],
-      [
-        "RepeatingĠtheseĠlinesĠgiv",
-        "esĠtheĠtokenizerĠmoreĠpractice.čĊ"
-      ],
-      [
-        "ShortĠblocksĠofĠ256Ġtoken",
-        "sĠkeepĠmemoryĠsmall.čĊ"
-      ],
-      [
-        "TrainingĠminimizesĠc",
-        "ross-entropyĠonĠsmallĠtext.čĊ"
-      ],
-      [
-        "ValidationĠlossĠtel",
-        "lsĠusĠifĠlearningĠgeneralizes.čĊ"
-      ],
-      [
-        "TheĠtokenizerĠsplitsĠwordsĠintoĠsubwor",
-        "dĠpieces.čĊ"
-      ],
-      [
-        "TheĠtinyĠtransformerĠlearnsĠbyĠpredictingĠt",
-        "heĠnextĠtoken.čĊ"
-      ]
-    ]
-  }
-}
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

tokenizer_config.json

@@ -1,7 +1,30 @@
-{
-  "tokenizer_class": "PreTrainedTokenizerFast",
-  "bos_token": "<bos>",
-  "eos_token": "<eos>",
-  "unk_token": "<unk>",
-  "pad_token": "<pad>"
-}
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}