alexey-recursive commited on
Commit
2cb2bdc
·
0 Parent(s):

Release Flint quantized checkpoint

Browse files
.gitattributes ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ assets/flint-logo.png filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: text-generation
4
+ tags:
5
+ - mlx
6
+ - int4
7
+ - safetensors
8
+ ---
9
+
10
+ <p align="center">
11
+ <img src="assets/flint-logo.png" width="160" alt="Flint logo">
12
+ </p>
13
+
14
+ <h1 align="center">Flint INT4</h1>
15
+
16
+ MLX-format INT4 weights for [Recursive Flint](https://huggingface.co/RecursiveSI/flint), primarily intended for local inference on Apple Silicon. Projections and experts use **group-64 affine INT4**; embeddings use INT8. The weights occupy approximately **22.6 GB**.
17
+
18
+ See [Flint’s model card](https://huggingface.co/RecursiveSI/flint) for model details, benchmarks, chat conventions, and limitations.
19
+
20
+ ## Use with Rec
21
+
22
+ Follow [Rec’s requirements and installation guide](https://github.com/recursive-oss/rec#requirements), then launch pi or OpenCode:
23
+
24
+ ```bash
25
+ rec launch pi
26
+ # Or use OpenCode:
27
+ rec launch opencode
28
+ ```
29
+
30
+ See [Rec’s usage guide](https://github.com/recursive-oss/rec#usage) for configuration options.
31
+
32
+ ## Download the weights separately
33
+
34
+ With the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) installed:
35
+
36
+ ```bash
37
+ hf download RecursiveSI/flint-int4 --local-dir flint-int4
38
+ (cd flint-int4 && shasum -a 256 -c SHA256SUMS)
39
+ ```
40
+
41
+ For llama.cpp on Metal, use [Flint INT4 GGUF](https://huggingface.co/RecursiveSI/flint-int4-gguf).
42
+
43
+ ## License
44
+
45
+ [Apache License 2.0](LICENSE).
SHA256SUMS ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 43070e2d4e532684de521b885f385d0841030efa2b1a20bafb76133a5e1379c1 LICENSE
2
+ 0f820c3b0fb601cb8ff0b27ef1c2f5d4e2713dc0305ec5f150153446df735d56 README.md
3
+ b95290bec100def989fd410999f8129d43068900705a0609a78cdbafa930ad61 assets/flint-logo.png
4
+ 7940f4ffa3275c36caa4a582f845aaad79da11018b3961e8fbcad6ae29733466 chat_template.jinja
5
+ 80f577a984db1bd40dfff88457b611d4c13e807704a945616be27f28809eed21 config.json
6
+ 281aa3bdcf2aa8cf5e402c3c46ee5008245c013aeb30a2f96ce62b9f43e58763 flint-int4.json
7
+ a499c71895e008abd572558a5ad5904feaed92cf2a3b6e2cb0e9cabf03b96ed3 generation_config.json
8
+ 30a239f0781dde2668cf0ed0893d0d9c6f6f285127285274c96277fcd68962f7 merges.txt
9
+ 1a362742a0136c95ed751e2c56d83e63d4ed9ac674984b23b368f105d16cfe7a model-00000.safetensors
10
+ 12979140fa019042ff5cdec4c0237cc87db13b56428ce7fe093dd9d33893ea32 model-00001.safetensors
11
+ 01805c6b882fdec008b0184ec68e7516102205b521e63ca5126f9d52b9aed87b model-00002.safetensors
12
+ 9380ed032d90e0ff566d49672faf292f354b3b0e7320820c150eeed8c75e9d42 model-00003.safetensors
13
+ c1ba12d2c2a80feb5e554554a01cfde2b71c2638e27fc0ee597b4121399eff99 model-00004.safetensors
14
+ be37b302338dda189bc70bfbe902d8faebeb5986d60c1342713439eefc4ca182 special_tokens_map.json
15
+ a263cce0b7e2b03f7ee07ce809783ef4b47f160c3f268a4ea1524f638da45b68 tokenizer.json
16
+ 159ac88217e90ffd0a4a294726048bb302829e82e1958dc85bb203aa79138fbf tokenizer_config.json
17
+ 4003918aadec997451fe63c3d3b50cc65c9e3bdb9278e7ceb166d2ed9a3d56aa vocab.json
assets/flint-logo.png ADDED

Git LFS Details

  • SHA256: b95290bec100def989fd410999f8129d43068900705a0609a78cdbafa930ad61
  • Pointer size: 131 Bytes
  • Size of remote file: 410 kB
chat_template.jinja ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- macro flint_text(t) -%}
2
+ {%- if t is string -%}
3
+ {{- t | replace("\x00", "") | replace("<think>", "") | replace("</think>", "") | replace("<|begin_of_text|>", "&lt;|begin_of_text|&gt;") | replace("<|end_of_text|>", "&lt;|end_of_text|&gt;") | replace("<|system|>", "&lt;|system|&gt;") | replace("<|developer|>", "&lt;|developer|&gt;") | replace("<|user|>", "&lt;|user|&gt;") | replace("<|assistant|>", "&lt;|assistant|&gt;") | replace("<|observation|>", "&lt;|observation|&gt;") -}}
4
+ {%- else -%}
5
+ {{- t -}}
6
+ {%- endif -%}
7
+ {%- endmacro -%}
8
+ {%- set tool_preamble -%}
9
+ {%- if tools %}<|system|>
10
+ # Tools
11
+
12
+ You may call one or more functions to assist with the user query.
13
+
14
+ You are provided with function signatures within <tools></tools> XML tags:
15
+ <tools>
16
+ {%- for tool in tools %}
17
+ {%- set fn = tool.function if tool.function is defined else tool %}
18
+ {{ fn | tojson(ensure_ascii=False) }}
19
+ {%- endfor %}
20
+ </tools>
21
+
22
+ For each function call, output the function name and arguments within the following XML format:
23
+ <tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value>...</tool_call>
24
+ {% endif -%}
25
+ {%- endset -%}
26
+ {%- set ns = namespace(tools_pending=true) -%}
27
+ {%- for m in messages -%}
28
+ {%- if ns.tools_pending and m.role != 'system' -%}
29
+ {{ tool_preamble }}
30
+ {%- set ns.tools_pending = false -%}
31
+ {%- endif -%}
32
+ {%- if m.role == 'system' -%}
33
+ <|system|>{{ flint_text(m.content) }}
34
+ {%- elif m.role == 'developer' -%}
35
+ <|developer|>{{ flint_text(m.content) }}
36
+ {%- elif m.role == 'user' -%}
37
+ <|user|>{{ flint_text(m.content) }}
38
+ {%- elif m.role == 'assistant' -%}
39
+ <|assistant|>
40
+ {%- if m.reasoning_content %}<think>{{ m.reasoning_content }}</think>{% endif -%}
41
+ {%- if m.content %}{{ m.content }}{% endif -%}
42
+ {%- if m.tool_calls -%}
43
+ {%- for tc in m.tool_calls -%}
44
+ {%- set fn = tc.function if tc.function is defined else tc -%}
45
+ <tool_call>{{ fn.name }}
46
+ {%- for k, v in fn.arguments.items() -%}
47
+ <arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>
48
+ {%- endfor -%}
49
+ </tool_call>
50
+ {%- endfor -%}
51
+ {%- endif -%}
52
+ {%- elif m.role == 'tool' -%}
53
+ <|observation|><tool_response>{{ flint_text(m.content) }}</tool_response>
54
+ {%- endif -%}
55
+ {%- endfor -%}
56
+ {%- if ns.tools_pending -%}
57
+ {{ tool_preamble }}
58
+ {%- endif -%}
59
+ {%- if add_generation_prompt -%}
60
+ <|assistant|>
61
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,152 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "FlintForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "attention_sink": true,
8
+ "bos_token_id": 150000,
9
+ "decoder_sparse_step": 1,
10
+ "dtype": "bfloat16",
11
+ "eos_token_id": [
12
+ 150001,
13
+ 150003,
14
+ 150005,
15
+ 150018
16
+ ],
17
+ "head_dim": 128,
18
+ "hidden_act": "silu",
19
+ "hidden_size": 2048,
20
+ "intermediate_size": 6144,
21
+ "kv_group_size": 64,
22
+ "kv_key_bits": 8,
23
+ "kv_value_bits": 8,
24
+ "layer_types": [
25
+ "full_attention",
26
+ "full_attention",
27
+ "sliding_attention",
28
+ "sliding_attention",
29
+ "sliding_attention",
30
+ "full_attention",
31
+ "sliding_attention",
32
+ "sliding_attention",
33
+ "sliding_attention",
34
+ "full_attention",
35
+ "sliding_attention",
36
+ "sliding_attention",
37
+ "sliding_attention",
38
+ "full_attention",
39
+ "sliding_attention",
40
+ "sliding_attention",
41
+ "sliding_attention",
42
+ "full_attention",
43
+ "sliding_attention",
44
+ "sliding_attention",
45
+ "sliding_attention",
46
+ "full_attention",
47
+ "sliding_attention",
48
+ "sliding_attention",
49
+ "sliding_attention",
50
+ "full_attention",
51
+ "sliding_attention",
52
+ "sliding_attention",
53
+ "sliding_attention",
54
+ "full_attention",
55
+ "sliding_attention",
56
+ "sliding_attention",
57
+ "sliding_attention",
58
+ "full_attention",
59
+ "sliding_attention",
60
+ "sliding_attention",
61
+ "sliding_attention",
62
+ "full_attention",
63
+ "sliding_attention",
64
+ "sliding_attention",
65
+ "sliding_attention",
66
+ "full_attention",
67
+ "sliding_attention",
68
+ "sliding_attention",
69
+ "sliding_attention",
70
+ "full_attention",
71
+ "sliding_attention",
72
+ "sliding_attention",
73
+ "full_attention"
74
+ ],
75
+ "max_position_embeddings": 262144,
76
+ "mlp_only_layers": [
77
+ 0,
78
+ 1
79
+ ],
80
+ "mlx_export": {
81
+ "bits": 4,
82
+ "embedding_bits": 8,
83
+ "full_attention_layers": [
84
+ 0,
85
+ 1,
86
+ 5,
87
+ 9,
88
+ 13,
89
+ 17,
90
+ 21,
91
+ 25,
92
+ 29,
93
+ 33,
94
+ 37,
95
+ 41,
96
+ 45,
97
+ 48
98
+ ],
99
+ "routed_expert_bits": 4,
100
+ "scale_rule": "mse"
101
+ },
102
+ "model_type": "flint",
103
+ "moe_intermediate_size": 512,
104
+ "no_rope_layer_indices": [
105
+ 0,
106
+ 1,
107
+ 5,
108
+ 9,
109
+ 13,
110
+ 17,
111
+ 21,
112
+ 25,
113
+ 29,
114
+ 33,
115
+ 37,
116
+ 41,
117
+ 45,
118
+ 48
119
+ ],
120
+ "norm_topk_prob": true,
121
+ "num_attention_heads": 32,
122
+ "num_experts": 256,
123
+ "num_experts_per_tok": 6,
124
+ "num_hidden_layers": 49,
125
+ "num_key_value_heads": 4,
126
+ "num_local_experts": 256,
127
+ "output_router_logits": false,
128
+ "quantization": {
129
+ "bits": 4,
130
+ "group_size": 64
131
+ },
132
+ "rms_norm_eps": 1e-06,
133
+ "rope_parameters": {
134
+ "rope_theta": 10000.0,
135
+ "rope_type": "default"
136
+ },
137
+ "rope_theta": 10000.0,
138
+ "router_aux_loss_coef": 0.0,
139
+ "router_expert_bias": false,
140
+ "scoring_func": "sigmoid",
141
+ "shared_expert_intermediate_size": 1024,
142
+ "sliding_window": 513,
143
+ "softmax_type": "learnable",
144
+ "softmax_type_source": "model.attention_pattern.softmax_type",
145
+ "tie_word_embeddings": false,
146
+ "use_cache": true,
147
+ "use_sliding_window": true,
148
+ "vocab_size": 150144,
149
+ "router_score_function": "sigmoid",
150
+ "torch_dtype": "bfloat16",
151
+ "pad_token_id": 150001
152
+ }
flint-int4.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_id": "flint-int4",
3
+ "weight_format": "mlx-affine-int4",
4
+ "kv_format": "bf16",
5
+ "max_context": 262144,
6
+ "checkpoint_files": {
7
+ "chat_template.jinja": "7940f4ffa3275c36caa4a582f845aaad79da11018b3961e8fbcad6ae29733466",
8
+ "config.json": "80f577a984db1bd40dfff88457b611d4c13e807704a945616be27f28809eed21",
9
+ "generation_config.json": "a499c71895e008abd572558a5ad5904feaed92cf2a3b6e2cb0e9cabf03b96ed3",
10
+ "merges.txt": "30a239f0781dde2668cf0ed0893d0d9c6f6f285127285274c96277fcd68962f7",
11
+ "model-00000.safetensors": "1a362742a0136c95ed751e2c56d83e63d4ed9ac674984b23b368f105d16cfe7a",
12
+ "model-00001.safetensors": "12979140fa019042ff5cdec4c0237cc87db13b56428ce7fe093dd9d33893ea32",
13
+ "model-00002.safetensors": "01805c6b882fdec008b0184ec68e7516102205b521e63ca5126f9d52b9aed87b",
14
+ "model-00003.safetensors": "9380ed032d90e0ff566d49672faf292f354b3b0e7320820c150eeed8c75e9d42",
15
+ "model-00004.safetensors": "c1ba12d2c2a80feb5e554554a01cfde2b71c2638e27fc0ee597b4121399eff99",
16
+ "special_tokens_map.json": "be37b302338dda189bc70bfbe902d8faebeb5986d60c1342713439eefc4ca182",
17
+ "tokenizer.json": "a263cce0b7e2b03f7ee07ce809783ef4b47f160c3f268a4ea1524f638da45b68",
18
+ "tokenizer_config.json": "159ac88217e90ffd0a4a294726048bb302829e82e1958dc85bb203aa79138fbf",
19
+ "vocab.json": "4003918aadec997451fe63c3d3b50cc65c9e3bdb9278e7ceb166d2ed9a3d56aa"
20
+ }
21
+ }
generation_config.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 150000,
3
+ "eos_token_id": [
4
+ 150001,
5
+ 150003,
6
+ 150005,
7
+ 150018
8
+ ],
9
+ "pad_token_id": 150001,
10
+ "do_sample": true,
11
+ "temperature": 1.0,
12
+ "top_p": 0.95,
13
+ "top_k": 20
14
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model-00000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a362742a0136c95ed751e2c56d83e63d4ed9ac674984b23b368f105d16cfe7a
3
+ size 4745746152
model-00001.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12979140fa019042ff5cdec4c0237cc87db13b56428ce7fe093dd9d33893ea32
3
+ size 4682022280
model-00002.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01805c6b882fdec008b0184ec68e7516102205b521e63ca5126f9d52b9aed87b
3
+ size 4682022280
model-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9380ed032d90e0ff566d49672faf292f354b3b0e7320820c150eeed8c75e9d42
3
+ size 4682022280
model-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1ba12d2c2a80feb5e554554a01cfde2b71c2638e27fc0ee597b4121399eff99
3
+ size 3777099608
special_tokens_map.json ADDED
@@ -0,0 +1,928 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<|system|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<|user|>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ },
17
+ {
18
+ "content": "<|assistant|>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ {
25
+ "content": "<|observation|>",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ {
32
+ "content": "<|developer|>",
33
+ "lstrip": false,
34
+ "normalized": false,
35
+ "rstrip": false,
36
+ "single_word": false
37
+ },
38
+ {
39
+ "content": "<|reserved_150019|>",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false
44
+ },
45
+ {
46
+ "content": "<|reserved_150020|>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": false,
50
+ "single_word": false
51
+ },
52
+ {
53
+ "content": "<|reserved_150021|>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false
58
+ },
59
+ {
60
+ "content": "<|reserved_150022|>",
61
+ "lstrip": false,
62
+ "normalized": false,
63
+ "rstrip": false,
64
+ "single_word": false
65
+ },
66
+ {
67
+ "content": "<|reserved_150023|>",
68
+ "lstrip": false,
69
+ "normalized": false,
70
+ "rstrip": false,
71
+ "single_word": false
72
+ },
73
+ {
74
+ "content": "<|reserved_150024|>",
75
+ "lstrip": false,
76
+ "normalized": false,
77
+ "rstrip": false,
78
+ "single_word": false
79
+ },
80
+ {
81
+ "content": "<|reserved_150025|>",
82
+ "lstrip": false,
83
+ "normalized": false,
84
+ "rstrip": false,
85
+ "single_word": false
86
+ },
87
+ {
88
+ "content": "<|reserved_150026|>",
89
+ "lstrip": false,
90
+ "normalized": false,
91
+ "rstrip": false,
92
+ "single_word": false
93
+ },
94
+ {
95
+ "content": "<|reserved_150027|>",
96
+ "lstrip": false,
97
+ "normalized": false,
98
+ "rstrip": false,
99
+ "single_word": false
100
+ },
101
+ {
102
+ "content": "<|reserved_150028|>",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": false,
106
+ "single_word": false
107
+ },
108
+ {
109
+ "content": "<|reserved_150029|>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false
114
+ },
115
+ {
116
+ "content": "<|reserved_150030|>",
117
+ "lstrip": false,
118
+ "normalized": false,
119
+ "rstrip": false,
120
+ "single_word": false
121
+ },
122
+ {
123
+ "content": "<|reserved_150031|>",
124
+ "lstrip": false,
125
+ "normalized": false,
126
+ "rstrip": false,
127
+ "single_word": false
128
+ },
129
+ {
130
+ "content": "<|reserved_150032|>",
131
+ "lstrip": false,
132
+ "normalized": false,
133
+ "rstrip": false,
134
+ "single_word": false
135
+ },
136
+ {
137
+ "content": "<|reserved_150033|>",
138
+ "lstrip": false,
139
+ "normalized": false,
140
+ "rstrip": false,
141
+ "single_word": false
142
+ },
143
+ {
144
+ "content": "<|reserved_150034|>",
145
+ "lstrip": false,
146
+ "normalized": false,
147
+ "rstrip": false,
148
+ "single_word": false
149
+ },
150
+ {
151
+ "content": "<|reserved_150035|>",
152
+ "lstrip": false,
153
+ "normalized": false,
154
+ "rstrip": false,
155
+ "single_word": false
156
+ },
157
+ {
158
+ "content": "<|reserved_150036|>",
159
+ "lstrip": false,
160
+ "normalized": false,
161
+ "rstrip": false,
162
+ "single_word": false
163
+ },
164
+ {
165
+ "content": "<|reserved_150037|>",
166
+ "lstrip": false,
167
+ "normalized": false,
168
+ "rstrip": false,
169
+ "single_word": false
170
+ },
171
+ {
172
+ "content": "<|reserved_150038|>",
173
+ "lstrip": false,
174
+ "normalized": false,
175
+ "rstrip": false,
176
+ "single_word": false
177
+ },
178
+ {
179
+ "content": "<|reserved_150039|>",
180
+ "lstrip": false,
181
+ "normalized": false,
182
+ "rstrip": false,
183
+ "single_word": false
184
+ },
185
+ {
186
+ "content": "<|reserved_150040|>",
187
+ "lstrip": false,
188
+ "normalized": false,
189
+ "rstrip": false,
190
+ "single_word": false
191
+ },
192
+ {
193
+ "content": "<|reserved_150041|>",
194
+ "lstrip": false,
195
+ "normalized": false,
196
+ "rstrip": false,
197
+ "single_word": false
198
+ },
199
+ {
200
+ "content": "<|reserved_150042|>",
201
+ "lstrip": false,
202
+ "normalized": false,
203
+ "rstrip": false,
204
+ "single_word": false
205
+ },
206
+ {
207
+ "content": "<|reserved_150043|>",
208
+ "lstrip": false,
209
+ "normalized": false,
210
+ "rstrip": false,
211
+ "single_word": false
212
+ },
213
+ {
214
+ "content": "<|reserved_150044|>",
215
+ "lstrip": false,
216
+ "normalized": false,
217
+ "rstrip": false,
218
+ "single_word": false
219
+ },
220
+ {
221
+ "content": "<|reserved_150045|>",
222
+ "lstrip": false,
223
+ "normalized": false,
224
+ "rstrip": false,
225
+ "single_word": false
226
+ },
227
+ {
228
+ "content": "<|reserved_150046|>",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false
233
+ },
234
+ {
235
+ "content": "<|reserved_150047|>",
236
+ "lstrip": false,
237
+ "normalized": false,
238
+ "rstrip": false,
239
+ "single_word": false
240
+ },
241
+ {
242
+ "content": "<|reserved_150048|>",
243
+ "lstrip": false,
244
+ "normalized": false,
245
+ "rstrip": false,
246
+ "single_word": false
247
+ },
248
+ {
249
+ "content": "<|reserved_150049|>",
250
+ "lstrip": false,
251
+ "normalized": false,
252
+ "rstrip": false,
253
+ "single_word": false
254
+ },
255
+ {
256
+ "content": "<|reserved_150050|>",
257
+ "lstrip": false,
258
+ "normalized": false,
259
+ "rstrip": false,
260
+ "single_word": false
261
+ },
262
+ {
263
+ "content": "<|reserved_150051|>",
264
+ "lstrip": false,
265
+ "normalized": false,
266
+ "rstrip": false,
267
+ "single_word": false
268
+ },
269
+ {
270
+ "content": "<|reserved_150052|>",
271
+ "lstrip": false,
272
+ "normalized": false,
273
+ "rstrip": false,
274
+ "single_word": false
275
+ },
276
+ {
277
+ "content": "<|reserved_150053|>",
278
+ "lstrip": false,
279
+ "normalized": false,
280
+ "rstrip": false,
281
+ "single_word": false
282
+ },
283
+ {
284
+ "content": "<|reserved_150054|>",
285
+ "lstrip": false,
286
+ "normalized": false,
287
+ "rstrip": false,
288
+ "single_word": false
289
+ },
290
+ {
291
+ "content": "<|reserved_150055|>",
292
+ "lstrip": false,
293
+ "normalized": false,
294
+ "rstrip": false,
295
+ "single_word": false
296
+ },
297
+ {
298
+ "content": "<|reserved_150056|>",
299
+ "lstrip": false,
300
+ "normalized": false,
301
+ "rstrip": false,
302
+ "single_word": false
303
+ },
304
+ {
305
+ "content": "<|reserved_150057|>",
306
+ "lstrip": false,
307
+ "normalized": false,
308
+ "rstrip": false,
309
+ "single_word": false
310
+ },
311
+ {
312
+ "content": "<|reserved_150058|>",
313
+ "lstrip": false,
314
+ "normalized": false,
315
+ "rstrip": false,
316
+ "single_word": false
317
+ },
318
+ {
319
+ "content": "<|reserved_150059|>",
320
+ "lstrip": false,
321
+ "normalized": false,
322
+ "rstrip": false,
323
+ "single_word": false
324
+ },
325
+ {
326
+ "content": "<|reserved_150060|>",
327
+ "lstrip": false,
328
+ "normalized": false,
329
+ "rstrip": false,
330
+ "single_word": false
331
+ },
332
+ {
333
+ "content": "<|reserved_150061|>",
334
+ "lstrip": false,
335
+ "normalized": false,
336
+ "rstrip": false,
337
+ "single_word": false
338
+ },
339
+ {
340
+ "content": "<|reserved_150062|>",
341
+ "lstrip": false,
342
+ "normalized": false,
343
+ "rstrip": false,
344
+ "single_word": false
345
+ },
346
+ {
347
+ "content": "<|reserved_150063|>",
348
+ "lstrip": false,
349
+ "normalized": false,
350
+ "rstrip": false,
351
+ "single_word": false
352
+ },
353
+ {
354
+ "content": "<|reserved_150064|>",
355
+ "lstrip": false,
356
+ "normalized": false,
357
+ "rstrip": false,
358
+ "single_word": false
359
+ },
360
+ {
361
+ "content": "<|reserved_150065|>",
362
+ "lstrip": false,
363
+ "normalized": false,
364
+ "rstrip": false,
365
+ "single_word": false
366
+ },
367
+ {
368
+ "content": "<|reserved_150066|>",
369
+ "lstrip": false,
370
+ "normalized": false,
371
+ "rstrip": false,
372
+ "single_word": false
373
+ },
374
+ {
375
+ "content": "<|reserved_150067|>",
376
+ "lstrip": false,
377
+ "normalized": false,
378
+ "rstrip": false,
379
+ "single_word": false
380
+ },
381
+ {
382
+ "content": "<|reserved_150068|>",
383
+ "lstrip": false,
384
+ "normalized": false,
385
+ "rstrip": false,
386
+ "single_word": false
387
+ },
388
+ {
389
+ "content": "<|reserved_150069|>",
390
+ "lstrip": false,
391
+ "normalized": false,
392
+ "rstrip": false,
393
+ "single_word": false
394
+ },
395
+ {
396
+ "content": "<|reserved_150070|>",
397
+ "lstrip": false,
398
+ "normalized": false,
399
+ "rstrip": false,
400
+ "single_word": false
401
+ },
402
+ {
403
+ "content": "<|reserved_150071|>",
404
+ "lstrip": false,
405
+ "normalized": false,
406
+ "rstrip": false,
407
+ "single_word": false
408
+ },
409
+ {
410
+ "content": "<|reserved_150072|>",
411
+ "lstrip": false,
412
+ "normalized": false,
413
+ "rstrip": false,
414
+ "single_word": false
415
+ },
416
+ {
417
+ "content": "<|reserved_150073|>",
418
+ "lstrip": false,
419
+ "normalized": false,
420
+ "rstrip": false,
421
+ "single_word": false
422
+ },
423
+ {
424
+ "content": "<|reserved_150074|>",
425
+ "lstrip": false,
426
+ "normalized": false,
427
+ "rstrip": false,
428
+ "single_word": false
429
+ },
430
+ {
431
+ "content": "<|reserved_150075|>",
432
+ "lstrip": false,
433
+ "normalized": false,
434
+ "rstrip": false,
435
+ "single_word": false
436
+ },
437
+ {
438
+ "content": "<|reserved_150076|>",
439
+ "lstrip": false,
440
+ "normalized": false,
441
+ "rstrip": false,
442
+ "single_word": false
443
+ },
444
+ {
445
+ "content": "<|reserved_150077|>",
446
+ "lstrip": false,
447
+ "normalized": false,
448
+ "rstrip": false,
449
+ "single_word": false
450
+ },
451
+ {
452
+ "content": "<|reserved_150078|>",
453
+ "lstrip": false,
454
+ "normalized": false,
455
+ "rstrip": false,
456
+ "single_word": false
457
+ },
458
+ {
459
+ "content": "<|reserved_150079|>",
460
+ "lstrip": false,
461
+ "normalized": false,
462
+ "rstrip": false,
463
+ "single_word": false
464
+ },
465
+ {
466
+ "content": "<|reserved_150080|>",
467
+ "lstrip": false,
468
+ "normalized": false,
469
+ "rstrip": false,
470
+ "single_word": false
471
+ },
472
+ {
473
+ "content": "<|reserved_150081|>",
474
+ "lstrip": false,
475
+ "normalized": false,
476
+ "rstrip": false,
477
+ "single_word": false
478
+ },
479
+ {
480
+ "content": "<|reserved_150082|>",
481
+ "lstrip": false,
482
+ "normalized": false,
483
+ "rstrip": false,
484
+ "single_word": false
485
+ },
486
+ {
487
+ "content": "<|reserved_150083|>",
488
+ "lstrip": false,
489
+ "normalized": false,
490
+ "rstrip": false,
491
+ "single_word": false
492
+ },
493
+ {
494
+ "content": "<|reserved_150084|>",
495
+ "lstrip": false,
496
+ "normalized": false,
497
+ "rstrip": false,
498
+ "single_word": false
499
+ },
500
+ {
501
+ "content": "<|reserved_150085|>",
502
+ "lstrip": false,
503
+ "normalized": false,
504
+ "rstrip": false,
505
+ "single_word": false
506
+ },
507
+ {
508
+ "content": "<|reserved_150086|>",
509
+ "lstrip": false,
510
+ "normalized": false,
511
+ "rstrip": false,
512
+ "single_word": false
513
+ },
514
+ {
515
+ "content": "<|reserved_150087|>",
516
+ "lstrip": false,
517
+ "normalized": false,
518
+ "rstrip": false,
519
+ "single_word": false
520
+ },
521
+ {
522
+ "content": "<|reserved_150088|>",
523
+ "lstrip": false,
524
+ "normalized": false,
525
+ "rstrip": false,
526
+ "single_word": false
527
+ },
528
+ {
529
+ "content": "<|reserved_150089|>",
530
+ "lstrip": false,
531
+ "normalized": false,
532
+ "rstrip": false,
533
+ "single_word": false
534
+ },
535
+ {
536
+ "content": "<|reserved_150090|>",
537
+ "lstrip": false,
538
+ "normalized": false,
539
+ "rstrip": false,
540
+ "single_word": false
541
+ },
542
+ {
543
+ "content": "<|reserved_150091|>",
544
+ "lstrip": false,
545
+ "normalized": false,
546
+ "rstrip": false,
547
+ "single_word": false
548
+ },
549
+ {
550
+ "content": "<|reserved_150092|>",
551
+ "lstrip": false,
552
+ "normalized": false,
553
+ "rstrip": false,
554
+ "single_word": false
555
+ },
556
+ {
557
+ "content": "<|reserved_150093|>",
558
+ "lstrip": false,
559
+ "normalized": false,
560
+ "rstrip": false,
561
+ "single_word": false
562
+ },
563
+ {
564
+ "content": "<|reserved_150094|>",
565
+ "lstrip": false,
566
+ "normalized": false,
567
+ "rstrip": false,
568
+ "single_word": false
569
+ },
570
+ {
571
+ "content": "<|reserved_150095|>",
572
+ "lstrip": false,
573
+ "normalized": false,
574
+ "rstrip": false,
575
+ "single_word": false
576
+ },
577
+ {
578
+ "content": "<|reserved_150096|>",
579
+ "lstrip": false,
580
+ "normalized": false,
581
+ "rstrip": false,
582
+ "single_word": false
583
+ },
584
+ {
585
+ "content": "<|reserved_150097|>",
586
+ "lstrip": false,
587
+ "normalized": false,
588
+ "rstrip": false,
589
+ "single_word": false
590
+ },
591
+ {
592
+ "content": "<|reserved_150098|>",
593
+ "lstrip": false,
594
+ "normalized": false,
595
+ "rstrip": false,
596
+ "single_word": false
597
+ },
598
+ {
599
+ "content": "<|reserved_150099|>",
600
+ "lstrip": false,
601
+ "normalized": false,
602
+ "rstrip": false,
603
+ "single_word": false
604
+ },
605
+ {
606
+ "content": "<|reserved_150100|>",
607
+ "lstrip": false,
608
+ "normalized": false,
609
+ "rstrip": false,
610
+ "single_word": false
611
+ },
612
+ {
613
+ "content": "<|reserved_150101|>",
614
+ "lstrip": false,
615
+ "normalized": false,
616
+ "rstrip": false,
617
+ "single_word": false
618
+ },
619
+ {
620
+ "content": "<|reserved_150102|>",
621
+ "lstrip": false,
622
+ "normalized": false,
623
+ "rstrip": false,
624
+ "single_word": false
625
+ },
626
+ {
627
+ "content": "<|reserved_150103|>",
628
+ "lstrip": false,
629
+ "normalized": false,
630
+ "rstrip": false,
631
+ "single_word": false
632
+ },
633
+ {
634
+ "content": "<|reserved_150104|>",
635
+ "lstrip": false,
636
+ "normalized": false,
637
+ "rstrip": false,
638
+ "single_word": false
639
+ },
640
+ {
641
+ "content": "<|reserved_150105|>",
642
+ "lstrip": false,
643
+ "normalized": false,
644
+ "rstrip": false,
645
+ "single_word": false
646
+ },
647
+ {
648
+ "content": "<|reserved_150106|>",
649
+ "lstrip": false,
650
+ "normalized": false,
651
+ "rstrip": false,
652
+ "single_word": false
653
+ },
654
+ {
655
+ "content": "<|reserved_150107|>",
656
+ "lstrip": false,
657
+ "normalized": false,
658
+ "rstrip": false,
659
+ "single_word": false
660
+ },
661
+ {
662
+ "content": "<|reserved_150108|>",
663
+ "lstrip": false,
664
+ "normalized": false,
665
+ "rstrip": false,
666
+ "single_word": false
667
+ },
668
+ {
669
+ "content": "<|reserved_150109|>",
670
+ "lstrip": false,
671
+ "normalized": false,
672
+ "rstrip": false,
673
+ "single_word": false
674
+ },
675
+ {
676
+ "content": "<|reserved_150110|>",
677
+ "lstrip": false,
678
+ "normalized": false,
679
+ "rstrip": false,
680
+ "single_word": false
681
+ },
682
+ {
683
+ "content": "<|reserved_150111|>",
684
+ "lstrip": false,
685
+ "normalized": false,
686
+ "rstrip": false,
687
+ "single_word": false
688
+ },
689
+ {
690
+ "content": "<|reserved_150112|>",
691
+ "lstrip": false,
692
+ "normalized": false,
693
+ "rstrip": false,
694
+ "single_word": false
695
+ },
696
+ {
697
+ "content": "<|reserved_150113|>",
698
+ "lstrip": false,
699
+ "normalized": false,
700
+ "rstrip": false,
701
+ "single_word": false
702
+ },
703
+ {
704
+ "content": "<|reserved_150114|>",
705
+ "lstrip": false,
706
+ "normalized": false,
707
+ "rstrip": false,
708
+ "single_word": false
709
+ },
710
+ {
711
+ "content": "<|reserved_150115|>",
712
+ "lstrip": false,
713
+ "normalized": false,
714
+ "rstrip": false,
715
+ "single_word": false
716
+ },
717
+ {
718
+ "content": "<|reserved_150116|>",
719
+ "lstrip": false,
720
+ "normalized": false,
721
+ "rstrip": false,
722
+ "single_word": false
723
+ },
724
+ {
725
+ "content": "<|reserved_150117|>",
726
+ "lstrip": false,
727
+ "normalized": false,
728
+ "rstrip": false,
729
+ "single_word": false
730
+ },
731
+ {
732
+ "content": "<|reserved_150118|>",
733
+ "lstrip": false,
734
+ "normalized": false,
735
+ "rstrip": false,
736
+ "single_word": false
737
+ },
738
+ {
739
+ "content": "<|reserved_150119|>",
740
+ "lstrip": false,
741
+ "normalized": false,
742
+ "rstrip": false,
743
+ "single_word": false
744
+ },
745
+ {
746
+ "content": "<|reserved_150120|>",
747
+ "lstrip": false,
748
+ "normalized": false,
749
+ "rstrip": false,
750
+ "single_word": false
751
+ },
752
+ {
753
+ "content": "<|reserved_150121|>",
754
+ "lstrip": false,
755
+ "normalized": false,
756
+ "rstrip": false,
757
+ "single_word": false
758
+ },
759
+ {
760
+ "content": "<|reserved_150122|>",
761
+ "lstrip": false,
762
+ "normalized": false,
763
+ "rstrip": false,
764
+ "single_word": false
765
+ },
766
+ {
767
+ "content": "<|reserved_150123|>",
768
+ "lstrip": false,
769
+ "normalized": false,
770
+ "rstrip": false,
771
+ "single_word": false
772
+ },
773
+ {
774
+ "content": "<|reserved_150124|>",
775
+ "lstrip": false,
776
+ "normalized": false,
777
+ "rstrip": false,
778
+ "single_word": false
779
+ },
780
+ {
781
+ "content": "<|reserved_150125|>",
782
+ "lstrip": false,
783
+ "normalized": false,
784
+ "rstrip": false,
785
+ "single_word": false
786
+ },
787
+ {
788
+ "content": "<|reserved_150126|>",
789
+ "lstrip": false,
790
+ "normalized": false,
791
+ "rstrip": false,
792
+ "single_word": false
793
+ },
794
+ {
795
+ "content": "<|reserved_150127|>",
796
+ "lstrip": false,
797
+ "normalized": false,
798
+ "rstrip": false,
799
+ "single_word": false
800
+ },
801
+ {
802
+ "content": "<|reserved_150128|>",
803
+ "lstrip": false,
804
+ "normalized": false,
805
+ "rstrip": false,
806
+ "single_word": false
807
+ },
808
+ {
809
+ "content": "<|reserved_150129|>",
810
+ "lstrip": false,
811
+ "normalized": false,
812
+ "rstrip": false,
813
+ "single_word": false
814
+ },
815
+ {
816
+ "content": "<|reserved_150130|>",
817
+ "lstrip": false,
818
+ "normalized": false,
819
+ "rstrip": false,
820
+ "single_word": false
821
+ },
822
+ {
823
+ "content": "<|reserved_150131|>",
824
+ "lstrip": false,
825
+ "normalized": false,
826
+ "rstrip": false,
827
+ "single_word": false
828
+ },
829
+ {
830
+ "content": "<|reserved_150132|>",
831
+ "lstrip": false,
832
+ "normalized": false,
833
+ "rstrip": false,
834
+ "single_word": false
835
+ },
836
+ {
837
+ "content": "<|reserved_150133|>",
838
+ "lstrip": false,
839
+ "normalized": false,
840
+ "rstrip": false,
841
+ "single_word": false
842
+ },
843
+ {
844
+ "content": "<|reserved_150134|>",
845
+ "lstrip": false,
846
+ "normalized": false,
847
+ "rstrip": false,
848
+ "single_word": false
849
+ },
850
+ {
851
+ "content": "<|reserved_150135|>",
852
+ "lstrip": false,
853
+ "normalized": false,
854
+ "rstrip": false,
855
+ "single_word": false
856
+ },
857
+ {
858
+ "content": "<|reserved_150136|>",
859
+ "lstrip": false,
860
+ "normalized": false,
861
+ "rstrip": false,
862
+ "single_word": false
863
+ },
864
+ {
865
+ "content": "<|reserved_150137|>",
866
+ "lstrip": false,
867
+ "normalized": false,
868
+ "rstrip": false,
869
+ "single_word": false
870
+ },
871
+ {
872
+ "content": "<|reserved_150138|>",
873
+ "lstrip": false,
874
+ "normalized": false,
875
+ "rstrip": false,
876
+ "single_word": false
877
+ },
878
+ {
879
+ "content": "<|reserved_150139|>",
880
+ "lstrip": false,
881
+ "normalized": false,
882
+ "rstrip": false,
883
+ "single_word": false
884
+ },
885
+ {
886
+ "content": "<|reserved_150140|>",
887
+ "lstrip": false,
888
+ "normalized": false,
889
+ "rstrip": false,
890
+ "single_word": false
891
+ },
892
+ {
893
+ "content": "<|reserved_150141|>",
894
+ "lstrip": false,
895
+ "normalized": false,
896
+ "rstrip": false,
897
+ "single_word": false
898
+ },
899
+ {
900
+ "content": "<|reserved_150142|>",
901
+ "lstrip": false,
902
+ "normalized": false,
903
+ "rstrip": false,
904
+ "single_word": false
905
+ },
906
+ {
907
+ "content": "<|reserved_150143|>",
908
+ "lstrip": false,
909
+ "normalized": false,
910
+ "rstrip": false,
911
+ "single_word": false
912
+ }
913
+ ],
914
+ "bos_token": {
915
+ "content": "<|begin_of_text|>",
916
+ "lstrip": false,
917
+ "normalized": false,
918
+ "rstrip": false,
919
+ "single_word": false
920
+ },
921
+ "eos_token": {
922
+ "content": "<|end_of_text|>",
923
+ "lstrip": false,
924
+ "normalized": false,
925
+ "rstrip": false,
926
+ "single_word": false
927
+ }
928
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a263cce0b7e2b03f7ee07ce809783ef4b47f160c3f268a4ea1524f638da45b68
3
+ size 10879128
tokenizer_config.json ADDED
@@ -0,0 +1,1295 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": false,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "150000": {
6
+ "content": "<|begin_of_text|>",
7
+ "single_word": false,
8
+ "lstrip": false,
9
+ "rstrip": false,
10
+ "normalized": false,
11
+ "special": true
12
+ },
13
+ "150001": {
14
+ "content": "<|end_of_text|>",
15
+ "single_word": false,
16
+ "lstrip": false,
17
+ "rstrip": false,
18
+ "normalized": false,
19
+ "special": true
20
+ },
21
+ "150002": {
22
+ "content": "<|system|>",
23
+ "single_word": false,
24
+ "lstrip": false,
25
+ "rstrip": false,
26
+ "normalized": false,
27
+ "special": true
28
+ },
29
+ "150003": {
30
+ "content": "<|user|>",
31
+ "single_word": false,
32
+ "lstrip": false,
33
+ "rstrip": false,
34
+ "normalized": false,
35
+ "special": true
36
+ },
37
+ "150004": {
38
+ "content": "<|assistant|>",
39
+ "single_word": false,
40
+ "lstrip": false,
41
+ "rstrip": false,
42
+ "normalized": false,
43
+ "special": true
44
+ },
45
+ "150005": {
46
+ "content": "<|observation|>",
47
+ "single_word": false,
48
+ "lstrip": false,
49
+ "rstrip": false,
50
+ "normalized": false,
51
+ "special": true
52
+ },
53
+ "150006": {
54
+ "content": "<think>",
55
+ "single_word": false,
56
+ "lstrip": false,
57
+ "rstrip": false,
58
+ "normalized": false,
59
+ "special": false
60
+ },
61
+ "150007": {
62
+ "content": "</think>",
63
+ "single_word": false,
64
+ "lstrip": false,
65
+ "rstrip": false,
66
+ "normalized": false,
67
+ "special": false
68
+ },
69
+ "150008": {
70
+ "content": "<tools>",
71
+ "single_word": false,
72
+ "lstrip": false,
73
+ "rstrip": false,
74
+ "normalized": false,
75
+ "special": false
76
+ },
77
+ "150009": {
78
+ "content": "</tools>",
79
+ "single_word": false,
80
+ "lstrip": false,
81
+ "rstrip": false,
82
+ "normalized": false,
83
+ "special": false
84
+ },
85
+ "150010": {
86
+ "content": "<tool_call>",
87
+ "single_word": false,
88
+ "lstrip": false,
89
+ "rstrip": false,
90
+ "normalized": false,
91
+ "special": false
92
+ },
93
+ "150011": {
94
+ "content": "</tool_call>",
95
+ "single_word": false,
96
+ "lstrip": false,
97
+ "rstrip": false,
98
+ "normalized": false,
99
+ "special": false
100
+ },
101
+ "150012": {
102
+ "content": "<arg_key>",
103
+ "single_word": false,
104
+ "lstrip": false,
105
+ "rstrip": false,
106
+ "normalized": false,
107
+ "special": false
108
+ },
109
+ "150013": {
110
+ "content": "</arg_key>",
111
+ "single_word": false,
112
+ "lstrip": false,
113
+ "rstrip": false,
114
+ "normalized": false,
115
+ "special": false
116
+ },
117
+ "150014": {
118
+ "content": "<arg_value>",
119
+ "single_word": false,
120
+ "lstrip": false,
121
+ "rstrip": false,
122
+ "normalized": false,
123
+ "special": false
124
+ },
125
+ "150015": {
126
+ "content": "</arg_value>",
127
+ "single_word": false,
128
+ "lstrip": false,
129
+ "rstrip": false,
130
+ "normalized": false,
131
+ "special": false
132
+ },
133
+ "150016": {
134
+ "content": "<tool_response>",
135
+ "single_word": false,
136
+ "lstrip": false,
137
+ "rstrip": false,
138
+ "normalized": false,
139
+ "special": false
140
+ },
141
+ "150017": {
142
+ "content": "</tool_response>",
143
+ "single_word": false,
144
+ "lstrip": false,
145
+ "rstrip": false,
146
+ "normalized": false,
147
+ "special": false
148
+ },
149
+ "150018": {
150
+ "content": "<|developer|>",
151
+ "single_word": false,
152
+ "lstrip": false,
153
+ "rstrip": false,
154
+ "normalized": false,
155
+ "special": true
156
+ },
157
+ "150019": {
158
+ "content": "<|reserved_150019|>",
159
+ "single_word": false,
160
+ "lstrip": false,
161
+ "rstrip": false,
162
+ "normalized": false,
163
+ "special": true
164
+ },
165
+ "150020": {
166
+ "content": "<|reserved_150020|>",
167
+ "single_word": false,
168
+ "lstrip": false,
169
+ "rstrip": false,
170
+ "normalized": false,
171
+ "special": true
172
+ },
173
+ "150021": {
174
+ "content": "<|reserved_150021|>",
175
+ "single_word": false,
176
+ "lstrip": false,
177
+ "rstrip": false,
178
+ "normalized": false,
179
+ "special": true
180
+ },
181
+ "150022": {
182
+ "content": "<|reserved_150022|>",
183
+ "single_word": false,
184
+ "lstrip": false,
185
+ "rstrip": false,
186
+ "normalized": false,
187
+ "special": true
188
+ },
189
+ "150023": {
190
+ "content": "<|reserved_150023|>",
191
+ "single_word": false,
192
+ "lstrip": false,
193
+ "rstrip": false,
194
+ "normalized": false,
195
+ "special": true
196
+ },
197
+ "150024": {
198
+ "content": "<|reserved_150024|>",
199
+ "single_word": false,
200
+ "lstrip": false,
201
+ "rstrip": false,
202
+ "normalized": false,
203
+ "special": true
204
+ },
205
+ "150025": {
206
+ "content": "<|reserved_150025|>",
207
+ "single_word": false,
208
+ "lstrip": false,
209
+ "rstrip": false,
210
+ "normalized": false,
211
+ "special": true
212
+ },
213
+ "150026": {
214
+ "content": "<|reserved_150026|>",
215
+ "single_word": false,
216
+ "lstrip": false,
217
+ "rstrip": false,
218
+ "normalized": false,
219
+ "special": true
220
+ },
221
+ "150027": {
222
+ "content": "<|reserved_150027|>",
223
+ "single_word": false,
224
+ "lstrip": false,
225
+ "rstrip": false,
226
+ "normalized": false,
227
+ "special": true
228
+ },
229
+ "150028": {
230
+ "content": "<|reserved_150028|>",
231
+ "single_word": false,
232
+ "lstrip": false,
233
+ "rstrip": false,
234
+ "normalized": false,
235
+ "special": true
236
+ },
237
+ "150029": {
238
+ "content": "<|reserved_150029|>",
239
+ "single_word": false,
240
+ "lstrip": false,
241
+ "rstrip": false,
242
+ "normalized": false,
243
+ "special": true
244
+ },
245
+ "150030": {
246
+ "content": "<|reserved_150030|>",
247
+ "single_word": false,
248
+ "lstrip": false,
249
+ "rstrip": false,
250
+ "normalized": false,
251
+ "special": true
252
+ },
253
+ "150031": {
254
+ "content": "<|reserved_150031|>",
255
+ "single_word": false,
256
+ "lstrip": false,
257
+ "rstrip": false,
258
+ "normalized": false,
259
+ "special": true
260
+ },
261
+ "150032": {
262
+ "content": "<|reserved_150032|>",
263
+ "single_word": false,
264
+ "lstrip": false,
265
+ "rstrip": false,
266
+ "normalized": false,
267
+ "special": true
268
+ },
269
+ "150033": {
270
+ "content": "<|reserved_150033|>",
271
+ "single_word": false,
272
+ "lstrip": false,
273
+ "rstrip": false,
274
+ "normalized": false,
275
+ "special": true
276
+ },
277
+ "150034": {
278
+ "content": "<|reserved_150034|>",
279
+ "single_word": false,
280
+ "lstrip": false,
281
+ "rstrip": false,
282
+ "normalized": false,
283
+ "special": true
284
+ },
285
+ "150035": {
286
+ "content": "<|reserved_150035|>",
287
+ "single_word": false,
288
+ "lstrip": false,
289
+ "rstrip": false,
290
+ "normalized": false,
291
+ "special": true
292
+ },
293
+ "150036": {
294
+ "content": "<|reserved_150036|>",
295
+ "single_word": false,
296
+ "lstrip": false,
297
+ "rstrip": false,
298
+ "normalized": false,
299
+ "special": true
300
+ },
301
+ "150037": {
302
+ "content": "<|reserved_150037|>",
303
+ "single_word": false,
304
+ "lstrip": false,
305
+ "rstrip": false,
306
+ "normalized": false,
307
+ "special": true
308
+ },
309
+ "150038": {
310
+ "content": "<|reserved_150038|>",
311
+ "single_word": false,
312
+ "lstrip": false,
313
+ "rstrip": false,
314
+ "normalized": false,
315
+ "special": true
316
+ },
317
+ "150039": {
318
+ "content": "<|reserved_150039|>",
319
+ "single_word": false,
320
+ "lstrip": false,
321
+ "rstrip": false,
322
+ "normalized": false,
323
+ "special": true
324
+ },
325
+ "150040": {
326
+ "content": "<|reserved_150040|>",
327
+ "single_word": false,
328
+ "lstrip": false,
329
+ "rstrip": false,
330
+ "normalized": false,
331
+ "special": true
332
+ },
333
+ "150041": {
334
+ "content": "<|reserved_150041|>",
335
+ "single_word": false,
336
+ "lstrip": false,
337
+ "rstrip": false,
338
+ "normalized": false,
339
+ "special": true
340
+ },
341
+ "150042": {
342
+ "content": "<|reserved_150042|>",
343
+ "single_word": false,
344
+ "lstrip": false,
345
+ "rstrip": false,
346
+ "normalized": false,
347
+ "special": true
348
+ },
349
+ "150043": {
350
+ "content": "<|reserved_150043|>",
351
+ "single_word": false,
352
+ "lstrip": false,
353
+ "rstrip": false,
354
+ "normalized": false,
355
+ "special": true
356
+ },
357
+ "150044": {
358
+ "content": "<|reserved_150044|>",
359
+ "single_word": false,
360
+ "lstrip": false,
361
+ "rstrip": false,
362
+ "normalized": false,
363
+ "special": true
364
+ },
365
+ "150045": {
366
+ "content": "<|reserved_150045|>",
367
+ "single_word": false,
368
+ "lstrip": false,
369
+ "rstrip": false,
370
+ "normalized": false,
371
+ "special": true
372
+ },
373
+ "150046": {
374
+ "content": "<|reserved_150046|>",
375
+ "single_word": false,
376
+ "lstrip": false,
377
+ "rstrip": false,
378
+ "normalized": false,
379
+ "special": true
380
+ },
381
+ "150047": {
382
+ "content": "<|reserved_150047|>",
383
+ "single_word": false,
384
+ "lstrip": false,
385
+ "rstrip": false,
386
+ "normalized": false,
387
+ "special": true
388
+ },
389
+ "150048": {
390
+ "content": "<|reserved_150048|>",
391
+ "single_word": false,
392
+ "lstrip": false,
393
+ "rstrip": false,
394
+ "normalized": false,
395
+ "special": true
396
+ },
397
+ "150049": {
398
+ "content": "<|reserved_150049|>",
399
+ "single_word": false,
400
+ "lstrip": false,
401
+ "rstrip": false,
402
+ "normalized": false,
403
+ "special": true
404
+ },
405
+ "150050": {
406
+ "content": "<|reserved_150050|>",
407
+ "single_word": false,
408
+ "lstrip": false,
409
+ "rstrip": false,
410
+ "normalized": false,
411
+ "special": true
412
+ },
413
+ "150051": {
414
+ "content": "<|reserved_150051|>",
415
+ "single_word": false,
416
+ "lstrip": false,
417
+ "rstrip": false,
418
+ "normalized": false,
419
+ "special": true
420
+ },
421
+ "150052": {
422
+ "content": "<|reserved_150052|>",
423
+ "single_word": false,
424
+ "lstrip": false,
425
+ "rstrip": false,
426
+ "normalized": false,
427
+ "special": true
428
+ },
429
+ "150053": {
430
+ "content": "<|reserved_150053|>",
431
+ "single_word": false,
432
+ "lstrip": false,
433
+ "rstrip": false,
434
+ "normalized": false,
435
+ "special": true
436
+ },
437
+ "150054": {
438
+ "content": "<|reserved_150054|>",
439
+ "single_word": false,
440
+ "lstrip": false,
441
+ "rstrip": false,
442
+ "normalized": false,
443
+ "special": true
444
+ },
445
+ "150055": {
446
+ "content": "<|reserved_150055|>",
447
+ "single_word": false,
448
+ "lstrip": false,
449
+ "rstrip": false,
450
+ "normalized": false,
451
+ "special": true
452
+ },
453
+ "150056": {
454
+ "content": "<|reserved_150056|>",
455
+ "single_word": false,
456
+ "lstrip": false,
457
+ "rstrip": false,
458
+ "normalized": false,
459
+ "special": true
460
+ },
461
+ "150057": {
462
+ "content": "<|reserved_150057|>",
463
+ "single_word": false,
464
+ "lstrip": false,
465
+ "rstrip": false,
466
+ "normalized": false,
467
+ "special": true
468
+ },
469
+ "150058": {
470
+ "content": "<|reserved_150058|>",
471
+ "single_word": false,
472
+ "lstrip": false,
473
+ "rstrip": false,
474
+ "normalized": false,
475
+ "special": true
476
+ },
477
+ "150059": {
478
+ "content": "<|reserved_150059|>",
479
+ "single_word": false,
480
+ "lstrip": false,
481
+ "rstrip": false,
482
+ "normalized": false,
483
+ "special": true
484
+ },
485
+ "150060": {
486
+ "content": "<|reserved_150060|>",
487
+ "single_word": false,
488
+ "lstrip": false,
489
+ "rstrip": false,
490
+ "normalized": false,
491
+ "special": true
492
+ },
493
+ "150061": {
494
+ "content": "<|reserved_150061|>",
495
+ "single_word": false,
496
+ "lstrip": false,
497
+ "rstrip": false,
498
+ "normalized": false,
499
+ "special": true
500
+ },
501
+ "150062": {
502
+ "content": "<|reserved_150062|>",
503
+ "single_word": false,
504
+ "lstrip": false,
505
+ "rstrip": false,
506
+ "normalized": false,
507
+ "special": true
508
+ },
509
+ "150063": {
510
+ "content": "<|reserved_150063|>",
511
+ "single_word": false,
512
+ "lstrip": false,
513
+ "rstrip": false,
514
+ "normalized": false,
515
+ "special": true
516
+ },
517
+ "150064": {
518
+ "content": "<|reserved_150064|>",
519
+ "single_word": false,
520
+ "lstrip": false,
521
+ "rstrip": false,
522
+ "normalized": false,
523
+ "special": true
524
+ },
525
+ "150065": {
526
+ "content": "<|reserved_150065|>",
527
+ "single_word": false,
528
+ "lstrip": false,
529
+ "rstrip": false,
530
+ "normalized": false,
531
+ "special": true
532
+ },
533
+ "150066": {
534
+ "content": "<|reserved_150066|>",
535
+ "single_word": false,
536
+ "lstrip": false,
537
+ "rstrip": false,
538
+ "normalized": false,
539
+ "special": true
540
+ },
541
+ "150067": {
542
+ "content": "<|reserved_150067|>",
543
+ "single_word": false,
544
+ "lstrip": false,
545
+ "rstrip": false,
546
+ "normalized": false,
547
+ "special": true
548
+ },
549
+ "150068": {
550
+ "content": "<|reserved_150068|>",
551
+ "single_word": false,
552
+ "lstrip": false,
553
+ "rstrip": false,
554
+ "normalized": false,
555
+ "special": true
556
+ },
557
+ "150069": {
558
+ "content": "<|reserved_150069|>",
559
+ "single_word": false,
560
+ "lstrip": false,
561
+ "rstrip": false,
562
+ "normalized": false,
563
+ "special": true
564
+ },
565
+ "150070": {
566
+ "content": "<|reserved_150070|>",
567
+ "single_word": false,
568
+ "lstrip": false,
569
+ "rstrip": false,
570
+ "normalized": false,
571
+ "special": true
572
+ },
573
+ "150071": {
574
+ "content": "<|reserved_150071|>",
575
+ "single_word": false,
576
+ "lstrip": false,
577
+ "rstrip": false,
578
+ "normalized": false,
579
+ "special": true
580
+ },
581
+ "150072": {
582
+ "content": "<|reserved_150072|>",
583
+ "single_word": false,
584
+ "lstrip": false,
585
+ "rstrip": false,
586
+ "normalized": false,
587
+ "special": true
588
+ },
589
+ "150073": {
590
+ "content": "<|reserved_150073|>",
591
+ "single_word": false,
592
+ "lstrip": false,
593
+ "rstrip": false,
594
+ "normalized": false,
595
+ "special": true
596
+ },
597
+ "150074": {
598
+ "content": "<|reserved_150074|>",
599
+ "single_word": false,
600
+ "lstrip": false,
601
+ "rstrip": false,
602
+ "normalized": false,
603
+ "special": true
604
+ },
605
+ "150075": {
606
+ "content": "<|reserved_150075|>",
607
+ "single_word": false,
608
+ "lstrip": false,
609
+ "rstrip": false,
610
+ "normalized": false,
611
+ "special": true
612
+ },
613
+ "150076": {
614
+ "content": "<|reserved_150076|>",
615
+ "single_word": false,
616
+ "lstrip": false,
617
+ "rstrip": false,
618
+ "normalized": false,
619
+ "special": true
620
+ },
621
+ "150077": {
622
+ "content": "<|reserved_150077|>",
623
+ "single_word": false,
624
+ "lstrip": false,
625
+ "rstrip": false,
626
+ "normalized": false,
627
+ "special": true
628
+ },
629
+ "150078": {
630
+ "content": "<|reserved_150078|>",
631
+ "single_word": false,
632
+ "lstrip": false,
633
+ "rstrip": false,
634
+ "normalized": false,
635
+ "special": true
636
+ },
637
+ "150079": {
638
+ "content": "<|reserved_150079|>",
639
+ "single_word": false,
640
+ "lstrip": false,
641
+ "rstrip": false,
642
+ "normalized": false,
643
+ "special": true
644
+ },
645
+ "150080": {
646
+ "content": "<|reserved_150080|>",
647
+ "single_word": false,
648
+ "lstrip": false,
649
+ "rstrip": false,
650
+ "normalized": false,
651
+ "special": true
652
+ },
653
+ "150081": {
654
+ "content": "<|reserved_150081|>",
655
+ "single_word": false,
656
+ "lstrip": false,
657
+ "rstrip": false,
658
+ "normalized": false,
659
+ "special": true
660
+ },
661
+ "150082": {
662
+ "content": "<|reserved_150082|>",
663
+ "single_word": false,
664
+ "lstrip": false,
665
+ "rstrip": false,
666
+ "normalized": false,
667
+ "special": true
668
+ },
669
+ "150083": {
670
+ "content": "<|reserved_150083|>",
671
+ "single_word": false,
672
+ "lstrip": false,
673
+ "rstrip": false,
674
+ "normalized": false,
675
+ "special": true
676
+ },
677
+ "150084": {
678
+ "content": "<|reserved_150084|>",
679
+ "single_word": false,
680
+ "lstrip": false,
681
+ "rstrip": false,
682
+ "normalized": false,
683
+ "special": true
684
+ },
685
+ "150085": {
686
+ "content": "<|reserved_150085|>",
687
+ "single_word": false,
688
+ "lstrip": false,
689
+ "rstrip": false,
690
+ "normalized": false,
691
+ "special": true
692
+ },
693
+ "150086": {
694
+ "content": "<|reserved_150086|>",
695
+ "single_word": false,
696
+ "lstrip": false,
697
+ "rstrip": false,
698
+ "normalized": false,
699
+ "special": true
700
+ },
701
+ "150087": {
702
+ "content": "<|reserved_150087|>",
703
+ "single_word": false,
704
+ "lstrip": false,
705
+ "rstrip": false,
706
+ "normalized": false,
707
+ "special": true
708
+ },
709
+ "150088": {
710
+ "content": "<|reserved_150088|>",
711
+ "single_word": false,
712
+ "lstrip": false,
713
+ "rstrip": false,
714
+ "normalized": false,
715
+ "special": true
716
+ },
717
+ "150089": {
718
+ "content": "<|reserved_150089|>",
719
+ "single_word": false,
720
+ "lstrip": false,
721
+ "rstrip": false,
722
+ "normalized": false,
723
+ "special": true
724
+ },
725
+ "150090": {
726
+ "content": "<|reserved_150090|>",
727
+ "single_word": false,
728
+ "lstrip": false,
729
+ "rstrip": false,
730
+ "normalized": false,
731
+ "special": true
732
+ },
733
+ "150091": {
734
+ "content": "<|reserved_150091|>",
735
+ "single_word": false,
736
+ "lstrip": false,
737
+ "rstrip": false,
738
+ "normalized": false,
739
+ "special": true
740
+ },
741
+ "150092": {
742
+ "content": "<|reserved_150092|>",
743
+ "single_word": false,
744
+ "lstrip": false,
745
+ "rstrip": false,
746
+ "normalized": false,
747
+ "special": true
748
+ },
749
+ "150093": {
750
+ "content": "<|reserved_150093|>",
751
+ "single_word": false,
752
+ "lstrip": false,
753
+ "rstrip": false,
754
+ "normalized": false,
755
+ "special": true
756
+ },
757
+ "150094": {
758
+ "content": "<|reserved_150094|>",
759
+ "single_word": false,
760
+ "lstrip": false,
761
+ "rstrip": false,
762
+ "normalized": false,
763
+ "special": true
764
+ },
765
+ "150095": {
766
+ "content": "<|reserved_150095|>",
767
+ "single_word": false,
768
+ "lstrip": false,
769
+ "rstrip": false,
770
+ "normalized": false,
771
+ "special": true
772
+ },
773
+ "150096": {
774
+ "content": "<|reserved_150096|>",
775
+ "single_word": false,
776
+ "lstrip": false,
777
+ "rstrip": false,
778
+ "normalized": false,
779
+ "special": true
780
+ },
781
+ "150097": {
782
+ "content": "<|reserved_150097|>",
783
+ "single_word": false,
784
+ "lstrip": false,
785
+ "rstrip": false,
786
+ "normalized": false,
787
+ "special": true
788
+ },
789
+ "150098": {
790
+ "content": "<|reserved_150098|>",
791
+ "single_word": false,
792
+ "lstrip": false,
793
+ "rstrip": false,
794
+ "normalized": false,
795
+ "special": true
796
+ },
797
+ "150099": {
798
+ "content": "<|reserved_150099|>",
799
+ "single_word": false,
800
+ "lstrip": false,
801
+ "rstrip": false,
802
+ "normalized": false,
803
+ "special": true
804
+ },
805
+ "150100": {
806
+ "content": "<|reserved_150100|>",
807
+ "single_word": false,
808
+ "lstrip": false,
809
+ "rstrip": false,
810
+ "normalized": false,
811
+ "special": true
812
+ },
813
+ "150101": {
814
+ "content": "<|reserved_150101|>",
815
+ "single_word": false,
816
+ "lstrip": false,
817
+ "rstrip": false,
818
+ "normalized": false,
819
+ "special": true
820
+ },
821
+ "150102": {
822
+ "content": "<|reserved_150102|>",
823
+ "single_word": false,
824
+ "lstrip": false,
825
+ "rstrip": false,
826
+ "normalized": false,
827
+ "special": true
828
+ },
829
+ "150103": {
830
+ "content": "<|reserved_150103|>",
831
+ "single_word": false,
832
+ "lstrip": false,
833
+ "rstrip": false,
834
+ "normalized": false,
835
+ "special": true
836
+ },
837
+ "150104": {
838
+ "content": "<|reserved_150104|>",
839
+ "single_word": false,
840
+ "lstrip": false,
841
+ "rstrip": false,
842
+ "normalized": false,
843
+ "special": true
844
+ },
845
+ "150105": {
846
+ "content": "<|reserved_150105|>",
847
+ "single_word": false,
848
+ "lstrip": false,
849
+ "rstrip": false,
850
+ "normalized": false,
851
+ "special": true
852
+ },
853
+ "150106": {
854
+ "content": "<|reserved_150106|>",
855
+ "single_word": false,
856
+ "lstrip": false,
857
+ "rstrip": false,
858
+ "normalized": false,
859
+ "special": true
860
+ },
861
+ "150107": {
862
+ "content": "<|reserved_150107|>",
863
+ "single_word": false,
864
+ "lstrip": false,
865
+ "rstrip": false,
866
+ "normalized": false,
867
+ "special": true
868
+ },
869
+ "150108": {
870
+ "content": "<|reserved_150108|>",
871
+ "single_word": false,
872
+ "lstrip": false,
873
+ "rstrip": false,
874
+ "normalized": false,
875
+ "special": true
876
+ },
877
+ "150109": {
878
+ "content": "<|reserved_150109|>",
879
+ "single_word": false,
880
+ "lstrip": false,
881
+ "rstrip": false,
882
+ "normalized": false,
883
+ "special": true
884
+ },
885
+ "150110": {
886
+ "content": "<|reserved_150110|>",
887
+ "single_word": false,
888
+ "lstrip": false,
889
+ "rstrip": false,
890
+ "normalized": false,
891
+ "special": true
892
+ },
893
+ "150111": {
894
+ "content": "<|reserved_150111|>",
895
+ "single_word": false,
896
+ "lstrip": false,
897
+ "rstrip": false,
898
+ "normalized": false,
899
+ "special": true
900
+ },
901
+ "150112": {
902
+ "content": "<|reserved_150112|>",
903
+ "single_word": false,
904
+ "lstrip": false,
905
+ "rstrip": false,
906
+ "normalized": false,
907
+ "special": true
908
+ },
909
+ "150113": {
910
+ "content": "<|reserved_150113|>",
911
+ "single_word": false,
912
+ "lstrip": false,
913
+ "rstrip": false,
914
+ "normalized": false,
915
+ "special": true
916
+ },
917
+ "150114": {
918
+ "content": "<|reserved_150114|>",
919
+ "single_word": false,
920
+ "lstrip": false,
921
+ "rstrip": false,
922
+ "normalized": false,
923
+ "special": true
924
+ },
925
+ "150115": {
926
+ "content": "<|reserved_150115|>",
927
+ "single_word": false,
928
+ "lstrip": false,
929
+ "rstrip": false,
930
+ "normalized": false,
931
+ "special": true
932
+ },
933
+ "150116": {
934
+ "content": "<|reserved_150116|>",
935
+ "single_word": false,
936
+ "lstrip": false,
937
+ "rstrip": false,
938
+ "normalized": false,
939
+ "special": true
940
+ },
941
+ "150117": {
942
+ "content": "<|reserved_150117|>",
943
+ "single_word": false,
944
+ "lstrip": false,
945
+ "rstrip": false,
946
+ "normalized": false,
947
+ "special": true
948
+ },
949
+ "150118": {
950
+ "content": "<|reserved_150118|>",
951
+ "single_word": false,
952
+ "lstrip": false,
953
+ "rstrip": false,
954
+ "normalized": false,
955
+ "special": true
956
+ },
957
+ "150119": {
958
+ "content": "<|reserved_150119|>",
959
+ "single_word": false,
960
+ "lstrip": false,
961
+ "rstrip": false,
962
+ "normalized": false,
963
+ "special": true
964
+ },
965
+ "150120": {
966
+ "content": "<|reserved_150120|>",
967
+ "single_word": false,
968
+ "lstrip": false,
969
+ "rstrip": false,
970
+ "normalized": false,
971
+ "special": true
972
+ },
973
+ "150121": {
974
+ "content": "<|reserved_150121|>",
975
+ "single_word": false,
976
+ "lstrip": false,
977
+ "rstrip": false,
978
+ "normalized": false,
979
+ "special": true
980
+ },
981
+ "150122": {
982
+ "content": "<|reserved_150122|>",
983
+ "single_word": false,
984
+ "lstrip": false,
985
+ "rstrip": false,
986
+ "normalized": false,
987
+ "special": true
988
+ },
989
+ "150123": {
990
+ "content": "<|reserved_150123|>",
991
+ "single_word": false,
992
+ "lstrip": false,
993
+ "rstrip": false,
994
+ "normalized": false,
995
+ "special": true
996
+ },
997
+ "150124": {
998
+ "content": "<|reserved_150124|>",
999
+ "single_word": false,
1000
+ "lstrip": false,
1001
+ "rstrip": false,
1002
+ "normalized": false,
1003
+ "special": true
1004
+ },
1005
+ "150125": {
1006
+ "content": "<|reserved_150125|>",
1007
+ "single_word": false,
1008
+ "lstrip": false,
1009
+ "rstrip": false,
1010
+ "normalized": false,
1011
+ "special": true
1012
+ },
1013
+ "150126": {
1014
+ "content": "<|reserved_150126|>",
1015
+ "single_word": false,
1016
+ "lstrip": false,
1017
+ "rstrip": false,
1018
+ "normalized": false,
1019
+ "special": true
1020
+ },
1021
+ "150127": {
1022
+ "content": "<|reserved_150127|>",
1023
+ "single_word": false,
1024
+ "lstrip": false,
1025
+ "rstrip": false,
1026
+ "normalized": false,
1027
+ "special": true
1028
+ },
1029
+ "150128": {
1030
+ "content": "<|reserved_150128|>",
1031
+ "single_word": false,
1032
+ "lstrip": false,
1033
+ "rstrip": false,
1034
+ "normalized": false,
1035
+ "special": true
1036
+ },
1037
+ "150129": {
1038
+ "content": "<|reserved_150129|>",
1039
+ "single_word": false,
1040
+ "lstrip": false,
1041
+ "rstrip": false,
1042
+ "normalized": false,
1043
+ "special": true
1044
+ },
1045
+ "150130": {
1046
+ "content": "<|reserved_150130|>",
1047
+ "single_word": false,
1048
+ "lstrip": false,
1049
+ "rstrip": false,
1050
+ "normalized": false,
1051
+ "special": true
1052
+ },
1053
+ "150131": {
1054
+ "content": "<|reserved_150131|>",
1055
+ "single_word": false,
1056
+ "lstrip": false,
1057
+ "rstrip": false,
1058
+ "normalized": false,
1059
+ "special": true
1060
+ },
1061
+ "150132": {
1062
+ "content": "<|reserved_150132|>",
1063
+ "single_word": false,
1064
+ "lstrip": false,
1065
+ "rstrip": false,
1066
+ "normalized": false,
1067
+ "special": true
1068
+ },
1069
+ "150133": {
1070
+ "content": "<|reserved_150133|>",
1071
+ "single_word": false,
1072
+ "lstrip": false,
1073
+ "rstrip": false,
1074
+ "normalized": false,
1075
+ "special": true
1076
+ },
1077
+ "150134": {
1078
+ "content": "<|reserved_150134|>",
1079
+ "single_word": false,
1080
+ "lstrip": false,
1081
+ "rstrip": false,
1082
+ "normalized": false,
1083
+ "special": true
1084
+ },
1085
+ "150135": {
1086
+ "content": "<|reserved_150135|>",
1087
+ "single_word": false,
1088
+ "lstrip": false,
1089
+ "rstrip": false,
1090
+ "normalized": false,
1091
+ "special": true
1092
+ },
1093
+ "150136": {
1094
+ "content": "<|reserved_150136|>",
1095
+ "single_word": false,
1096
+ "lstrip": false,
1097
+ "rstrip": false,
1098
+ "normalized": false,
1099
+ "special": true
1100
+ },
1101
+ "150137": {
1102
+ "content": "<|reserved_150137|>",
1103
+ "single_word": false,
1104
+ "lstrip": false,
1105
+ "rstrip": false,
1106
+ "normalized": false,
1107
+ "special": true
1108
+ },
1109
+ "150138": {
1110
+ "content": "<|reserved_150138|>",
1111
+ "single_word": false,
1112
+ "lstrip": false,
1113
+ "rstrip": false,
1114
+ "normalized": false,
1115
+ "special": true
1116
+ },
1117
+ "150139": {
1118
+ "content": "<|reserved_150139|>",
1119
+ "single_word": false,
1120
+ "lstrip": false,
1121
+ "rstrip": false,
1122
+ "normalized": false,
1123
+ "special": true
1124
+ },
1125
+ "150140": {
1126
+ "content": "<|reserved_150140|>",
1127
+ "single_word": false,
1128
+ "lstrip": false,
1129
+ "rstrip": false,
1130
+ "normalized": false,
1131
+ "special": true
1132
+ },
1133
+ "150141": {
1134
+ "content": "<|reserved_150141|>",
1135
+ "single_word": false,
1136
+ "lstrip": false,
1137
+ "rstrip": false,
1138
+ "normalized": false,
1139
+ "special": true
1140
+ },
1141
+ "150142": {
1142
+ "content": "<|reserved_150142|>",
1143
+ "single_word": false,
1144
+ "lstrip": false,
1145
+ "rstrip": false,
1146
+ "normalized": false,
1147
+ "special": true
1148
+ },
1149
+ "150143": {
1150
+ "content": "<|reserved_150143|>",
1151
+ "single_word": false,
1152
+ "lstrip": false,
1153
+ "rstrip": false,
1154
+ "normalized": false,
1155
+ "special": true
1156
+ }
1157
+ },
1158
+ "additional_special_tokens": [
1159
+ "<|system|>",
1160
+ "<|user|>",
1161
+ "<|assistant|>",
1162
+ "<|observation|>",
1163
+ "<|developer|>",
1164
+ "<|reserved_150019|>",
1165
+ "<|reserved_150020|>",
1166
+ "<|reserved_150021|>",
1167
+ "<|reserved_150022|>",
1168
+ "<|reserved_150023|>",
1169
+ "<|reserved_150024|>",
1170
+ "<|reserved_150025|>",
1171
+ "<|reserved_150026|>",
1172
+ "<|reserved_150027|>",
1173
+ "<|reserved_150028|>",
1174
+ "<|reserved_150029|>",
1175
+ "<|reserved_150030|>",
1176
+ "<|reserved_150031|>",
1177
+ "<|reserved_150032|>",
1178
+ "<|reserved_150033|>",
1179
+ "<|reserved_150034|>",
1180
+ "<|reserved_150035|>",
1181
+ "<|reserved_150036|>",
1182
+ "<|reserved_150037|>",
1183
+ "<|reserved_150038|>",
1184
+ "<|reserved_150039|>",
1185
+ "<|reserved_150040|>",
1186
+ "<|reserved_150041|>",
1187
+ "<|reserved_150042|>",
1188
+ "<|reserved_150043|>",
1189
+ "<|reserved_150044|>",
1190
+ "<|reserved_150045|>",
1191
+ "<|reserved_150046|>",
1192
+ "<|reserved_150047|>",
1193
+ "<|reserved_150048|>",
1194
+ "<|reserved_150049|>",
1195
+ "<|reserved_150050|>",
1196
+ "<|reserved_150051|>",
1197
+ "<|reserved_150052|>",
1198
+ "<|reserved_150053|>",
1199
+ "<|reserved_150054|>",
1200
+ "<|reserved_150055|>",
1201
+ "<|reserved_150056|>",
1202
+ "<|reserved_150057|>",
1203
+ "<|reserved_150058|>",
1204
+ "<|reserved_150059|>",
1205
+ "<|reserved_150060|>",
1206
+ "<|reserved_150061|>",
1207
+ "<|reserved_150062|>",
1208
+ "<|reserved_150063|>",
1209
+ "<|reserved_150064|>",
1210
+ "<|reserved_150065|>",
1211
+ "<|reserved_150066|>",
1212
+ "<|reserved_150067|>",
1213
+ "<|reserved_150068|>",
1214
+ "<|reserved_150069|>",
1215
+ "<|reserved_150070|>",
1216
+ "<|reserved_150071|>",
1217
+ "<|reserved_150072|>",
1218
+ "<|reserved_150073|>",
1219
+ "<|reserved_150074|>",
1220
+ "<|reserved_150075|>",
1221
+ "<|reserved_150076|>",
1222
+ "<|reserved_150077|>",
1223
+ "<|reserved_150078|>",
1224
+ "<|reserved_150079|>",
1225
+ "<|reserved_150080|>",
1226
+ "<|reserved_150081|>",
1227
+ "<|reserved_150082|>",
1228
+ "<|reserved_150083|>",
1229
+ "<|reserved_150084|>",
1230
+ "<|reserved_150085|>",
1231
+ "<|reserved_150086|>",
1232
+ "<|reserved_150087|>",
1233
+ "<|reserved_150088|>",
1234
+ "<|reserved_150089|>",
1235
+ "<|reserved_150090|>",
1236
+ "<|reserved_150091|>",
1237
+ "<|reserved_150092|>",
1238
+ "<|reserved_150093|>",
1239
+ "<|reserved_150094|>",
1240
+ "<|reserved_150095|>",
1241
+ "<|reserved_150096|>",
1242
+ "<|reserved_150097|>",
1243
+ "<|reserved_150098|>",
1244
+ "<|reserved_150099|>",
1245
+ "<|reserved_150100|>",
1246
+ "<|reserved_150101|>",
1247
+ "<|reserved_150102|>",
1248
+ "<|reserved_150103|>",
1249
+ "<|reserved_150104|>",
1250
+ "<|reserved_150105|>",
1251
+ "<|reserved_150106|>",
1252
+ "<|reserved_150107|>",
1253
+ "<|reserved_150108|>",
1254
+ "<|reserved_150109|>",
1255
+ "<|reserved_150110|>",
1256
+ "<|reserved_150111|>",
1257
+ "<|reserved_150112|>",
1258
+ "<|reserved_150113|>",
1259
+ "<|reserved_150114|>",
1260
+ "<|reserved_150115|>",
1261
+ "<|reserved_150116|>",
1262
+ "<|reserved_150117|>",
1263
+ "<|reserved_150118|>",
1264
+ "<|reserved_150119|>",
1265
+ "<|reserved_150120|>",
1266
+ "<|reserved_150121|>",
1267
+ "<|reserved_150122|>",
1268
+ "<|reserved_150123|>",
1269
+ "<|reserved_150124|>",
1270
+ "<|reserved_150125|>",
1271
+ "<|reserved_150126|>",
1272
+ "<|reserved_150127|>",
1273
+ "<|reserved_150128|>",
1274
+ "<|reserved_150129|>",
1275
+ "<|reserved_150130|>",
1276
+ "<|reserved_150131|>",
1277
+ "<|reserved_150132|>",
1278
+ "<|reserved_150133|>",
1279
+ "<|reserved_150134|>",
1280
+ "<|reserved_150135|>",
1281
+ "<|reserved_150136|>",
1282
+ "<|reserved_150137|>",
1283
+ "<|reserved_150138|>",
1284
+ "<|reserved_150139|>",
1285
+ "<|reserved_150140|>",
1286
+ "<|reserved_150141|>",
1287
+ "<|reserved_150142|>",
1288
+ "<|reserved_150143|>"
1289
+ ],
1290
+ "bos_token": "<|begin_of_text|>",
1291
+ "clean_up_tokenization_spaces": false,
1292
+ "eos_token": "<|end_of_text|>",
1293
+ "model_max_length": 262144,
1294
+ "tokenizer_class": "PreTrainedTokenizerFast"
1295
+ }
vocab.json ADDED
The diff for this file is too large to render. See raw diff