Add files using upload-large-folder tool

README.md

@@ -0,0 +1,112 @@
+---
+license: apache-2.0
+license_link: https://huggingface.co/tencent/Hy3/blob/main/LICENSE
+base_model: tencent/Hy3
+base_model_relation: quantized
+library_name: mlx
+pipeline_tag: text-generation
+tags:
+- mlx
+- oq
+- oqe
+- imatrix
+- quantized
+- moe
+- hunyuan
+---
+# Hy3 (MLX, oQ2e)
+
+Calibrated 2-bit MLX quantization of [tencent/Hy3](https://huggingface.co/tencent/Hy3) (Hunyuan 3.0, 295B-A21B MoE), produced with **omlx** oQe at level 2 — **2.44 bits/weight effective, 90 GB on disk**. For Apple Silicon.
+
+Same family as [mlx-community/Hy3-oQ2](https://huggingface.co/mlx-community/Hy3-oQ2), but **9 GB smaller** (90 GB vs 99 GB). Two changes get there:
+
+- **`group_size=128` on the routed experts** instead of 64. Affine quantization stores an fp16 scale and bias per group, so the metadata costs `32 / group_size` bits per weight. Doubling the group halves that overhead: 0.50 → 0.25 bpw. The experts are 97.6% of the model (288B of 295B params), so this alone is the entire saving.
+- **imatrix-weighted quantization (oQe)** to pay for it. A larger group means four quantization levels now have to cover 128 weights instead of 64 — one outlier stretches the scale for twice as many neighbours. oQe weights the error by a measured importance matrix, so within each group the bits land on the weights that actually matter.
+
+Attention, embeddings and `lm_head` stay at **8-bit, group_size 64** — untouched from oQ2. The size reduction comes entirely from the expert FFN.
+
+## Regular oQ2 goes cuckoo at group_size=128
+
+Without the imatrix, `group_size=128` at the same 90 GB is worse. Same prompt, same greedy decoding — plain gs=128 emits a dead statement and an off-by-one loop where this model (and oQ2) get it right:
+
+```python
+    # plain 2-bit gs=128, no imatrix — broken
+    if n == 1:
+        scalar = 1              # dead statement, no return
+    else:
+        a, b = 0, 1
+        for _ in range(2, n):   # off-by-one: returns F(n-1)
+            a, b = b, a + b
+        return b
+```
+
+The imatrix is what makes the larger group survivable.
+
+## Requirements
+
+mlx-lm doesn't support the `hy_v3` architecture yet — there's an open PR: [mlx-lm#1211](https://github.com/ml-explore/mlx-lm/pull/1211). Until it lands, install mlx-lm from the PR branch, otherwise the model won't load:
+
+```bash
+uv pip install "mlx-lm @ git+https://github.com/kernelpool/mlx-lm.git@add-hy3-preview"
+```
+
+If you’re using `oMLX`, Hy3 is supported already.
+
+## How it was quantized
+
+Hy3 is ~590 GB in BF16 — larger than RAM on a 128 GB machine — so neither of oQe's two calibration passes can load the source model. Both have to be routed around, and they fail differently.
+
+1. **Sensitivity** — pass an existing quant as `sensitivity_model_path` (I used oQ2). omlx's automatic fallback builds a uniform 4-bit proxy of a 295B model, which does not survive on this hardware.
+
+2. **imatrix** — the collection pass has the *same* problem, but `sensitivity_model_path` does not cover it: it is a separate code path with its own proxy builder. The way out is a cache hit, since a cached imatrix skips collection entirely. The cache key is the source checkpoint, the calibration set and the sample/sequence counts — **not** the bits or the group size, because per-column importance is orthogonal to how the weights are later grouped. So an imatrix collected once for oQ2e at `group_size=64` is valid, unchanged, for `group_size=128`.
+
+   One catch: Hy3 declares `num_nextn_predict_layers: 1`, and omlx force-recollects when a model declares MTP heads but the cache has no `mtp.*` entries — even on a signature match. With `preserve_mtp=False` the MTP tensors are stripped from the output anyway, so calibrating that head is work thrown away; skipping the recollect is the correct behaviour here, not a workaround.
+
+With both passes served from cache, the quantization itself streams tensor-by-tensor and never holds the full model.
+
+## Memory and speed
+
+Measured with the oMLX benchmark (Engine: Auto) on a **MacBook Pro M5 Max, 128 GB, 40-core GPU**. Generating 128 tokens, single request:
+
+| context | TTFT | prompt | generation | peak memory |
+|---|---|---|---|---|
+| 1024 | 1.7 s | 618 tok/s | 29.7 tok/s | 84.7 GB |
+| 4096 | 7.8 s | 523 tok/s | 25.2 tok/s | 86.2 GB |
+| 8192 | 16.2 s | 504 tok/s | 23.0 tok/s | 87.9 GB |
+| 16384 | 31.6 s | 519 tok/s | 19.8 tok/s | 89.6 GB |
+
+The 1024 row is the mean of three runs; the longer contexts are single measurements.
+
+Continuous batching at 1024-token prompts scales to **68 tok/s aggregate at 8 concurrent requests**:
+
+| batch | generation | speedup |
+|---|---|---|
+| 1× | 29.7 tok/s | 1.00× |
+| 2× | 42.2 tok/s | 1.44× |
+| 4× | 54.6 tok/s | 1.83× |
+| 8× | 67.6 tok/s | 2.28× |
+
+The headroom is what the 9 GB bought. oQ2 peaks at 99 GB against the default Metal working-set cap of 107.5 GB on a 128 GB machine, so the KV cache runs out of room at long context; this one is still at 89.6 GB with a 16k prompt.
+
+**TurboQuant KV did not pay off here** — it slowed down batched prompt processing, so these numbers are all without it. I don't have an explanation yet. With this much headroom you likely don't need it anyway.
+
+## Conversion check
+
+Smoke-tested with `mlx_lm.generate`, greedy: an iterative Fibonacci function (correct, no corrupted tokens) and a backend-agnostic Keras 3 autoencoder (real bottleneck, `keras.ops` throughout, parses clean). oQ2 passes both as well; plain 2-bit `group_size=128` without the imatrix fails the first.
+
+This is a smoke test, not a benchmark — **it does not establish parity with oQ2**. If you run an executable code benchmark on both, please post the numbers in a discussion.
+
+## Usage
+
+```bash
+python -m mlx_lm generate --model mlx-community/Hy3-oQ2e --prompt "Explain Bayes' theorem in two sentences." --max-tokens 300
+```
+
+```python
+from mlx_lm import load, generate
+model, tokenizer = load("mlx-community/Hy3-oQ2e")
+```
+
+## License
+
+[Apache-2.0](https://huggingface.co/tencent/Hy3/blob/main/LICENSE), inherited from the base model. Refer to the original model card for architecture, benchmarks, and intended use.

chat_template.jinja

@@ -0,0 +1,222 @@
+{#- ----------‑‑‑ special token variables ‑‑‑---------- -#}
+{%- set HYTK = ':opensource' %}
+{%- set eos_token = '<｜hy_eos{}｜>'.format(HYTK) %}
+{%- set bos_token = '<｜hy_begin_of_sentence{}｜>'.format(HYTK) %}
+{%- set pad_token = '<｜hy_pad{}｜>'.format(HYTK) %}
+{%- set user_token = '<｜hy_User{}｜>'.format(HYTK) %}
+{%- set assistant_token = '<｜hy_Assistant{}｜>'.format(HYTK) %}
+{%- set think_begin_token = '<think{}>'.format(HYTK) %}
+{%- set think_end_token = '</think{}>'.format(HYTK) %}
+{%- set toolcalls_begin_token = '<tool_calls{}>'.format(HYTK) %}
+{%- set toolcalls_end_token = '</tool_calls{}>'.format(HYTK) %}
+{%- set toolcall_begin_token = '<tool_call{}>'.format(HYTK) %}
+{%- set toolcall_end_token = '</tool_call{}>'.format(HYTK) %}
+{%- set toolsep_token = '<tool_sep{}>'.format(HYTK) %}
+{%- set argkey_begin_token = '<arg_key{}>'.format(HYTK) %}
+{%- set argkey_end_token = '</arg_key{}>'.format(HYTK) %}
+{%- set argvalue_begin_token = '<arg_value{}>'.format(HYTK) %}
+{%- set argvalue_end_token = '</arg_value{}>'.format(HYTK) %}
+{%- set toolresponses_begin_token = '<tool_responses{}>'.format(HYTK) %}
+{%- set toolresponses_end_token = '</tool_responses{}>'.format(HYTK) %}
+{%- set toolresponse_begin_token = '<tool_response{}>'.format(HYTK) %}
+{%- set toolresponse_end_token = '</tool_response{}>'.format(HYTK) %}
+{%- set reasoning_mode_token = '<｜reasoning_mode{}｜>'.format(HYTK) %}
+
+{#- ----------‑‑‑ hyperparameters variables ‑‑‑---------- -#}
+{%- if not add_generation_prompt is defined %}
+    {%- set add_generation_prompt = false %}
+{%- endif %}
+{%- if not preserved_thinking is defined %}
+    {%- if not tools %}
+        {%- set preserved_thinking = false %}
+    {%- else %}
+        {%- set preserved_thinking = true %}
+    {%- endif %}
+{%- endif %}
+{%- if not is_training is defined %}
+    {%- set is_training = false %}
+{%- endif %}
+
+{%- if not reasoning_effort is defined %}
+    {%- set reasoning_effort = 'no_think' %}
+{%- elif reasoning_effort not in ['high', 'low', 'no_think'] %}
+    {%- if reasoning_effort is none %}
+        {{- raise_exception('reasoning_effort error : None, should be no_think/low/high') }}
+    {%- else %}
+        {{- raise_exception('reasoning_effort error : ' + reasoning_effort + ', should be no_think/low/high') }}
+    {%- endif %}
+{%- endif %}
+
+{%- if fallback_strategy is defined and fallback_strategy == 'reasoning_toolcall_retry' %}
+    {%- set reasoning_effort = 'high' %}
+    {%- set add_generation_prompt = false %}
+{%- endif %}
+{%- if not raw_last_assistant is defined %}
+    {%- set raw_last_assistant = false %}
+{%- endif %}
+
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- elif content is none -%}
+        {{- '' }}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- set ns = namespace(last_user_index=-1) %}
+{%- set sp_ns = namespace(system_prompt='', is_first_sp=true) %}
+{%- for message in messages %}
+    {%- if message['role'] == 'system' %}
+        {%- set sp_ns.system_prompt = sp_ns.system_prompt + visible_text(message['content']) %}
+    {%- endif %}
+    {%- if message['role'] == 'user' %}
+        {%- set ns.last_user_index = loop.index0 %}
+    {%- endif %}
+{%- endfor %}
+{%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' and not tools %}
+    {%- set sp_ns.system_prompt = sp_ns.system_prompt + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort %}
+{%- endif %}
+{{- bos_token }}
+{{- sp_ns.system_prompt }}
+{%- if tools %}
+    {%- if sp_ns.system_prompt != '' %}
+        {{- '\n\n# Tools\n\nYou may call one or more functions to assist with the user query.' }}
+    {%- else %}
+        {{- '# Tools\n\nYou may call one or more functions to assist with the user query.' }}
+    {%- endif %}
+    {{- '\n\nYou are provided with function signatures within <tools></tools> XML tags:' }}
+    {{- '\n<tools>\n' }}
+    {%- for tool in tools %}
+        {%- if loop.index0 > 0 %}
+            {{- '\n' }}
+        {%- endif %}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- '\n</tools>\n\n' }}
+    {{- 'For function call returns, you should first print ' + toolcalls_begin_token + '\n' }}
+    {{- 'For each function call, you should return object like:\n' }}
+    {{- toolcall_begin_token + '{function-name}' + toolsep_token + '\n' }}
+    {{- argkey_begin_token + '{arg-key-1}' + argkey_end_token + '\n' }}
+    {{- argvalue_begin_token + '{arg-value-1}' + argvalue_end_token + '\n' }}
+    {{- argkey_begin_token + '{arg-key-2}' + argkey_end_token + '\n' }}
+    {{- argvalue_begin_token + '{arg-value-2}' + argvalue_end_token + '\n' }}
+    {{- '...\n' }}
+    {{- toolcall_end_token + '\n' }}
+    {%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' %}
+        {{- 'At the end of function call returns, you should print ' + toolcalls_end_token + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort }}
+    {%- else %}
+        {{- 'At the end of function call returns, you should print ' + toolcalls_end_token }}
+    {%- endif %}
+{%- endif %}
+
+{%- set prev_ns = namespace(is_tool=false, is_tool_first=true) %}
+{%- set last_ns = namespace(last_is_assistant=false) %}
+{%- for message in messages %}
+    {%- if message['role'] == 'user' %}
+        {%- if prev_ns.is_tool %}
+            {{- toolresponses_end_token }}
+        {%- endif %}
+        {{- user_token + visible_text(message['content']) }}
+        {%- set prev_ns.is_tool = false %}
+    {%- endif %}
+    {%- if message['role'] == 'assistant' %}
+        {%- if is_training %}
+            {%- if 'reasoning_content' in message and message['reasoning_content'] is string %}
+                {%- set rc = message['reasoning_content'] %}
+            {%- elif 'reasoning' in message and message['reasoning'] is string %}
+                {%- set rc = message['reasoning'] %}
+            {%- else %}
+                {%- set rc = none %}
+            {%- endif %}
+            {%- if rc is not none %}
+                {%- set content = think_begin_token + rc + think_end_token + visible_text(message['content']) %}
+            {%- else %}
+                {%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
+            {%- endif %}
+        {%- else %}
+            {%- if ((preserved_thinking is defined and preserved_thinking) or loop.index0 > ns.last_user_index) %}
+                {%- if 'reasoning_content' in message and message['reasoning_content'] is string %}
+                    {%- set rc = message['reasoning_content'] %}
+                {%- elif 'reasoning' in message and message['reasoning'] is string %}
+                    {%- set rc = message['reasoning'] %}
+                {%- else %}
+                    {%- set rc = none %}
+                {%- endif %}
+                {%- if rc is not none %}
+                    {%- set content = think_begin_token + rc + think_end_token + visible_text(message['content']) %}
+                {%- else %}
+                    {%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
+                {%- endif %}
+            {%- else %}
+                {%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
+            {%- endif %}
+        {%- endif %}
+        {%- if prev_ns.is_tool %}
+            {{- toolresponses_end_token }}
+        {%- endif %}
+        {{- assistant_token }}
+        {%- if message['tool_calls'] is defined and message['tool_calls'] %}
+            {%- set prev_ns.is_tool_first = true %}
+            {{- content }}
+            {{- toolcalls_begin_token + '\n' }}
+            {%- for tool in message['tool_calls'] %}
+                {%- set arguments = tool['function']['arguments'] %}
+                {{- toolcall_begin_token + tool['function']['name'] + toolsep_token + '\n' }}
+                {%- for key, value in arguments.items() %}
+                    {{- argkey_begin_token + key + argkey_end_token + '\n' }}
+                    {%- if value is not string %}
+                        {%- set value = value | tojson(ensure_ascii=False) %}
+                    {%- endif %}
+                    {{- argvalue_begin_token + value + argvalue_end_token + '\n' }}
+                {%- endfor %}
+                {{- toolcall_end_token + '\n' }}
+            {%- endfor %}
+            {{- toolcalls_end_token + eos_token }}
+        {%- else %}
+            {%- if loop.last and raw_last_assistant %}
+                {{- visible_text(message['content']) }}
+            {%- elif not loop.last or is_training %}
+                {{- content + eos_token }}
+            {%- else %}
+                {{- content }}
+            {%- endif %}
+        {%- endif %}
+        {%- set prev_ns.is_tool = false %}
+    {%- endif %}
+    {%- if message['role'] == 'tool' %}
+        {%- set prev_ns.is_tool = true %}
+        {%- if prev_ns.is_tool_first %}
+            {{- toolresponses_begin_token + '\n' }}
+            {%- set prev_ns.is_tool_first = false %}
+        {%- endif %}
+        {{- toolresponse_begin_token + '\n' + visible_text(message['content']) + '\n' + toolresponse_end_token + '\n' }}
+    {%- endif %}
+    {%- if loop.last and message['role'] == 'assistant' %}
+        {%- set last_ns.last_is_assistant = true %}
+    {%- endif %}
+
+{%- endfor %}
+{%- if prev_ns.is_tool %}
+    {{- toolresponses_end_token }}
+{%- endif %}
+{%- if add_generation_prompt %}
+    {%- if not last_ns.last_is_assistant %}
+        {%- if reasoning_effort is defined and reasoning_effort in ['low', 'high'] %}
+            {{- assistant_token + think_begin_token }}
+        {%- elif reasoning_effort is defined and reasoning_effort == 'no_think' %}
+            {{- assistant_token + think_begin_token + think_end_token }}
+        {%- else %}
+            {{- assistant_token }}
+        {%- endif %}
+    {%- endif %}
+{%- endif %}

generation_config.json

@@ -0,0 +1,10 @@
+{
+    "bos_token_id": 120000,
+    "do_sample": true,
+    "eos_token_id": 120025,
+    "pad_token_id": 120002,
+    "temperature": 0.9,
+    "top_k": -1,
+    "top_p": 1,
+    "transformers_version": "5.6.0"
+}

model-00001-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53c5c237aa53ddb67108e39bde49be009f45ee823a701703cf1bfb6bb32208d9
+size 5264213231

model-00002-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb394dc163a3b5088d1f8ee1074794c4306e77227284db619dff6e808b6c7e06
+size 5006144011

model-00003-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3afec9110c5bff343d595fca0fb98635f211906c0e7ea037124f3b070896cd24
+size 5083583062

model-00004-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:418ac6133b90617162e60298a60532282275382fd4a10e4f5c8b3ddfcbdc4dcd
+size 5244027753

model-00005-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3de4f4e00f47a6453fcce0606a575bec11e53b50ec5a00cf2208da65f0047ac
+size 5185446870

model-00006-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e8081df8ceda99341a0f39c7aa44e88ae4addaf24d26a428a17954a6d6be021
+size 5265656705

model-00007-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c26ac73320c0b517533e53697c942cacb0fbc550cedcffb894772ebd06b8803e
+size 5244027773

model-00008-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eebc90d7262b7cb798e4287734801114f7c4614dd180b0d314b85a28ed1345f
+size 5185446872

model-00009-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1760cc0f11bb3bd609b0dc3f53f20d2be9a500536a2bf65ec72c07d7f7e50a6
+size 5265656735

model-00010-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67824283951c8f4b6a015c8918c52241af5f1651c6babd2bed3f1b89c830a74e
+size 5244027705

model-00011-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29122def5115f920e28b9c9422ec419652fcfdc14623583d14e5d76dc7ea5ca4
+size 5185446880

model-00012-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11e1d5acc35ce1af71c2fd0ade932b50a60207d2e1d4ef19caf35b2657c8fd51
+size 5265656717

model-00013-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1affa0deb282cb273add64909df5285a9a721a8bc73cd31781fe34b76e430a08
+size 5244027757

model-00014-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:187b5f96ffaf993d2778813b3f18de1a62d53848a81953cfe3624a44bb38a4b7
+size 5185446892

model-00015-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5bbd011a8e8bb3ac0a50974dd61a5c0d21cfcadbfd11918fb25cb5d15d302e1
+size 5265656749

model-00016-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a743fbc48d19911479c81a4cd28ef9f2e831812fc0f695022c049f66bc975e3
+size 5244027753

model-00017-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82968cf3cf155e0025c83eed76b3c0930086f41a4b9372556f08ff87cbd997e2
+size 5185446836

model-00018-of-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24ac1746ec4c5be0928035ea31f71545fb9ef4ad20f29db708f05c896565b843
+size 1307211223

oq_imatrix_report.json

@@ -0,0 +1,897 @@
+{
+  "enabled": true,
+  "cache_path": "/Users/gabfssilva/.cache/huggingface/hub/models--local--Hy3-mlx-oQ2e/snapshots/.oqe_imatrix/f25e17585fa97688711042f822a8bedf975fb80e-oQe-s128-l512.npz",
+  "cache_reused": true,
+  "entry_count": 876,
+  "calib_dataset": "oqe_code_multilingual",
+  "collection": {
+    "dataset": "oqe_code_multilingual",
+    "requested_samples": 128,
+    "seq_length": 512,
+    "adaptive": true,
+    "adaptive_step_samples": 128,
+    "adaptive_max_samples": 1024,
+    "available_samples": 1024,
+    "micro_batch_size": 12,
+    "micro_batches": 11,
+    "batch_plan": {
+      "micro_batch_size": 12,
+      "estimated_sample_bytes": 67108864,
+      "capture_budget_bytes": 805306368,
+      "system_available_bytes": 116966981632,
+      "metal_available_bytes": 115444302520,
+      "live_available_bytes": 115444302520,
+      "hidden_size": 4096,
+      "num_experts": 192,
+      "top_k": 8
+    },
+    "processed_samples": 128,
+    "installed_modules": 877,
+    "capture_module_classes": {
+      "Linear": 79,
+      "QuantizedLinear": 561,
+      "QuantizedSwitchLinear": 237
+    },
+    "switch_capture_modules": 237,
+    "requires_expert_counts": true,
+    "coverage_sufficient": true,
+    "collection_sufficient": true,
+    "coverage": {
+      "has_expert_counts": true,
+      "expert_modules": 237,
+      "total_experts": 45504,
+      "active_experts": 45504,
+      "zero_count_experts": 0,
+      "active_ratio": 1.0,
+      "min_count": 24,
+      "p05_count": 1045.0,
+      "p10_count": 1331.0,
+      "median_count": 2472.0,
+      "max_count": 24939,
+      "min_required_count": 16,
+      "required_percentile": 5
+    },
+    "rounds": [
+      {
+        "processed_samples": 128,
+        "coverage_sufficient": true,
+        "collection_sufficient": true,
+        "coverage": {
+          "has_expert_counts": true,
+          "expert_modules": 237,
+          "total_experts": 45504,
+          "active_experts": 45504,
+          "zero_count_experts": 0,
+          "active_ratio": 1.0,
+          "min_count": 24,
+          "p05_count": 1045.0,
+          "p10_count": 1331.0,
+          "median_count": 2472.0,
+          "max_count": 24939,
+          "min_required_count": 16,
+          "required_percentile": 5
+        }
+      }
+    ]
+  },
+  "expert_coverage": {
+    "has_expert_counts": true,
+    "expert_modules": 237,
+    "total_experts": 45504,
+    "active_experts": 45504,
+    "zero_count_experts": 0,
+    "active_ratio": 1.0,
+    "min_count": 24,
+    "p05_count": 1045.0,
+    "p10_count": 1331.0,
+    "median_count": 2472.0,
+    "max_count": 24939,
+    "min_required_count": 16,
+    "required_percentile": 5
+  },
+  "applied": [
+    "model.layers.0.mlp.down_proj",
+    "model.layers.0.mlp.gate_proj",
+    "model.layers.0.mlp.up_proj",
+    "model.layers.0.self_attn.k_proj",
+    "model.layers.0.self_attn.o_proj",
+    "model.layers.0.self_attn.q_proj",
+    "model.layers.0.self_attn.v_proj",
+    "model.layers.1.mlp.shared_mlp.down_proj",
+    "model.layers.1.mlp.shared_mlp.gate_proj",
+    "model.layers.1.mlp.shared_mlp.up_proj",
+    "model.layers.1.mlp.switch_mlp.down_proj",
+    "model.layers.1.mlp.switch_mlp.gate_proj",
+    "model.layers.1.mlp.switch_mlp.up_proj",
+    "model.layers.1.self_attn.k_proj",
+    "model.layers.1.self_attn.o_proj",
+    "model.layers.1.self_attn.q_proj",
+    "model.layers.1.self_attn.v_proj",
+    "model.layers.10.mlp.shared_mlp.down_proj",
+    "model.layers.10.mlp.shared_mlp.gate_proj",
+    "model.layers.10.mlp.shared_mlp.up_proj",
+    "model.layers.10.mlp.switch_mlp.down_proj",
+    "model.layers.10.mlp.switch_mlp.gate_proj",
+    "model.layers.10.mlp.switch_mlp.up_proj",
+    "model.layers.10.self_attn.k_proj",
+    "model.layers.10.self_attn.o_proj",
+    "model.layers.10.self_attn.q_proj",
+    "model.layers.10.self_attn.v_proj",
+    "model.layers.11.mlp.shared_mlp.down_proj",
+    "model.layers.11.mlp.shared_mlp.gate_proj",
+    "model.layers.11.mlp.shared_mlp.up_proj",
+    "model.layers.11.mlp.switch_mlp.down_proj",
+    "model.layers.11.mlp.switch_mlp.gate_proj",
+    "model.layers.11.mlp.switch_mlp.up_proj",
+    "model.layers.11.self_attn.k_proj",
+    "model.layers.11.self_attn.o_proj",
+    "model.layers.11.self_attn.q_proj",
+    "model.layers.11.self_attn.v_proj",
+    "model.layers.12.mlp.shared_mlp.down_proj",
+    "model.layers.12.mlp.shared_mlp.gate_proj",
+    "model.layers.12.mlp.shared_mlp.up_proj",
+    "model.layers.12.mlp.switch_mlp.down_proj",
+    "model.layers.12.mlp.switch_mlp.gate_proj",
+    "model.layers.12.mlp.switch_mlp.up_proj",
+    "model.layers.12.self_attn.k_proj",
+    "model.layers.12.self_attn.o_proj",
+    "model.layers.12.self_attn.q_proj",
+    "model.layers.12.self_attn.v_proj",
+    "model.layers.13.mlp.shared_mlp.down_proj",
+    "model.layers.13.mlp.shared_mlp.gate_proj",
+    "model.layers.13.mlp.shared_mlp.up_proj",
+    "model.layers.13.mlp.switch_mlp.down_proj",
+    "model.layers.13.mlp.switch_mlp.gate_proj",
+    "model.layers.13.mlp.switch_mlp.up_proj",
+    "model.layers.13.self_attn.k_proj",
+    "model.layers.13.self_attn.o_proj",
+    "model.layers.13.self_attn.q_proj",
+    "model.layers.13.self_attn.v_proj",
+    "model.layers.14.mlp.shared_mlp.down_proj",
+    "model.layers.14.mlp.shared_mlp.gate_proj",
+    "model.layers.14.mlp.shared_mlp.up_proj",
+    "model.layers.14.mlp.switch_mlp.down_proj",
+    "model.layers.14.mlp.switch_mlp.gate_proj",
+    "model.layers.14.mlp.switch_mlp.up_proj",
+    "model.layers.14.self_attn.k_proj",
+    "model.layers.14.self_attn.o_proj",
+    "model.layers.14.self_attn.q_proj",
+    "model.layers.14.self_attn.v_proj",
+    "model.layers.15.mlp.shared_mlp.down_proj",
+    "model.layers.15.mlp.shared_mlp.gate_proj",
+    "model.layers.15.mlp.shared_mlp.up_proj",
+    "model.layers.15.mlp.switch_mlp.down_proj",
+    "model.layers.15.mlp.switch_mlp.gate_proj",
+    "model.layers.15.mlp.switch_mlp.up_proj",
+    "model.layers.15.self_attn.k_proj",
+    "model.layers.15.self_attn.o_proj",
+    "model.layers.15.self_attn.q_proj",
+    "model.layers.15.self_attn.v_proj",
+    "model.layers.16.mlp.shared_mlp.down_proj",
+    "model.layers.16.mlp.shared_mlp.gate_proj",
+    "model.layers.16.mlp.shared_mlp.up_proj",
+    "model.layers.16.mlp.switch_mlp.down_proj",
+    "model.layers.16.mlp.switch_mlp.gate_proj",
+    "model.layers.16.mlp.switch_mlp.up_proj",
+    "model.layers.16.self_attn.k_proj",
+    "model.layers.16.self_attn.o_proj",
+    "model.layers.16.self_attn.q_proj",
+    "model.layers.16.self_attn.v_proj",
+    "model.layers.17.mlp.shared_mlp.down_proj",
+    "model.layers.17.mlp.shared_mlp.gate_proj",
+    "model.layers.17.mlp.shared_mlp.up_proj",
+    "model.layers.17.mlp.switch_mlp.down_proj",
+    "model.layers.17.mlp.switch_mlp.gate_proj",
+    "model.layers.17.mlp.switch_mlp.up_proj",
+    "model.layers.17.self_attn.k_proj",
+    "model.layers.17.self_attn.o_proj",
+    "model.layers.17.self_attn.q_proj",
+    "model.layers.17.self_attn.v_proj",
+    "model.layers.18.mlp.shared_mlp.down_proj",
+    "model.layers.18.mlp.shared_mlp.gate_proj",
+    "model.layers.18.mlp.shared_mlp.up_proj",
+    "model.layers.18.mlp.switch_mlp.down_proj",
+    "model.layers.18.mlp.switch_mlp.gate_proj",
+    "model.layers.18.mlp.switch_mlp.up_proj",
+    "model.layers.18.self_attn.k_proj",
+    "model.layers.18.self_attn.o_proj",
+    "model.layers.18.self_attn.q_proj",
+    "model.layers.18.self_attn.v_proj",
+    "model.layers.19.mlp.shared_mlp.down_proj",
+    "model.layers.19.mlp.shared_mlp.gate_proj",
+    "model.layers.19.mlp.shared_mlp.up_proj",
+    "model.layers.19.mlp.switch_mlp.down_proj",
+    "model.layers.19.mlp.switch_mlp.gate_proj",
+    "model.layers.19.mlp.switch_mlp.up_proj",
+    "model.layers.19.self_attn.k_proj",
+    "model.layers.19.self_attn.o_proj",
+    "model.layers.19.self_attn.q_proj",
+    "model.layers.19.self_attn.v_proj",
+    "model.layers.2.mlp.shared_mlp.down_proj",
+    "model.layers.2.mlp.shared_mlp.gate_proj",
+    "model.layers.2.mlp.shared_mlp.up_proj",
+    "model.layers.2.mlp.switch_mlp.down_proj",
+    "model.layers.2.mlp.switch_mlp.gate_proj",
+    "model.layers.2.mlp.switch_mlp.up_proj",
+    "model.layers.2.self_attn.k_proj",
+    "model.layers.2.self_attn.o_proj",
+    "model.layers.2.self_attn.q_proj",
+    "model.layers.2.self_attn.v_proj",
+    "model.layers.20.mlp.shared_mlp.down_proj",
+    "model.layers.20.mlp.shared_mlp.gate_proj",
+    "model.layers.20.mlp.shared_mlp.up_proj",
+    "model.layers.20.mlp.switch_mlp.down_proj",
+    "model.layers.20.mlp.switch_mlp.gate_proj",
+    "model.layers.20.mlp.switch_mlp.up_proj",
+    "model.layers.20.self_attn.k_proj",
+    "model.layers.20.self_attn.o_proj",
+    "model.layers.20.self_attn.q_proj",
+    "model.layers.20.self_attn.v_proj",
+    "model.layers.21.mlp.shared_mlp.down_proj",
+    "model.layers.21.mlp.shared_mlp.gate_proj",
+    "model.layers.21.mlp.shared_mlp.up_proj",
+    "model.layers.21.mlp.switch_mlp.down_proj",
+    "model.layers.21.mlp.switch_mlp.gate_proj",
+    "model.layers.21.mlp.switch_mlp.up_proj",
+    "model.layers.21.self_attn.k_proj",
+    "model.layers.21.self_attn.o_proj",
+    "model.layers.21.self_attn.q_proj",
+    "model.layers.21.self_attn.v_proj",
+    "model.layers.22.mlp.shared_mlp.down_proj",
+    "model.layers.22.mlp.shared_mlp.gate_proj",
+    "model.layers.22.mlp.shared_mlp.up_proj",
+    "model.layers.22.mlp.switch_mlp.down_proj",
+    "model.layers.22.mlp.switch_mlp.gate_proj",
+    "model.layers.22.mlp.switch_mlp.up_proj",
+    "model.layers.22.self_attn.k_proj",
+    "model.layers.22.self_attn.o_proj",
+    "model.layers.22.self_attn.q_proj",
+    "model.layers.22.self_attn.v_proj",
+    "model.layers.23.mlp.shared_mlp.down_proj",
+    "model.layers.23.mlp.shared_mlp.gate_proj",
+    "model.layers.23.mlp.shared_mlp.up_proj",
+    "model.layers.23.mlp.switch_mlp.down_proj",
+    "model.layers.23.mlp.switch_mlp.gate_proj",
+    "model.layers.23.mlp.switch_mlp.up_proj",
+    "model.layers.23.self_attn.k_proj",
+    "model.layers.23.self_attn.o_proj",
+    "model.layers.23.self_attn.q_proj",
+    "model.layers.23.self_attn.v_proj",
+    "model.layers.24.mlp.shared_mlp.down_proj",
+    "model.layers.24.mlp.shared_mlp.gate_proj",
+    "model.layers.24.mlp.shared_mlp.up_proj",
+    "model.layers.24.mlp.switch_mlp.down_proj",
+    "model.layers.24.mlp.switch_mlp.gate_proj",
+    "model.layers.24.mlp.switch_mlp.up_proj",
+    "model.layers.24.self_attn.k_proj",
+    "model.layers.24.self_attn.o_proj",
+    "model.layers.24.self_attn.q_proj",
+    "model.layers.24.self_attn.v_proj",
+    "model.layers.25.mlp.shared_mlp.down_proj",
+    "model.layers.25.mlp.shared_mlp.gate_proj",
+    "model.layers.25.mlp.shared_mlp.up_proj",
+    "model.layers.25.mlp.switch_mlp.down_proj",
+    "model.layers.25.mlp.switch_mlp.gate_proj",
+    "model.layers.25.mlp.switch_mlp.up_proj",
+    "model.layers.25.self_attn.k_proj",
+    "model.layers.25.self_attn.o_proj",
+    "model.layers.25.self_attn.q_proj",
+    "model.layers.25.self_attn.v_proj",
+    "model.layers.26.mlp.shared_mlp.down_proj",
+    "model.layers.26.mlp.shared_mlp.gate_proj",
+    "model.layers.26.mlp.shared_mlp.up_proj",
+    "model.layers.26.mlp.switch_mlp.down_proj",
+    "model.layers.26.mlp.switch_mlp.gate_proj",
+    "model.layers.26.mlp.switch_mlp.up_proj",
+    "model.layers.26.self_attn.k_proj",
+    "model.layers.26.self_attn.o_proj",
+    "model.layers.26.self_attn.q_proj",
+    "model.layers.26.self_attn.v_proj",
+    "model.layers.27.mlp.shared_mlp.down_proj",
+    "model.layers.27.mlp.shared_mlp.gate_proj",
+    "model.layers.27.mlp.shared_mlp.up_proj",
+    "model.layers.27.mlp.switch_mlp.down_proj",
+    "model.layers.27.mlp.switch_mlp.gate_proj",
+    "model.layers.27.mlp.switch_mlp.up_proj",
+    "model.layers.27.self_attn.k_proj",
+    "model.layers.27.self_attn.o_proj",
+    "model.layers.27.self_attn.q_proj",
+    "model.layers.27.self_attn.v_proj",
+    "model.layers.28.mlp.shared_mlp.down_proj",
+    "model.layers.28.mlp.shared_mlp.gate_proj",
+    "model.layers.28.mlp.shared_mlp.up_proj",
+    "model.layers.28.mlp.switch_mlp.down_proj",
+    "model.layers.28.mlp.switch_mlp.gate_proj",
+    "model.layers.28.mlp.switch_mlp.up_proj",
+    "model.layers.28.self_attn.k_proj",
+    "model.layers.28.self_attn.o_proj",
+    "model.layers.28.self_attn.q_proj",
+    "model.layers.28.self_attn.v_proj",
+    "model.layers.29.mlp.shared_mlp.down_proj",
+    "model.layers.29.mlp.shared_mlp.gate_proj",
+    "model.layers.29.mlp.shared_mlp.up_proj",
+    "model.layers.29.mlp.switch_mlp.down_proj",
+    "model.layers.29.mlp.switch_mlp.gate_proj",
+    "model.layers.29.mlp.switch_mlp.up_proj",
+    "model.layers.29.self_attn.k_proj",
+    "model.layers.29.self_attn.o_proj",
+    "model.layers.29.self_attn.q_proj",
+    "model.layers.29.self_attn.v_proj",
+    "model.layers.3.mlp.shared_mlp.down_proj",
+    "model.layers.3.mlp.shared_mlp.gate_proj",
+    "model.layers.3.mlp.shared_mlp.up_proj",
+    "model.layers.3.mlp.switch_mlp.down_proj",
+    "model.layers.3.mlp.switch_mlp.gate_proj",
+    "model.layers.3.mlp.switch_mlp.up_proj",
+    "model.layers.3.self_attn.k_proj",
+    "model.layers.3.self_attn.o_proj",
+    "model.layers.3.self_attn.q_proj",
+    "model.layers.3.self_attn.v_proj",
+    "model.layers.30.mlp.shared_mlp.down_proj",
+    "model.layers.30.mlp.shared_mlp.gate_proj",
+    "model.layers.30.mlp.shared_mlp.up_proj",
+    "model.layers.30.mlp.switch_mlp.down_proj",
+    "model.layers.30.mlp.switch_mlp.gate_proj",
+    "model.layers.30.mlp.switch_mlp.up_proj",
+    "model.layers.30.self_attn.k_proj",
+    "model.layers.30.self_attn.o_proj",
+    "model.layers.30.self_attn.q_proj",
+    "model.layers.30.self_attn.v_proj",
+    "model.layers.31.mlp.shared_mlp.down_proj",
+    "model.layers.31.mlp.shared_mlp.gate_proj",
+    "model.layers.31.mlp.shared_mlp.up_proj",
+    "model.layers.31.mlp.switch_mlp.down_proj",
+    "model.layers.31.mlp.switch_mlp.gate_proj",
+    "model.layers.31.mlp.switch_mlp.up_proj",
+    "model.layers.31.self_attn.k_proj",
+    "model.layers.31.self_attn.o_proj",
+    "model.layers.31.self_attn.q_proj",
+    "model.layers.31.self_attn.v_proj",
+    "model.layers.32.mlp.shared_mlp.down_proj",
+    "model.layers.32.mlp.shared_mlp.gate_proj",
+    "model.layers.32.mlp.shared_mlp.up_proj",
+    "model.layers.32.mlp.switch_mlp.down_proj",
+    "model.layers.32.mlp.switch_mlp.gate_proj",
+    "model.layers.32.mlp.switch_mlp.up_proj",
+    "model.layers.32.self_attn.k_proj",
+    "model.layers.32.self_attn.o_proj",
+    "model.layers.32.self_attn.q_proj",
+    "model.layers.32.self_attn.v_proj",
+    "model.layers.33.mlp.shared_mlp.down_proj",
+    "model.layers.33.mlp.shared_mlp.gate_proj",
+    "model.layers.33.mlp.shared_mlp.up_proj",
+    "model.layers.33.mlp.switch_mlp.down_proj",
+    "model.layers.33.mlp.switch_mlp.gate_proj",
+    "model.layers.33.mlp.switch_mlp.up_proj",
+    "model.layers.33.self_attn.k_proj",
+    "model.layers.33.self_attn.o_proj",
+    "model.layers.33.self_attn.q_proj",
+    "model.layers.33.self_attn.v_proj",
+    "model.layers.34.mlp.shared_mlp.down_proj",
+    "model.layers.34.mlp.shared_mlp.gate_proj",
+    "model.layers.34.mlp.shared_mlp.up_proj",
+    "model.layers.34.mlp.switch_mlp.down_proj",
+    "model.layers.34.mlp.switch_mlp.gate_proj",
+    "model.layers.34.mlp.switch_mlp.up_proj",
+    "model.layers.34.self_attn.k_proj",
+    "model.layers.34.self_attn.o_proj",
+    "model.layers.34.self_attn.q_proj",
+    "model.layers.34.self_attn.v_proj",
+    "model.layers.35.mlp.shared_mlp.down_proj",
+    "model.layers.35.mlp.shared_mlp.gate_proj",
+    "model.layers.35.mlp.shared_mlp.up_proj",
+    "model.layers.35.mlp.switch_mlp.down_proj",
+    "model.layers.35.mlp.switch_mlp.gate_proj",
+    "model.layers.35.mlp.switch_mlp.up_proj",
+    "model.layers.35.self_attn.k_proj",
+    "model.layers.35.self_attn.o_proj",
+    "model.layers.35.self_attn.q_proj",
+    "model.layers.35.self_attn.v_proj",
+    "model.layers.36.mlp.shared_mlp.down_proj",
+    "model.layers.36.mlp.shared_mlp.gate_proj",
+    "model.layers.36.mlp.shared_mlp.up_proj",
+    "model.layers.36.mlp.switch_mlp.down_proj",
+    "model.layers.36.mlp.switch_mlp.gate_proj",
+    "model.layers.36.mlp.switch_mlp.up_proj",
+    "model.layers.36.self_attn.k_proj",
+    "model.layers.36.self_attn.o_proj",
+    "model.layers.36.self_attn.q_proj",
+    "model.layers.36.self_attn.v_proj",
+    "model.layers.37.mlp.shared_mlp.down_proj",
+    "model.layers.37.mlp.shared_mlp.gate_proj",
+    "model.layers.37.mlp.shared_mlp.up_proj",
+    "model.layers.37.mlp.switch_mlp.down_proj",
+    "model.layers.37.mlp.switch_mlp.gate_proj",
+    "model.layers.37.mlp.switch_mlp.up_proj",
+    "model.layers.37.self_attn.k_proj",
+    "model.layers.37.self_attn.o_proj",
+    "model.layers.37.self_attn.q_proj",
+    "model.layers.37.self_attn.v_proj",
+    "model.layers.38.mlp.shared_mlp.down_proj",
+    "model.layers.38.mlp.shared_mlp.gate_proj",
+    "model.layers.38.mlp.shared_mlp.up_proj",
+    "model.layers.38.mlp.switch_mlp.down_proj",
+    "model.layers.38.mlp.switch_mlp.gate_proj",
+    "model.layers.38.mlp.switch_mlp.up_proj",
+    "model.layers.38.self_attn.k_proj",
+    "model.layers.38.self_attn.o_proj",
+    "model.layers.38.self_attn.q_proj",
+    "model.layers.38.self_attn.v_proj",
+    "model.layers.39.mlp.shared_mlp.down_proj",
+    "model.layers.39.mlp.shared_mlp.gate_proj",
+    "model.layers.39.mlp.shared_mlp.up_proj",
+    "model.layers.39.mlp.switch_mlp.down_proj",
+    "model.layers.39.mlp.switch_mlp.gate_proj",
+    "model.layers.39.mlp.switch_mlp.up_proj",
+    "model.layers.39.self_attn.k_proj",
+    "model.layers.39.self_attn.o_proj",
+    "model.layers.39.self_attn.q_proj",
+    "model.layers.39.self_attn.v_proj",
+    "model.layers.4.mlp.shared_mlp.down_proj",
+    "model.layers.4.mlp.shared_mlp.gate_proj",
+    "model.layers.4.mlp.shared_mlp.up_proj",
+    "model.layers.4.mlp.switch_mlp.down_proj",
+    "model.layers.4.mlp.switch_mlp.gate_proj",
+    "model.layers.4.mlp.switch_mlp.up_proj",
+    "model.layers.4.self_attn.k_proj",
+    "model.layers.4.self_attn.o_proj",
+    "model.layers.4.self_attn.q_proj",
+    "model.layers.4.self_attn.v_proj",
+    "model.layers.40.mlp.shared_mlp.down_proj",
+    "model.layers.40.mlp.shared_mlp.gate_proj",
+    "model.layers.40.mlp.shared_mlp.up_proj",
+    "model.layers.40.mlp.switch_mlp.down_proj",
+    "model.layers.40.mlp.switch_mlp.gate_proj",
+    "model.layers.40.mlp.switch_mlp.up_proj",
+    "model.layers.40.self_attn.k_proj",
+    "model.layers.40.self_attn.o_proj",
+    "model.layers.40.self_attn.q_proj",
+    "model.layers.40.self_attn.v_proj",
+    "model.layers.41.mlp.shared_mlp.down_proj",
+    "model.layers.41.mlp.shared_mlp.gate_proj",
+    "model.layers.41.mlp.shared_mlp.up_proj",
+    "model.layers.41.mlp.switch_mlp.down_proj",
+    "model.layers.41.mlp.switch_mlp.gate_proj",
+    "model.layers.41.mlp.switch_mlp.up_proj",
+    "model.layers.41.self_attn.k_proj",
+    "model.layers.41.self_attn.o_proj",
+    "model.layers.41.self_attn.q_proj",
+    "model.layers.41.self_attn.v_proj",
+    "model.layers.42.mlp.shared_mlp.down_proj",
+    "model.layers.42.mlp.shared_mlp.gate_proj",
+    "model.layers.42.mlp.shared_mlp.up_proj",
+    "model.layers.42.mlp.switch_mlp.down_proj",
+    "model.layers.42.mlp.switch_mlp.gate_proj",
+    "model.layers.42.mlp.switch_mlp.up_proj",
+    "model.layers.42.self_attn.k_proj",
+    "model.layers.42.self_attn.o_proj",
+    "model.layers.42.self_attn.q_proj",
+    "model.layers.42.self_attn.v_proj",
+    "model.layers.43.mlp.shared_mlp.down_proj",
+    "model.layers.43.mlp.shared_mlp.gate_proj",
+    "model.layers.43.mlp.shared_mlp.up_proj",
+    "model.layers.43.mlp.switch_mlp.down_proj",
+    "model.layers.43.mlp.switch_mlp.gate_proj",
+    "model.layers.43.mlp.switch_mlp.up_proj",
+    "model.layers.43.self_attn.k_proj",
+    "model.layers.43.self_attn.o_proj",
+    "model.layers.43.self_attn.q_proj",
+    "model.layers.43.self_attn.v_proj",
+    "model.layers.44.mlp.shared_mlp.down_proj",
+    "model.layers.44.mlp.shared_mlp.gate_proj",
+    "model.layers.44.mlp.shared_mlp.up_proj",
+    "model.layers.44.mlp.switch_mlp.down_proj",
+    "model.layers.44.mlp.switch_mlp.gate_proj",
+    "model.layers.44.mlp.switch_mlp.up_proj",
+    "model.layers.44.self_attn.k_proj",
+    "model.layers.44.self_attn.o_proj",
+    "model.layers.44.self_attn.q_proj",
+    "model.layers.44.self_attn.v_proj",
+    "model.layers.45.mlp.shared_mlp.down_proj",
+    "model.layers.45.mlp.shared_mlp.gate_proj",
+    "model.layers.45.mlp.shared_mlp.up_proj",
+    "model.layers.45.mlp.switch_mlp.down_proj",
+    "model.layers.45.mlp.switch_mlp.gate_proj",
+    "model.layers.45.mlp.switch_mlp.up_proj",
+    "model.layers.45.self_attn.k_proj",
+    "model.layers.45.self_attn.o_proj",
+    "model.layers.45.self_attn.q_proj",
+    "model.layers.45.self_attn.v_proj",
+    "model.layers.46.mlp.shared_mlp.down_proj",
+    "model.layers.46.mlp.shared_mlp.gate_proj",
+    "model.layers.46.mlp.shared_mlp.up_proj",
+    "model.layers.46.mlp.switch_mlp.down_proj",
+    "model.layers.46.mlp.switch_mlp.gate_proj",
+    "model.layers.46.mlp.switch_mlp.up_proj",
+    "model.layers.46.self_attn.k_proj",
+    "model.layers.46.self_attn.o_proj",
+    "model.layers.46.self_attn.q_proj",
+    "model.layers.46.self_attn.v_proj",
+    "model.layers.47.mlp.shared_mlp.down_proj",
+    "model.layers.47.mlp.shared_mlp.gate_proj",
+    "model.layers.47.mlp.shared_mlp.up_proj",
+    "model.layers.47.mlp.switch_mlp.down_proj",
+    "model.layers.47.mlp.switch_mlp.gate_proj",
+    "model.layers.47.mlp.switch_mlp.up_proj",
+    "model.layers.47.self_attn.k_proj",
+    "model.layers.47.self_attn.o_proj",
+    "model.layers.47.self_attn.q_proj",
+    "model.layers.47.self_attn.v_proj",
+    "model.layers.48.mlp.shared_mlp.down_proj",
+    "model.layers.48.mlp.shared_mlp.gate_proj",
+    "model.layers.48.mlp.shared_mlp.up_proj",
+    "model.layers.48.mlp.switch_mlp.down_proj",
+    "model.layers.48.mlp.switch_mlp.gate_proj",
+    "model.layers.48.mlp.switch_mlp.up_proj",
+    "model.layers.48.self_attn.k_proj",
+    "model.layers.48.self_attn.o_proj",
+    "model.layers.48.self_attn.q_proj",
+    "model.layers.48.self_attn.v_proj",
+    "model.layers.49.mlp.shared_mlp.down_proj",
+    "model.layers.49.mlp.shared_mlp.gate_proj",
+    "model.layers.49.mlp.shared_mlp.up_proj",
+    "model.layers.49.mlp.switch_mlp.down_proj",
+    "model.layers.49.mlp.switch_mlp.gate_proj",
+    "model.layers.49.mlp.switch_mlp.up_proj",
+    "model.layers.49.self_attn.k_proj",
+    "model.layers.49.self_attn.o_proj",
+    "model.layers.49.self_attn.q_proj",
+    "model.layers.49.self_attn.v_proj",
+    "model.layers.5.mlp.shared_mlp.down_proj",
+    "model.layers.5.mlp.shared_mlp.gate_proj",
+    "model.layers.5.mlp.shared_mlp.up_proj",
+    "model.layers.5.mlp.switch_mlp.down_proj",
+    "model.layers.5.mlp.switch_mlp.gate_proj",
+    "model.layers.5.mlp.switch_mlp.up_proj",
+    "model.layers.5.self_attn.k_proj",
+    "model.layers.5.self_attn.o_proj",
+    "model.layers.5.self_attn.q_proj",
+    "model.layers.5.self_attn.v_proj",
+    "model.layers.50.mlp.shared_mlp.down_proj",
+    "model.layers.50.mlp.shared_mlp.gate_proj",
+    "model.layers.50.mlp.shared_mlp.up_proj",
+    "model.layers.50.mlp.switch_mlp.down_proj",
+    "model.layers.50.mlp.switch_mlp.gate_proj",
+    "model.layers.50.mlp.switch_mlp.up_proj",
+    "model.layers.50.self_attn.k_proj",
+    "model.layers.50.self_attn.o_proj",
+    "model.layers.50.self_attn.q_proj",
+    "model.layers.50.self_attn.v_proj",
+    "model.layers.51.mlp.shared_mlp.down_proj",
+    "model.layers.51.mlp.shared_mlp.gate_proj",
+    "model.layers.51.mlp.shared_mlp.up_proj",
+    "model.layers.51.mlp.switch_mlp.down_proj",
+    "model.layers.51.mlp.switch_mlp.gate_proj",
+    "model.layers.51.mlp.switch_mlp.up_proj",
+    "model.layers.51.self_attn.k_proj",
+    "model.layers.51.self_attn.o_proj",
+    "model.layers.51.self_attn.q_proj",
+    "model.layers.51.self_attn.v_proj",
+    "model.layers.52.mlp.shared_mlp.down_proj",
+    "model.layers.52.mlp.shared_mlp.gate_proj",
+    "model.layers.52.mlp.shared_mlp.up_proj",
+    "model.layers.52.mlp.switch_mlp.down_proj",
+    "model.layers.52.mlp.switch_mlp.gate_proj",
+    "model.layers.52.mlp.switch_mlp.up_proj",
+    "model.layers.52.self_attn.k_proj",
+    "model.layers.52.self_attn.o_proj",
+    "model.layers.52.self_attn.q_proj",
+    "model.layers.52.self_attn.v_proj",
+    "model.layers.53.mlp.shared_mlp.down_proj",
+    "model.layers.53.mlp.shared_mlp.gate_proj",
+    "model.layers.53.mlp.shared_mlp.up_proj",
+    "model.layers.53.mlp.switch_mlp.down_proj",
+    "model.layers.53.mlp.switch_mlp.gate_proj",
+    "model.layers.53.mlp.switch_mlp.up_proj",
+    "model.layers.53.self_attn.k_proj",
+    "model.layers.53.self_attn.o_proj",
+    "model.layers.53.self_attn.q_proj",
+    "model.layers.53.self_attn.v_proj",
+    "model.layers.54.mlp.shared_mlp.down_proj",
+    "model.layers.54.mlp.shared_mlp.gate_proj",
+    "model.layers.54.mlp.shared_mlp.up_proj",
+    "model.layers.54.mlp.switch_mlp.down_proj",
+    "model.layers.54.mlp.switch_mlp.gate_proj",
+    "model.layers.54.mlp.switch_mlp.up_proj",
+    "model.layers.54.self_attn.k_proj",
+    "model.layers.54.self_attn.o_proj",
+    "model.layers.54.self_attn.q_proj",
+    "model.layers.54.self_attn.v_proj",
+    "model.layers.55.mlp.shared_mlp.down_proj",
+    "model.layers.55.mlp.shared_mlp.gate_proj",
+    "model.layers.55.mlp.shared_mlp.up_proj",
+    "model.layers.55.mlp.switch_mlp.down_proj",
+    "model.layers.55.mlp.switch_mlp.gate_proj",
+    "model.layers.55.mlp.switch_mlp.up_proj",
+    "model.layers.55.self_attn.k_proj",
+    "model.layers.55.self_attn.o_proj",
+    "model.layers.55.self_attn.q_proj",
+    "model.layers.55.self_attn.v_proj",
+    "model.layers.56.mlp.shared_mlp.down_proj",
+    "model.layers.56.mlp.shared_mlp.gate_proj",
+    "model.layers.56.mlp.shared_mlp.up_proj",
+    "model.layers.56.mlp.switch_mlp.down_proj",
+    "model.layers.56.mlp.switch_mlp.gate_proj",
+    "model.layers.56.mlp.switch_mlp.up_proj",
+    "model.layers.56.self_attn.k_proj",
+    "model.layers.56.self_attn.o_proj",
+    "model.layers.56.self_attn.q_proj",
+    "model.layers.56.self_attn.v_proj",
+    "model.layers.57.mlp.shared_mlp.down_proj",
+    "model.layers.57.mlp.shared_mlp.gate_proj",
+    "model.layers.57.mlp.shared_mlp.up_proj",
+    "model.layers.57.mlp.switch_mlp.down_proj",
+    "model.layers.57.mlp.switch_mlp.gate_proj",
+    "model.layers.57.mlp.switch_mlp.up_proj",
+    "model.layers.57.self_attn.k_proj",
+    "model.layers.57.self_attn.o_proj",
+    "model.layers.57.self_attn.q_proj",
+    "model.layers.57.self_attn.v_proj",
+    "model.layers.58.mlp.shared_mlp.down_proj",
+    "model.layers.58.mlp.shared_mlp.gate_proj",
+    "model.layers.58.mlp.shared_mlp.up_proj",
+    "model.layers.58.mlp.switch_mlp.down_proj",
+    "model.layers.58.mlp.switch_mlp.gate_proj",
+    "model.layers.58.mlp.switch_mlp.up_proj",
+    "model.layers.58.self_attn.k_proj",
+    "model.layers.58.self_attn.o_proj",
+    "model.layers.58.self_attn.q_proj",
+    "model.layers.58.self_attn.v_proj",
+    "model.layers.59.mlp.shared_mlp.down_proj",
+    "model.layers.59.mlp.shared_mlp.gate_proj",
+    "model.layers.59.mlp.shared_mlp.up_proj",
+    "model.layers.59.mlp.switch_mlp.down_proj",
+    "model.layers.59.mlp.switch_mlp.gate_proj",
+    "model.layers.59.mlp.switch_mlp.up_proj",
+    "model.layers.59.self_attn.k_proj",
+    "model.layers.59.self_attn.o_proj",
+    "model.layers.59.self_attn.q_proj",
+    "model.layers.59.self_attn.v_proj",
+    "model.layers.6.mlp.shared_mlp.down_proj",
+    "model.layers.6.mlp.shared_mlp.gate_proj",
+    "model.layers.6.mlp.shared_mlp.up_proj",
+    "model.layers.6.mlp.switch_mlp.down_proj",
+    "model.layers.6.mlp.switch_mlp.gate_proj",
+    "model.layers.6.mlp.switch_mlp.up_proj",
+    "model.layers.6.self_attn.k_proj",
+    "model.layers.6.self_attn.o_proj",
+    "model.layers.6.self_attn.q_proj",
+    "model.layers.6.self_attn.v_proj",
+    "model.layers.60.mlp.shared_mlp.down_proj",
+    "model.layers.60.mlp.shared_mlp.gate_proj",
+    "model.layers.60.mlp.shared_mlp.up_proj",
+    "model.layers.60.mlp.switch_mlp.down_proj",
+    "model.layers.60.mlp.switch_mlp.gate_proj",
+    "model.layers.60.mlp.switch_mlp.up_proj",
+    "model.layers.60.self_attn.k_proj",
+    "model.layers.60.self_attn.o_proj",
+    "model.layers.60.self_attn.q_proj",
+    "model.layers.60.self_attn.v_proj",
+    "model.layers.61.mlp.shared_mlp.down_proj",
+    "model.layers.61.mlp.shared_mlp.gate_proj",
+    "model.layers.61.mlp.shared_mlp.up_proj",
+    "model.layers.61.mlp.switch_mlp.down_proj",
+    "model.layers.61.mlp.switch_mlp.gate_proj",
+    "model.layers.61.mlp.switch_mlp.up_proj",
+    "model.layers.61.self_attn.k_proj",
+    "model.layers.61.self_attn.o_proj",
+    "model.layers.61.self_attn.q_proj",
+    "model.layers.61.self_attn.v_proj",
+    "model.layers.62.mlp.shared_mlp.down_proj",
+    "model.layers.62.mlp.shared_mlp.gate_proj",
+    "model.layers.62.mlp.shared_mlp.up_proj",
+    "model.layers.62.mlp.switch_mlp.down_proj",
+    "model.layers.62.mlp.switch_mlp.gate_proj",
+    "model.layers.62.mlp.switch_mlp.up_proj",
+    "model.layers.62.self_attn.k_proj",
+    "model.layers.62.self_attn.o_proj",
+    "model.layers.62.self_attn.q_proj",
+    "model.layers.62.self_attn.v_proj",
+    "model.layers.63.mlp.shared_mlp.down_proj",
+    "model.layers.63.mlp.shared_mlp.gate_proj",
+    "model.layers.63.mlp.shared_mlp.up_proj",
+    "model.layers.63.mlp.switch_mlp.down_proj",
+    "model.layers.63.mlp.switch_mlp.gate_proj",
+    "model.layers.63.mlp.switch_mlp.up_proj",
+    "model.layers.63.self_attn.k_proj",
+    "model.layers.63.self_attn.o_proj",
+    "model.layers.63.self_attn.q_proj",
+    "model.layers.63.self_attn.v_proj",
+    "model.layers.64.mlp.shared_mlp.down_proj",
+    "model.layers.64.mlp.shared_mlp.gate_proj",
+    "model.layers.64.mlp.shared_mlp.up_proj",
+    "model.layers.64.mlp.switch_mlp.down_proj",
+    "model.layers.64.mlp.switch_mlp.gate_proj",
+    "model.layers.64.mlp.switch_mlp.up_proj",
+    "model.layers.64.self_attn.k_proj",
+    "model.layers.64.self_attn.o_proj",
+    "model.layers.64.self_attn.q_proj",
+    "model.layers.64.self_attn.v_proj",
+    "model.layers.65.mlp.shared_mlp.down_proj",
+    "model.layers.65.mlp.shared_mlp.gate_proj",
+    "model.layers.65.mlp.shared_mlp.up_proj",
+    "model.layers.65.mlp.switch_mlp.down_proj",
+    "model.layers.65.mlp.switch_mlp.gate_proj",
+    "model.layers.65.mlp.switch_mlp.up_proj",
+    "model.layers.65.self_attn.k_proj",
+    "model.layers.65.self_attn.o_proj",
+    "model.layers.65.self_attn.q_proj",
+    "model.layers.65.self_attn.v_proj",
+    "model.layers.66.mlp.shared_mlp.down_proj",
+    "model.layers.66.mlp.shared_mlp.gate_proj",
+    "model.layers.66.mlp.shared_mlp.up_proj",
+    "model.layers.66.mlp.switch_mlp.down_proj",
+    "model.layers.66.mlp.switch_mlp.gate_proj",
+    "model.layers.66.mlp.switch_mlp.up_proj",
+    "model.layers.66.self_attn.k_proj",
+    "model.layers.66.self_attn.o_proj",
+    "model.layers.66.self_attn.q_proj",
+    "model.layers.66.self_attn.v_proj",
+    "model.layers.67.mlp.shared_mlp.down_proj",
+    "model.layers.67.mlp.shared_mlp.gate_proj",
+    "model.layers.67.mlp.shared_mlp.up_proj",
+    "model.layers.67.mlp.switch_mlp.down_proj",
+    "model.layers.67.mlp.switch_mlp.gate_proj",
+    "model.layers.67.mlp.switch_mlp.up_proj",
+    "model.layers.67.self_attn.k_proj",
+    "model.layers.67.self_attn.o_proj",
+    "model.layers.67.self_attn.q_proj",
+    "model.layers.67.self_attn.v_proj",
+    "model.layers.68.mlp.shared_mlp.down_proj",
+    "model.layers.68.mlp.shared_mlp.gate_proj",
+    "model.layers.68.mlp.shared_mlp.up_proj",
+    "model.layers.68.mlp.switch_mlp.down_proj",
+    "model.layers.68.mlp.switch_mlp.gate_proj",
+    "model.layers.68.mlp.switch_mlp.up_proj",
+    "model.layers.68.self_attn.k_proj",
+    "model.layers.68.self_attn.o_proj",
+    "model.layers.68.self_attn.q_proj",
+    "model.layers.68.self_attn.v_proj",
+    "model.layers.69.mlp.shared_mlp.down_proj",
+    "model.layers.69.mlp.shared_mlp.gate_proj",
+    "model.layers.69.mlp.shared_mlp.up_proj",
+    "model.layers.69.mlp.switch_mlp.down_proj",
+    "model.layers.69.mlp.switch_mlp.gate_proj",
+    "model.layers.69.mlp.switch_mlp.up_proj",
+    "model.layers.69.self_attn.k_proj",
+    "model.layers.69.self_attn.o_proj",
+    "model.layers.69.self_attn.q_proj",
+    "model.layers.69.self_attn.v_proj",
+    "model.layers.7.mlp.shared_mlp.down_proj",
+    "model.layers.7.mlp.shared_mlp.gate_proj",
+    "model.layers.7.mlp.shared_mlp.up_proj",
+    "model.layers.7.mlp.switch_mlp.down_proj",
+    "model.layers.7.mlp.switch_mlp.gate_proj",
+    "model.layers.7.mlp.switch_mlp.up_proj",
+    "model.layers.7.self_attn.k_proj",
+    "model.layers.7.self_attn.o_proj",
+    "model.layers.7.self_attn.q_proj",
+    "model.layers.7.self_attn.v_proj",
+    "model.layers.70.mlp.shared_mlp.down_proj",
+    "model.layers.70.mlp.shared_mlp.gate_proj",
+    "model.layers.70.mlp.shared_mlp.up_proj",
+    "model.layers.70.mlp.switch_mlp.down_proj",
+    "model.layers.70.mlp.switch_mlp.gate_proj",
+    "model.layers.70.mlp.switch_mlp.up_proj",
+    "model.layers.70.self_attn.k_proj",
+    "model.layers.70.self_attn.o_proj",
+    "model.layers.70.self_attn.q_proj",
+    "model.layers.70.self_attn.v_proj",
+    "model.layers.71.mlp.shared_mlp.down_proj",
+    "model.layers.71.mlp.shared_mlp.gate_proj",
+    "model.layers.71.mlp.shared_mlp.up_proj",
+    "model.layers.71.mlp.switch_mlp.down_proj",
+    "model.layers.71.mlp.switch_mlp.gate_proj",
+    "model.layers.71.mlp.switch_mlp.up_proj",
+    "model.layers.71.self_attn.k_proj",
+    "model.layers.71.self_attn.o_proj",
+    "model.layers.71.self_attn.q_proj",
+    "model.layers.71.self_attn.v_proj",
+    "model.layers.72.mlp.shared_mlp.down_proj",
+    "model.layers.72.mlp.shared_mlp.gate_proj",
+    "model.layers.72.mlp.shared_mlp.up_proj",
+    "model.layers.72.mlp.switch_mlp.down_proj",
+    "model.layers.72.mlp.switch_mlp.gate_proj",
+    "model.layers.72.mlp.switch_mlp.up_proj",
+    "model.layers.72.self_attn.k_proj",
+    "model.layers.72.self_attn.o_proj",
+    "model.layers.72.self_attn.q_proj",
+    "model.layers.72.self_attn.v_proj",
+    "model.layers.73.mlp.shared_mlp.down_proj",
+    "model.layers.73.mlp.shared_mlp.gate_proj",
+    "model.layers.73.mlp.shared_mlp.up_proj",
+    "model.layers.73.mlp.switch_mlp.down_proj",
+    "model.layers.73.mlp.switch_mlp.gate_proj",
+    "model.layers.73.mlp.switch_mlp.up_proj",
+    "model.layers.73.self_attn.k_proj",
+    "model.layers.73.self_attn.o_proj",
+    "model.layers.73.self_attn.q_proj",
+    "model.layers.73.self_attn.v_proj",
+    "model.layers.74.mlp.shared_mlp.down_proj",
+    "model.layers.74.mlp.shared_mlp.gate_proj",
+    "model.layers.74.mlp.shared_mlp.up_proj",
+    "model.layers.74.mlp.switch_mlp.down_proj",
+    "model.layers.74.mlp.switch_mlp.gate_proj",
+    "model.layers.74.mlp.switch_mlp.up_proj",
+    "model.layers.74.self_attn.k_proj",
+    "model.layers.74.self_attn.o_proj",
+    "model.layers.74.self_attn.q_proj",
+    "model.layers.74.self_attn.v_proj",
+    "model.layers.75.mlp.shared_mlp.down_proj",
+    "model.layers.75.mlp.shared_mlp.gate_proj",
+    "model.layers.75.mlp.shared_mlp.up_proj",
+    "model.layers.75.mlp.switch_mlp.down_proj",
+    "model.layers.75.mlp.switch_mlp.gate_proj",
+    "model.layers.75.mlp.switch_mlp.up_proj",
+    "model.layers.75.self_attn.k_proj",
+    "model.layers.75.self_attn.o_proj",
+    "model.layers.75.self_attn.q_proj",
+    "model.layers.75.self_attn.v_proj",
+    "model.layers.76.mlp.shared_mlp.down_proj",
+    "model.layers.76.mlp.shared_mlp.gate_proj",
+    "model.layers.76.mlp.shared_mlp.up_proj",
+    "model.layers.76.mlp.switch_mlp.down_proj",
+    "model.layers.76.mlp.switch_mlp.gate_proj",
+    "model.layers.76.mlp.switch_mlp.up_proj",
+    "model.layers.76.self_attn.k_proj",
+    "model.layers.76.self_attn.o_proj",
+    "model.layers.76.self_attn.q_proj",
+    "model.layers.76.self_attn.v_proj",
+    "model.layers.77.mlp.shared_mlp.down_proj",
+    "model.layers.77.mlp.shared_mlp.gate_proj",
+    "model.layers.77.mlp.shared_mlp.up_proj",
+    "model.layers.77.mlp.switch_mlp.down_proj",
+    "model.layers.77.mlp.switch_mlp.gate_proj",
+    "model.layers.77.mlp.switch_mlp.up_proj",
+    "model.layers.77.self_attn.k_proj",
+    "model.layers.77.self_attn.o_proj",
+    "model.layers.77.self_attn.q_proj",
+    "model.layers.77.self_attn.v_proj",
+    "model.layers.78.mlp.shared_mlp.down_proj",
+    "model.layers.78.mlp.shared_mlp.gate_proj",
+    "model.layers.78.mlp.shared_mlp.up_proj",
+    "model.layers.78.mlp.switch_mlp.down_proj",
+    "model.layers.78.mlp.switch_mlp.gate_proj",
+    "model.layers.78.mlp.switch_mlp.up_proj",
+    "model.layers.78.self_attn.k_proj",
+    "model.layers.78.self_attn.o_proj",
+    "model.layers.78.self_attn.q_proj",
+    "model.layers.78.self_attn.v_proj",
+    "model.layers.79.mlp.shared_mlp.down_proj",
+    "model.layers.79.mlp.shared_mlp.gate_proj",
+    "model.layers.79.mlp.shared_mlp.up_proj",
+    "model.layers.79.mlp.switch_mlp.down_proj",
+    "model.layers.79.mlp.switch_mlp.gate_proj",
+    "model.layers.79.mlp.switch_mlp.up_proj",
+    "model.layers.79.self_attn.k_proj",
+    "model.layers.79.self_attn.o_proj",
+    "model.layers.79.self_attn.q_proj",
+    "model.layers.79.self_attn.v_proj",
+    "model.layers.8.mlp.shared_mlp.down_proj",
+    "model.layers.8.mlp.shared_mlp.gate_proj",
+    "model.layers.8.mlp.shared_mlp.up_proj",
+    "model.layers.8.mlp.switch_mlp.down_proj",
+    "model.layers.8.mlp.switch_mlp.gate_proj",
+    "model.layers.8.mlp.switch_mlp.up_proj",
+    "model.layers.8.self_attn.k_proj",
+    "model.layers.8.self_attn.o_proj",
+    "model.layers.8.self_attn.q_proj",
+    "model.layers.8.self_attn.v_proj",
+    "model.layers.9.mlp.shared_mlp.down_proj",
+    "model.layers.9.mlp.shared_mlp.gate_proj",
+    "model.layers.9.mlp.shared_mlp.up_proj",
+    "model.layers.9.mlp.switch_mlp.down_proj",
+    "model.layers.9.mlp.switch_mlp.gate_proj",
+    "model.layers.9.mlp.switch_mlp.up_proj",
+    "model.layers.9.self_attn.k_proj",
+    "model.layers.9.self_attn.o_proj",
+    "model.layers.9.self_attn.q_proj",
+    "model.layers.9.self_attn.v_proj"
+  ],
+  "missing": [
+    "lm_head",
+    "model.embed_tokens"
+  ],
+  "mismatched": [],
+  "zero_count_experts": 0
+}

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<｜hy_begin_of_sentence:opensource｜>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<｜hy_eos:opensource｜>",
+  "is_local": true,
+  "local_files_only": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<｜hy_pad:opensource｜>",
+  "token_suffix": ":opensource",
+  "tokenizer_class": "TokenizersBackend",
+  "tool_parser_type": "hy_v3_opensource"
+}