IMvision12 commited on
Commit
add1dee
·
0 Parent(s):

Super-squash branch 'main' using huggingface_hub

Browse files
.gitattributes ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ license: apache-2.0
4
+ base_model: Qwen/Qwen3-8B
5
+ library_name: kerasformers
6
+ language:
7
+ - en
8
+ tags:
9
+ - keras
10
+ - kerasformers
11
+ - qwen3
12
+ - qwen3-dense
13
+ - text-generation
14
+ - pytorch
15
+ - jax
16
+ - tf
17
+ ---
18
+
19
+ # Run Qwen3 with Keras 3: JAX, PyTorch, or TensorFlow
20
+
21
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-181717?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-Qwen3-1f6feb)](https://imvision12.github.io/KerasFormers/qwen3/) [![HuggingFace](https://img.shields.io/badge/HuggingFace-Qwen3-ffd21e?logo=huggingface&logoColor=black)](https://huggingface.co/collections/kerasformers/qwen3-6a7d3fcc4e56b32e86f5b2c4)
22
+
23
+ # kerasformers/qwen3-8b
24
+
25
+ Pure-**Keras 3** conversion of [`Qwen/Qwen3-8B`](https://huggingface.co/Qwen/Qwen3-8B) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**. This is a **dense** Qwen3; weights are stored in **bfloat16**.
26
+
27
+ For model details, license, and usage terms, see the upstream [model card](https://huggingface.co/Qwen/Qwen3-8B).
28
+
29
+ Paper: [Qwen3 Technical Report (arXiv:2505.09388)](https://arxiv.org/abs/2505.09388) · [HF Papers](https://huggingface.co/papers/2505.09388)
30
+
31
+ Paper: [YaRN: Efficient Context Window Extension of Large Language Models (arXiv:2309.00071)](https://arxiv.org/abs/2309.00071) · [HF Papers](https://huggingface.co/papers/2309.00071)
32
+
33
+ ## ✨ Quick start
34
+
35
+ ```python
36
+ import os
37
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
38
+
39
+ from kerasformers.models.qwen3 import Qwen3TextGenerate, Qwen3Tokenizer
40
+
41
+ model = Qwen3TextGenerate.from_weights("kerasformers/qwen3-8b")
42
+ tokenizer = Qwen3Tokenizer.from_weights("kerasformers/qwen3-8b")
43
+
44
+ inputs = tokenizer("Give me a short introduction to large language models.")
45
+ outputs = model.generate(**inputs, max_new_tokens=128)
46
+ print(tokenizer.decode(outputs[0]))
47
+ ```
48
+
49
+ Load any Qwen3 variant the same way with `from_weights("kerasformers/<variant>")`:
50
+
51
+ | Variant | Hub |
52
+ | --- | --- |
53
+ | `qwen3-0.6b` | [kerasformers/qwen3-0.6b](https://huggingface.co/kerasformers/qwen3-0.6b) |
54
+ | `qwen3-0.6b-base` | [kerasformers/qwen3-0.6b-base](https://huggingface.co/kerasformers/qwen3-0.6b-base) |
55
+ | `qwen3-1.7b` | [kerasformers/qwen3-1.7b](https://huggingface.co/kerasformers/qwen3-1.7b) |
56
+ | `qwen3-1.7b-base` | [kerasformers/qwen3-1.7b-base](https://huggingface.co/kerasformers/qwen3-1.7b-base) |
57
+ | `qwen3-4b` | [kerasformers/qwen3-4b](https://huggingface.co/kerasformers/qwen3-4b) |
58
+ | `qwen3-4b-base` | [kerasformers/qwen3-4b-base](https://huggingface.co/kerasformers/qwen3-4b-base) |
59
+ | `qwen3-8b` | [kerasformers/qwen3-8b](https://huggingface.co/kerasformers/qwen3-8b) |
60
+ | `qwen3-8b-base` | [kerasformers/qwen3-8b-base](https://huggingface.co/kerasformers/qwen3-8b-base) |
61
+ | `qwen3-14b` | [kerasformers/qwen3-14b](https://huggingface.co/kerasformers/qwen3-14b) |
62
+ | `qwen3-14b-base` | [kerasformers/qwen3-14b-base](https://huggingface.co/kerasformers/qwen3-14b-base) |
63
+ | `qwen3-32b` | [kerasformers/qwen3-32b](https://huggingface.co/kerasformers/qwen3-32b) |
64
+ | `qwen3-30b-a3b` | [kerasformers/qwen3-30b-a3b](https://huggingface.co/kerasformers/qwen3-30b-a3b) |
65
+ | `qwen3-30b-a3b-instruct-2507` | [kerasformers/qwen3-30b-a3b-instruct-2507](https://huggingface.co/kerasformers/qwen3-30b-a3b-instruct-2507) |
66
+
67
+ ## Special Thanks
68
+
69
+ A huge thank you to the Qwen team at Alibaba for creating and releasing these models.
70
+
71
+ License: Apache 2.0.
kf_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "library_name": "kerasformers",
3
+ "kerasformers_version": "1.2.2",
4
+ "model_module": "kerasformers.models.qwen3",
5
+ "model_class": "Qwen3TextGenerate",
6
+ "variant": "qwen3-8b",
7
+ "weights": "model.weights.json",
8
+ "schema_version": 2,
9
+ "weight_dtype": "bfloat16",
10
+ "model_type": "qwen3",
11
+ "text_config": {
12
+ "vocab_size": 151936,
13
+ "embed_dim": 4096,
14
+ "mlp_dim": 12288,
15
+ "num_layers": 36,
16
+ "num_heads": 32,
17
+ "num_kv_heads": 8,
18
+ "head_dim": 128,
19
+ "norm_eps": 1e-06,
20
+ "rope_theta": 1000000,
21
+ "tie_embeddings": false
22
+ }
23
+ }
model.weights.json ADDED
@@ -0,0 +1,1204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 16381470720.0
4
+ },
5
+ "weight_map": {
6
+ "/layers/embedding/vars": [
7
+ "model_00000.weights.h5"
8
+ ],
9
+ "/layers/qwen3_decoder_layer/attention_norm/vars": [
10
+ "model_00000.weights.h5"
11
+ ],
12
+ "/layers/qwen3_decoder_layer/mlp/down/vars": [
13
+ "model_00000.weights.h5"
14
+ ],
15
+ "/layers/qwen3_decoder_layer/mlp/gate/vars": [
16
+ "model_00000.weights.h5"
17
+ ],
18
+ "/layers/qwen3_decoder_layer/mlp/up/vars": [
19
+ "model_00000.weights.h5"
20
+ ],
21
+ "/layers/qwen3_decoder_layer/mlp_norm/vars": [
22
+ "model_00000.weights.h5"
23
+ ],
24
+ "/layers/qwen3_decoder_layer/attention/key_norm/vars": [
25
+ "model_00000.weights.h5"
26
+ ],
27
+ "/layers/qwen3_decoder_layer/attention/key/vars": [
28
+ "model_00000.weights.h5"
29
+ ],
30
+ "/layers/qwen3_decoder_layer/attention/output_proj/vars": [
31
+ "model_00000.weights.h5"
32
+ ],
33
+ "/layers/qwen3_decoder_layer/attention/query_norm/vars": [
34
+ "model_00000.weights.h5"
35
+ ],
36
+ "/layers/qwen3_decoder_layer/attention/query/vars": [
37
+ "model_00000.weights.h5"
38
+ ],
39
+ "/layers/qwen3_decoder_layer/attention/value/vars": [
40
+ "model_00000.weights.h5"
41
+ ],
42
+ "/layers/qwen3_decoder_layer_1/attention_norm/vars": [
43
+ "model_00000.weights.h5"
44
+ ],
45
+ "/layers/qwen3_decoder_layer_1/mlp/down/vars": [
46
+ "model_00000.weights.h5"
47
+ ],
48
+ "/layers/qwen3_decoder_layer_1/mlp/gate/vars": [
49
+ "model_00000.weights.h5"
50
+ ],
51
+ "/layers/qwen3_decoder_layer_1/mlp/up/vars": [
52
+ "model_00000.weights.h5"
53
+ ],
54
+ "/layers/qwen3_decoder_layer_1/mlp_norm/vars": [
55
+ "model_00000.weights.h5"
56
+ ],
57
+ "/layers/qwen3_decoder_layer_1/attention/key_norm/vars": [
58
+ "model_00000.weights.h5"
59
+ ],
60
+ "/layers/qwen3_decoder_layer_1/attention/key/vars": [
61
+ "model_00000.weights.h5"
62
+ ],
63
+ "/layers/qwen3_decoder_layer_1/attention/output_proj/vars": [
64
+ "model_00000.weights.h5"
65
+ ],
66
+ "/layers/qwen3_decoder_layer_1/attention/query_norm/vars": [
67
+ "model_00000.weights.h5"
68
+ ],
69
+ "/layers/qwen3_decoder_layer_1/attention/query/vars": [
70
+ "model_00000.weights.h5"
71
+ ],
72
+ "/layers/qwen3_decoder_layer_1/attention/value/vars": [
73
+ "model_00000.weights.h5"
74
+ ],
75
+ "/layers/qwen3_decoder_layer_2/attention_norm/vars": [
76
+ "model_00000.weights.h5"
77
+ ],
78
+ "/layers/qwen3_decoder_layer_2/mlp/down/vars": [
79
+ "model_00000.weights.h5"
80
+ ],
81
+ "/layers/qwen3_decoder_layer_2/mlp/gate/vars": [
82
+ "model_00000.weights.h5"
83
+ ],
84
+ "/layers/qwen3_decoder_layer_2/mlp/up/vars": [
85
+ "model_00000.weights.h5"
86
+ ],
87
+ "/layers/qwen3_decoder_layer_2/mlp_norm/vars": [
88
+ "model_00000.weights.h5"
89
+ ],
90
+ "/layers/qwen3_decoder_layer_2/attention/key_norm/vars": [
91
+ "model_00000.weights.h5"
92
+ ],
93
+ "/layers/qwen3_decoder_layer_2/attention/key/vars": [
94
+ "model_00000.weights.h5"
95
+ ],
96
+ "/layers/qwen3_decoder_layer_2/attention/output_proj/vars": [
97
+ "model_00000.weights.h5"
98
+ ],
99
+ "/layers/qwen3_decoder_layer_2/attention/query_norm/vars": [
100
+ "model_00000.weights.h5"
101
+ ],
102
+ "/layers/qwen3_decoder_layer_2/attention/query/vars": [
103
+ "model_00000.weights.h5"
104
+ ],
105
+ "/layers/qwen3_decoder_layer_2/attention/value/vars": [
106
+ "model_00000.weights.h5"
107
+ ],
108
+ "/layers/qwen3_decoder_layer_3/attention_norm/vars": [
109
+ "model_00000.weights.h5"
110
+ ],
111
+ "/layers/qwen3_decoder_layer_3/mlp/down/vars": [
112
+ "model_00000.weights.h5"
113
+ ],
114
+ "/layers/qwen3_decoder_layer_3/mlp/gate/vars": [
115
+ "model_00000.weights.h5"
116
+ ],
117
+ "/layers/qwen3_decoder_layer_3/mlp/up/vars": [
118
+ "model_00000.weights.h5"
119
+ ],
120
+ "/layers/qwen3_decoder_layer_3/mlp_norm/vars": [
121
+ "model_00000.weights.h5"
122
+ ],
123
+ "/layers/qwen3_decoder_layer_3/attention/key_norm/vars": [
124
+ "model_00000.weights.h5"
125
+ ],
126
+ "/layers/qwen3_decoder_layer_3/attention/key/vars": [
127
+ "model_00000.weights.h5"
128
+ ],
129
+ "/layers/qwen3_decoder_layer_3/attention/output_proj/vars": [
130
+ "model_00000.weights.h5"
131
+ ],
132
+ "/layers/qwen3_decoder_layer_3/attention/query_norm/vars": [
133
+ "model_00000.weights.h5"
134
+ ],
135
+ "/layers/qwen3_decoder_layer_3/attention/query/vars": [
136
+ "model_00000.weights.h5"
137
+ ],
138
+ "/layers/qwen3_decoder_layer_3/attention/value/vars": [
139
+ "model_00000.weights.h5"
140
+ ],
141
+ "/layers/qwen3_decoder_layer_4/attention_norm/vars": [
142
+ "model_00000.weights.h5"
143
+ ],
144
+ "/layers/qwen3_decoder_layer_4/mlp/down/vars": [
145
+ "model_00000.weights.h5"
146
+ ],
147
+ "/layers/qwen3_decoder_layer_4/mlp/gate/vars": [
148
+ "model_00000.weights.h5"
149
+ ],
150
+ "/layers/qwen3_decoder_layer_4/mlp/up/vars": [
151
+ "model_00000.weights.h5"
152
+ ],
153
+ "/layers/qwen3_decoder_layer_4/mlp_norm/vars": [
154
+ "model_00000.weights.h5"
155
+ ],
156
+ "/layers/qwen3_decoder_layer_4/attention/key_norm/vars": [
157
+ "model_00000.weights.h5"
158
+ ],
159
+ "/layers/qwen3_decoder_layer_4/attention/key/vars": [
160
+ "model_00000.weights.h5"
161
+ ],
162
+ "/layers/qwen3_decoder_layer_4/attention/output_proj/vars": [
163
+ "model_00000.weights.h5"
164
+ ],
165
+ "/layers/qwen3_decoder_layer_4/attention/query_norm/vars": [
166
+ "model_00000.weights.h5"
167
+ ],
168
+ "/layers/qwen3_decoder_layer_4/attention/query/vars": [
169
+ "model_00000.weights.h5"
170
+ ],
171
+ "/layers/qwen3_decoder_layer_4/attention/value/vars": [
172
+ "model_00000.weights.h5"
173
+ ],
174
+ "/layers/qwen3_decoder_layer_5/attention_norm/vars": [
175
+ "model_00000.weights.h5"
176
+ ],
177
+ "/layers/qwen3_decoder_layer_5/mlp/down/vars": [
178
+ "model_00000.weights.h5"
179
+ ],
180
+ "/layers/qwen3_decoder_layer_5/mlp/gate/vars": [
181
+ "model_00000.weights.h5"
182
+ ],
183
+ "/layers/qwen3_decoder_layer_5/mlp/up/vars": [
184
+ "model_00000.weights.h5"
185
+ ],
186
+ "/layers/qwen3_decoder_layer_5/mlp_norm/vars": [
187
+ "model_00000.weights.h5"
188
+ ],
189
+ "/layers/qwen3_decoder_layer_5/attention/key_norm/vars": [
190
+ "model_00000.weights.h5"
191
+ ],
192
+ "/layers/qwen3_decoder_layer_5/attention/key/vars": [
193
+ "model_00000.weights.h5"
194
+ ],
195
+ "/layers/qwen3_decoder_layer_5/attention/output_proj/vars": [
196
+ "model_00000.weights.h5"
197
+ ],
198
+ "/layers/qwen3_decoder_layer_5/attention/query_norm/vars": [
199
+ "model_00000.weights.h5"
200
+ ],
201
+ "/layers/qwen3_decoder_layer_5/attention/query/vars": [
202
+ "model_00000.weights.h5"
203
+ ],
204
+ "/layers/qwen3_decoder_layer_5/attention/value/vars": [
205
+ "model_00000.weights.h5"
206
+ ],
207
+ "/layers/qwen3_decoder_layer_6/attention_norm/vars": [
208
+ "model_00000.weights.h5"
209
+ ],
210
+ "/layers/qwen3_decoder_layer_6/mlp/down/vars": [
211
+ "model_00000.weights.h5"
212
+ ],
213
+ "/layers/qwen3_decoder_layer_6/mlp/gate/vars": [
214
+ "model_00000.weights.h5"
215
+ ],
216
+ "/layers/qwen3_decoder_layer_6/mlp/up/vars": [
217
+ "model_00000.weights.h5"
218
+ ],
219
+ "/layers/qwen3_decoder_layer_6/mlp_norm/vars": [
220
+ "model_00000.weights.h5"
221
+ ],
222
+ "/layers/qwen3_decoder_layer_6/attention/key_norm/vars": [
223
+ "model_00000.weights.h5"
224
+ ],
225
+ "/layers/qwen3_decoder_layer_6/attention/key/vars": [
226
+ "model_00000.weights.h5"
227
+ ],
228
+ "/layers/qwen3_decoder_layer_6/attention/output_proj/vars": [
229
+ "model_00000.weights.h5"
230
+ ],
231
+ "/layers/qwen3_decoder_layer_6/attention/query_norm/vars": [
232
+ "model_00000.weights.h5"
233
+ ],
234
+ "/layers/qwen3_decoder_layer_6/attention/query/vars": [
235
+ "model_00000.weights.h5"
236
+ ],
237
+ "/layers/qwen3_decoder_layer_6/attention/value/vars": [
238
+ "model_00000.weights.h5"
239
+ ],
240
+ "/layers/qwen3_decoder_layer_7/attention/key/vars": [
241
+ "model_00000.weights.h5"
242
+ ],
243
+ "/layers/qwen3_decoder_layer_7/attention/query/vars": [
244
+ "model_00000.weights.h5"
245
+ ],
246
+ "/layers/qwen3_decoder_layer_7/attention/value/vars": [
247
+ "model_00000.weights.h5"
248
+ ],
249
+ "/layers/qwen3_decoder_layer_10/attention_norm/vars": [
250
+ "model_00000.weights.h5"
251
+ ],
252
+ "/layers/qwen3_decoder_layer_10/mlp/down/vars": [
253
+ "model_00000.weights.h5"
254
+ ],
255
+ "/layers/qwen3_decoder_layer_10/mlp/gate/vars": [
256
+ "model_00000.weights.h5"
257
+ ],
258
+ "/layers/qwen3_decoder_layer_10/mlp/up/vars": [
259
+ "model_00000.weights.h5"
260
+ ],
261
+ "/layers/qwen3_decoder_layer_10/mlp_norm/vars": [
262
+ "model_00000.weights.h5"
263
+ ],
264
+ "/layers/qwen3_decoder_layer_10/attention/key_norm/vars": [
265
+ "model_00000.weights.h5"
266
+ ],
267
+ "/layers/qwen3_decoder_layer_10/attention/key/vars": [
268
+ "model_00000.weights.h5"
269
+ ],
270
+ "/layers/qwen3_decoder_layer_10/attention/output_proj/vars": [
271
+ "model_00000.weights.h5"
272
+ ],
273
+ "/layers/qwen3_decoder_layer_10/attention/query_norm/vars": [
274
+ "model_00000.weights.h5"
275
+ ],
276
+ "/layers/qwen3_decoder_layer_10/attention/query/vars": [
277
+ "model_00000.weights.h5"
278
+ ],
279
+ "/layers/qwen3_decoder_layer_10/attention/value/vars": [
280
+ "model_00000.weights.h5"
281
+ ],
282
+ "/layers/qwen3_decoder_layer_11/attention_norm/vars": [
283
+ "model_00000.weights.h5"
284
+ ],
285
+ "/layers/qwen3_decoder_layer_11/mlp/down/vars": [
286
+ "model_00000.weights.h5"
287
+ ],
288
+ "/layers/qwen3_decoder_layer_11/mlp/gate/vars": [
289
+ "model_00000.weights.h5"
290
+ ],
291
+ "/layers/qwen3_decoder_layer_11/mlp/up/vars": [
292
+ "model_00000.weights.h5"
293
+ ],
294
+ "/layers/qwen3_decoder_layer_11/mlp_norm/vars": [
295
+ "model_00000.weights.h5"
296
+ ],
297
+ "/layers/qwen3_decoder_layer_11/attention/key_norm/vars": [
298
+ "model_00000.weights.h5"
299
+ ],
300
+ "/layers/qwen3_decoder_layer_11/attention/key/vars": [
301
+ "model_00000.weights.h5"
302
+ ],
303
+ "/layers/qwen3_decoder_layer_11/attention/output_proj/vars": [
304
+ "model_00000.weights.h5"
305
+ ],
306
+ "/layers/qwen3_decoder_layer_11/attention/query_norm/vars": [
307
+ "model_00000.weights.h5"
308
+ ],
309
+ "/layers/qwen3_decoder_layer_11/attention/query/vars": [
310
+ "model_00000.weights.h5"
311
+ ],
312
+ "/layers/qwen3_decoder_layer_11/attention/value/vars": [
313
+ "model_00000.weights.h5"
314
+ ],
315
+ "/layers/qwen3_decoder_layer_12/attention_norm/vars": [
316
+ "model_00000.weights.h5"
317
+ ],
318
+ "/layers/qwen3_decoder_layer_12/mlp/down/vars": [
319
+ "model_00000.weights.h5"
320
+ ],
321
+ "/layers/qwen3_decoder_layer_12/mlp/gate/vars": [
322
+ "model_00000.weights.h5"
323
+ ],
324
+ "/layers/qwen3_decoder_layer_12/mlp/up/vars": [
325
+ "model_00000.weights.h5"
326
+ ],
327
+ "/layers/qwen3_decoder_layer_12/mlp_norm/vars": [
328
+ "model_00000.weights.h5"
329
+ ],
330
+ "/layers/qwen3_decoder_layer_12/attention/key_norm/vars": [
331
+ "model_00000.weights.h5"
332
+ ],
333
+ "/layers/qwen3_decoder_layer_12/attention/key/vars": [
334
+ "model_00000.weights.h5"
335
+ ],
336
+ "/layers/qwen3_decoder_layer_12/attention/output_proj/vars": [
337
+ "model_00000.weights.h5"
338
+ ],
339
+ "/layers/qwen3_decoder_layer_12/attention/query_norm/vars": [
340
+ "model_00000.weights.h5"
341
+ ],
342
+ "/layers/qwen3_decoder_layer_12/attention/query/vars": [
343
+ "model_00000.weights.h5"
344
+ ],
345
+ "/layers/qwen3_decoder_layer_12/attention/value/vars": [
346
+ "model_00000.weights.h5"
347
+ ],
348
+ "/layers/qwen3_decoder_layer_13/attention_norm/vars": [
349
+ "model_00000.weights.h5"
350
+ ],
351
+ "/layers/qwen3_decoder_layer_13/mlp/down/vars": [
352
+ "model_00000.weights.h5"
353
+ ],
354
+ "/layers/qwen3_decoder_layer_13/mlp/gate/vars": [
355
+ "model_00000.weights.h5"
356
+ ],
357
+ "/layers/qwen3_decoder_layer_13/mlp/up/vars": [
358
+ "model_00001.weights.h5"
359
+ ],
360
+ "/layers/qwen3_decoder_layer_13/mlp_norm/vars": [
361
+ "model_00001.weights.h5"
362
+ ],
363
+ "/layers/qwen3_decoder_layer_13/attention/key_norm/vars": [
364
+ "model_00001.weights.h5"
365
+ ],
366
+ "/layers/qwen3_decoder_layer_13/attention/key/vars": [
367
+ "model_00001.weights.h5"
368
+ ],
369
+ "/layers/qwen3_decoder_layer_13/attention/output_proj/vars": [
370
+ "model_00001.weights.h5"
371
+ ],
372
+ "/layers/qwen3_decoder_layer_13/attention/query_norm/vars": [
373
+ "model_00001.weights.h5"
374
+ ],
375
+ "/layers/qwen3_decoder_layer_13/attention/query/vars": [
376
+ "model_00001.weights.h5"
377
+ ],
378
+ "/layers/qwen3_decoder_layer_13/attention/value/vars": [
379
+ "model_00001.weights.h5"
380
+ ],
381
+ "/layers/qwen3_decoder_layer_14/attention_norm/vars": [
382
+ "model_00001.weights.h5"
383
+ ],
384
+ "/layers/qwen3_decoder_layer_14/mlp/down/vars": [
385
+ "model_00001.weights.h5"
386
+ ],
387
+ "/layers/qwen3_decoder_layer_14/mlp/gate/vars": [
388
+ "model_00001.weights.h5"
389
+ ],
390
+ "/layers/qwen3_decoder_layer_14/mlp/up/vars": [
391
+ "model_00001.weights.h5"
392
+ ],
393
+ "/layers/qwen3_decoder_layer_14/mlp_norm/vars": [
394
+ "model_00001.weights.h5"
395
+ ],
396
+ "/layers/qwen3_decoder_layer_14/attention/key_norm/vars": [
397
+ "model_00001.weights.h5"
398
+ ],
399
+ "/layers/qwen3_decoder_layer_14/attention/key/vars": [
400
+ "model_00001.weights.h5"
401
+ ],
402
+ "/layers/qwen3_decoder_layer_14/attention/output_proj/vars": [
403
+ "model_00001.weights.h5"
404
+ ],
405
+ "/layers/qwen3_decoder_layer_14/attention/query_norm/vars": [
406
+ "model_00001.weights.h5"
407
+ ],
408
+ "/layers/qwen3_decoder_layer_14/attention/query/vars": [
409
+ "model_00001.weights.h5"
410
+ ],
411
+ "/layers/qwen3_decoder_layer_14/attention/value/vars": [
412
+ "model_00001.weights.h5"
413
+ ],
414
+ "/layers/qwen3_decoder_layer_15/attention_norm/vars": [
415
+ "model_00001.weights.h5"
416
+ ],
417
+ "/layers/qwen3_decoder_layer_15/mlp/down/vars": [
418
+ "model_00001.weights.h5"
419
+ ],
420
+ "/layers/qwen3_decoder_layer_15/mlp/gate/vars": [
421
+ "model_00001.weights.h5"
422
+ ],
423
+ "/layers/qwen3_decoder_layer_15/mlp/up/vars": [
424
+ "model_00001.weights.h5"
425
+ ],
426
+ "/layers/qwen3_decoder_layer_15/mlp_norm/vars": [
427
+ "model_00001.weights.h5"
428
+ ],
429
+ "/layers/qwen3_decoder_layer_15/attention/key_norm/vars": [
430
+ "model_00001.weights.h5"
431
+ ],
432
+ "/layers/qwen3_decoder_layer_15/attention/key/vars": [
433
+ "model_00001.weights.h5"
434
+ ],
435
+ "/layers/qwen3_decoder_layer_15/attention/output_proj/vars": [
436
+ "model_00001.weights.h5"
437
+ ],
438
+ "/layers/qwen3_decoder_layer_15/attention/query_norm/vars": [
439
+ "model_00001.weights.h5"
440
+ ],
441
+ "/layers/qwen3_decoder_layer_15/attention/query/vars": [
442
+ "model_00001.weights.h5"
443
+ ],
444
+ "/layers/qwen3_decoder_layer_15/attention/value/vars": [
445
+ "model_00001.weights.h5"
446
+ ],
447
+ "/layers/qwen3_decoder_layer_16/attention_norm/vars": [
448
+ "model_00001.weights.h5"
449
+ ],
450
+ "/layers/qwen3_decoder_layer_16/mlp/down/vars": [
451
+ "model_00001.weights.h5"
452
+ ],
453
+ "/layers/qwen3_decoder_layer_16/mlp/gate/vars": [
454
+ "model_00001.weights.h5"
455
+ ],
456
+ "/layers/qwen3_decoder_layer_16/mlp/up/vars": [
457
+ "model_00001.weights.h5"
458
+ ],
459
+ "/layers/qwen3_decoder_layer_16/mlp_norm/vars": [
460
+ "model_00001.weights.h5"
461
+ ],
462
+ "/layers/qwen3_decoder_layer_16/attention/key_norm/vars": [
463
+ "model_00001.weights.h5"
464
+ ],
465
+ "/layers/qwen3_decoder_layer_16/attention/key/vars": [
466
+ "model_00001.weights.h5"
467
+ ],
468
+ "/layers/qwen3_decoder_layer_16/attention/output_proj/vars": [
469
+ "model_00001.weights.h5"
470
+ ],
471
+ "/layers/qwen3_decoder_layer_16/attention/query_norm/vars": [
472
+ "model_00001.weights.h5"
473
+ ],
474
+ "/layers/qwen3_decoder_layer_16/attention/query/vars": [
475
+ "model_00001.weights.h5"
476
+ ],
477
+ "/layers/qwen3_decoder_layer_16/attention/value/vars": [
478
+ "model_00001.weights.h5"
479
+ ],
480
+ "/layers/qwen3_decoder_layer_17/mlp/gate/vars": [
481
+ "model_00001.weights.h5"
482
+ ],
483
+ "/layers/qwen3_decoder_layer_17/attention/key_norm/vars": [
484
+ "model_00001.weights.h5"
485
+ ],
486
+ "/layers/qwen3_decoder_layer_17/attention/key/vars": [
487
+ "model_00001.weights.h5"
488
+ ],
489
+ "/layers/qwen3_decoder_layer_17/attention/output_proj/vars": [
490
+ "model_00001.weights.h5"
491
+ ],
492
+ "/layers/qwen3_decoder_layer_17/attention/query_norm/vars": [
493
+ "model_00001.weights.h5"
494
+ ],
495
+ "/layers/qwen3_decoder_layer_17/attention/query/vars": [
496
+ "model_00001.weights.h5"
497
+ ],
498
+ "/layers/qwen3_decoder_layer_17/attention/value/vars": [
499
+ "model_00001.weights.h5"
500
+ ],
501
+ "/layers/qwen3_decoder_layer_7/attention_norm/vars": [
502
+ "model_00001.weights.h5"
503
+ ],
504
+ "/layers/qwen3_decoder_layer_7/mlp/down/vars": [
505
+ "model_00001.weights.h5"
506
+ ],
507
+ "/layers/qwen3_decoder_layer_7/mlp/gate/vars": [
508
+ "model_00001.weights.h5"
509
+ ],
510
+ "/layers/qwen3_decoder_layer_7/mlp/up/vars": [
511
+ "model_00001.weights.h5"
512
+ ],
513
+ "/layers/qwen3_decoder_layer_7/mlp_norm/vars": [
514
+ "model_00001.weights.h5"
515
+ ],
516
+ "/layers/qwen3_decoder_layer_7/attention/key_norm/vars": [
517
+ "model_00001.weights.h5"
518
+ ],
519
+ "/layers/qwen3_decoder_layer_7/attention/output_proj/vars": [
520
+ "model_00001.weights.h5"
521
+ ],
522
+ "/layers/qwen3_decoder_layer_7/attention/query_norm/vars": [
523
+ "model_00001.weights.h5"
524
+ ],
525
+ "/layers/qwen3_decoder_layer_8/attention_norm/vars": [
526
+ "model_00001.weights.h5"
527
+ ],
528
+ "/layers/qwen3_decoder_layer_8/mlp/down/vars": [
529
+ "model_00001.weights.h5"
530
+ ],
531
+ "/layers/qwen3_decoder_layer_8/mlp/gate/vars": [
532
+ "model_00001.weights.h5"
533
+ ],
534
+ "/layers/qwen3_decoder_layer_8/mlp/up/vars": [
535
+ "model_00001.weights.h5"
536
+ ],
537
+ "/layers/qwen3_decoder_layer_8/mlp_norm/vars": [
538
+ "model_00001.weights.h5"
539
+ ],
540
+ "/layers/qwen3_decoder_layer_8/attention/key_norm/vars": [
541
+ "model_00001.weights.h5"
542
+ ],
543
+ "/layers/qwen3_decoder_layer_8/attention/key/vars": [
544
+ "model_00001.weights.h5"
545
+ ],
546
+ "/layers/qwen3_decoder_layer_8/attention/output_proj/vars": [
547
+ "model_00001.weights.h5"
548
+ ],
549
+ "/layers/qwen3_decoder_layer_8/attention/query_norm/vars": [
550
+ "model_00001.weights.h5"
551
+ ],
552
+ "/layers/qwen3_decoder_layer_8/attention/query/vars": [
553
+ "model_00001.weights.h5"
554
+ ],
555
+ "/layers/qwen3_decoder_layer_8/attention/value/vars": [
556
+ "model_00001.weights.h5"
557
+ ],
558
+ "/layers/qwen3_decoder_layer_9/attention_norm/vars": [
559
+ "model_00001.weights.h5"
560
+ ],
561
+ "/layers/qwen3_decoder_layer_9/mlp/down/vars": [
562
+ "model_00001.weights.h5"
563
+ ],
564
+ "/layers/qwen3_decoder_layer_9/mlp/gate/vars": [
565
+ "model_00001.weights.h5"
566
+ ],
567
+ "/layers/qwen3_decoder_layer_9/mlp/up/vars": [
568
+ "model_00001.weights.h5"
569
+ ],
570
+ "/layers/qwen3_decoder_layer_9/mlp_norm/vars": [
571
+ "model_00001.weights.h5"
572
+ ],
573
+ "/layers/qwen3_decoder_layer_9/attention/key_norm/vars": [
574
+ "model_00001.weights.h5"
575
+ ],
576
+ "/layers/qwen3_decoder_layer_9/attention/key/vars": [
577
+ "model_00001.weights.h5"
578
+ ],
579
+ "/layers/qwen3_decoder_layer_9/attention/output_proj/vars": [
580
+ "model_00001.weights.h5"
581
+ ],
582
+ "/layers/qwen3_decoder_layer_9/attention/query_norm/vars": [
583
+ "model_00001.weights.h5"
584
+ ],
585
+ "/layers/qwen3_decoder_layer_9/attention/query/vars": [
586
+ "model_00001.weights.h5"
587
+ ],
588
+ "/layers/qwen3_decoder_layer_9/attention/value/vars": [
589
+ "model_00001.weights.h5"
590
+ ],
591
+ "/layers/qwen3_decoder_layer_17/attention_norm/vars": [
592
+ "model_00001.weights.h5"
593
+ ],
594
+ "/layers/qwen3_decoder_layer_17/mlp/down/vars": [
595
+ "model_00001.weights.h5"
596
+ ],
597
+ "/layers/qwen3_decoder_layer_17/mlp/up/vars": [
598
+ "model_00001.weights.h5"
599
+ ],
600
+ "/layers/qwen3_decoder_layer_17/mlp_norm/vars": [
601
+ "model_00001.weights.h5"
602
+ ],
603
+ "/layers/qwen3_decoder_layer_18/attention_norm/vars": [
604
+ "model_00001.weights.h5"
605
+ ],
606
+ "/layers/qwen3_decoder_layer_18/mlp/down/vars": [
607
+ "model_00001.weights.h5"
608
+ ],
609
+ "/layers/qwen3_decoder_layer_18/mlp/gate/vars": [
610
+ "model_00001.weights.h5"
611
+ ],
612
+ "/layers/qwen3_decoder_layer_18/mlp/up/vars": [
613
+ "model_00001.weights.h5"
614
+ ],
615
+ "/layers/qwen3_decoder_layer_18/mlp_norm/vars": [
616
+ "model_00001.weights.h5"
617
+ ],
618
+ "/layers/qwen3_decoder_layer_18/attention/key_norm/vars": [
619
+ "model_00001.weights.h5"
620
+ ],
621
+ "/layers/qwen3_decoder_layer_18/attention/key/vars": [
622
+ "model_00001.weights.h5"
623
+ ],
624
+ "/layers/qwen3_decoder_layer_18/attention/output_proj/vars": [
625
+ "model_00001.weights.h5"
626
+ ],
627
+ "/layers/qwen3_decoder_layer_18/attention/query_norm/vars": [
628
+ "model_00001.weights.h5"
629
+ ],
630
+ "/layers/qwen3_decoder_layer_18/attention/query/vars": [
631
+ "model_00001.weights.h5"
632
+ ],
633
+ "/layers/qwen3_decoder_layer_18/attention/value/vars": [
634
+ "model_00001.weights.h5"
635
+ ],
636
+ "/layers/qwen3_decoder_layer_19/attention_norm/vars": [
637
+ "model_00001.weights.h5"
638
+ ],
639
+ "/layers/qwen3_decoder_layer_19/mlp/down/vars": [
640
+ "model_00001.weights.h5"
641
+ ],
642
+ "/layers/qwen3_decoder_layer_19/mlp/gate/vars": [
643
+ "model_00001.weights.h5"
644
+ ],
645
+ "/layers/qwen3_decoder_layer_19/mlp/up/vars": [
646
+ "model_00001.weights.h5"
647
+ ],
648
+ "/layers/qwen3_decoder_layer_19/mlp_norm/vars": [
649
+ "model_00001.weights.h5"
650
+ ],
651
+ "/layers/qwen3_decoder_layer_19/attention/key_norm/vars": [
652
+ "model_00001.weights.h5"
653
+ ],
654
+ "/layers/qwen3_decoder_layer_19/attention/key/vars": [
655
+ "model_00001.weights.h5"
656
+ ],
657
+ "/layers/qwen3_decoder_layer_19/attention/output_proj/vars": [
658
+ "model_00001.weights.h5"
659
+ ],
660
+ "/layers/qwen3_decoder_layer_19/attention/query_norm/vars": [
661
+ "model_00001.weights.h5"
662
+ ],
663
+ "/layers/qwen3_decoder_layer_19/attention/query/vars": [
664
+ "model_00001.weights.h5"
665
+ ],
666
+ "/layers/qwen3_decoder_layer_19/attention/value/vars": [
667
+ "model_00001.weights.h5"
668
+ ],
669
+ "/layers/qwen3_decoder_layer_20/attention_norm/vars": [
670
+ "model_00001.weights.h5"
671
+ ],
672
+ "/layers/qwen3_decoder_layer_20/mlp/down/vars": [
673
+ "model_00001.weights.h5"
674
+ ],
675
+ "/layers/qwen3_decoder_layer_20/mlp/gate/vars": [
676
+ "model_00001.weights.h5"
677
+ ],
678
+ "/layers/qwen3_decoder_layer_20/mlp/up/vars": [
679
+ "model_00001.weights.h5"
680
+ ],
681
+ "/layers/qwen3_decoder_layer_20/mlp_norm/vars": [
682
+ "model_00001.weights.h5"
683
+ ],
684
+ "/layers/qwen3_decoder_layer_20/attention/key_norm/vars": [
685
+ "model_00001.weights.h5"
686
+ ],
687
+ "/layers/qwen3_decoder_layer_20/attention/key/vars": [
688
+ "model_00001.weights.h5"
689
+ ],
690
+ "/layers/qwen3_decoder_layer_20/attention/output_proj/vars": [
691
+ "model_00001.weights.h5"
692
+ ],
693
+ "/layers/qwen3_decoder_layer_20/attention/query_norm/vars": [
694
+ "model_00001.weights.h5"
695
+ ],
696
+ "/layers/qwen3_decoder_layer_20/attention/query/vars": [
697
+ "model_00001.weights.h5"
698
+ ],
699
+ "/layers/qwen3_decoder_layer_20/attention/value/vars": [
700
+ "model_00001.weights.h5"
701
+ ],
702
+ "/layers/qwen3_decoder_layer_21/attention_norm/vars": [
703
+ "model_00001.weights.h5"
704
+ ],
705
+ "/layers/qwen3_decoder_layer_21/mlp/down/vars": [
706
+ "model_00001.weights.h5"
707
+ ],
708
+ "/layers/qwen3_decoder_layer_21/mlp/gate/vars": [
709
+ "model_00001.weights.h5"
710
+ ],
711
+ "/layers/qwen3_decoder_layer_21/mlp/up/vars": [
712
+ "model_00001.weights.h5"
713
+ ],
714
+ "/layers/qwen3_decoder_layer_21/mlp_norm/vars": [
715
+ "model_00001.weights.h5"
716
+ ],
717
+ "/layers/qwen3_decoder_layer_21/attention/key_norm/vars": [
718
+ "model_00001.weights.h5"
719
+ ],
720
+ "/layers/qwen3_decoder_layer_21/attention/key/vars": [
721
+ "model_00001.weights.h5"
722
+ ],
723
+ "/layers/qwen3_decoder_layer_21/attention/output_proj/vars": [
724
+ "model_00001.weights.h5"
725
+ ],
726
+ "/layers/qwen3_decoder_layer_21/attention/query_norm/vars": [
727
+ "model_00001.weights.h5"
728
+ ],
729
+ "/layers/qwen3_decoder_layer_21/attention/query/vars": [
730
+ "model_00001.weights.h5"
731
+ ],
732
+ "/layers/qwen3_decoder_layer_21/attention/value/vars": [
733
+ "model_00001.weights.h5"
734
+ ],
735
+ "/layers/qwen3_decoder_layer_22/attention_norm/vars": [
736
+ "model_00001.weights.h5"
737
+ ],
738
+ "/layers/qwen3_decoder_layer_22/mlp/down/vars": [
739
+ "model_00001.weights.h5"
740
+ ],
741
+ "/layers/qwen3_decoder_layer_22/mlp/gate/vars": [
742
+ "model_00001.weights.h5"
743
+ ],
744
+ "/layers/qwen3_decoder_layer_22/mlp/up/vars": [
745
+ "model_00001.weights.h5"
746
+ ],
747
+ "/layers/qwen3_decoder_layer_22/mlp_norm/vars": [
748
+ "model_00001.weights.h5"
749
+ ],
750
+ "/layers/qwen3_decoder_layer_22/attention/key_norm/vars": [
751
+ "model_00001.weights.h5"
752
+ ],
753
+ "/layers/qwen3_decoder_layer_22/attention/key/vars": [
754
+ "model_00001.weights.h5"
755
+ ],
756
+ "/layers/qwen3_decoder_layer_22/attention/output_proj/vars": [
757
+ "model_00001.weights.h5"
758
+ ],
759
+ "/layers/qwen3_decoder_layer_22/attention/query_norm/vars": [
760
+ "model_00001.weights.h5"
761
+ ],
762
+ "/layers/qwen3_decoder_layer_22/attention/query/vars": [
763
+ "model_00001.weights.h5"
764
+ ],
765
+ "/layers/qwen3_decoder_layer_22/attention/value/vars": [
766
+ "model_00001.weights.h5"
767
+ ],
768
+ "/layers/qwen3_decoder_layer_23/attention_norm/vars": [
769
+ "model_00001.weights.h5"
770
+ ],
771
+ "/layers/qwen3_decoder_layer_23/mlp/down/vars": [
772
+ "model_00001.weights.h5"
773
+ ],
774
+ "/layers/qwen3_decoder_layer_23/mlp/gate/vars": [
775
+ "model_00001.weights.h5"
776
+ ],
777
+ "/layers/qwen3_decoder_layer_23/mlp/up/vars": [
778
+ "model_00001.weights.h5"
779
+ ],
780
+ "/layers/qwen3_decoder_layer_23/mlp_norm/vars": [
781
+ "model_00001.weights.h5"
782
+ ],
783
+ "/layers/qwen3_decoder_layer_23/attention/key_norm/vars": [
784
+ "model_00001.weights.h5"
785
+ ],
786
+ "/layers/qwen3_decoder_layer_23/attention/key/vars": [
787
+ "model_00001.weights.h5"
788
+ ],
789
+ "/layers/qwen3_decoder_layer_23/attention/output_proj/vars": [
790
+ "model_00001.weights.h5"
791
+ ],
792
+ "/layers/qwen3_decoder_layer_23/attention/query_norm/vars": [
793
+ "model_00001.weights.h5"
794
+ ],
795
+ "/layers/qwen3_decoder_layer_23/attention/query/vars": [
796
+ "model_00001.weights.h5"
797
+ ],
798
+ "/layers/qwen3_decoder_layer_23/attention/value/vars": [
799
+ "model_00001.weights.h5"
800
+ ],
801
+ "/layers/qwen3_decoder_layer_24/attention_norm/vars": [
802
+ "model_00001.weights.h5"
803
+ ],
804
+ "/layers/qwen3_decoder_layer_24/mlp/down/vars": [
805
+ "model_00001.weights.h5"
806
+ ],
807
+ "/layers/qwen3_decoder_layer_24/mlp/gate/vars": [
808
+ "model_00001.weights.h5"
809
+ ],
810
+ "/layers/qwen3_decoder_layer_24/mlp/up/vars": [
811
+ "model_00002.weights.h5"
812
+ ],
813
+ "/layers/qwen3_decoder_layer_24/mlp_norm/vars": [
814
+ "model_00002.weights.h5"
815
+ ],
816
+ "/layers/qwen3_decoder_layer_24/attention/key_norm/vars": [
817
+ "model_00002.weights.h5"
818
+ ],
819
+ "/layers/qwen3_decoder_layer_24/attention/key/vars": [
820
+ "model_00002.weights.h5"
821
+ ],
822
+ "/layers/qwen3_decoder_layer_24/attention/output_proj/vars": [
823
+ "model_00002.weights.h5"
824
+ ],
825
+ "/layers/qwen3_decoder_layer_24/attention/query_norm/vars": [
826
+ "model_00002.weights.h5"
827
+ ],
828
+ "/layers/qwen3_decoder_layer_24/attention/query/vars": [
829
+ "model_00002.weights.h5"
830
+ ],
831
+ "/layers/qwen3_decoder_layer_24/attention/value/vars": [
832
+ "model_00002.weights.h5"
833
+ ],
834
+ "/layers/qwen3_decoder_layer_25/attention_norm/vars": [
835
+ "model_00002.weights.h5"
836
+ ],
837
+ "/layers/qwen3_decoder_layer_25/mlp/down/vars": [
838
+ "model_00002.weights.h5"
839
+ ],
840
+ "/layers/qwen3_decoder_layer_25/mlp/gate/vars": [
841
+ "model_00002.weights.h5"
842
+ ],
843
+ "/layers/qwen3_decoder_layer_25/mlp/up/vars": [
844
+ "model_00002.weights.h5"
845
+ ],
846
+ "/layers/qwen3_decoder_layer_25/mlp_norm/vars": [
847
+ "model_00002.weights.h5"
848
+ ],
849
+ "/layers/qwen3_decoder_layer_25/attention/key_norm/vars": [
850
+ "model_00002.weights.h5"
851
+ ],
852
+ "/layers/qwen3_decoder_layer_25/attention/key/vars": [
853
+ "model_00002.weights.h5"
854
+ ],
855
+ "/layers/qwen3_decoder_layer_25/attention/output_proj/vars": [
856
+ "model_00002.weights.h5"
857
+ ],
858
+ "/layers/qwen3_decoder_layer_25/attention/query_norm/vars": [
859
+ "model_00002.weights.h5"
860
+ ],
861
+ "/layers/qwen3_decoder_layer_25/attention/query/vars": [
862
+ "model_00002.weights.h5"
863
+ ],
864
+ "/layers/qwen3_decoder_layer_25/attention/value/vars": [
865
+ "model_00002.weights.h5"
866
+ ],
867
+ "/layers/qwen3_decoder_layer_26/attention_norm/vars": [
868
+ "model_00002.weights.h5"
869
+ ],
870
+ "/layers/qwen3_decoder_layer_26/mlp/down/vars": [
871
+ "model_00002.weights.h5"
872
+ ],
873
+ "/layers/qwen3_decoder_layer_26/mlp/gate/vars": [
874
+ "model_00002.weights.h5"
875
+ ],
876
+ "/layers/qwen3_decoder_layer_26/mlp/up/vars": [
877
+ "model_00002.weights.h5"
878
+ ],
879
+ "/layers/qwen3_decoder_layer_26/mlp_norm/vars": [
880
+ "model_00002.weights.h5"
881
+ ],
882
+ "/layers/qwen3_decoder_layer_26/attention/key_norm/vars": [
883
+ "model_00002.weights.h5"
884
+ ],
885
+ "/layers/qwen3_decoder_layer_26/attention/key/vars": [
886
+ "model_00002.weights.h5"
887
+ ],
888
+ "/layers/qwen3_decoder_layer_26/attention/output_proj/vars": [
889
+ "model_00002.weights.h5"
890
+ ],
891
+ "/layers/qwen3_decoder_layer_26/attention/query_norm/vars": [
892
+ "model_00002.weights.h5"
893
+ ],
894
+ "/layers/qwen3_decoder_layer_26/attention/query/vars": [
895
+ "model_00002.weights.h5"
896
+ ],
897
+ "/layers/qwen3_decoder_layer_26/attention/value/vars": [
898
+ "model_00002.weights.h5"
899
+ ],
900
+ "/layers/qwen3_decoder_layer_27/mlp/gate/vars": [
901
+ "model_00002.weights.h5"
902
+ ],
903
+ "/layers/qwen3_decoder_layer_27/mlp/up/vars": [
904
+ "model_00002.weights.h5"
905
+ ],
906
+ "/layers/qwen3_decoder_layer_27/attention/key_norm/vars": [
907
+ "model_00002.weights.h5"
908
+ ],
909
+ "/layers/qwen3_decoder_layer_27/attention/key/vars": [
910
+ "model_00002.weights.h5"
911
+ ],
912
+ "/layers/qwen3_decoder_layer_27/attention/output_proj/vars": [
913
+ "model_00002.weights.h5"
914
+ ],
915
+ "/layers/qwen3_decoder_layer_27/attention/query_norm/vars": [
916
+ "model_00002.weights.h5"
917
+ ],
918
+ "/layers/qwen3_decoder_layer_27/attention/query/vars": [
919
+ "model_00002.weights.h5"
920
+ ],
921
+ "/layers/qwen3_decoder_layer_27/attention/value/vars": [
922
+ "model_00002.weights.h5"
923
+ ],
924
+ "/layers/qwen3_decoder_layer_27/attention_norm/vars": [
925
+ "model_00002.weights.h5"
926
+ ],
927
+ "/layers/qwen3_decoder_layer_27/mlp/down/vars": [
928
+ "model_00002.weights.h5"
929
+ ],
930
+ "/layers/qwen3_decoder_layer_27/mlp_norm/vars": [
931
+ "model_00002.weights.h5"
932
+ ],
933
+ "/layers/qwen3_decoder_layer_28/attention_norm/vars": [
934
+ "model_00002.weights.h5"
935
+ ],
936
+ "/layers/qwen3_decoder_layer_28/mlp/down/vars": [
937
+ "model_00002.weights.h5"
938
+ ],
939
+ "/layers/qwen3_decoder_layer_28/mlp/gate/vars": [
940
+ "model_00002.weights.h5"
941
+ ],
942
+ "/layers/qwen3_decoder_layer_28/mlp/up/vars": [
943
+ "model_00002.weights.h5"
944
+ ],
945
+ "/layers/qwen3_decoder_layer_28/mlp_norm/vars": [
946
+ "model_00002.weights.h5"
947
+ ],
948
+ "/layers/qwen3_decoder_layer_28/attention/key_norm/vars": [
949
+ "model_00002.weights.h5"
950
+ ],
951
+ "/layers/qwen3_decoder_layer_28/attention/key/vars": [
952
+ "model_00002.weights.h5"
953
+ ],
954
+ "/layers/qwen3_decoder_layer_28/attention/output_proj/vars": [
955
+ "model_00002.weights.h5"
956
+ ],
957
+ "/layers/qwen3_decoder_layer_28/attention/query_norm/vars": [
958
+ "model_00002.weights.h5"
959
+ ],
960
+ "/layers/qwen3_decoder_layer_28/attention/query/vars": [
961
+ "model_00002.weights.h5"
962
+ ],
963
+ "/layers/qwen3_decoder_layer_28/attention/value/vars": [
964
+ "model_00002.weights.h5"
965
+ ],
966
+ "/layers/qwen3_decoder_layer_29/attention_norm/vars": [
967
+ "model_00002.weights.h5"
968
+ ],
969
+ "/layers/qwen3_decoder_layer_29/mlp/down/vars": [
970
+ "model_00002.weights.h5"
971
+ ],
972
+ "/layers/qwen3_decoder_layer_29/mlp/gate/vars": [
973
+ "model_00002.weights.h5"
974
+ ],
975
+ "/layers/qwen3_decoder_layer_29/mlp/up/vars": [
976
+ "model_00002.weights.h5"
977
+ ],
978
+ "/layers/qwen3_decoder_layer_29/mlp_norm/vars": [
979
+ "model_00002.weights.h5"
980
+ ],
981
+ "/layers/qwen3_decoder_layer_29/attention/key_norm/vars": [
982
+ "model_00002.weights.h5"
983
+ ],
984
+ "/layers/qwen3_decoder_layer_29/attention/key/vars": [
985
+ "model_00002.weights.h5"
986
+ ],
987
+ "/layers/qwen3_decoder_layer_29/attention/output_proj/vars": [
988
+ "model_00002.weights.h5"
989
+ ],
990
+ "/layers/qwen3_decoder_layer_29/attention/query_norm/vars": [
991
+ "model_00002.weights.h5"
992
+ ],
993
+ "/layers/qwen3_decoder_layer_29/attention/query/vars": [
994
+ "model_00002.weights.h5"
995
+ ],
996
+ "/layers/qwen3_decoder_layer_29/attention/value/vars": [
997
+ "model_00002.weights.h5"
998
+ ],
999
+ "/layers/qwen3_decoder_layer_30/attention_norm/vars": [
1000
+ "model_00002.weights.h5"
1001
+ ],
1002
+ "/layers/qwen3_decoder_layer_30/mlp/down/vars": [
1003
+ "model_00002.weights.h5"
1004
+ ],
1005
+ "/layers/qwen3_decoder_layer_30/mlp/gate/vars": [
1006
+ "model_00002.weights.h5"
1007
+ ],
1008
+ "/layers/qwen3_decoder_layer_30/mlp/up/vars": [
1009
+ "model_00002.weights.h5"
1010
+ ],
1011
+ "/layers/qwen3_decoder_layer_30/mlp_norm/vars": [
1012
+ "model_00002.weights.h5"
1013
+ ],
1014
+ "/layers/qwen3_decoder_layer_30/attention/key_norm/vars": [
1015
+ "model_00002.weights.h5"
1016
+ ],
1017
+ "/layers/qwen3_decoder_layer_30/attention/key/vars": [
1018
+ "model_00002.weights.h5"
1019
+ ],
1020
+ "/layers/qwen3_decoder_layer_30/attention/output_proj/vars": [
1021
+ "model_00002.weights.h5"
1022
+ ],
1023
+ "/layers/qwen3_decoder_layer_30/attention/query_norm/vars": [
1024
+ "model_00002.weights.h5"
1025
+ ],
1026
+ "/layers/qwen3_decoder_layer_30/attention/query/vars": [
1027
+ "model_00002.weights.h5"
1028
+ ],
1029
+ "/layers/qwen3_decoder_layer_30/attention/value/vars": [
1030
+ "model_00002.weights.h5"
1031
+ ],
1032
+ "/layers/qwen3_decoder_layer_31/attention_norm/vars": [
1033
+ "model_00002.weights.h5"
1034
+ ],
1035
+ "/layers/qwen3_decoder_layer_31/mlp/down/vars": [
1036
+ "model_00002.weights.h5"
1037
+ ],
1038
+ "/layers/qwen3_decoder_layer_31/mlp/gate/vars": [
1039
+ "model_00002.weights.h5"
1040
+ ],
1041
+ "/layers/qwen3_decoder_layer_31/mlp/up/vars": [
1042
+ "model_00002.weights.h5"
1043
+ ],
1044
+ "/layers/qwen3_decoder_layer_31/mlp_norm/vars": [
1045
+ "model_00002.weights.h5"
1046
+ ],
1047
+ "/layers/qwen3_decoder_layer_31/attention/key_norm/vars": [
1048
+ "model_00002.weights.h5"
1049
+ ],
1050
+ "/layers/qwen3_decoder_layer_31/attention/key/vars": [
1051
+ "model_00002.weights.h5"
1052
+ ],
1053
+ "/layers/qwen3_decoder_layer_31/attention/output_proj/vars": [
1054
+ "model_00002.weights.h5"
1055
+ ],
1056
+ "/layers/qwen3_decoder_layer_31/attention/query_norm/vars": [
1057
+ "model_00002.weights.h5"
1058
+ ],
1059
+ "/layers/qwen3_decoder_layer_31/attention/query/vars": [
1060
+ "model_00002.weights.h5"
1061
+ ],
1062
+ "/layers/qwen3_decoder_layer_31/attention/value/vars": [
1063
+ "model_00002.weights.h5"
1064
+ ],
1065
+ "/layers/qwen3_decoder_layer_32/attention_norm/vars": [
1066
+ "model_00002.weights.h5"
1067
+ ],
1068
+ "/layers/qwen3_decoder_layer_32/mlp/down/vars": [
1069
+ "model_00002.weights.h5"
1070
+ ],
1071
+ "/layers/qwen3_decoder_layer_32/mlp/gate/vars": [
1072
+ "model_00002.weights.h5"
1073
+ ],
1074
+ "/layers/qwen3_decoder_layer_32/mlp/up/vars": [
1075
+ "model_00002.weights.h5"
1076
+ ],
1077
+ "/layers/qwen3_decoder_layer_32/mlp_norm/vars": [
1078
+ "model_00002.weights.h5"
1079
+ ],
1080
+ "/layers/qwen3_decoder_layer_32/attention/key_norm/vars": [
1081
+ "model_00002.weights.h5"
1082
+ ],
1083
+ "/layers/qwen3_decoder_layer_32/attention/key/vars": [
1084
+ "model_00002.weights.h5"
1085
+ ],
1086
+ "/layers/qwen3_decoder_layer_32/attention/output_proj/vars": [
1087
+ "model_00002.weights.h5"
1088
+ ],
1089
+ "/layers/qwen3_decoder_layer_32/attention/query_norm/vars": [
1090
+ "model_00002.weights.h5"
1091
+ ],
1092
+ "/layers/qwen3_decoder_layer_32/attention/query/vars": [
1093
+ "model_00002.weights.h5"
1094
+ ],
1095
+ "/layers/qwen3_decoder_layer_32/attention/value/vars": [
1096
+ "model_00002.weights.h5"
1097
+ ],
1098
+ "/layers/qwen3_decoder_layer_33/attention_norm/vars": [
1099
+ "model_00002.weights.h5"
1100
+ ],
1101
+ "/layers/qwen3_decoder_layer_33/mlp/down/vars": [
1102
+ "model_00002.weights.h5"
1103
+ ],
1104
+ "/layers/qwen3_decoder_layer_33/mlp/gate/vars": [
1105
+ "model_00002.weights.h5"
1106
+ ],
1107
+ "/layers/qwen3_decoder_layer_33/mlp/up/vars": [
1108
+ "model_00002.weights.h5"
1109
+ ],
1110
+ "/layers/qwen3_decoder_layer_33/mlp_norm/vars": [
1111
+ "model_00002.weights.h5"
1112
+ ],
1113
+ "/layers/qwen3_decoder_layer_33/attention/key_norm/vars": [
1114
+ "model_00002.weights.h5"
1115
+ ],
1116
+ "/layers/qwen3_decoder_layer_33/attention/key/vars": [
1117
+ "model_00002.weights.h5"
1118
+ ],
1119
+ "/layers/qwen3_decoder_layer_33/attention/output_proj/vars": [
1120
+ "model_00002.weights.h5"
1121
+ ],
1122
+ "/layers/qwen3_decoder_layer_33/attention/query_norm/vars": [
1123
+ "model_00002.weights.h5"
1124
+ ],
1125
+ "/layers/qwen3_decoder_layer_33/attention/query/vars": [
1126
+ "model_00002.weights.h5"
1127
+ ],
1128
+ "/layers/qwen3_decoder_layer_33/attention/value/vars": [
1129
+ "model_00002.weights.h5"
1130
+ ],
1131
+ "/layers/qwen3_decoder_layer_34/attention_norm/vars": [
1132
+ "model_00002.weights.h5"
1133
+ ],
1134
+ "/layers/qwen3_decoder_layer_34/mlp/down/vars": [
1135
+ "model_00002.weights.h5"
1136
+ ],
1137
+ "/layers/qwen3_decoder_layer_34/mlp/gate/vars": [
1138
+ "model_00002.weights.h5"
1139
+ ],
1140
+ "/layers/qwen3_decoder_layer_34/mlp/up/vars": [
1141
+ "model_00002.weights.h5"
1142
+ ],
1143
+ "/layers/qwen3_decoder_layer_34/mlp_norm/vars": [
1144
+ "model_00002.weights.h5"
1145
+ ],
1146
+ "/layers/qwen3_decoder_layer_34/attention/key_norm/vars": [
1147
+ "model_00002.weights.h5"
1148
+ ],
1149
+ "/layers/qwen3_decoder_layer_34/attention/key/vars": [
1150
+ "model_00002.weights.h5"
1151
+ ],
1152
+ "/layers/qwen3_decoder_layer_34/attention/output_proj/vars": [
1153
+ "model_00002.weights.h5"
1154
+ ],
1155
+ "/layers/qwen3_decoder_layer_34/attention/query_norm/vars": [
1156
+ "model_00002.weights.h5"
1157
+ ],
1158
+ "/layers/qwen3_decoder_layer_34/attention/query/vars": [
1159
+ "model_00002.weights.h5"
1160
+ ],
1161
+ "/layers/qwen3_decoder_layer_34/attention/value/vars": [
1162
+ "model_00002.weights.h5"
1163
+ ],
1164
+ "/layers/qwen3_decoder_layer_35/attention_norm/vars": [
1165
+ "model_00002.weights.h5"
1166
+ ],
1167
+ "/layers/qwen3_decoder_layer_35/mlp/down/vars": [
1168
+ "model_00002.weights.h5"
1169
+ ],
1170
+ "/layers/qwen3_decoder_layer_35/mlp/gate/vars": [
1171
+ "model_00002.weights.h5"
1172
+ ],
1173
+ "/layers/qwen3_decoder_layer_35/mlp/up/vars": [
1174
+ "model_00002.weights.h5"
1175
+ ],
1176
+ "/layers/qwen3_decoder_layer_35/mlp_norm/vars": [
1177
+ "model_00002.weights.h5"
1178
+ ],
1179
+ "/layers/qwen3_decoder_layer_35/attention/key_norm/vars": [
1180
+ "model_00002.weights.h5"
1181
+ ],
1182
+ "/layers/qwen3_decoder_layer_35/attention/key/vars": [
1183
+ "model_00002.weights.h5"
1184
+ ],
1185
+ "/layers/qwen3_decoder_layer_35/attention/output_proj/vars": [
1186
+ "model_00002.weights.h5"
1187
+ ],
1188
+ "/layers/qwen3_decoder_layer_35/attention/query_norm/vars": [
1189
+ "model_00002.weights.h5"
1190
+ ],
1191
+ "/layers/qwen3_decoder_layer_35/attention/query/vars": [
1192
+ "model_00002.weights.h5"
1193
+ ],
1194
+ "/layers/qwen3_decoder_layer_35/attention/value/vars": [
1195
+ "model_00002.weights.h5"
1196
+ ],
1197
+ "/layers/qwen3rms_norm/vars": [
1198
+ "model_00002.weights.h5"
1199
+ ],
1200
+ "/layers/dense/vars": [
1201
+ "model_00003.weights.h5"
1202
+ ]
1203
+ }
1204
+ }
model_00000.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f02a48ae40dd059a44f66bcc02dd12d9f6a111934ff4cdeec9668640adbd9a3
3
+ size 5357886448
model_00001.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:900da7bf5faf3583f6d3b7f71c0d3d6537c4660c9e7f8daadd94bdd29290702a
3
+ size 5354824176
model_00002.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1eaeaee67fcff9e6476ecadc34ef3df9ed4f853e1ecedeb3549229c5d129286
3
+ size 4432029168
model_00003.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:699c08ce6364cef2175dba345f3a4890545a60cd889ef7a0616b9e253ae1c8b3
3
+ size 1247213040
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
3
+ size 11422654