Add files using upload-large-folder tool
Browse files- .gitattributes +2 -0
- LICENSE +210 -0
- README.md +231 -0
- README_CN.md +221 -0
- chat_template.jinja +222 -0
- config.json +1044 -0
- generation_config.json +10 -0
- model-00002-of-00099.safetensors +3 -0
- model-00003-of-00099.safetensors +3 -0
- model-00005-of-00099.safetensors +3 -0
- model-00006-of-00099.safetensors +3 -0
- model-00012-of-00099.safetensors +3 -0
- model-00013-of-00099.safetensors +3 -0
- model-00015-of-00099.safetensors +3 -0
- model-00016-of-00099.safetensors +3 -0
- model-00023-of-00099.safetensors +3 -0
- model-00025-of-00099.safetensors +3 -0
- model-00028-of-00099.safetensors +3 -0
- model-00029-of-00099.safetensors +3 -0
- model-00032-of-00099.safetensors +3 -0
- model-00036-of-00099.safetensors +3 -0
- model-00041-of-00099.safetensors +3 -0
- model-00042-of-00099.safetensors +3 -0
- model-00043-of-00099.safetensors +3 -0
- model-00048-of-00099.safetensors +3 -0
- model-00050-of-00099.safetensors +3 -0
- model-00051-of-00099.safetensors +3 -0
- model-00052-of-00099.safetensors +3 -0
- model-00053-of-00099.safetensors +3 -0
- model-00058-of-00099.safetensors +3 -0
- model-00064-of-00099.safetensors +3 -0
- model-00069-of-00099.safetensors +3 -0
- model-00070-of-00099.safetensors +3 -0
- model-00074-of-00099.safetensors +3 -0
- model-00075-of-00099.safetensors +3 -0
- model-00076-of-00099.safetensors +3 -0
- model-00078-of-00099.safetensors +3 -0
- model-00081-of-00099.safetensors +3 -0
- model-00087-of-00099.safetensors +3 -0
- model-00090-of-00099.safetensors +3 -0
- model-00091-of-00099.safetensors +3 -0
- model-00092-of-00099.safetensors +3 -0
- model-00093-of-00099.safetensors +3 -0
- model-00094-of-00099.safetensors +3 -0
- model-00095-of-00099.safetensors +3 -0
- model-00096-of-00099.safetensors +3 -0
- model-00098-of-00099.safetensors +3 -0
- model.safetensors.index.json +0 -0
- tokenizer.json +0 -0
- tokenizer_config.json +0 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/benchmark-appendix.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/benchmark.png filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,210 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Tencent is pleased to support the open source community by making Tencent Hy3 available.
|
| 2 |
+
|
| 3 |
+
Copyright (C) 2026 Tencent. All rights reserved.
|
| 4 |
+
|
| 5 |
+
Tencent Hy3 is licensed under the Apache-2.0.
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
Terms of the Apache-2.0:
|
| 9 |
+
--------------------------------------------------------------------
|
| 10 |
+
Apache License
|
| 11 |
+
Version 2.0, January 2004
|
| 12 |
+
http://www.apache.org/licenses/
|
| 13 |
+
|
| 14 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 15 |
+
|
| 16 |
+
1. Definitions.
|
| 17 |
+
|
| 18 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 19 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 20 |
+
|
| 21 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 22 |
+
the copyright owner that is granting the License.
|
| 23 |
+
|
| 24 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 25 |
+
other entities that control, are controlled by, or are under common
|
| 26 |
+
control with that entity. For the purposes of this definition,
|
| 27 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 28 |
+
direction or management of such entity, whether by contract or
|
| 29 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 30 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 31 |
+
|
| 32 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 33 |
+
exercising permissions granted by this License.
|
| 34 |
+
|
| 35 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 36 |
+
including but not limited to software source code, documentation
|
| 37 |
+
source, and configuration files.
|
| 38 |
+
|
| 39 |
+
"Object" form shall mean any form resulting from mechanical
|
| 40 |
+
transformation or translation of a Source form, including but
|
| 41 |
+
not limited to compiled object code, generated documentation,
|
| 42 |
+
and conversions to other media types.
|
| 43 |
+
|
| 44 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 45 |
+
Object form, made available under the License, as indicated by a
|
| 46 |
+
copyright notice that is included in or attached to the work
|
| 47 |
+
(an example is provided in the Appendix below).
|
| 48 |
+
|
| 49 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 50 |
+
form, that is based on (or derived from) the Work and for which the
|
| 51 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 52 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 53 |
+
of this License, Derivative Works shall not include works that remain
|
| 54 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 55 |
+
the Work and Derivative Works thereof.
|
| 56 |
+
|
| 57 |
+
"Contribution" shall mean any work of authorship, including
|
| 58 |
+
the original version of the Work and any modifications or additions
|
| 59 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 60 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 61 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 62 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 63 |
+
means any form of electronic, verbal, or written communication sent
|
| 64 |
+
to the Licensor or its representatives, including but not limited to
|
| 65 |
+
communication on electronic mailing lists, source code control systems,
|
| 66 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 67 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 68 |
+
excluding communication that is conspicuously marked or otherwise
|
| 69 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 70 |
+
|
| 71 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 72 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 73 |
+
subsequently incorporated within the Work.
|
| 74 |
+
|
| 75 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 76 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 77 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 78 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 79 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 80 |
+
Work and such Derivative Works in Source or Object form.
|
| 81 |
+
|
| 82 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 83 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 84 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 85 |
+
(except as stated in this section) patent license to make, have made,
|
| 86 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 87 |
+
where such license applies only to those patent claims licensable
|
| 88 |
+
by such Contributor that are necessarily infringed by their
|
| 89 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 90 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 91 |
+
institute patent litigation against any entity (including a
|
| 92 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 93 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 94 |
+
or contributory patent infringement, then any patent licenses
|
| 95 |
+
granted to You under this License for that Work shall terminate
|
| 96 |
+
as of the date such litigation is filed.
|
| 97 |
+
|
| 98 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 99 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 100 |
+
modifications, and in Source or Object form, provided that You
|
| 101 |
+
meet the following conditions:
|
| 102 |
+
|
| 103 |
+
(a) You must give any other recipients of the Work or
|
| 104 |
+
Derivative Works a copy of this License; and
|
| 105 |
+
|
| 106 |
+
(b) You must cause any modified files to carry prominent notices
|
| 107 |
+
stating that You changed the files; and
|
| 108 |
+
|
| 109 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 110 |
+
that You distribute, all copyright, patent, trademark, and
|
| 111 |
+
attribution notices from the Source form of the Work,
|
| 112 |
+
excluding those notices that do not pertain to any part of
|
| 113 |
+
the Derivative Works; and
|
| 114 |
+
|
| 115 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 116 |
+
distribution, then any Derivative Works that You distribute must
|
| 117 |
+
include a readable copy of the attribution notices contained
|
| 118 |
+
within such NOTICE file, excluding those notices that do not
|
| 119 |
+
pertain to any part of the Derivative Works, in at least one
|
| 120 |
+
of the following places: within a NOTICE text file distributed
|
| 121 |
+
as part of the Derivative Works; within the Source form or
|
| 122 |
+
documentation, if provided along with the Derivative Works; or,
|
| 123 |
+
within a display generated by the Derivative Works, if and
|
| 124 |
+
wherever such third-party notices normally appear. The contents
|
| 125 |
+
of the NOTICE file are for informational purposes only and
|
| 126 |
+
do not modify the License. You may add Your own attribution
|
| 127 |
+
notices within Derivative Works that You distribute, alongside
|
| 128 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 129 |
+
that such additional attribution notices cannot be construed
|
| 130 |
+
as modifying the License.
|
| 131 |
+
|
| 132 |
+
You may add Your own copyright statement to Your modifications and
|
| 133 |
+
may provide additional or different license terms and conditions
|
| 134 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 135 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 136 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 137 |
+
the conditions stated in this License.
|
| 138 |
+
|
| 139 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 140 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 141 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 142 |
+
this License, without any additional terms or conditions.
|
| 143 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 144 |
+
the terms of any separate license agreement you may have executed
|
| 145 |
+
with Licensor regarding such Contributions.
|
| 146 |
+
|
| 147 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 148 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 149 |
+
except as required for reasonable and customary use in describing the
|
| 150 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 151 |
+
|
| 152 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 153 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 154 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 155 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 156 |
+
implied, including, without limitation, any warranties or conditions
|
| 157 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 158 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 159 |
+
appropriateness of using or redistributing the Work and assume any
|
| 160 |
+
risks associated with Your exercise of permissions under this License.
|
| 161 |
+
|
| 162 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 163 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 164 |
+
unless required by applicable law (such as deliberate and grossly
|
| 165 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 166 |
+
liable to You for damages, including any direct, indirect, special,
|
| 167 |
+
incidental, or consequential damages of any character arising as a
|
| 168 |
+
result of this License or out of the use or inability to use the
|
| 169 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 170 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 171 |
+
other commercial damages or losses), even if such Contributor
|
| 172 |
+
has been advised of the possibility of such damages.
|
| 173 |
+
|
| 174 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 175 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 176 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 177 |
+
or other liability obligations and/or rights consistent with this
|
| 178 |
+
License. However, in accepting such obligations, You may act only
|
| 179 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 180 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 181 |
+
defend, and hold each Contributor harmless for any liability
|
| 182 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 183 |
+
of your accepting any such warranty or additional liability.
|
| 184 |
+
|
| 185 |
+
END OF TERMS AND CONDITIONS
|
| 186 |
+
|
| 187 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 188 |
+
|
| 189 |
+
To apply the Apache License to your work, attach the following
|
| 190 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 191 |
+
replaced with your own identifying information. (Don't include
|
| 192 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 193 |
+
comment syntax for the file format. We also recommend that a
|
| 194 |
+
file or class name and description of purpose be included on the
|
| 195 |
+
same "printed page" as the copyright notice for easier
|
| 196 |
+
identification within third-party archives.
|
| 197 |
+
|
| 198 |
+
Copyright [yyyy] [name of copyright owner]
|
| 199 |
+
|
| 200 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 201 |
+
you may not use this file except in compliance with the License.
|
| 202 |
+
You may obtain a copy of the License at
|
| 203 |
+
|
| 204 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 205 |
+
|
| 206 |
+
Unless required by applicable law or agreed to in writing, software
|
| 207 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 208 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 209 |
+
See the License for the specific language governing permissions and
|
| 210 |
+
limitations under the License.
|
README.md
ADDED
|
@@ -0,0 +1,231 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: transformers
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- hunyuan
|
| 7 |
+
- hy3
|
| 8 |
+
- moe
|
| 9 |
+
- text-generation
|
| 10 |
+
---
|
| 11 |
+
<p align="left">
|
| 12 |
+
<a href="https://huggingface.co/tencent/Hy3/blob/main/README_CN.md">中文</a> ｜ English
|
| 13 |
+
</p>
|
| 14 |
+
<br>
|
| 15 |
+
|
| 16 |
+
<p align="center">
|
| 17 |
+
<img src="assets/logo-en.png" width="400"/> <br>
|
| 18 |
+
</p>
|
| 19 |
+
|
| 20 |
+
<div align="center" style="line-height: 1;">
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
[](#license)
|
| 24 |
+
|
| 25 |
+
[](https://huggingface.co/tencent/Hy3)
|
| 26 |
+
|
| 27 |
+
[](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)
|
| 28 |
+
|
| 29 |
+
[](https://cnb.cool/ai-models/tencent/Hy3)
|
| 30 |
+
|
| 31 |
+
[](https://ai.gitcode.com/tencent_hunyuan/Hy3)
|
| 32 |
+
|
| 33 |
+
</div>
|
| 34 |
+
|
| 35 |
+
<p align="center">
|
| 36 |
+
🖥️ <a href="https://aistudio.tencent.com/"><b>Official Website</b></a> |
|
| 37 |
+
💬 <a href="https://github.com/Tencent-Hunyuan/Hy3"><b>GitHub</b></a></p>
|
| 38 |
+
|
| 39 |
+
---
|
| 40 |
+
|
| 41 |
+
## Table of Contents
|
| 42 |
+
|
| 43 |
+
- [Model Introduction](#model-introduction)
|
| 44 |
+
- [Stronger Agent Performance](#stronger-agent-performance)
|
| 45 |
+
- [Product Experience: More Reliable, More Cost-Effective](#product-experience-more-reliable-more-cost-effective)
|
| 46 |
+
- [Benchmark Appendix](#benchmark-appendix)
|
| 47 |
+
- [News](#news)
|
| 48 |
+
- [Model Links](#model-links)
|
| 49 |
+
- [Quickstart](#quickstart)
|
| 50 |
+
- [Deployment](#deployment)
|
| 51 |
+
- [vLLM](#vllm)
|
| 52 |
+
- [SGLang](#sglang)
|
| 53 |
+
- [Finetuning](#finetuning)
|
| 54 |
+
- [Quantization](#quantization)
|
| 55 |
+
- [License](#license)
|
| 56 |
+
- [Contact Us](#contact-us)
|
| 57 |
+
|
| 58 |
+
---
|
| 59 |
+
|
| 60 |
+
## Model Introduction
|
| 61 |
+
|
| 62 |
+
**Hy3** is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Following the Hy3 Preview launch in late April, we gathered feedback from 50+ product teams. We fixed various issues in task execution and interaction, and improved both the quality and scale of our post-training pipeline. Today, we are launching Hy3. It significantly outperforms similar-size models and rivals flagship open-source models with 2-5x the parameters. It also shows solid gains in utility across productivity tasks and real-world applications.
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
| Property | Value |
|
| 66 |
+
|:---|:---|
|
| 67 |
+
| Architecture | Mixture-of-Experts (MoE) |
|
| 68 |
+
| Total Parameters | 295B |
|
| 69 |
+
| Activated Parameters | 21B |
|
| 70 |
+
| MTP Layer Parameters | 3.8B |
|
| 71 |
+
| Number of Layers (excluding MTP layer) | 80 |
|
| 72 |
+
| Number of MTP Layers | 1 |
|
| 73 |
+
| Attention Heads | 64 (GQA, 8 KV heads, head dim 128) |
|
| 74 |
+
| Hidden Size | 4096 |
|
| 75 |
+
| Intermediate Size | 13312 |
|
| 76 |
+
| Context Length | 256K |
|
| 77 |
+
| Vocabulary Size | 120832 |
|
| 78 |
+
| Number of Experts | 192 experts, top-8 activated |
|
| 79 |
+
| Supported Precisions | BF16 |
|
| 80 |
+
|
| 81 |
+
## Stronger Agent Performance
|
| 82 |
+
|
| 83 |
+
Building on Hy3 Preview, we improved post-training data quality and diversity while scaling up RL training. Hy3 shows solid gains across reasoning, agentic workflows, and long-context tasks. Its performance is close to leading flagship models, both domestic and international.
|
| 84 |
+
|
| 85 |
+
<p align="center">
|
| 86 |
+
<img src="assets/benchmark.png" width="100%"/>
|
| 87 |
+
</p>
|
| 88 |
+
|
| 89 |
+
In productivity scenarios such as coding, document processing, financial analysis, game development, and frontend design, Hy3 has made solid gains, positioning it as a reliable, cost-effective option.
|
| 90 |
+
|
| 91 |
+
We don't think public benchmark scores tell the full story. So we ran a blind test with 270 experts from various disciplines, working on real-world workflows, and collected 312 valid comparisons. Hy3 scored 2.67/4, outperforming GLM-5.1 at 2.51/4. The advantage was clearest in frontend development, CI/CD, and data & storage.
|
| 92 |
+
|
| 93 |
+
## Product Experience: More Reliable, More Cost-Effective
|
| 94 |
+
|
| 95 |
+
Utility in production is not fully captured by benchmarks. Based on extensive user feedback and product telemetry, we identified real-world behavior issues that break product experience and improved the model's capabilities in those areas, earning uniformly positive feedback from product teams.
|
| 96 |
+
|
| 97 |
+
**Output Formatting and Tool Calling Stability**: We fixed multiple baseline reliability issues, bringing the model to production-grade standards across tool configurations and output constraints. Tool-call success rates and error recovery improved, and invalid calls that trigger infinite loops dropped. Hy3 also generalizes across different agent scaffoldings. On SWE-Bench Verified, accuracy variance across scaffoldings like CodeBuddy, Cline, and KiloCode remains within 4%.
|
| 98 |
+
|
| 99 |
+
**World Knowledge and Anti-Hallucination**: Internal knowledge and external hallucination are interconnected and critical to real-world product experience. Guided by the ideal behavior pattern: "answer when grounded, state when evidence is missing, do not conflate sources, do not fabricate data," we implemented fine-grained data cleaning and specific training constraints. In internal evaluations on real-world scenarios, Hy3's hallucination rate dropped from 12.5% to 5.4%, and commonsense error rates fell from 25.4% to 12.7%. These improvements materially reduce fact conflation, fabrication, and logical contradiction.
|
| 100 |
+
|
| 101 |
+
**Complex Context Retention and Multi-turn Intent Tracking**: Through joint optimization of SFT and RL, Hy3 improved on operational pain points like coreference resolution, ellipsis recovery, and multi-turn constraint inheritance. On internal comprehensive multi-turn tests, the issue rate dropped from 17.4% to 7.9%. It also posted significant gains on open-source long-dialogue benchmarks like MRCR, from 42.9% to 75.1%. Overall outputs are more concise while ensuring complex intents do not decay or drift over long-horizon interactions.
|
| 102 |
+
|
| 103 |
+
## Benchmark Appendix
|
| 104 |
+
|
| 105 |
+
<p align="center">
|
| 106 |
+
<img src="assets/benchmark-appendix.png" width="100%"/>
|
| 107 |
+
</p>
|
| 108 |
+
|
| 109 |
+
## News
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
* 🔥 We open-source **Hy3** and **Hy3-FP8** model weights on [Hugging Face](https://huggingface.co/tencent/Hy3), [ModelScope](https://modelscope.cn/models/Tencent-Hunyuan/Hy3), [GitCode](https://ai.gitcode.com/tencent_hunyuan/Hy3), and [CNB](https://cnb.cool/ai-models/tencent/Hy3).
|
| 113 |
+
|
| 114 |
+
## Model Links
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
| Model Name | Description | Hugging Face | ModelScope | GitCode | CNB |
|
| 118 |
+
|:---|:---|:---:|:---:|:---:|:---:|
|
| 119 |
+
| Hy3 | Instruct model | 🤗 [Model](https://huggingface.co/tencent/Hy3) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3) | [Model](https://cnb.cool/ai-models/tencent/Hy3) |
|
| 120 |
+
| Hy3-FP8 | FP8 quantized instruct model | 🤗 [Model](https://huggingface.co/tencent/Hy3-FP8) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-FP8) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-FP8) | [Model](https://cnb.cool/ai-models/tencent/Hy3-FP8) |
|
| 121 |
+
|
| 122 |
+
## Quickstart
|
| 123 |
+
|
| 124 |
+
Deploy Hy3 with [vLLM](#vllm) or [SGLang](#sglang) first, then call the OpenAI-compatible API:
|
| 125 |
+
|
| 126 |
+
```python
|
| 127 |
+
from openai import OpenAI
|
| 128 |
+
|
| 129 |
+
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
|
| 130 |
+
|
| 131 |
+
response = client.chat.completions.create(
|
| 132 |
+
model="hy3",
|
| 133 |
+
messages=[
|
| 134 |
+
{"role": "user", "content": "Hello! Can you briefly introduce yourself?"},
|
| 135 |
+
],
|
| 136 |
+
temperature=0.9,
|
| 137 |
+
top_p=1.0,
|
| 138 |
+
# reasoning_effort: "no_think" (default, direct response), "low", "high" (deep chain-of-thought)
|
| 139 |
+
extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
|
| 140 |
+
)
|
| 141 |
+
print(response.choices[0].message.content)
|
| 142 |
+
```
|
| 143 |
+
|
| 144 |
+
> **Recommended parameters**: `temperature=0.9`, `top_p=1.0`.
|
| 145 |
+
>
|
| 146 |
+
> **Reasoning mode**: Set `reasoning_effort` to `"high"` for complex tasks (math, coding, reasoning) or `"no_think"` for direct responses.
|
| 147 |
+
|
| 148 |
+
See the [Deployment](#deployment) section below for how to start the API server.
|
| 149 |
+
|
| 150 |
+
## Deployment
|
| 151 |
+
|
| 152 |
+
Hy3 has 295B parameters in total. To serve it on 8 GPUs, we recommend using H20-3e or other GPUs with larger memory capacity.
|
| 153 |
+
|
| 154 |
+
### vLLM
|
| 155 |
+
|
| 156 |
+
Build vLLM from source:
|
| 157 |
+
```bash
|
| 158 |
+
uv venv --python 3.12 --seed --managed-python
|
| 159 |
+
source .venv/bin/activate
|
| 160 |
+
git clone https://github.com/vllm-project/vllm.git
|
| 161 |
+
cd vllm
|
| 162 |
+
uv pip install --editable . --torch-backend=auto
|
| 163 |
+
```
|
| 164 |
+
|
| 165 |
+
Start the vLLM server with MTP enabled:
|
| 166 |
+
|
| 167 |
+
```bash
|
| 168 |
+
# Switch to trtllm backend to work-around mnnvl workspace size issue.
|
| 169 |
+
export VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm
|
| 170 |
+
vllm serve tencent/Hy3 \
|
| 171 |
+
--tensor-parallel-size 8 \
|
| 172 |
+
--speculative-config.method mtp \
|
| 173 |
+
--speculative-config.num_speculative_tokens 2 \
|
| 174 |
+
--tool-call-parser hy_v3 \
|
| 175 |
+
--reasoning-parser hy_v3 \
|
| 176 |
+
--enable-auto-tool-choice \
|
| 177 |
+
--port 8000 \
|
| 178 |
+
--served-model-name hy3
|
| 179 |
+
```
|
| 180 |
+
|
| 181 |
+
### SGLang
|
| 182 |
+
|
| 183 |
+
Build SGLang from source:
|
| 184 |
+
```bash
|
| 185 |
+
git clone https://github.com/sgl-project/sglang
|
| 186 |
+
cd sglang
|
| 187 |
+
pip3 install pip --upgrade
|
| 188 |
+
pip3 install "transformers>=5.6.0"
|
| 189 |
+
pip3 install -e "python"
|
| 190 |
+
```
|
| 191 |
+
|
| 192 |
+
Launch SGLang server with MTP enabled:
|
| 193 |
+
|
| 194 |
+
```bash
|
| 195 |
+
python3 -m sglang.launch_server \
|
| 196 |
+
--model tencent/Hy3 \
|
| 197 |
+
--tp-size 8 \
|
| 198 |
+
--tool-call-parser hunyuan \
|
| 199 |
+
--reasoning-parser hunyuan \
|
| 200 |
+
--speculative-num-steps 2 \
|
| 201 |
+
--speculative-eagle-topk 1 \
|
| 202 |
+
--speculative-num-draft-tokens 3 \
|
| 203 |
+
--speculative-algorithm EAGLE \
|
| 204 |
+
--port 8000 \
|
| 205 |
+
--served-model-name hy3
|
| 206 |
+
```
|
| 207 |
+
|
| 208 |
+
## Finetuning
|
| 209 |
+
|
| 210 |
+
Hy3 provides a complete model finetuning pipeline. For detailed documentation, please refer to: [Finetuning Guide](https://huggingface.co/tencent/Hy3/blob/main/finetune/README.md)
|
| 211 |
+
|
| 212 |
+
## Quantization
|
| 213 |
+
|
| 214 |
+
We provide [AngelSlim](https://github.com/tencent/AngelSlim), a more accessible, comprehensive, and efficient toolkit for large model compression. AngelSlim supports a comprehensive suite of compression tools for large-scale multimodal models, including common quantization algorithms, low-bit quantization, and speculative sampling.
|
| 215 |
+
|
| 216 |
+
## License
|
| 217 |
+
|
| 218 |
+
|
| 219 |
+
Hy3 is released under the **Apache License 2.0**. See [LICENSE](https://huggingface.co/tencent/Hy3/blob/main/LICENSE) for details.
|
| 220 |
+
|
| 221 |
+
## Contact Us
|
| 222 |
+
|
| 223 |
+
If you would like to leave a message for our R&D and product teams, welcome to contact us. You can also reach us via email:
|
| 224 |
+
|
| 225 |
+
📧 **hunyuan_opensource@tencent.com**
|
| 226 |
+
|
| 227 |
+
---
|
| 228 |
+
|
| 229 |
+
<p align="center">
|
| 230 |
+
<i>Hy3 is developed by the Tencent Hy Team.</i>
|
| 231 |
+
</p>
|
README_CN.md
ADDED
|
@@ -0,0 +1,221 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<p align="left">
|
| 2 |
+
<a href="https://huggingface.co/tencent/Hy3">English</a> ｜ 中文
|
| 3 |
+
</p>
|
| 4 |
+
<br>
|
| 5 |
+
|
| 6 |
+
<p align="center">
|
| 7 |
+
<img src="assets/logo-zh.png" width="400"/> <br>
|
| 8 |
+
</p>
|
| 9 |
+
|
| 10 |
+
<div align="center" style="line-height: 1;">
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
[](#许可证)
|
| 14 |
+
|
| 15 |
+
[](https://huggingface.co/tencent/Hy3)
|
| 16 |
+
|
| 17 |
+
[](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)
|
| 18 |
+
|
| 19 |
+
[](https://cnb.cool/ai-models/tencent/Hy3)
|
| 20 |
+
|
| 21 |
+
[](https://ai.gitcode.com/tencent_hunyuan/Hy3)
|
| 22 |
+
|
| 23 |
+
</div>
|
| 24 |
+
|
| 25 |
+
<p align="center">
|
| 26 |
+
🖥️ <a href="https://aistudio.tencent.com/"><b>官方网站</b></a> |
|
| 27 |
+
💬 <a href="https://github.com/Tencent-Hunyuan/Hy3"><b>GitHub</b></a></p>
|
| 28 |
+
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
## 目录
|
| 32 |
+
|
| 33 |
+
- [模型介绍](#模型介绍)
|
| 34 |
+
- [更强大的智能体能力](#更强大的智能体能力)
|
| 35 |
+
- [更可靠的产品体验](#更可靠的产品体验)
|
| 36 |
+
- [Benchmark 附录](#benchmark-附录)
|
| 37 |
+
- [新闻](#新闻)
|
| 38 |
+
- [模型链接](#模型链接)
|
| 39 |
+
- [快速开始](#快速开始)
|
| 40 |
+
- [推理和部署](#推理和部署)
|
| 41 |
+
- [vLLM](#使用-vllm-推理)
|
| 42 |
+
- [SGLang](#使用-sglang-推理)
|
| 43 |
+
- [模型微调](#模型微调)
|
| 44 |
+
- [量化工具](#量化工具)
|
| 45 |
+
- [许可证](#许可证)
|
| 46 |
+
- [联系我们](#联系我们)
|
| 47 |
+
|
| 48 |
+
---
|
| 49 |
+
|
| 50 |
+
## 模型介绍
|
| 51 |
+
|
| 52 |
+
**Hy3** 是由腾讯混元团队研发的快慢思考融合的混合专家模型，总参数量 295B，激活参数 21B，MTP 层参数 3.8B。4 月底发布 Hy3 Preview 后，我们在 50 多个业务中获得了广泛的反馈，修复了各种体验问题，进一步提升了后训练的质量和规模。今天，我们发布 Hy3。它展现出显著强于同尺寸并比肩旗舰（参数规模往往是 Hy3 的 2～5 倍）开源模型的智能水平，显著提升了在各类产品和生产力任务中的实用价值。
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
| 属性 | 值 |
|
| 56 |
+
|:---|:---|
|
| 57 |
+
| 架构 | 混合专家（MoE） |
|
| 58 |
+
| 总参数量 | 295B |
|
| 59 |
+
| 激活参数量 | 21B |
|
| 60 |
+
| MTP层参数量 | 3.8B |
|
| 61 |
+
| 层数（不含MTP层） | 80 |
|
| 62 |
+
| MTP层数 | 1 |
|
| 63 |
+
| 注意力头 | 64（GQA，8 个 KV 头，head dim 128） |
|
| 64 |
+
| 隐藏层维度 | 4096 |
|
| 65 |
+
| FFN 中间层维度 | 13312 |
|
| 66 |
+
| 上下文长度 | 256K |
|
| 67 |
+
| 词表大小 | 120832 |
|
| 68 |
+
| 专家数量 | 192 个专家，top-8 激活 |
|
| 69 |
+
| 支持精度 | BF16 |
|
| 70 |
+
|
| 71 |
+
## 更强大的智能体能力
|
| 72 |
+
|
| 73 |
+
Hy3 基于 Preview 进一步提升了后训练数据的质量和多样性，扩大了 RL 算力规模，在推理、智能体、长上下文等任务上显著进步，取得了比肩国内外更大尺寸旗舰模型的效果。
|
| 74 |
+
|
| 75 |
+
<p align="center">
|
| 76 |
+
<img src="assets/benchmark.png" width="100%"/>
|
| 77 |
+
</p>
|
| 78 |
+
|
| 79 |
+
在代码、办公、金融、前端、游戏制作等生产力任务中，Hy3 进步显著，可以成为可靠且高性价比的模型选择。我们在内部组织了来自不同工种的 270 位专家在真实工作中盲测模型效果，产生了 312 条有效比较。结果表明，Hy3（均分 2.67 / 4）展现出优于 GLM5.1（均分 2.51 / 4）的表现，尤其在前端、CI/CD、数据与存储等类别优势显著。
|
| 80 |
+
|
| 81 |
+
## 更可靠的产品体验
|
| 82 |
+
|
| 83 |
+
模型的实用体验不完全与榜单成绩挂钩。基于广泛的用户反馈和分析，我们定位并优化了一系列体验向能力，获得了产品侧一致且积极的评价。
|
| 84 |
+
|
| 85 |
+
**输出格式和工具调用稳定性**：我们显著改善了一系列基础底线问题，确保模型在各种工具设置和输出要求下达到生产级标准，工具调用的错误恢复能力和效率大幅提升。另外，Hy3 还增强了跨智能体环境的泛化性，跨框架泛化能力增强，不同脚手架（如 CodeBuddy、Cline、KiloCode）都能稳定发挥，在 SWE Bench Verified 上标准差整体控制在 4 个百分点以内。
|
| 86 |
+
|
| 87 |
+
**知识常识和抗幻觉能力**：内生知识和外部幻觉问题相互关联，且对于真实体验至关重要。基于“有依据才回答、无依据明示缺失，多来源信息不乱拼，数据和状态不乱编”的理想态，我们进行了细粒度的数据清洗和训练约束。在基于真实产品的内部评测中，Hy3 的幻觉率从 12.5% 降至 5.4%，常识错误率从 25.4% 降至 12.7%，显著改善了“张冠李戴”、无中生有、逻辑矛盾等问题。
|
| 88 |
+
|
| 89 |
+
**复杂上下文承接与多轮意图保持能力**：Hy3 在 SFT 与 RL 阶段联合优化了指代消解、省略还原及多轮约束继承等业务痛点问题，内部评测的多轮问题率从 17.4% 降至 7.9%。同时 Hy3 在长对话理解基准中取得显著跨越（如 MRCR 从 42.9% 升至 75.1%），整体输出更加精炼的同时确保复杂意图在长程交互中不衰减、不跑偏。
|
| 90 |
+
|
| 91 |
+
## Benchmark 附录
|
| 92 |
+
|
| 93 |
+
<p align="center">
|
| 94 |
+
<img src="assets/benchmark-appendix.png" width="100%"/>
|
| 95 |
+
</p>
|
| 96 |
+
|
| 97 |
+
## 新闻
|
| 98 |
+
|
| 99 |
+
* 🔥 我们在 [Hugging Face](https://huggingface.co/tencent/Hy3)、[ModelScope](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)、[GitCode](https://ai.gitcode.com/tencent_hunyuan/Hy3) 和 [CNB](https://cnb.cool/ai-models/tencent/Hy3) 开源了 **Hy3** 和 **Hy3-FP8** 模型权重。
|
| 100 |
+
|
| 101 |
+
## 模型链接
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
| 模型名 | 简介 | Hugging Face | ModelScope | GitCode | CNB |
|
| 105 |
+
|:---|:---|:---:|:---:|:---:|:---:|
|
| 106 |
+
| Hy3 | Instruct 模型 | 🤗 [Model](https://huggingface.co/tencent/Hy3) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3) | [Model](https://cnb.cool/ai-models/tencent/Hy3) |
|
| 107 |
+
| Hy3-FP8 | FP8 量化 Instruct 模型 | 🤗 [Model](https://huggingface.co/tencent/Hy3-FP8) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-FP8) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-FP8) | [Model](https://cnb.cool/ai-models/tencent/Hy3-FP8) |
|
| 108 |
+
|
| 109 |
+
## 快速开始
|
| 110 |
+
|
| 111 |
+
建议先通过 [vLLM](#使用-vllm-推理) 或 [SGLang](#使用-sglang-推理) 部署服务，然后通过 OpenAI 兼容 API 调用：
|
| 112 |
+
|
| 113 |
+
```python
|
| 114 |
+
from openai import OpenAI
|
| 115 |
+
|
| 116 |
+
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
|
| 117 |
+
|
| 118 |
+
response = client.chat.completions.create(
|
| 119 |
+
model="hy3",
|
| 120 |
+
messages=[
|
| 121 |
+
{"role": "user", "content": "你好！请简单介绍一下你自己。"},
|
| 122 |
+
],
|
| 123 |
+
temperature=0.9,
|
| 124 |
+
top_p=1.0,
|
| 125 |
+
# reasoning_effort: "no_think"（默认，直接回复）、"low"、"high"（深度思维链）
|
| 126 |
+
extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
|
| 127 |
+
)
|
| 128 |
+
print(response.choices[0].message.content)
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
+
> **推荐参数**：`temperature=0.9`，`top_p=1.0`。
|
| 132 |
+
>
|
| 133 |
+
> **推理模式**：复杂任务（数学、编程、推理）建议设置 `reasoning_effort="high"`，日常对话可使用默认的 `"no_think"` 直接回复。
|
| 134 |
+
|
| 135 |
+
具体部署方式请参考下方[推理和部署](#推理和部署)章节。
|
| 136 |
+
|
| 137 |
+
## 推理和部署
|
| 138 |
+
|
| 139 |
+
Hy3 总参数量为 295B，当使用 8 张 GPU 时，建议使用 H20-3e 或其他有更大显存的卡型。
|
| 140 |
+
|
| 141 |
+
### vLLM
|
| 142 |
+
|
| 143 |
+
从源码构建 vLLM：
|
| 144 |
+
|
| 145 |
+
```bash
|
| 146 |
+
uv venv --python 3.12 --seed --managed-python
|
| 147 |
+
source .venv/bin/activate
|
| 148 |
+
git clone https://github.com/vllm-project/vllm.git
|
| 149 |
+
cd vllm
|
| 150 |
+
uv pip install --editable . --torch-backend=auto
|
| 151 |
+
```
|
| 152 |
+
|
| 153 |
+
启动 vLLM 服务，开启 MTP：
|
| 154 |
+
|
| 155 |
+
```bash
|
| 156 |
+
# Switch to trtllm backend to work-around mnnvl workspace size issue.
|
| 157 |
+
export VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm
|
| 158 |
+
|
| 159 |
+
vllm serve tencent/Hy3 \
|
| 160 |
+
--tensor-parallel-size 8 \
|
| 161 |
+
--speculative-config.method mtp \
|
| 162 |
+
--speculative-config.num_speculative_tokens 2 \
|
| 163 |
+
--tool-call-parser hy_v3 \
|
| 164 |
+
--reasoning-parser hy_v3 \
|
| 165 |
+
--enable-auto-tool-choice \
|
| 166 |
+
--port 8000 \
|
| 167 |
+
--served-model-name hy3
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
### SGLang
|
| 171 |
+
|
| 172 |
+
从源码构建 SGLang：
|
| 173 |
+
|
| 174 |
+
```bash
|
| 175 |
+
git clone https://github.com/sgl-project/sglang
|
| 176 |
+
cd sglang
|
| 177 |
+
pip3 install pip --upgrade
|
| 178 |
+
pip3 install "transformers>=5.6.0"
|
| 179 |
+
pip3 install -e "python"
|
| 180 |
+
```
|
| 181 |
+
|
| 182 |
+
启动 SGLang 服务，开启 MTP：
|
| 183 |
+
|
| 184 |
+
```bash
|
| 185 |
+
python3 -m sglang.launch_server \
|
| 186 |
+
--model tencent/Hy3 \
|
| 187 |
+
--tp-size 8 \
|
| 188 |
+
--tool-call-parser hunyuan \
|
| 189 |
+
--reasoning-parser hunyuan \
|
| 190 |
+
--speculative-num-steps 2 \
|
| 191 |
+
--speculative-eagle-topk 1 \
|
| 192 |
+
--speculative-num-draft-tokens 3 \
|
| 193 |
+
--speculative-algorithm EAGLE \
|
| 194 |
+
--port 8000 \
|
| 195 |
+
--served-model-name hy3
|
| 196 |
+
```
|
| 197 |
+
|
| 198 |
+
## 模型微调
|
| 199 |
+
|
| 200 |
+
Hy3 提供了完整的模型微调流程，详细的微调文档请参考：[模型微调指南](https://huggingface.co/tencent/Hy3/blob/main/finetune/README_CN.md)
|
| 201 |
+
|
| 202 |
+
## 量化工具
|
| 203 |
+
|
| 204 |
+
我们提供了 [AngelSlim](https://github.com/tencent/AngelSlim)——一套易用、全面、高效的大模型压缩工具包，涵盖常用量化算法、低比特量化和投机采样等能力。
|
| 205 |
+
|
| 206 |
+
## 许可证
|
| 207 |
+
|
| 208 |
+
|
| 209 |
+
Hy3 基于 **Apache 2.0 许可证** 发布。详情请参阅 [LICENSE](https://huggingface.co/tencent/Hy3/blob/main/LICENSE)。
|
| 210 |
+
|
| 211 |
+
## 联系我们
|
| 212 |
+
|
| 213 |
+
如有问题或建议，欢迎通过邮件联系我们：
|
| 214 |
+
|
| 215 |
+
📧 **hunyuan_opensource@tencent.com**
|
| 216 |
+
|
| 217 |
+
---
|
| 218 |
+
|
| 219 |
+
<p align="center">
|
| 220 |
+
<i>Hy3 由腾讯混元团队研发。</i>
|
| 221 |
+
</p>
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,222 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{#- ----------‑‑‑ special token variables ‑‑‑---------- -#}
|
| 2 |
+
{%- set HYTK = ':opensource' %}
|
| 3 |
+
{%- set eos_token = '<｜hy_eos{}｜>'.format(HYTK) %}
|
| 4 |
+
{%- set bos_token = '<｜hy_begin_of_sentence{}｜>'.format(HYTK) %}
|
| 5 |
+
{%- set pad_token = '<｜hy_pad{}｜>'.format(HYTK) %}
|
| 6 |
+
{%- set user_token = '<｜hy_User{}｜>'.format(HYTK) %}
|
| 7 |
+
{%- set assistant_token = '<｜hy_Assistant{}｜>'.format(HYTK) %}
|
| 8 |
+
{%- set think_begin_token = '<think{}>'.format(HYTK) %}
|
| 9 |
+
{%- set think_end_token = '</think{}>'.format(HYTK) %}
|
| 10 |
+
{%- set toolcalls_begin_token = '<tool_calls{}>'.format(HYTK) %}
|
| 11 |
+
{%- set toolcalls_end_token = '</tool_calls{}>'.format(HYTK) %}
|
| 12 |
+
{%- set toolcall_begin_token = '<tool_call{}>'.format(HYTK) %}
|
| 13 |
+
{%- set toolcall_end_token = '</tool_call{}>'.format(HYTK) %}
|
| 14 |
+
{%- set toolsep_token = '<tool_sep{}>'.format(HYTK) %}
|
| 15 |
+
{%- set argkey_begin_token = '<arg_key{}>'.format(HYTK) %}
|
| 16 |
+
{%- set argkey_end_token = '</arg_key{}>'.format(HYTK) %}
|
| 17 |
+
{%- set argvalue_begin_token = '<arg_value{}>'.format(HYTK) %}
|
| 18 |
+
{%- set argvalue_end_token = '</arg_value{}>'.format(HYTK) %}
|
| 19 |
+
{%- set toolresponses_begin_token = '<tool_responses{}>'.format(HYTK) %}
|
| 20 |
+
{%- set toolresponses_end_token = '</tool_responses{}>'.format(HYTK) %}
|
| 21 |
+
{%- set toolresponse_begin_token = '<tool_response{}>'.format(HYTK) %}
|
| 22 |
+
{%- set toolresponse_end_token = '</tool_response{}>'.format(HYTK) %}
|
| 23 |
+
{%- set reasoning_mode_token = '<｜reasoning_mode{}｜>'.format(HYTK) %}
|
| 24 |
+
|
| 25 |
+
{#- ----------‑‑‑ hyperparameters variables ‑‑‑---------- -#}
|
| 26 |
+
{%- if not add_generation_prompt is defined %}
|
| 27 |
+
{%- set add_generation_prompt = false %}
|
| 28 |
+
{%- endif %}
|
| 29 |
+
{%- if not preserved_thinking is defined %}
|
| 30 |
+
{%- if not tools %}
|
| 31 |
+
{%- set preserved_thinking = false %}
|
| 32 |
+
{%- else %}
|
| 33 |
+
{%- set preserved_thinking = true %}
|
| 34 |
+
{%- endif %}
|
| 35 |
+
{%- endif %}
|
| 36 |
+
{%- if not is_training is defined %}
|
| 37 |
+
{%- set is_training = false %}
|
| 38 |
+
{%- endif %}
|
| 39 |
+
|
| 40 |
+
{%- if not reasoning_effort is defined %}
|
| 41 |
+
{%- set reasoning_effort = 'no_think' %}
|
| 42 |
+
{%- elif reasoning_effort not in ['high', 'low', 'no_think'] %}
|
| 43 |
+
{%- if reasoning_effort is none %}
|
| 44 |
+
{{- raise_exception('reasoning_effort error : None, should be no_think/low/high') }}
|
| 45 |
+
{%- else %}
|
| 46 |
+
{{- raise_exception('reasoning_effort error : ' + reasoning_effort + ', should be no_think/low/high') }}
|
| 47 |
+
{%- endif %}
|
| 48 |
+
{%- endif %}
|
| 49 |
+
|
| 50 |
+
{%- if fallback_strategy is defined and fallback_strategy == 'reasoning_toolcall_retry' %}
|
| 51 |
+
{%- set reasoning_effort = 'high' %}
|
| 52 |
+
{%- set add_generation_prompt = false %}
|
| 53 |
+
{%- endif %}
|
| 54 |
+
{%- if not raw_last_assistant is defined %}
|
| 55 |
+
{%- set raw_last_assistant = false %}
|
| 56 |
+
{%- endif %}
|
| 57 |
+
|
| 58 |
+
{%- macro visible_text(content) -%}
|
| 59 |
+
{%- if content is string -%}
|
| 60 |
+
{{- content }}
|
| 61 |
+
{%- elif content is iterable and content is not mapping -%}
|
| 62 |
+
{%- for item in content -%}
|
| 63 |
+
{%- if item is mapping and item.type == 'text' -%}
|
| 64 |
+
{{- item.text }}
|
| 65 |
+
{%- elif item is string -%}
|
| 66 |
+
{{- item }}
|
| 67 |
+
{%- endif -%}
|
| 68 |
+
{%- endfor -%}
|
| 69 |
+
{%- elif content is none -%}
|
| 70 |
+
{{- '' }}
|
| 71 |
+
{%- else -%}
|
| 72 |
+
{{- content }}
|
| 73 |
+
{%- endif -%}
|
| 74 |
+
{%- endmacro -%}
|
| 75 |
+
|
| 76 |
+
{%- set ns = namespace(last_user_index=-1) %}
|
| 77 |
+
{%- set sp_ns = namespace(system_prompt='', is_first_sp=true) %}
|
| 78 |
+
{%- for message in messages %}
|
| 79 |
+
{%- if message['role'] == 'system' %}
|
| 80 |
+
{%- set sp_ns.system_prompt = sp_ns.system_prompt + visible_text(message['content']) %}
|
| 81 |
+
{%- endif %}
|
| 82 |
+
{%- if message['role'] == 'user' %}
|
| 83 |
+
{%- set ns.last_user_index = loop.index0 %}
|
| 84 |
+
{%- endif %}
|
| 85 |
+
{%- endfor %}
|
| 86 |
+
{%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' and not tools %}
|
| 87 |
+
{%- set sp_ns.system_prompt = sp_ns.system_prompt + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort %}
|
| 88 |
+
{%- endif %}
|
| 89 |
+
{{- bos_token }}
|
| 90 |
+
{{- sp_ns.system_prompt }}
|
| 91 |
+
{%- if tools %}
|
| 92 |
+
{%- if sp_ns.system_prompt != '' %}
|
| 93 |
+
{{- '\n\n# Tools\n\nYou may call one or more functions to assist with the user query.' }}
|
| 94 |
+
{%- else %}
|
| 95 |
+
{{- '# Tools\n\nYou may call one or more functions to assist with the user query.' }}
|
| 96 |
+
{%- endif %}
|
| 97 |
+
{{- '\n\nYou are provided with function signatures within <tools></tools> XML tags:' }}
|
| 98 |
+
{{- '\n<tools>\n' }}
|
| 99 |
+
{%- for tool in tools %}
|
| 100 |
+
{%- if loop.index0 > 0 %}
|
| 101 |
+
{{- '\n' }}
|
| 102 |
+
{%- endif %}
|
| 103 |
+
{{- tool | tojson }}
|
| 104 |
+
{%- endfor %}
|
| 105 |
+
{{- '\n</tools>\n\n' }}
|
| 106 |
+
{{- 'For function call returns, you should first print ' + toolcalls_begin_token + '\n' }}
|
| 107 |
+
{{- 'For each function call, you should return object like:\n' }}
|
| 108 |
+
{{- toolcall_begin_token + '{function-name}' + toolsep_token + '\n' }}
|
| 109 |
+
{{- argkey_begin_token + '{arg-key-1}' + argkey_end_token + '\n' }}
|
| 110 |
+
{{- argvalue_begin_token + '{arg-value-1}' + argvalue_end_token + '\n' }}
|
| 111 |
+
{{- argkey_begin_token + '{arg-key-2}' + argkey_end_token + '\n' }}
|
| 112 |
+
{{- argvalue_begin_token + '{arg-value-2}' + argvalue_end_token + '\n' }}
|
| 113 |
+
{{- '...\n' }}
|
| 114 |
+
{{- toolcall_end_token + '\n' }}
|
| 115 |
+
{%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' %}
|
| 116 |
+
{{- 'At the end of function call returns, you should print ' + toolcalls_end_token + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort }}
|
| 117 |
+
{%- else %}
|
| 118 |
+
{{- 'At the end of function call returns, you should print ' + toolcalls_end_token }}
|
| 119 |
+
{%- endif %}
|
| 120 |
+
{%- endif %}
|
| 121 |
+
|
| 122 |
+
{%- set prev_ns = namespace(is_tool=false, is_tool_first=true) %}
|
| 123 |
+
{%- set last_ns = namespace(last_is_assistant=false) %}
|
| 124 |
+
{%- for message in messages %}
|
| 125 |
+
{%- if message['role'] == 'user' %}
|
| 126 |
+
{%- if prev_ns.is_tool %}
|
| 127 |
+
{{- toolresponses_end_token }}
|
| 128 |
+
{%- endif %}
|
| 129 |
+
{{- user_token + visible_text(message['content']) }}
|
| 130 |
+
{%- set prev_ns.is_tool = false %}
|
| 131 |
+
{%- endif %}
|
| 132 |
+
{%- if message['role'] == 'assistant' %}
|
| 133 |
+
{%- if is_training %}
|
| 134 |
+
{%- if 'reasoning_content' in message and message['reasoning_content'] is string %}
|
| 135 |
+
{%- set rc = message['reasoning_content'] %}
|
| 136 |
+
{%- elif 'reasoning' in message and message['reasoning'] is string %}
|
| 137 |
+
{%- set rc = message['reasoning'] %}
|
| 138 |
+
{%- else %}
|
| 139 |
+
{%- set rc = none %}
|
| 140 |
+
{%- endif %}
|
| 141 |
+
{%- if rc is not none %}
|
| 142 |
+
{%- set content = think_begin_token + rc + think_end_token + visible_text(message['content']) %}
|
| 143 |
+
{%- else %}
|
| 144 |
+
{%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
|
| 145 |
+
{%- endif %}
|
| 146 |
+
{%- else %}
|
| 147 |
+
{%- if ((preserved_thinking is defined and preserved_thinking) or loop.index0 > ns.last_user_index) %}
|
| 148 |
+
{%- if 'reasoning_content' in message and message['reasoning_content'] is string %}
|
| 149 |
+
{%- set rc = message['reasoning_content'] %}
|
| 150 |
+
{%- elif 'reasoning' in message and message['reasoning'] is string %}
|
| 151 |
+
{%- set rc = message['reasoning'] %}
|
| 152 |
+
{%- else %}
|
| 153 |
+
{%- set rc = none %}
|
| 154 |
+
{%- endif %}
|
| 155 |
+
{%- if rc is not none %}
|
| 156 |
+
{%- set content = think_begin_token + rc + think_end_token + visible_text(message['content']) %}
|
| 157 |
+
{%- else %}
|
| 158 |
+
{%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
|
| 159 |
+
{%- endif %}
|
| 160 |
+
{%- else %}
|
| 161 |
+
{%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
|
| 162 |
+
{%- endif %}
|
| 163 |
+
{%- endif %}
|
| 164 |
+
{%- if prev_ns.is_tool %}
|
| 165 |
+
{{- toolresponses_end_token }}
|
| 166 |
+
{%- endif %}
|
| 167 |
+
{{- assistant_token }}
|
| 168 |
+
{%- if message['tool_calls'] is defined and message['tool_calls'] %}
|
| 169 |
+
{%- set prev_ns.is_tool_first = true %}
|
| 170 |
+
{{- content }}
|
| 171 |
+
{{- toolcalls_begin_token + '\n' }}
|
| 172 |
+
{%- for tool in message['tool_calls'] %}
|
| 173 |
+
{%- set arguments = tool['function']['arguments'] %}
|
| 174 |
+
{{- toolcall_begin_token + tool['function']['name'] + toolsep_token + '\n' }}
|
| 175 |
+
{%- for key, value in arguments.items() %}
|
| 176 |
+
{{- argkey_begin_token + key + argkey_end_token + '\n' }}
|
| 177 |
+
{%- if value is not string %}
|
| 178 |
+
{%- set value = value | tojson(ensure_ascii=False) %}
|
| 179 |
+
{%- endif %}
|
| 180 |
+
{{- argvalue_begin_token + value + argvalue_end_token + '\n' }}
|
| 181 |
+
{%- endfor %}
|
| 182 |
+
{{- toolcall_end_token + '\n' }}
|
| 183 |
+
{%- endfor %}
|
| 184 |
+
{{- toolcalls_end_token + eos_token }}
|
| 185 |
+
{%- else %}
|
| 186 |
+
{%- if loop.last and raw_last_assistant %}
|
| 187 |
+
{{- visible_text(message['content']) }}
|
| 188 |
+
{%- elif not loop.last or is_training %}
|
| 189 |
+
{{- content + eos_token }}
|
| 190 |
+
{%- else %}
|
| 191 |
+
{{- content }}
|
| 192 |
+
{%- endif %}
|
| 193 |
+
{%- endif %}
|
| 194 |
+
{%- set prev_ns.is_tool = false %}
|
| 195 |
+
{%- endif %}
|
| 196 |
+
{%- if message['role'] == 'tool' %}
|
| 197 |
+
{%- set prev_ns.is_tool = true %}
|
| 198 |
+
{%- if prev_ns.is_tool_first %}
|
| 199 |
+
{{- toolresponses_begin_token + '\n' }}
|
| 200 |
+
{%- set prev_ns.is_tool_first = false %}
|
| 201 |
+
{%- endif %}
|
| 202 |
+
{{- toolresponse_begin_token + '\n' + visible_text(message['content']) + '\n' + toolresponse_end_token + '\n' }}
|
| 203 |
+
{%- endif %}
|
| 204 |
+
{%- if loop.last and message['role'] == 'assistant' %}
|
| 205 |
+
{%- set last_ns.last_is_assistant = true %}
|
| 206 |
+
{%- endif %}
|
| 207 |
+
|
| 208 |
+
{%- endfor %}
|
| 209 |
+
{%- if prev_ns.is_tool %}
|
| 210 |
+
{{- toolresponses_end_token }}
|
| 211 |
+
{%- endif %}
|
| 212 |
+
{%- if add_generation_prompt %}
|
| 213 |
+
{%- if not last_ns.last_is_assistant %}
|
| 214 |
+
{%- if reasoning_effort is defined and reasoning_effort in ['low', 'high'] %}
|
| 215 |
+
{{- assistant_token + think_begin_token }}
|
| 216 |
+
{%- elif reasoning_effort is defined and reasoning_effort == 'no_think' %}
|
| 217 |
+
{{- assistant_token + think_begin_token + think_end_token }}
|
| 218 |
+
{%- else %}
|
| 219 |
+
{{- assistant_token }}
|
| 220 |
+
{%- endif %}
|
| 221 |
+
{%- endif %}
|
| 222 |
+
{%- endif %}
|
config.json
ADDED
|
@@ -0,0 +1,1044 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"HYV3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"bos_token_id": 120000,
|
| 6 |
+
"enable_attention_fp32_softmax": false,
|
| 7 |
+
"enable_lm_head_fp32": true,
|
| 8 |
+
"enable_moe_fp32_combine": false,
|
| 9 |
+
"eod_token_id": 120026,
|
| 10 |
+
"eos_token_id": 120025,
|
| 11 |
+
"expert_hidden_dim": 1536,
|
| 12 |
+
"moe_intermediate_size": 1536,
|
| 13 |
+
"first_k_dense_replace": 1,
|
| 14 |
+
"head_dim": 128,
|
| 15 |
+
"hidden_act": "silu",
|
| 16 |
+
"hidden_size": 4096,
|
| 17 |
+
"initializer_range": 0.006,
|
| 18 |
+
"intermediate_size": 13312,
|
| 19 |
+
"max_position_embeddings": 262144,
|
| 20 |
+
"model_type": "hy_v3",
|
| 21 |
+
"moe_router_enable_expert_bias": true,
|
| 22 |
+
"moe_router_use_sigmoid": true,
|
| 23 |
+
"num_attention_heads": 64,
|
| 24 |
+
"num_experts": 192,
|
| 25 |
+
"num_experts_per_tok": 8,
|
| 26 |
+
"num_hidden_layers": 80,
|
| 27 |
+
"num_key_value_heads": 8,
|
| 28 |
+
"num_shared_experts": 1,
|
| 29 |
+
"output_router_logits": true,
|
| 30 |
+
"pad_token_id": 120002,
|
| 31 |
+
"qk_norm": true,
|
| 32 |
+
"rms_norm_eps": 1e-05,
|
| 33 |
+
"rope_parameters": {
|
| 34 |
+
"rope_theta": 11158840.0,
|
| 35 |
+
"rope_type": "default"
|
| 36 |
+
},
|
| 37 |
+
"route_norm": true,
|
| 38 |
+
"router_scaling_factor": 2.826,
|
| 39 |
+
"sep_token_id": 120007,
|
| 40 |
+
"tie_word_embeddings": false,
|
| 41 |
+
"transformers_version": "5.6.0",
|
| 42 |
+
"use_cache": true,
|
| 43 |
+
"use_grouped_mm": false,
|
| 44 |
+
"vocab_size": 120832,
|
| 45 |
+
"num_nextn_predict_layers": 1,
|
| 46 |
+
"quantization_config": {
|
| 47 |
+
"quant_method": "compressed-tensors",
|
| 48 |
+
"format": "mxfp4-pack-quantized",
|
| 49 |
+
"config_groups": {
|
| 50 |
+
"group_0": {
|
| 51 |
+
"targets": [
|
| 52 |
+
"Linear"
|
| 53 |
+
],
|
| 54 |
+
"weights": {
|
| 55 |
+
"num_bits": 4,
|
| 56 |
+
"type": "float",
|
| 57 |
+
"strategy": "group",
|
| 58 |
+
"group_size": 32,
|
| 59 |
+
"symmetric": true
|
| 60 |
+
}
|
| 61 |
+
}
|
| 62 |
+
},
|
| 63 |
+
"ignore": [
|
| 64 |
+
"lm_head",
|
| 65 |
+
"model.embed_tokens",
|
| 66 |
+
"model.layers.0.input_layernorm",
|
| 67 |
+
"model.layers.0.mlp.down_proj",
|
| 68 |
+
"model.layers.0.mlp.gate_proj",
|
| 69 |
+
"model.layers.0.mlp.up_proj",
|
| 70 |
+
"model.layers.0.post_attention_layernorm",
|
| 71 |
+
"model.layers.0.self_attn.k_norm",
|
| 72 |
+
"model.layers.0.self_attn.k_proj",
|
| 73 |
+
"model.layers.0.self_attn.o_proj",
|
| 74 |
+
"model.layers.0.self_attn.q_norm",
|
| 75 |
+
"model.layers.0.self_attn.q_proj",
|
| 76 |
+
"model.layers.0.self_attn.v_proj",
|
| 77 |
+
"model.layers.1.input_layernorm",
|
| 78 |
+
"model.layers.1.mlp.router.gate",
|
| 79 |
+
"model.layers.1.mlp.shared_mlp.down_proj",
|
| 80 |
+
"model.layers.1.mlp.shared_mlp.gate_proj",
|
| 81 |
+
"model.layers.1.mlp.shared_mlp.up_proj",
|
| 82 |
+
"model.layers.1.post_attention_layernorm",
|
| 83 |
+
"model.layers.1.self_attn.k_norm",
|
| 84 |
+
"model.layers.1.self_attn.k_proj",
|
| 85 |
+
"model.layers.1.self_attn.o_proj",
|
| 86 |
+
"model.layers.1.self_attn.q_norm",
|
| 87 |
+
"model.layers.1.self_attn.q_proj",
|
| 88 |
+
"model.layers.1.self_attn.v_proj",
|
| 89 |
+
"model.layers.10.input_layernorm",
|
| 90 |
+
"model.layers.10.mlp.router.gate",
|
| 91 |
+
"model.layers.10.mlp.shared_mlp.down_proj",
|
| 92 |
+
"model.layers.10.mlp.shared_mlp.gate_proj",
|
| 93 |
+
"model.layers.10.mlp.shared_mlp.up_proj",
|
| 94 |
+
"model.layers.10.post_attention_layernorm",
|
| 95 |
+
"model.layers.10.self_attn.k_norm",
|
| 96 |
+
"model.layers.10.self_attn.k_proj",
|
| 97 |
+
"model.layers.10.self_attn.o_proj",
|
| 98 |
+
"model.layers.10.self_attn.q_norm",
|
| 99 |
+
"model.layers.10.self_attn.q_proj",
|
| 100 |
+
"model.layers.10.self_attn.v_proj",
|
| 101 |
+
"model.layers.11.input_layernorm",
|
| 102 |
+
"model.layers.11.mlp.router.gate",
|
| 103 |
+
"model.layers.11.mlp.shared_mlp.down_proj",
|
| 104 |
+
"model.layers.11.mlp.shared_mlp.gate_proj",
|
| 105 |
+
"model.layers.11.mlp.shared_mlp.up_proj",
|
| 106 |
+
"model.layers.11.post_attention_layernorm",
|
| 107 |
+
"model.layers.11.self_attn.k_norm",
|
| 108 |
+
"model.layers.11.self_attn.k_proj",
|
| 109 |
+
"model.layers.11.self_attn.o_proj",
|
| 110 |
+
"model.layers.11.self_attn.q_norm",
|
| 111 |
+
"model.layers.11.self_attn.q_proj",
|
| 112 |
+
"model.layers.11.self_attn.v_proj",
|
| 113 |
+
"model.layers.12.input_layernorm",
|
| 114 |
+
"model.layers.12.mlp.router.gate",
|
| 115 |
+
"model.layers.12.mlp.shared_mlp.down_proj",
|
| 116 |
+
"model.layers.12.mlp.shared_mlp.gate_proj",
|
| 117 |
+
"model.layers.12.mlp.shared_mlp.up_proj",
|
| 118 |
+
"model.layers.12.post_attention_layernorm",
|
| 119 |
+
"model.layers.12.self_attn.k_norm",
|
| 120 |
+
"model.layers.12.self_attn.k_proj",
|
| 121 |
+
"model.layers.12.self_attn.o_proj",
|
| 122 |
+
"model.layers.12.self_attn.q_norm",
|
| 123 |
+
"model.layers.12.self_attn.q_proj",
|
| 124 |
+
"model.layers.12.self_attn.v_proj",
|
| 125 |
+
"model.layers.13.input_layernorm",
|
| 126 |
+
"model.layers.13.mlp.router.gate",
|
| 127 |
+
"model.layers.13.mlp.shared_mlp.down_proj",
|
| 128 |
+
"model.layers.13.mlp.shared_mlp.gate_proj",
|
| 129 |
+
"model.layers.13.mlp.shared_mlp.up_proj",
|
| 130 |
+
"model.layers.13.post_attention_layernorm",
|
| 131 |
+
"model.layers.13.self_attn.k_norm",
|
| 132 |
+
"model.layers.13.self_attn.k_proj",
|
| 133 |
+
"model.layers.13.self_attn.o_proj",
|
| 134 |
+
"model.layers.13.self_attn.q_norm",
|
| 135 |
+
"model.layers.13.self_attn.q_proj",
|
| 136 |
+
"model.layers.13.self_attn.v_proj",
|
| 137 |
+
"model.layers.14.input_layernorm",
|
| 138 |
+
"model.layers.14.mlp.router.gate",
|
| 139 |
+
"model.layers.14.mlp.shared_mlp.down_proj",
|
| 140 |
+
"model.layers.14.mlp.shared_mlp.gate_proj",
|
| 141 |
+
"model.layers.14.mlp.shared_mlp.up_proj",
|
| 142 |
+
"model.layers.14.post_attention_layernorm",
|
| 143 |
+
"model.layers.14.self_attn.k_norm",
|
| 144 |
+
"model.layers.14.self_attn.k_proj",
|
| 145 |
+
"model.layers.14.self_attn.o_proj",
|
| 146 |
+
"model.layers.14.self_attn.q_norm",
|
| 147 |
+
"model.layers.14.self_attn.q_proj",
|
| 148 |
+
"model.layers.14.self_attn.v_proj",
|
| 149 |
+
"model.layers.15.input_layernorm",
|
| 150 |
+
"model.layers.15.mlp.router.gate",
|
| 151 |
+
"model.layers.15.mlp.shared_mlp.down_proj",
|
| 152 |
+
"model.layers.15.mlp.shared_mlp.gate_proj",
|
| 153 |
+
"model.layers.15.mlp.shared_mlp.up_proj",
|
| 154 |
+
"model.layers.15.post_attention_layernorm",
|
| 155 |
+
"model.layers.15.self_attn.k_norm",
|
| 156 |
+
"model.layers.15.self_attn.k_proj",
|
| 157 |
+
"model.layers.15.self_attn.o_proj",
|
| 158 |
+
"model.layers.15.self_attn.q_norm",
|
| 159 |
+
"model.layers.15.self_attn.q_proj",
|
| 160 |
+
"model.layers.15.self_attn.v_proj",
|
| 161 |
+
"model.layers.16.input_layernorm",
|
| 162 |
+
"model.layers.16.mlp.router.gate",
|
| 163 |
+
"model.layers.16.mlp.shared_mlp.down_proj",
|
| 164 |
+
"model.layers.16.mlp.shared_mlp.gate_proj",
|
| 165 |
+
"model.layers.16.mlp.shared_mlp.up_proj",
|
| 166 |
+
"model.layers.16.post_attention_layernorm",
|
| 167 |
+
"model.layers.16.self_attn.k_norm",
|
| 168 |
+
"model.layers.16.self_attn.k_proj",
|
| 169 |
+
"model.layers.16.self_attn.o_proj",
|
| 170 |
+
"model.layers.16.self_attn.q_norm",
|
| 171 |
+
"model.layers.16.self_attn.q_proj",
|
| 172 |
+
"model.layers.16.self_attn.v_proj",
|
| 173 |
+
"model.layers.17.input_layernorm",
|
| 174 |
+
"model.layers.17.mlp.router.gate",
|
| 175 |
+
"model.layers.17.mlp.shared_mlp.down_proj",
|
| 176 |
+
"model.layers.17.mlp.shared_mlp.gate_proj",
|
| 177 |
+
"model.layers.17.mlp.shared_mlp.up_proj",
|
| 178 |
+
"model.layers.17.post_attention_layernorm",
|
| 179 |
+
"model.layers.17.self_attn.k_norm",
|
| 180 |
+
"model.layers.17.self_attn.k_proj",
|
| 181 |
+
"model.layers.17.self_attn.o_proj",
|
| 182 |
+
"model.layers.17.self_attn.q_norm",
|
| 183 |
+
"model.layers.17.self_attn.q_proj",
|
| 184 |
+
"model.layers.17.self_attn.v_proj",
|
| 185 |
+
"model.layers.18.input_layernorm",
|
| 186 |
+
"model.layers.18.mlp.router.gate",
|
| 187 |
+
"model.layers.18.mlp.shared_mlp.down_proj",
|
| 188 |
+
"model.layers.18.mlp.shared_mlp.gate_proj",
|
| 189 |
+
"model.layers.18.mlp.shared_mlp.up_proj",
|
| 190 |
+
"model.layers.18.post_attention_layernorm",
|
| 191 |
+
"model.layers.18.self_attn.k_norm",
|
| 192 |
+
"model.layers.18.self_attn.k_proj",
|
| 193 |
+
"model.layers.18.self_attn.o_proj",
|
| 194 |
+
"model.layers.18.self_attn.q_norm",
|
| 195 |
+
"model.layers.18.self_attn.q_proj",
|
| 196 |
+
"model.layers.18.self_attn.v_proj",
|
| 197 |
+
"model.layers.19.input_layernorm",
|
| 198 |
+
"model.layers.19.mlp.router.gate",
|
| 199 |
+
"model.layers.19.mlp.shared_mlp.down_proj",
|
| 200 |
+
"model.layers.19.mlp.shared_mlp.gate_proj",
|
| 201 |
+
"model.layers.19.mlp.shared_mlp.up_proj",
|
| 202 |
+
"model.layers.19.post_attention_layernorm",
|
| 203 |
+
"model.layers.19.self_attn.k_norm",
|
| 204 |
+
"model.layers.19.self_attn.k_proj",
|
| 205 |
+
"model.layers.19.self_attn.o_proj",
|
| 206 |
+
"model.layers.19.self_attn.q_norm",
|
| 207 |
+
"model.layers.19.self_attn.q_proj",
|
| 208 |
+
"model.layers.19.self_attn.v_proj",
|
| 209 |
+
"model.layers.2.input_layernorm",
|
| 210 |
+
"model.layers.2.mlp.router.gate",
|
| 211 |
+
"model.layers.2.mlp.shared_mlp.down_proj",
|
| 212 |
+
"model.layers.2.mlp.shared_mlp.gate_proj",
|
| 213 |
+
"model.layers.2.mlp.shared_mlp.up_proj",
|
| 214 |
+
"model.layers.2.post_attention_layernorm",
|
| 215 |
+
"model.layers.2.self_attn.k_norm",
|
| 216 |
+
"model.layers.2.self_attn.k_proj",
|
| 217 |
+
"model.layers.2.self_attn.o_proj",
|
| 218 |
+
"model.layers.2.self_attn.q_norm",
|
| 219 |
+
"model.layers.2.self_attn.q_proj",
|
| 220 |
+
"model.layers.2.self_attn.v_proj",
|
| 221 |
+
"model.layers.20.input_layernorm",
|
| 222 |
+
"model.layers.20.mlp.router.gate",
|
| 223 |
+
"model.layers.20.mlp.shared_mlp.down_proj",
|
| 224 |
+
"model.layers.20.mlp.shared_mlp.gate_proj",
|
| 225 |
+
"model.layers.20.mlp.shared_mlp.up_proj",
|
| 226 |
+
"model.layers.20.post_attention_layernorm",
|
| 227 |
+
"model.layers.20.self_attn.k_norm",
|
| 228 |
+
"model.layers.20.self_attn.k_proj",
|
| 229 |
+
"model.layers.20.self_attn.o_proj",
|
| 230 |
+
"model.layers.20.self_attn.q_norm",
|
| 231 |
+
"model.layers.20.self_attn.q_proj",
|
| 232 |
+
"model.layers.20.self_attn.v_proj",
|
| 233 |
+
"model.layers.21.input_layernorm",
|
| 234 |
+
"model.layers.21.mlp.router.gate",
|
| 235 |
+
"model.layers.21.mlp.shared_mlp.down_proj",
|
| 236 |
+
"model.layers.21.mlp.shared_mlp.gate_proj",
|
| 237 |
+
"model.layers.21.mlp.shared_mlp.up_proj",
|
| 238 |
+
"model.layers.21.post_attention_layernorm",
|
| 239 |
+
"model.layers.21.self_attn.k_norm",
|
| 240 |
+
"model.layers.21.self_attn.k_proj",
|
| 241 |
+
"model.layers.21.self_attn.o_proj",
|
| 242 |
+
"model.layers.21.self_attn.q_norm",
|
| 243 |
+
"model.layers.21.self_attn.q_proj",
|
| 244 |
+
"model.layers.21.self_attn.v_proj",
|
| 245 |
+
"model.layers.22.input_layernorm",
|
| 246 |
+
"model.layers.22.mlp.router.gate",
|
| 247 |
+
"model.layers.22.mlp.shared_mlp.down_proj",
|
| 248 |
+
"model.layers.22.mlp.shared_mlp.gate_proj",
|
| 249 |
+
"model.layers.22.mlp.shared_mlp.up_proj",
|
| 250 |
+
"model.layers.22.post_attention_layernorm",
|
| 251 |
+
"model.layers.22.self_attn.k_norm",
|
| 252 |
+
"model.layers.22.self_attn.k_proj",
|
| 253 |
+
"model.layers.22.self_attn.o_proj",
|
| 254 |
+
"model.layers.22.self_attn.q_norm",
|
| 255 |
+
"model.layers.22.self_attn.q_proj",
|
| 256 |
+
"model.layers.22.self_attn.v_proj",
|
| 257 |
+
"model.layers.23.input_layernorm",
|
| 258 |
+
"model.layers.23.mlp.router.gate",
|
| 259 |
+
"model.layers.23.mlp.shared_mlp.down_proj",
|
| 260 |
+
"model.layers.23.mlp.shared_mlp.gate_proj",
|
| 261 |
+
"model.layers.23.mlp.shared_mlp.up_proj",
|
| 262 |
+
"model.layers.23.post_attention_layernorm",
|
| 263 |
+
"model.layers.23.self_attn.k_norm",
|
| 264 |
+
"model.layers.23.self_attn.k_proj",
|
| 265 |
+
"model.layers.23.self_attn.o_proj",
|
| 266 |
+
"model.layers.23.self_attn.q_norm",
|
| 267 |
+
"model.layers.23.self_attn.q_proj",
|
| 268 |
+
"model.layers.23.self_attn.v_proj",
|
| 269 |
+
"model.layers.24.input_layernorm",
|
| 270 |
+
"model.layers.24.mlp.router.gate",
|
| 271 |
+
"model.layers.24.mlp.shared_mlp.down_proj",
|
| 272 |
+
"model.layers.24.mlp.shared_mlp.gate_proj",
|
| 273 |
+
"model.layers.24.mlp.shared_mlp.up_proj",
|
| 274 |
+
"model.layers.24.post_attention_layernorm",
|
| 275 |
+
"model.layers.24.self_attn.k_norm",
|
| 276 |
+
"model.layers.24.self_attn.k_proj",
|
| 277 |
+
"model.layers.24.self_attn.o_proj",
|
| 278 |
+
"model.layers.24.self_attn.q_norm",
|
| 279 |
+
"model.layers.24.self_attn.q_proj",
|
| 280 |
+
"model.layers.24.self_attn.v_proj",
|
| 281 |
+
"model.layers.25.input_layernorm",
|
| 282 |
+
"model.layers.25.mlp.router.gate",
|
| 283 |
+
"model.layers.25.mlp.shared_mlp.down_proj",
|
| 284 |
+
"model.layers.25.mlp.shared_mlp.gate_proj",
|
| 285 |
+
"model.layers.25.mlp.shared_mlp.up_proj",
|
| 286 |
+
"model.layers.25.post_attention_layernorm",
|
| 287 |
+
"model.layers.25.self_attn.k_norm",
|
| 288 |
+
"model.layers.25.self_attn.k_proj",
|
| 289 |
+
"model.layers.25.self_attn.o_proj",
|
| 290 |
+
"model.layers.25.self_attn.q_norm",
|
| 291 |
+
"model.layers.25.self_attn.q_proj",
|
| 292 |
+
"model.layers.25.self_attn.v_proj",
|
| 293 |
+
"model.layers.26.input_layernorm",
|
| 294 |
+
"model.layers.26.mlp.router.gate",
|
| 295 |
+
"model.layers.26.mlp.shared_mlp.down_proj",
|
| 296 |
+
"model.layers.26.mlp.shared_mlp.gate_proj",
|
| 297 |
+
"model.layers.26.mlp.shared_mlp.up_proj",
|
| 298 |
+
"model.layers.26.post_attention_layernorm",
|
| 299 |
+
"model.layers.26.self_attn.k_norm",
|
| 300 |
+
"model.layers.26.self_attn.k_proj",
|
| 301 |
+
"model.layers.26.self_attn.o_proj",
|
| 302 |
+
"model.layers.26.self_attn.q_norm",
|
| 303 |
+
"model.layers.26.self_attn.q_proj",
|
| 304 |
+
"model.layers.26.self_attn.v_proj",
|
| 305 |
+
"model.layers.27.input_layernorm",
|
| 306 |
+
"model.layers.27.mlp.router.gate",
|
| 307 |
+
"model.layers.27.mlp.shared_mlp.down_proj",
|
| 308 |
+
"model.layers.27.mlp.shared_mlp.gate_proj",
|
| 309 |
+
"model.layers.27.mlp.shared_mlp.up_proj",
|
| 310 |
+
"model.layers.27.post_attention_layernorm",
|
| 311 |
+
"model.layers.27.self_attn.k_norm",
|
| 312 |
+
"model.layers.27.self_attn.k_proj",
|
| 313 |
+
"model.layers.27.self_attn.o_proj",
|
| 314 |
+
"model.layers.27.self_attn.q_norm",
|
| 315 |
+
"model.layers.27.self_attn.q_proj",
|
| 316 |
+
"model.layers.27.self_attn.v_proj",
|
| 317 |
+
"model.layers.28.input_layernorm",
|
| 318 |
+
"model.layers.28.mlp.router.gate",
|
| 319 |
+
"model.layers.28.mlp.shared_mlp.down_proj",
|
| 320 |
+
"model.layers.28.mlp.shared_mlp.gate_proj",
|
| 321 |
+
"model.layers.28.mlp.shared_mlp.up_proj",
|
| 322 |
+
"model.layers.28.post_attention_layernorm",
|
| 323 |
+
"model.layers.28.self_attn.k_norm",
|
| 324 |
+
"model.layers.28.self_attn.k_proj",
|
| 325 |
+
"model.layers.28.self_attn.o_proj",
|
| 326 |
+
"model.layers.28.self_attn.q_norm",
|
| 327 |
+
"model.layers.28.self_attn.q_proj",
|
| 328 |
+
"model.layers.28.self_attn.v_proj",
|
| 329 |
+
"model.layers.29.input_layernorm",
|
| 330 |
+
"model.layers.29.mlp.router.gate",
|
| 331 |
+
"model.layers.29.mlp.shared_mlp.down_proj",
|
| 332 |
+
"model.layers.29.mlp.shared_mlp.gate_proj",
|
| 333 |
+
"model.layers.29.mlp.shared_mlp.up_proj",
|
| 334 |
+
"model.layers.29.post_attention_layernorm",
|
| 335 |
+
"model.layers.29.self_attn.k_norm",
|
| 336 |
+
"model.layers.29.self_attn.k_proj",
|
| 337 |
+
"model.layers.29.self_attn.o_proj",
|
| 338 |
+
"model.layers.29.self_attn.q_norm",
|
| 339 |
+
"model.layers.29.self_attn.q_proj",
|
| 340 |
+
"model.layers.29.self_attn.v_proj",
|
| 341 |
+
"model.layers.3.input_layernorm",
|
| 342 |
+
"model.layers.3.mlp.router.gate",
|
| 343 |
+
"model.layers.3.mlp.shared_mlp.down_proj",
|
| 344 |
+
"model.layers.3.mlp.shared_mlp.gate_proj",
|
| 345 |
+
"model.layers.3.mlp.shared_mlp.up_proj",
|
| 346 |
+
"model.layers.3.post_attention_layernorm",
|
| 347 |
+
"model.layers.3.self_attn.k_norm",
|
| 348 |
+
"model.layers.3.self_attn.k_proj",
|
| 349 |
+
"model.layers.3.self_attn.o_proj",
|
| 350 |
+
"model.layers.3.self_attn.q_norm",
|
| 351 |
+
"model.layers.3.self_attn.q_proj",
|
| 352 |
+
"model.layers.3.self_attn.v_proj",
|
| 353 |
+
"model.layers.30.input_layernorm",
|
| 354 |
+
"model.layers.30.mlp.router.gate",
|
| 355 |
+
"model.layers.30.mlp.shared_mlp.down_proj",
|
| 356 |
+
"model.layers.30.mlp.shared_mlp.gate_proj",
|
| 357 |
+
"model.layers.30.mlp.shared_mlp.up_proj",
|
| 358 |
+
"model.layers.30.post_attention_layernorm",
|
| 359 |
+
"model.layers.30.self_attn.k_norm",
|
| 360 |
+
"model.layers.30.self_attn.k_proj",
|
| 361 |
+
"model.layers.30.self_attn.o_proj",
|
| 362 |
+
"model.layers.30.self_attn.q_norm",
|
| 363 |
+
"model.layers.30.self_attn.q_proj",
|
| 364 |
+
"model.layers.30.self_attn.v_proj",
|
| 365 |
+
"model.layers.31.input_layernorm",
|
| 366 |
+
"model.layers.31.mlp.router.gate",
|
| 367 |
+
"model.layers.31.mlp.shared_mlp.down_proj",
|
| 368 |
+
"model.layers.31.mlp.shared_mlp.gate_proj",
|
| 369 |
+
"model.layers.31.mlp.shared_mlp.up_proj",
|
| 370 |
+
"model.layers.31.post_attention_layernorm",
|
| 371 |
+
"model.layers.31.self_attn.k_norm",
|
| 372 |
+
"model.layers.31.self_attn.k_proj",
|
| 373 |
+
"model.layers.31.self_attn.o_proj",
|
| 374 |
+
"model.layers.31.self_attn.q_norm",
|
| 375 |
+
"model.layers.31.self_attn.q_proj",
|
| 376 |
+
"model.layers.31.self_attn.v_proj",
|
| 377 |
+
"model.layers.32.input_layernorm",
|
| 378 |
+
"model.layers.32.mlp.router.gate",
|
| 379 |
+
"model.layers.32.mlp.shared_mlp.down_proj",
|
| 380 |
+
"model.layers.32.mlp.shared_mlp.gate_proj",
|
| 381 |
+
"model.layers.32.mlp.shared_mlp.up_proj",
|
| 382 |
+
"model.layers.32.post_attention_layernorm",
|
| 383 |
+
"model.layers.32.self_attn.k_norm",
|
| 384 |
+
"model.layers.32.self_attn.k_proj",
|
| 385 |
+
"model.layers.32.self_attn.o_proj",
|
| 386 |
+
"model.layers.32.self_attn.q_norm",
|
| 387 |
+
"model.layers.32.self_attn.q_proj",
|
| 388 |
+
"model.layers.32.self_attn.v_proj",
|
| 389 |
+
"model.layers.33.input_layernorm",
|
| 390 |
+
"model.layers.33.mlp.router.gate",
|
| 391 |
+
"model.layers.33.mlp.shared_mlp.down_proj",
|
| 392 |
+
"model.layers.33.mlp.shared_mlp.gate_proj",
|
| 393 |
+
"model.layers.33.mlp.shared_mlp.up_proj",
|
| 394 |
+
"model.layers.33.post_attention_layernorm",
|
| 395 |
+
"model.layers.33.self_attn.k_norm",
|
| 396 |
+
"model.layers.33.self_attn.k_proj",
|
| 397 |
+
"model.layers.33.self_attn.o_proj",
|
| 398 |
+
"model.layers.33.self_attn.q_norm",
|
| 399 |
+
"model.layers.33.self_attn.q_proj",
|
| 400 |
+
"model.layers.33.self_attn.v_proj",
|
| 401 |
+
"model.layers.34.input_layernorm",
|
| 402 |
+
"model.layers.34.mlp.router.gate",
|
| 403 |
+
"model.layers.34.mlp.shared_mlp.down_proj",
|
| 404 |
+
"model.layers.34.mlp.shared_mlp.gate_proj",
|
| 405 |
+
"model.layers.34.mlp.shared_mlp.up_proj",
|
| 406 |
+
"model.layers.34.post_attention_layernorm",
|
| 407 |
+
"model.layers.34.self_attn.k_norm",
|
| 408 |
+
"model.layers.34.self_attn.k_proj",
|
| 409 |
+
"model.layers.34.self_attn.o_proj",
|
| 410 |
+
"model.layers.34.self_attn.q_norm",
|
| 411 |
+
"model.layers.34.self_attn.q_proj",
|
| 412 |
+
"model.layers.34.self_attn.v_proj",
|
| 413 |
+
"model.layers.35.input_layernorm",
|
| 414 |
+
"model.layers.35.mlp.router.gate",
|
| 415 |
+
"model.layers.35.mlp.shared_mlp.down_proj",
|
| 416 |
+
"model.layers.35.mlp.shared_mlp.gate_proj",
|
| 417 |
+
"model.layers.35.mlp.shared_mlp.up_proj",
|
| 418 |
+
"model.layers.35.post_attention_layernorm",
|
| 419 |
+
"model.layers.35.self_attn.k_norm",
|
| 420 |
+
"model.layers.35.self_attn.k_proj",
|
| 421 |
+
"model.layers.35.self_attn.o_proj",
|
| 422 |
+
"model.layers.35.self_attn.q_norm",
|
| 423 |
+
"model.layers.35.self_attn.q_proj",
|
| 424 |
+
"model.layers.35.self_attn.v_proj",
|
| 425 |
+
"model.layers.36.input_layernorm",
|
| 426 |
+
"model.layers.36.mlp.router.gate",
|
| 427 |
+
"model.layers.36.mlp.shared_mlp.down_proj",
|
| 428 |
+
"model.layers.36.mlp.shared_mlp.gate_proj",
|
| 429 |
+
"model.layers.36.mlp.shared_mlp.up_proj",
|
| 430 |
+
"model.layers.36.post_attention_layernorm",
|
| 431 |
+
"model.layers.36.self_attn.k_norm",
|
| 432 |
+
"model.layers.36.self_attn.k_proj",
|
| 433 |
+
"model.layers.36.self_attn.o_proj",
|
| 434 |
+
"model.layers.36.self_attn.q_norm",
|
| 435 |
+
"model.layers.36.self_attn.q_proj",
|
| 436 |
+
"model.layers.36.self_attn.v_proj",
|
| 437 |
+
"model.layers.37.input_layernorm",
|
| 438 |
+
"model.layers.37.mlp.router.gate",
|
| 439 |
+
"model.layers.37.mlp.shared_mlp.down_proj",
|
| 440 |
+
"model.layers.37.mlp.shared_mlp.gate_proj",
|
| 441 |
+
"model.layers.37.mlp.shared_mlp.up_proj",
|
| 442 |
+
"model.layers.37.post_attention_layernorm",
|
| 443 |
+
"model.layers.37.self_attn.k_norm",
|
| 444 |
+
"model.layers.37.self_attn.k_proj",
|
| 445 |
+
"model.layers.37.self_attn.o_proj",
|
| 446 |
+
"model.layers.37.self_attn.q_norm",
|
| 447 |
+
"model.layers.37.self_attn.q_proj",
|
| 448 |
+
"model.layers.37.self_attn.v_proj",
|
| 449 |
+
"model.layers.38.input_layernorm",
|
| 450 |
+
"model.layers.38.mlp.router.gate",
|
| 451 |
+
"model.layers.38.mlp.shared_mlp.down_proj",
|
| 452 |
+
"model.layers.38.mlp.shared_mlp.gate_proj",
|
| 453 |
+
"model.layers.38.mlp.shared_mlp.up_proj",
|
| 454 |
+
"model.layers.38.post_attention_layernorm",
|
| 455 |
+
"model.layers.38.self_attn.k_norm",
|
| 456 |
+
"model.layers.38.self_attn.k_proj",
|
| 457 |
+
"model.layers.38.self_attn.o_proj",
|
| 458 |
+
"model.layers.38.self_attn.q_norm",
|
| 459 |
+
"model.layers.38.self_attn.q_proj",
|
| 460 |
+
"model.layers.38.self_attn.v_proj",
|
| 461 |
+
"model.layers.39.input_layernorm",
|
| 462 |
+
"model.layers.39.mlp.router.gate",
|
| 463 |
+
"model.layers.39.mlp.shared_mlp.down_proj",
|
| 464 |
+
"model.layers.39.mlp.shared_mlp.gate_proj",
|
| 465 |
+
"model.layers.39.mlp.shared_mlp.up_proj",
|
| 466 |
+
"model.layers.39.post_attention_layernorm",
|
| 467 |
+
"model.layers.39.self_attn.k_norm",
|
| 468 |
+
"model.layers.39.self_attn.k_proj",
|
| 469 |
+
"model.layers.39.self_attn.o_proj",
|
| 470 |
+
"model.layers.39.self_attn.q_norm",
|
| 471 |
+
"model.layers.39.self_attn.q_proj",
|
| 472 |
+
"model.layers.39.self_attn.v_proj",
|
| 473 |
+
"model.layers.4.input_layernorm",
|
| 474 |
+
"model.layers.4.mlp.router.gate",
|
| 475 |
+
"model.layers.4.mlp.shared_mlp.down_proj",
|
| 476 |
+
"model.layers.4.mlp.shared_mlp.gate_proj",
|
| 477 |
+
"model.layers.4.mlp.shared_mlp.up_proj",
|
| 478 |
+
"model.layers.4.post_attention_layernorm",
|
| 479 |
+
"model.layers.4.self_attn.k_norm",
|
| 480 |
+
"model.layers.4.self_attn.k_proj",
|
| 481 |
+
"model.layers.4.self_attn.o_proj",
|
| 482 |
+
"model.layers.4.self_attn.q_norm",
|
| 483 |
+
"model.layers.4.self_attn.q_proj",
|
| 484 |
+
"model.layers.4.self_attn.v_proj",
|
| 485 |
+
"model.layers.40.input_layernorm",
|
| 486 |
+
"model.layers.40.mlp.router.gate",
|
| 487 |
+
"model.layers.40.mlp.shared_mlp.down_proj",
|
| 488 |
+
"model.layers.40.mlp.shared_mlp.gate_proj",
|
| 489 |
+
"model.layers.40.mlp.shared_mlp.up_proj",
|
| 490 |
+
"model.layers.40.post_attention_layernorm",
|
| 491 |
+
"model.layers.40.self_attn.k_norm",
|
| 492 |
+
"model.layers.40.self_attn.k_proj",
|
| 493 |
+
"model.layers.40.self_attn.o_proj",
|
| 494 |
+
"model.layers.40.self_attn.q_norm",
|
| 495 |
+
"model.layers.40.self_attn.q_proj",
|
| 496 |
+
"model.layers.40.self_attn.v_proj",
|
| 497 |
+
"model.layers.41.input_layernorm",
|
| 498 |
+
"model.layers.41.mlp.router.gate",
|
| 499 |
+
"model.layers.41.mlp.shared_mlp.down_proj",
|
| 500 |
+
"model.layers.41.mlp.shared_mlp.gate_proj",
|
| 501 |
+
"model.layers.41.mlp.shared_mlp.up_proj",
|
| 502 |
+
"model.layers.41.post_attention_layernorm",
|
| 503 |
+
"model.layers.41.self_attn.k_norm",
|
| 504 |
+
"model.layers.41.self_attn.k_proj",
|
| 505 |
+
"model.layers.41.self_attn.o_proj",
|
| 506 |
+
"model.layers.41.self_attn.q_norm",
|
| 507 |
+
"model.layers.41.self_attn.q_proj",
|
| 508 |
+
"model.layers.41.self_attn.v_proj",
|
| 509 |
+
"model.layers.42.input_layernorm",
|
| 510 |
+
"model.layers.42.mlp.router.gate",
|
| 511 |
+
"model.layers.42.mlp.shared_mlp.down_proj",
|
| 512 |
+
"model.layers.42.mlp.shared_mlp.gate_proj",
|
| 513 |
+
"model.layers.42.mlp.shared_mlp.up_proj",
|
| 514 |
+
"model.layers.42.post_attention_layernorm",
|
| 515 |
+
"model.layers.42.self_attn.k_norm",
|
| 516 |
+
"model.layers.42.self_attn.k_proj",
|
| 517 |
+
"model.layers.42.self_attn.o_proj",
|
| 518 |
+
"model.layers.42.self_attn.q_norm",
|
| 519 |
+
"model.layers.42.self_attn.q_proj",
|
| 520 |
+
"model.layers.42.self_attn.v_proj",
|
| 521 |
+
"model.layers.43.input_layernorm",
|
| 522 |
+
"model.layers.43.mlp.router.gate",
|
| 523 |
+
"model.layers.43.mlp.shared_mlp.down_proj",
|
| 524 |
+
"model.layers.43.mlp.shared_mlp.gate_proj",
|
| 525 |
+
"model.layers.43.mlp.shared_mlp.up_proj",
|
| 526 |
+
"model.layers.43.post_attention_layernorm",
|
| 527 |
+
"model.layers.43.self_attn.k_norm",
|
| 528 |
+
"model.layers.43.self_attn.k_proj",
|
| 529 |
+
"model.layers.43.self_attn.o_proj",
|
| 530 |
+
"model.layers.43.self_attn.q_norm",
|
| 531 |
+
"model.layers.43.self_attn.q_proj",
|
| 532 |
+
"model.layers.43.self_attn.v_proj",
|
| 533 |
+
"model.layers.44.input_layernorm",
|
| 534 |
+
"model.layers.44.mlp.router.gate",
|
| 535 |
+
"model.layers.44.mlp.shared_mlp.down_proj",
|
| 536 |
+
"model.layers.44.mlp.shared_mlp.gate_proj",
|
| 537 |
+
"model.layers.44.mlp.shared_mlp.up_proj",
|
| 538 |
+
"model.layers.44.post_attention_layernorm",
|
| 539 |
+
"model.layers.44.self_attn.k_norm",
|
| 540 |
+
"model.layers.44.self_attn.k_proj",
|
| 541 |
+
"model.layers.44.self_attn.o_proj",
|
| 542 |
+
"model.layers.44.self_attn.q_norm",
|
| 543 |
+
"model.layers.44.self_attn.q_proj",
|
| 544 |
+
"model.layers.44.self_attn.v_proj",
|
| 545 |
+
"model.layers.45.input_layernorm",
|
| 546 |
+
"model.layers.45.mlp.router.gate",
|
| 547 |
+
"model.layers.45.mlp.shared_mlp.down_proj",
|
| 548 |
+
"model.layers.45.mlp.shared_mlp.gate_proj",
|
| 549 |
+
"model.layers.45.mlp.shared_mlp.up_proj",
|
| 550 |
+
"model.layers.45.post_attention_layernorm",
|
| 551 |
+
"model.layers.45.self_attn.k_norm",
|
| 552 |
+
"model.layers.45.self_attn.k_proj",
|
| 553 |
+
"model.layers.45.self_attn.o_proj",
|
| 554 |
+
"model.layers.45.self_attn.q_norm",
|
| 555 |
+
"model.layers.45.self_attn.q_proj",
|
| 556 |
+
"model.layers.45.self_attn.v_proj",
|
| 557 |
+
"model.layers.46.input_layernorm",
|
| 558 |
+
"model.layers.46.mlp.router.gate",
|
| 559 |
+
"model.layers.46.mlp.shared_mlp.down_proj",
|
| 560 |
+
"model.layers.46.mlp.shared_mlp.gate_proj",
|
| 561 |
+
"model.layers.46.mlp.shared_mlp.up_proj",
|
| 562 |
+
"model.layers.46.post_attention_layernorm",
|
| 563 |
+
"model.layers.46.self_attn.k_norm",
|
| 564 |
+
"model.layers.46.self_attn.k_proj",
|
| 565 |
+
"model.layers.46.self_attn.o_proj",
|
| 566 |
+
"model.layers.46.self_attn.q_norm",
|
| 567 |
+
"model.layers.46.self_attn.q_proj",
|
| 568 |
+
"model.layers.46.self_attn.v_proj",
|
| 569 |
+
"model.layers.47.input_layernorm",
|
| 570 |
+
"model.layers.47.mlp.router.gate",
|
| 571 |
+
"model.layers.47.mlp.shared_mlp.down_proj",
|
| 572 |
+
"model.layers.47.mlp.shared_mlp.gate_proj",
|
| 573 |
+
"model.layers.47.mlp.shared_mlp.up_proj",
|
| 574 |
+
"model.layers.47.post_attention_layernorm",
|
| 575 |
+
"model.layers.47.self_attn.k_norm",
|
| 576 |
+
"model.layers.47.self_attn.k_proj",
|
| 577 |
+
"model.layers.47.self_attn.o_proj",
|
| 578 |
+
"model.layers.47.self_attn.q_norm",
|
| 579 |
+
"model.layers.47.self_attn.q_proj",
|
| 580 |
+
"model.layers.47.self_attn.v_proj",
|
| 581 |
+
"model.layers.48.input_layernorm",
|
| 582 |
+
"model.layers.48.mlp.router.gate",
|
| 583 |
+
"model.layers.48.mlp.shared_mlp.down_proj",
|
| 584 |
+
"model.layers.48.mlp.shared_mlp.gate_proj",
|
| 585 |
+
"model.layers.48.mlp.shared_mlp.up_proj",
|
| 586 |
+
"model.layers.48.post_attention_layernorm",
|
| 587 |
+
"model.layers.48.self_attn.k_norm",
|
| 588 |
+
"model.layers.48.self_attn.k_proj",
|
| 589 |
+
"model.layers.48.self_attn.o_proj",
|
| 590 |
+
"model.layers.48.self_attn.q_norm",
|
| 591 |
+
"model.layers.48.self_attn.q_proj",
|
| 592 |
+
"model.layers.48.self_attn.v_proj",
|
| 593 |
+
"model.layers.49.input_layernorm",
|
| 594 |
+
"model.layers.49.mlp.router.gate",
|
| 595 |
+
"model.layers.49.mlp.shared_mlp.down_proj",
|
| 596 |
+
"model.layers.49.mlp.shared_mlp.gate_proj",
|
| 597 |
+
"model.layers.49.mlp.shared_mlp.up_proj",
|
| 598 |
+
"model.layers.49.post_attention_layernorm",
|
| 599 |
+
"model.layers.49.self_attn.k_norm",
|
| 600 |
+
"model.layers.49.self_attn.k_proj",
|
| 601 |
+
"model.layers.49.self_attn.o_proj",
|
| 602 |
+
"model.layers.49.self_attn.q_norm",
|
| 603 |
+
"model.layers.49.self_attn.q_proj",
|
| 604 |
+
"model.layers.49.self_attn.v_proj",
|
| 605 |
+
"model.layers.5.input_layernorm",
|
| 606 |
+
"model.layers.5.mlp.router.gate",
|
| 607 |
+
"model.layers.5.mlp.shared_mlp.down_proj",
|
| 608 |
+
"model.layers.5.mlp.shared_mlp.gate_proj",
|
| 609 |
+
"model.layers.5.mlp.shared_mlp.up_proj",
|
| 610 |
+
"model.layers.5.post_attention_layernorm",
|
| 611 |
+
"model.layers.5.self_attn.k_norm",
|
| 612 |
+
"model.layers.5.self_attn.k_proj",
|
| 613 |
+
"model.layers.5.self_attn.o_proj",
|
| 614 |
+
"model.layers.5.self_attn.q_norm",
|
| 615 |
+
"model.layers.5.self_attn.q_proj",
|
| 616 |
+
"model.layers.5.self_attn.v_proj",
|
| 617 |
+
"model.layers.50.input_layernorm",
|
| 618 |
+
"model.layers.50.mlp.router.gate",
|
| 619 |
+
"model.layers.50.mlp.shared_mlp.down_proj",
|
| 620 |
+
"model.layers.50.mlp.shared_mlp.gate_proj",
|
| 621 |
+
"model.layers.50.mlp.shared_mlp.up_proj",
|
| 622 |
+
"model.layers.50.post_attention_layernorm",
|
| 623 |
+
"model.layers.50.self_attn.k_norm",
|
| 624 |
+
"model.layers.50.self_attn.k_proj",
|
| 625 |
+
"model.layers.50.self_attn.o_proj",
|
| 626 |
+
"model.layers.50.self_attn.q_norm",
|
| 627 |
+
"model.layers.50.self_attn.q_proj",
|
| 628 |
+
"model.layers.50.self_attn.v_proj",
|
| 629 |
+
"model.layers.51.input_layernorm",
|
| 630 |
+
"model.layers.51.mlp.router.gate",
|
| 631 |
+
"model.layers.51.mlp.shared_mlp.down_proj",
|
| 632 |
+
"model.layers.51.mlp.shared_mlp.gate_proj",
|
| 633 |
+
"model.layers.51.mlp.shared_mlp.up_proj",
|
| 634 |
+
"model.layers.51.post_attention_layernorm",
|
| 635 |
+
"model.layers.51.self_attn.k_norm",
|
| 636 |
+
"model.layers.51.self_attn.k_proj",
|
| 637 |
+
"model.layers.51.self_attn.o_proj",
|
| 638 |
+
"model.layers.51.self_attn.q_norm",
|
| 639 |
+
"model.layers.51.self_attn.q_proj",
|
| 640 |
+
"model.layers.51.self_attn.v_proj",
|
| 641 |
+
"model.layers.52.input_layernorm",
|
| 642 |
+
"model.layers.52.mlp.router.gate",
|
| 643 |
+
"model.layers.52.mlp.shared_mlp.down_proj",
|
| 644 |
+
"model.layers.52.mlp.shared_mlp.gate_proj",
|
| 645 |
+
"model.layers.52.mlp.shared_mlp.up_proj",
|
| 646 |
+
"model.layers.52.post_attention_layernorm",
|
| 647 |
+
"model.layers.52.self_attn.k_norm",
|
| 648 |
+
"model.layers.52.self_attn.k_proj",
|
| 649 |
+
"model.layers.52.self_attn.o_proj",
|
| 650 |
+
"model.layers.52.self_attn.q_norm",
|
| 651 |
+
"model.layers.52.self_attn.q_proj",
|
| 652 |
+
"model.layers.52.self_attn.v_proj",
|
| 653 |
+
"model.layers.53.input_layernorm",
|
| 654 |
+
"model.layers.53.mlp.router.gate",
|
| 655 |
+
"model.layers.53.mlp.shared_mlp.down_proj",
|
| 656 |
+
"model.layers.53.mlp.shared_mlp.gate_proj",
|
| 657 |
+
"model.layers.53.mlp.shared_mlp.up_proj",
|
| 658 |
+
"model.layers.53.post_attention_layernorm",
|
| 659 |
+
"model.layers.53.self_attn.k_norm",
|
| 660 |
+
"model.layers.53.self_attn.k_proj",
|
| 661 |
+
"model.layers.53.self_attn.o_proj",
|
| 662 |
+
"model.layers.53.self_attn.q_norm",
|
| 663 |
+
"model.layers.53.self_attn.q_proj",
|
| 664 |
+
"model.layers.53.self_attn.v_proj",
|
| 665 |
+
"model.layers.54.input_layernorm",
|
| 666 |
+
"model.layers.54.mlp.router.gate",
|
| 667 |
+
"model.layers.54.mlp.shared_mlp.down_proj",
|
| 668 |
+
"model.layers.54.mlp.shared_mlp.gate_proj",
|
| 669 |
+
"model.layers.54.mlp.shared_mlp.up_proj",
|
| 670 |
+
"model.layers.54.post_attention_layernorm",
|
| 671 |
+
"model.layers.54.self_attn.k_norm",
|
| 672 |
+
"model.layers.54.self_attn.k_proj",
|
| 673 |
+
"model.layers.54.self_attn.o_proj",
|
| 674 |
+
"model.layers.54.self_attn.q_norm",
|
| 675 |
+
"model.layers.54.self_attn.q_proj",
|
| 676 |
+
"model.layers.54.self_attn.v_proj",
|
| 677 |
+
"model.layers.55.input_layernorm",
|
| 678 |
+
"model.layers.55.mlp.router.gate",
|
| 679 |
+
"model.layers.55.mlp.shared_mlp.down_proj",
|
| 680 |
+
"model.layers.55.mlp.shared_mlp.gate_proj",
|
| 681 |
+
"model.layers.55.mlp.shared_mlp.up_proj",
|
| 682 |
+
"model.layers.55.post_attention_layernorm",
|
| 683 |
+
"model.layers.55.self_attn.k_norm",
|
| 684 |
+
"model.layers.55.self_attn.k_proj",
|
| 685 |
+
"model.layers.55.self_attn.o_proj",
|
| 686 |
+
"model.layers.55.self_attn.q_norm",
|
| 687 |
+
"model.layers.55.self_attn.q_proj",
|
| 688 |
+
"model.layers.55.self_attn.v_proj",
|
| 689 |
+
"model.layers.56.input_layernorm",
|
| 690 |
+
"model.layers.56.mlp.router.gate",
|
| 691 |
+
"model.layers.56.mlp.shared_mlp.down_proj",
|
| 692 |
+
"model.layers.56.mlp.shared_mlp.gate_proj",
|
| 693 |
+
"model.layers.56.mlp.shared_mlp.up_proj",
|
| 694 |
+
"model.layers.56.post_attention_layernorm",
|
| 695 |
+
"model.layers.56.self_attn.k_norm",
|
| 696 |
+
"model.layers.56.self_attn.k_proj",
|
| 697 |
+
"model.layers.56.self_attn.o_proj",
|
| 698 |
+
"model.layers.56.self_attn.q_norm",
|
| 699 |
+
"model.layers.56.self_attn.q_proj",
|
| 700 |
+
"model.layers.56.self_attn.v_proj",
|
| 701 |
+
"model.layers.57.input_layernorm",
|
| 702 |
+
"model.layers.57.mlp.router.gate",
|
| 703 |
+
"model.layers.57.mlp.shared_mlp.down_proj",
|
| 704 |
+
"model.layers.57.mlp.shared_mlp.gate_proj",
|
| 705 |
+
"model.layers.57.mlp.shared_mlp.up_proj",
|
| 706 |
+
"model.layers.57.post_attention_layernorm",
|
| 707 |
+
"model.layers.57.self_attn.k_norm",
|
| 708 |
+
"model.layers.57.self_attn.k_proj",
|
| 709 |
+
"model.layers.57.self_attn.o_proj",
|
| 710 |
+
"model.layers.57.self_attn.q_norm",
|
| 711 |
+
"model.layers.57.self_attn.q_proj",
|
| 712 |
+
"model.layers.57.self_attn.v_proj",
|
| 713 |
+
"model.layers.58.input_layernorm",
|
| 714 |
+
"model.layers.58.mlp.router.gate",
|
| 715 |
+
"model.layers.58.mlp.shared_mlp.down_proj",
|
| 716 |
+
"model.layers.58.mlp.shared_mlp.gate_proj",
|
| 717 |
+
"model.layers.58.mlp.shared_mlp.up_proj",
|
| 718 |
+
"model.layers.58.post_attention_layernorm",
|
| 719 |
+
"model.layers.58.self_attn.k_norm",
|
| 720 |
+
"model.layers.58.self_attn.k_proj",
|
| 721 |
+
"model.layers.58.self_attn.o_proj",
|
| 722 |
+
"model.layers.58.self_attn.q_norm",
|
| 723 |
+
"model.layers.58.self_attn.q_proj",
|
| 724 |
+
"model.layers.58.self_attn.v_proj",
|
| 725 |
+
"model.layers.59.input_layernorm",
|
| 726 |
+
"model.layers.59.mlp.router.gate",
|
| 727 |
+
"model.layers.59.mlp.shared_mlp.down_proj",
|
| 728 |
+
"model.layers.59.mlp.shared_mlp.gate_proj",
|
| 729 |
+
"model.layers.59.mlp.shared_mlp.up_proj",
|
| 730 |
+
"model.layers.59.post_attention_layernorm",
|
| 731 |
+
"model.layers.59.self_attn.k_norm",
|
| 732 |
+
"model.layers.59.self_attn.k_proj",
|
| 733 |
+
"model.layers.59.self_attn.o_proj",
|
| 734 |
+
"model.layers.59.self_attn.q_norm",
|
| 735 |
+
"model.layers.59.self_attn.q_proj",
|
| 736 |
+
"model.layers.59.self_attn.v_proj",
|
| 737 |
+
"model.layers.6.input_layernorm",
|
| 738 |
+
"model.layers.6.mlp.router.gate",
|
| 739 |
+
"model.layers.6.mlp.shared_mlp.down_proj",
|
| 740 |
+
"model.layers.6.mlp.shared_mlp.gate_proj",
|
| 741 |
+
"model.layers.6.mlp.shared_mlp.up_proj",
|
| 742 |
+
"model.layers.6.post_attention_layernorm",
|
| 743 |
+
"model.layers.6.self_attn.k_norm",
|
| 744 |
+
"model.layers.6.self_attn.k_proj",
|
| 745 |
+
"model.layers.6.self_attn.o_proj",
|
| 746 |
+
"model.layers.6.self_attn.q_norm",
|
| 747 |
+
"model.layers.6.self_attn.q_proj",
|
| 748 |
+
"model.layers.6.self_attn.v_proj",
|
| 749 |
+
"model.layers.60.input_layernorm",
|
| 750 |
+
"model.layers.60.mlp.router.gate",
|
| 751 |
+
"model.layers.60.mlp.shared_mlp.down_proj",
|
| 752 |
+
"model.layers.60.mlp.shared_mlp.gate_proj",
|
| 753 |
+
"model.layers.60.mlp.shared_mlp.up_proj",
|
| 754 |
+
"model.layers.60.post_attention_layernorm",
|
| 755 |
+
"model.layers.60.self_attn.k_norm",
|
| 756 |
+
"model.layers.60.self_attn.k_proj",
|
| 757 |
+
"model.layers.60.self_attn.o_proj",
|
| 758 |
+
"model.layers.60.self_attn.q_norm",
|
| 759 |
+
"model.layers.60.self_attn.q_proj",
|
| 760 |
+
"model.layers.60.self_attn.v_proj",
|
| 761 |
+
"model.layers.61.input_layernorm",
|
| 762 |
+
"model.layers.61.mlp.router.gate",
|
| 763 |
+
"model.layers.61.mlp.shared_mlp.down_proj",
|
| 764 |
+
"model.layers.61.mlp.shared_mlp.gate_proj",
|
| 765 |
+
"model.layers.61.mlp.shared_mlp.up_proj",
|
| 766 |
+
"model.layers.61.post_attention_layernorm",
|
| 767 |
+
"model.layers.61.self_attn.k_norm",
|
| 768 |
+
"model.layers.61.self_attn.k_proj",
|
| 769 |
+
"model.layers.61.self_attn.o_proj",
|
| 770 |
+
"model.layers.61.self_attn.q_norm",
|
| 771 |
+
"model.layers.61.self_attn.q_proj",
|
| 772 |
+
"model.layers.61.self_attn.v_proj",
|
| 773 |
+
"model.layers.62.input_layernorm",
|
| 774 |
+
"model.layers.62.mlp.router.gate",
|
| 775 |
+
"model.layers.62.mlp.shared_mlp.down_proj",
|
| 776 |
+
"model.layers.62.mlp.shared_mlp.gate_proj",
|
| 777 |
+
"model.layers.62.mlp.shared_mlp.up_proj",
|
| 778 |
+
"model.layers.62.post_attention_layernorm",
|
| 779 |
+
"model.layers.62.self_attn.k_norm",
|
| 780 |
+
"model.layers.62.self_attn.k_proj",
|
| 781 |
+
"model.layers.62.self_attn.o_proj",
|
| 782 |
+
"model.layers.62.self_attn.q_norm",
|
| 783 |
+
"model.layers.62.self_attn.q_proj",
|
| 784 |
+
"model.layers.62.self_attn.v_proj",
|
| 785 |
+
"model.layers.63.input_layernorm",
|
| 786 |
+
"model.layers.63.mlp.router.gate",
|
| 787 |
+
"model.layers.63.mlp.shared_mlp.down_proj",
|
| 788 |
+
"model.layers.63.mlp.shared_mlp.gate_proj",
|
| 789 |
+
"model.layers.63.mlp.shared_mlp.up_proj",
|
| 790 |
+
"model.layers.63.post_attention_layernorm",
|
| 791 |
+
"model.layers.63.self_attn.k_norm",
|
| 792 |
+
"model.layers.63.self_attn.k_proj",
|
| 793 |
+
"model.layers.63.self_attn.o_proj",
|
| 794 |
+
"model.layers.63.self_attn.q_norm",
|
| 795 |
+
"model.layers.63.self_attn.q_proj",
|
| 796 |
+
"model.layers.63.self_attn.v_proj",
|
| 797 |
+
"model.layers.64.input_layernorm",
|
| 798 |
+
"model.layers.64.mlp.router.gate",
|
| 799 |
+
"model.layers.64.mlp.shared_mlp.down_proj",
|
| 800 |
+
"model.layers.64.mlp.shared_mlp.gate_proj",
|
| 801 |
+
"model.layers.64.mlp.shared_mlp.up_proj",
|
| 802 |
+
"model.layers.64.post_attention_layernorm",
|
| 803 |
+
"model.layers.64.self_attn.k_norm",
|
| 804 |
+
"model.layers.64.self_attn.k_proj",
|
| 805 |
+
"model.layers.64.self_attn.o_proj",
|
| 806 |
+
"model.layers.64.self_attn.q_norm",
|
| 807 |
+
"model.layers.64.self_attn.q_proj",
|
| 808 |
+
"model.layers.64.self_attn.v_proj",
|
| 809 |
+
"model.layers.65.input_layernorm",
|
| 810 |
+
"model.layers.65.mlp.router.gate",
|
| 811 |
+
"model.layers.65.mlp.shared_mlp.down_proj",
|
| 812 |
+
"model.layers.65.mlp.shared_mlp.gate_proj",
|
| 813 |
+
"model.layers.65.mlp.shared_mlp.up_proj",
|
| 814 |
+
"model.layers.65.post_attention_layernorm",
|
| 815 |
+
"model.layers.65.self_attn.k_norm",
|
| 816 |
+
"model.layers.65.self_attn.k_proj",
|
| 817 |
+
"model.layers.65.self_attn.o_proj",
|
| 818 |
+
"model.layers.65.self_attn.q_norm",
|
| 819 |
+
"model.layers.65.self_attn.q_proj",
|
| 820 |
+
"model.layers.65.self_attn.v_proj",
|
| 821 |
+
"model.layers.66.input_layernorm",
|
| 822 |
+
"model.layers.66.mlp.router.gate",
|
| 823 |
+
"model.layers.66.mlp.shared_mlp.down_proj",
|
| 824 |
+
"model.layers.66.mlp.shared_mlp.gate_proj",
|
| 825 |
+
"model.layers.66.mlp.shared_mlp.up_proj",
|
| 826 |
+
"model.layers.66.post_attention_layernorm",
|
| 827 |
+
"model.layers.66.self_attn.k_norm",
|
| 828 |
+
"model.layers.66.self_attn.k_proj",
|
| 829 |
+
"model.layers.66.self_attn.o_proj",
|
| 830 |
+
"model.layers.66.self_attn.q_norm",
|
| 831 |
+
"model.layers.66.self_attn.q_proj",
|
| 832 |
+
"model.layers.66.self_attn.v_proj",
|
| 833 |
+
"model.layers.67.input_layernorm",
|
| 834 |
+
"model.layers.67.mlp.router.gate",
|
| 835 |
+
"model.layers.67.mlp.shared_mlp.down_proj",
|
| 836 |
+
"model.layers.67.mlp.shared_mlp.gate_proj",
|
| 837 |
+
"model.layers.67.mlp.shared_mlp.up_proj",
|
| 838 |
+
"model.layers.67.post_attention_layernorm",
|
| 839 |
+
"model.layers.67.self_attn.k_norm",
|
| 840 |
+
"model.layers.67.self_attn.k_proj",
|
| 841 |
+
"model.layers.67.self_attn.o_proj",
|
| 842 |
+
"model.layers.67.self_attn.q_norm",
|
| 843 |
+
"model.layers.67.self_attn.q_proj",
|
| 844 |
+
"model.layers.67.self_attn.v_proj",
|
| 845 |
+
"model.layers.68.input_layernorm",
|
| 846 |
+
"model.layers.68.mlp.router.gate",
|
| 847 |
+
"model.layers.68.mlp.shared_mlp.down_proj",
|
| 848 |
+
"model.layers.68.mlp.shared_mlp.gate_proj",
|
| 849 |
+
"model.layers.68.mlp.shared_mlp.up_proj",
|
| 850 |
+
"model.layers.68.post_attention_layernorm",
|
| 851 |
+
"model.layers.68.self_attn.k_norm",
|
| 852 |
+
"model.layers.68.self_attn.k_proj",
|
| 853 |
+
"model.layers.68.self_attn.o_proj",
|
| 854 |
+
"model.layers.68.self_attn.q_norm",
|
| 855 |
+
"model.layers.68.self_attn.q_proj",
|
| 856 |
+
"model.layers.68.self_attn.v_proj",
|
| 857 |
+
"model.layers.69.input_layernorm",
|
| 858 |
+
"model.layers.69.mlp.router.gate",
|
| 859 |
+
"model.layers.69.mlp.shared_mlp.down_proj",
|
| 860 |
+
"model.layers.69.mlp.shared_mlp.gate_proj",
|
| 861 |
+
"model.layers.69.mlp.shared_mlp.up_proj",
|
| 862 |
+
"model.layers.69.post_attention_layernorm",
|
| 863 |
+
"model.layers.69.self_attn.k_norm",
|
| 864 |
+
"model.layers.69.self_attn.k_proj",
|
| 865 |
+
"model.layers.69.self_attn.o_proj",
|
| 866 |
+
"model.layers.69.self_attn.q_norm",
|
| 867 |
+
"model.layers.69.self_attn.q_proj",
|
| 868 |
+
"model.layers.69.self_attn.v_proj",
|
| 869 |
+
"model.layers.7.input_layernorm",
|
| 870 |
+
"model.layers.7.mlp.router.gate",
|
| 871 |
+
"model.layers.7.mlp.shared_mlp.down_proj",
|
| 872 |
+
"model.layers.7.mlp.shared_mlp.gate_proj",
|
| 873 |
+
"model.layers.7.mlp.shared_mlp.up_proj",
|
| 874 |
+
"model.layers.7.post_attention_layernorm",
|
| 875 |
+
"model.layers.7.self_attn.k_norm",
|
| 876 |
+
"model.layers.7.self_attn.k_proj",
|
| 877 |
+
"model.layers.7.self_attn.o_proj",
|
| 878 |
+
"model.layers.7.self_attn.q_norm",
|
| 879 |
+
"model.layers.7.self_attn.q_proj",
|
| 880 |
+
"model.layers.7.self_attn.v_proj",
|
| 881 |
+
"model.layers.70.input_layernorm",
|
| 882 |
+
"model.layers.70.mlp.router.gate",
|
| 883 |
+
"model.layers.70.mlp.shared_mlp.down_proj",
|
| 884 |
+
"model.layers.70.mlp.shared_mlp.gate_proj",
|
| 885 |
+
"model.layers.70.mlp.shared_mlp.up_proj",
|
| 886 |
+
"model.layers.70.post_attention_layernorm",
|
| 887 |
+
"model.layers.70.self_attn.k_norm",
|
| 888 |
+
"model.layers.70.self_attn.k_proj",
|
| 889 |
+
"model.layers.70.self_attn.o_proj",
|
| 890 |
+
"model.layers.70.self_attn.q_norm",
|
| 891 |
+
"model.layers.70.self_attn.q_proj",
|
| 892 |
+
"model.layers.70.self_attn.v_proj",
|
| 893 |
+
"model.layers.71.input_layernorm",
|
| 894 |
+
"model.layers.71.mlp.router.gate",
|
| 895 |
+
"model.layers.71.mlp.shared_mlp.down_proj",
|
| 896 |
+
"model.layers.71.mlp.shared_mlp.gate_proj",
|
| 897 |
+
"model.layers.71.mlp.shared_mlp.up_proj",
|
| 898 |
+
"model.layers.71.post_attention_layernorm",
|
| 899 |
+
"model.layers.71.self_attn.k_norm",
|
| 900 |
+
"model.layers.71.self_attn.k_proj",
|
| 901 |
+
"model.layers.71.self_attn.o_proj",
|
| 902 |
+
"model.layers.71.self_attn.q_norm",
|
| 903 |
+
"model.layers.71.self_attn.q_proj",
|
| 904 |
+
"model.layers.71.self_attn.v_proj",
|
| 905 |
+
"model.layers.72.input_layernorm",
|
| 906 |
+
"model.layers.72.mlp.router.gate",
|
| 907 |
+
"model.layers.72.mlp.shared_mlp.down_proj",
|
| 908 |
+
"model.layers.72.mlp.shared_mlp.gate_proj",
|
| 909 |
+
"model.layers.72.mlp.shared_mlp.up_proj",
|
| 910 |
+
"model.layers.72.post_attention_layernorm",
|
| 911 |
+
"model.layers.72.self_attn.k_norm",
|
| 912 |
+
"model.layers.72.self_attn.k_proj",
|
| 913 |
+
"model.layers.72.self_attn.o_proj",
|
| 914 |
+
"model.layers.72.self_attn.q_norm",
|
| 915 |
+
"model.layers.72.self_attn.q_proj",
|
| 916 |
+
"model.layers.72.self_attn.v_proj",
|
| 917 |
+
"model.layers.73.input_layernorm",
|
| 918 |
+
"model.layers.73.mlp.router.gate",
|
| 919 |
+
"model.layers.73.mlp.shared_mlp.down_proj",
|
| 920 |
+
"model.layers.73.mlp.shared_mlp.gate_proj",
|
| 921 |
+
"model.layers.73.mlp.shared_mlp.up_proj",
|
| 922 |
+
"model.layers.73.post_attention_layernorm",
|
| 923 |
+
"model.layers.73.self_attn.k_norm",
|
| 924 |
+
"model.layers.73.self_attn.k_proj",
|
| 925 |
+
"model.layers.73.self_attn.o_proj",
|
| 926 |
+
"model.layers.73.self_attn.q_norm",
|
| 927 |
+
"model.layers.73.self_attn.q_proj",
|
| 928 |
+
"model.layers.73.self_attn.v_proj",
|
| 929 |
+
"model.layers.74.input_layernorm",
|
| 930 |
+
"model.layers.74.mlp.router.gate",
|
| 931 |
+
"model.layers.74.mlp.shared_mlp.down_proj",
|
| 932 |
+
"model.layers.74.mlp.shared_mlp.gate_proj",
|
| 933 |
+
"model.layers.74.mlp.shared_mlp.up_proj",
|
| 934 |
+
"model.layers.74.post_attention_layernorm",
|
| 935 |
+
"model.layers.74.self_attn.k_norm",
|
| 936 |
+
"model.layers.74.self_attn.k_proj",
|
| 937 |
+
"model.layers.74.self_attn.o_proj",
|
| 938 |
+
"model.layers.74.self_attn.q_norm",
|
| 939 |
+
"model.layers.74.self_attn.q_proj",
|
| 940 |
+
"model.layers.74.self_attn.v_proj",
|
| 941 |
+
"model.layers.75.input_layernorm",
|
| 942 |
+
"model.layers.75.mlp.router.gate",
|
| 943 |
+
"model.layers.75.mlp.shared_mlp.down_proj",
|
| 944 |
+
"model.layers.75.mlp.shared_mlp.gate_proj",
|
| 945 |
+
"model.layers.75.mlp.shared_mlp.up_proj",
|
| 946 |
+
"model.layers.75.post_attention_layernorm",
|
| 947 |
+
"model.layers.75.self_attn.k_norm",
|
| 948 |
+
"model.layers.75.self_attn.k_proj",
|
| 949 |
+
"model.layers.75.self_attn.o_proj",
|
| 950 |
+
"model.layers.75.self_attn.q_norm",
|
| 951 |
+
"model.layers.75.self_attn.q_proj",
|
| 952 |
+
"model.layers.75.self_attn.v_proj",
|
| 953 |
+
"model.layers.76.input_layernorm",
|
| 954 |
+
"model.layers.76.mlp.router.gate",
|
| 955 |
+
"model.layers.76.mlp.shared_mlp.down_proj",
|
| 956 |
+
"model.layers.76.mlp.shared_mlp.gate_proj",
|
| 957 |
+
"model.layers.76.mlp.shared_mlp.up_proj",
|
| 958 |
+
"model.layers.76.post_attention_layernorm",
|
| 959 |
+
"model.layers.76.self_attn.k_norm",
|
| 960 |
+
"model.layers.76.self_attn.k_proj",
|
| 961 |
+
"model.layers.76.self_attn.o_proj",
|
| 962 |
+
"model.layers.76.self_attn.q_norm",
|
| 963 |
+
"model.layers.76.self_attn.q_proj",
|
| 964 |
+
"model.layers.76.self_attn.v_proj",
|
| 965 |
+
"model.layers.77.input_layernorm",
|
| 966 |
+
"model.layers.77.mlp.router.gate",
|
| 967 |
+
"model.layers.77.mlp.shared_mlp.down_proj",
|
| 968 |
+
"model.layers.77.mlp.shared_mlp.gate_proj",
|
| 969 |
+
"model.layers.77.mlp.shared_mlp.up_proj",
|
| 970 |
+
"model.layers.77.post_attention_layernorm",
|
| 971 |
+
"model.layers.77.self_attn.k_norm",
|
| 972 |
+
"model.layers.77.self_attn.k_proj",
|
| 973 |
+
"model.layers.77.self_attn.o_proj",
|
| 974 |
+
"model.layers.77.self_attn.q_norm",
|
| 975 |
+
"model.layers.77.self_attn.q_proj",
|
| 976 |
+
"model.layers.77.self_attn.v_proj",
|
| 977 |
+
"model.layers.78.input_layernorm",
|
| 978 |
+
"model.layers.78.mlp.router.gate",
|
| 979 |
+
"model.layers.78.mlp.shared_mlp.down_proj",
|
| 980 |
+
"model.layers.78.mlp.shared_mlp.gate_proj",
|
| 981 |
+
"model.layers.78.mlp.shared_mlp.up_proj",
|
| 982 |
+
"model.layers.78.post_attention_layernorm",
|
| 983 |
+
"model.layers.78.self_attn.k_norm",
|
| 984 |
+
"model.layers.78.self_attn.k_proj",
|
| 985 |
+
"model.layers.78.self_attn.o_proj",
|
| 986 |
+
"model.layers.78.self_attn.q_norm",
|
| 987 |
+
"model.layers.78.self_attn.q_proj",
|
| 988 |
+
"model.layers.78.self_attn.v_proj",
|
| 989 |
+
"model.layers.79.input_layernorm",
|
| 990 |
+
"model.layers.79.mlp.router.gate",
|
| 991 |
+
"model.layers.79.mlp.shared_mlp.down_proj",
|
| 992 |
+
"model.layers.79.mlp.shared_mlp.gate_proj",
|
| 993 |
+
"model.layers.79.mlp.shared_mlp.up_proj",
|
| 994 |
+
"model.layers.79.post_attention_layernorm",
|
| 995 |
+
"model.layers.79.self_attn.k_norm",
|
| 996 |
+
"model.layers.79.self_attn.k_proj",
|
| 997 |
+
"model.layers.79.self_attn.o_proj",
|
| 998 |
+
"model.layers.79.self_attn.q_norm",
|
| 999 |
+
"model.layers.79.self_attn.q_proj",
|
| 1000 |
+
"model.layers.79.self_attn.v_proj",
|
| 1001 |
+
"model.layers.8.input_layernorm",
|
| 1002 |
+
"model.layers.8.mlp.router.gate",
|
| 1003 |
+
"model.layers.8.mlp.shared_mlp.down_proj",
|
| 1004 |
+
"model.layers.8.mlp.shared_mlp.gate_proj",
|
| 1005 |
+
"model.layers.8.mlp.shared_mlp.up_proj",
|
| 1006 |
+
"model.layers.8.post_attention_layernorm",
|
| 1007 |
+
"model.layers.8.self_attn.k_norm",
|
| 1008 |
+
"model.layers.8.self_attn.k_proj",
|
| 1009 |
+
"model.layers.8.self_attn.o_proj",
|
| 1010 |
+
"model.layers.8.self_attn.q_norm",
|
| 1011 |
+
"model.layers.8.self_attn.q_proj",
|
| 1012 |
+
"model.layers.8.self_attn.v_proj",
|
| 1013 |
+
"model.layers.80.eh_proj",
|
| 1014 |
+
"model.layers.80.enorm",
|
| 1015 |
+
"model.layers.80.final_layernorm",
|
| 1016 |
+
"model.layers.80.hnorm",
|
| 1017 |
+
"model.layers.80.input_layernorm",
|
| 1018 |
+
"model.layers.80.mlp.router.gate",
|
| 1019 |
+
"model.layers.80.mlp.shared_mlp.down_proj",
|
| 1020 |
+
"model.layers.80.mlp.shared_mlp.gate_proj",
|
| 1021 |
+
"model.layers.80.mlp.shared_mlp.up_proj",
|
| 1022 |
+
"model.layers.80.post_attention_layernorm",
|
| 1023 |
+
"model.layers.80.self_attn.k_norm",
|
| 1024 |
+
"model.layers.80.self_attn.k_proj",
|
| 1025 |
+
"model.layers.80.self_attn.o_proj",
|
| 1026 |
+
"model.layers.80.self_attn.q_norm",
|
| 1027 |
+
"model.layers.80.self_attn.q_proj",
|
| 1028 |
+
"model.layers.80.self_attn.v_proj",
|
| 1029 |
+
"model.layers.9.input_layernorm",
|
| 1030 |
+
"model.layers.9.mlp.router.gate",
|
| 1031 |
+
"model.layers.9.mlp.shared_mlp.down_proj",
|
| 1032 |
+
"model.layers.9.mlp.shared_mlp.gate_proj",
|
| 1033 |
+
"model.layers.9.mlp.shared_mlp.up_proj",
|
| 1034 |
+
"model.layers.9.post_attention_layernorm",
|
| 1035 |
+
"model.layers.9.self_attn.k_norm",
|
| 1036 |
+
"model.layers.9.self_attn.k_proj",
|
| 1037 |
+
"model.layers.9.self_attn.o_proj",
|
| 1038 |
+
"model.layers.9.self_attn.q_norm",
|
| 1039 |
+
"model.layers.9.self_attn.q_proj",
|
| 1040 |
+
"model.layers.9.self_attn.v_proj",
|
| 1041 |
+
"model.norm"
|
| 1042 |
+
]
|
| 1043 |
+
}
|
| 1044 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token_id": 120000,
|
| 3 |
+
"do_sample": true,
|
| 4 |
+
"eos_token_id": 120025,
|
| 5 |
+
"pad_token_id": 120002,
|
| 6 |
+
"temperature": 0.9,
|
| 7 |
+
"top_k": -1,
|
| 8 |
+
"top_p": 1,
|
| 9 |
+
"transformers_version": "5.6.0"
|
| 10 |
+
}
|
model-00002-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bcf18012a75cbe67c9edbacd6f844932ce5388052df105f7bca7fd04f19076e0
|
| 3 |
+
size 1925333320
|
model-00003-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8aba1af9a285e6701972c16bfe2567de2318bcc9b0afab147202ad76b396ad6d
|
| 3 |
+
size 1925333320
|
model-00005-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:56de5577f843fb6d0cf8a1fab86ab82b4f1961680cae587ff7f8be2c9b0085e3
|
| 3 |
+
size 1925333320
|
model-00006-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c63adc9dcbaebef3c6876a95b966f2880b3259f28e89093316299ac23499445e
|
| 3 |
+
size 1073745312
|
model-00012-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f03adb53cbf5e2ff5200b8358374f5109ec1ce996679875c3a75a2b7ea986077
|
| 3 |
+
size 1073742272
|
model-00013-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a01a04d48073df3c20e88d97d55765e63428658bc97425db351ca6e992d1e23d
|
| 3 |
+
size 1925332168
|
model-00015-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:67fedbfa68d5152282dae3966a9ded5023e076891f1e92452ac2beb14caa988a
|
| 3 |
+
size 1925333320
|
model-00016-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a8574409eb29460666349968c93aeb6b697770642c1172984a6d1d2839f2e92d
|
| 3 |
+
size 1925333320
|
model-00023-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cf1435462fb8407772f72d03a7d4f059606e752b18623c0d88f6be620f77db78
|
| 3 |
+
size 1925333320
|
model-00025-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f861ae6807f3eeacce1fb79fe9850366598a5465522041c11004580368682bc3
|
| 3 |
+
size 1925332168
|
model-00028-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:abeb1461810dabf07f29da84fb559efbd40a7f31a61f711a6d8910863b7ec445
|
| 3 |
+
size 1925333320
|
model-00029-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:74af5a445e19bcaf1b2a23aed45a0ab8b92e2d6a8a87fb2d8c0ce5530a95f9aa
|
| 3 |
+
size 1925333320
|
model-00032-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ddb1b63952fff51e0e46333109e3bab815da5e2fd16657679b0167fa1e97c8f7
|
| 3 |
+
size 1925333320
|
model-00036-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8f27a414abc0c63d17dd384efc22d6f567d9dd717132834e055a05a015fcbaf4
|
| 3 |
+
size 1057352200
|
model-00041-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f18d7ddc0d2008418f8c30886306e03130f03a5b811eab9f51f71ed0d60acbb5
|
| 3 |
+
size 1925333320
|
model-00042-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:50841884bd8a563a18ae65c946031fb7bde5efaf4cd76cbd2206a3befa4a804d
|
| 3 |
+
size 1056967864
|
model-00043-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:51f416f7164dc44b248bae30b0787958d4e88f0d48809eb6cd8e9d3408d9c504
|
| 3 |
+
size 1925332168
|
model-00048-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f3729f6082a9561adb9b928b031932f2557ec816969ffc149a486a397c3224a9
|
| 3 |
+
size 1056970080
|
model-00050-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:095afcd72d18681c29ac42f569eb996a3964ac7150965edaa156d340bc53d4a0
|
| 3 |
+
size 1925333320
|
model-00051-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ec843677c0ef2e3a1b11efa2509871cf24a357e62dfbfff2ce52b25ac852d2d
|
| 3 |
+
size 1925333320
|
model-00052-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac1f2ad98672fba48a28c0fc0fc9d6ec984ca678f07625b7e78e15a9dc305ab7
|
| 3 |
+
size 1925333320
|
model-00053-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e0c3a7baed883197948372cb7fe200aef68a598a309966a84727be9c2eced586
|
| 3 |
+
size 1925333320
|
model-00058-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:862666119cf674f93fa356550ed58806341acf5f6b55d1f704e3abfda13a2468
|
| 3 |
+
size 1925333320
|
model-00064-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:60f84a8bd4186e09ff017c97feac561ed6cf49843481b2ecd1bbd7a2623e447d
|
| 3 |
+
size 1925333320
|
model-00069-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f2162998a73d1ab3991f783928307fe70f367e95ed0b7a03d99c0d353b4b3404
|
| 3 |
+
size 1925333320
|
model-00070-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b97517bf958e675a615e807233ca8e20439caf056de2a611addc5a5bf833853f
|
| 3 |
+
size 1925333320
|
model-00074-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9dc5d0f6fb09dff5d8d0a79cc2836bd4299b02337629210bb1b6bee861b85955
|
| 3 |
+
size 1925333320
|
model-00075-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0c203d6c2ce249549749e3eb53d9a70828a864d023c81bf057d87b72a70def87
|
| 3 |
+
size 1925333320
|
model-00076-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2e6184471fb19cd5ab26cff7be1b8bcdd50fea2a1910431eb3dcf83c4f4107bf
|
| 3 |
+
size 1925333320
|
model-00078-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7d9e89fcd78d0e325ae4c841ad88f85604e42f3f8ffd2011b0e53edf7ea612f8
|
| 3 |
+
size 1006636320
|
model-00081-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:031d36469ddeeb7ddc2239338505a3dd71cbb37830a13a80bfc58a64eb618a46
|
| 3 |
+
size 1925333320
|
model-00087-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:baa5064088062c503679eac8263878040a5267f1cec213bcf3919deff4ef97ab
|
| 3 |
+
size 1925333320
|
model-00090-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cfed48109271c715475e91dd43c5a3bef46d3a750bb1459a27c5706d08ee93fc
|
| 3 |
+
size 998247464
|
model-00091-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:827012bb0331458343834e70c3c5965f74fc0e7dbbe57f8532a8049b4461454f
|
| 3 |
+
size 305501032
|
model-00092-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:27c013f7cf9a8e00a60c0338fc0fb9a26f0e8bba6e193460675d3d129911916e
|
| 3 |
+
size 1925333320
|
model-00093-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d6407371581a650e88cc8ab14f9081c36aaaafff31a3aca278c4cdfc7e51f630
|
| 3 |
+
size 1925333320
|
model-00094-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a5d3bee41d05731e4f43152a4139753d1c34e48c813b56a00acde4dc31b827a3
|
| 3 |
+
size 1925333320
|
model-00095-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9f805de157b79ae48e86c80a248eaeea3445970e6cd5478f93f796b28a7147e3
|
| 3 |
+
size 1925333320
|
model-00096-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4370c15c42236c63ddf4099926eac032b2a9af4888461013a5414a04b6f20607
|
| 3 |
+
size 1283555104
|
model-00098-of-00099.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3981cec5493da0ef184aa0d8b58c1a95d194abe2588749a694ebd48b53eab0e4
|
| 3 |
+
size 989855856
|
model.safetensors.index.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|