abdoelsayed commited on
Commit
5be4a8d
·
verified ·
1 Parent(s): f7ed37f

Replace adapter with the merged full-weight OBLIQ-IR-3B model (EMNLP 2026)

Browse files
LICENSE_nvidia_base_model.txt ADDED
@@ -0,0 +1,619 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ All binary model files are licensed under the Creative Commons Attribution Non Commercial 4.0 International (CC BY-NC 4.0) License.
2
+ All source code files are licensed under the Apache 2.0 License.
3
+
4
+
5
+ ------------
6
+ Attribution-NonCommercial 4.0 International
7
+
8
+ =======================================================================
9
+
10
+ Creative Commons Corporation ("Creative Commons") is not a law firm and
11
+ does not provide legal services or legal advice. Distribution of
12
+ Creative Commons public licenses does not create a lawyer-client or
13
+ other relationship. Creative Commons makes its licenses and related
14
+ information available on an "as-is" basis. Creative Commons gives no
15
+ warranties regarding its licenses, any material licensed under their
16
+ terms and conditions, or any related information. Creative Commons
17
+ disclaims all liability for damages resulting from their use to the
18
+ fullest extent possible.
19
+
20
+ Using Creative Commons Public Licenses
21
+
22
+ Creative Commons public licenses provide a standard set of terms and
23
+ conditions that creators and other rights holders may use to share
24
+ original works of authorship and other material subject to copyright
25
+ and certain other rights specified in the public license below. The
26
+ following considerations are for informational purposes only, are not
27
+ exhaustive, and do not form part of our licenses.
28
+
29
+ Considerations for licensors: Our public licenses are
30
+ intended for use by those authorized to give the public
31
+ permission to use material in ways otherwise restricted by
32
+ copyright and certain other rights. Our licenses are
33
+ irrevocable. Licensors should read and understand the terms
34
+ and conditions of the license they choose before applying it.
35
+ Licensors should also secure all rights necessary before
36
+ applying our licenses so that the public can reuse the
37
+ material as expected. Licensors should clearly mark any
38
+ material not subject to the license. This includes other CC-
39
+ licensed material, or material used under an exception or
40
+ limitation to copyright. More considerations for licensors:
41
+ wiki.creativecommons.org/Considerations_for_licensors
42
+
43
+ Considerations for the public: By using one of our public
44
+ licenses, a licensor grants the public permission to use the
45
+ licensed material under specified terms and conditions. If
46
+ the licensor's permission is not necessary for any reason--for
47
+ example, because of any applicable exception or limitation to
48
+ copyright--then that use is not regulated by the license. Our
49
+ licenses grant only permissions under copyright and certain
50
+ other rights that a licensor has authority to grant. Use of
51
+ the licensed material may still be restricted for other
52
+ reasons, including because others have copyright or other
53
+ rights in the material. A licensor may make special requests,
54
+ such as asking that all changes be marked or described.
55
+ Although not required by our licenses, you are encouraged to
56
+ respect those requests where reasonable. More considerations
57
+ for the public:
58
+ wiki.creativecommons.org/Considerations_for_licensees
59
+
60
+ =======================================================================
61
+
62
+ Creative Commons Attribution-NonCommercial 4.0 International Public
63
+ License
64
+
65
+ By exercising the Licensed Rights (defined below), You accept and agree
66
+ to be bound by the terms and conditions of this Creative Commons
67
+ Attribution-NonCommercial 4.0 International Public License ("Public
68
+ License"). To the extent this Public License may be interpreted as a
69
+ contract, You are granted the Licensed Rights in consideration of Your
70
+ acceptance of these terms and conditions, and the Licensor grants You
71
+ such rights in consideration of benefits the Licensor receives from
72
+ making the Licensed Material available under these terms and
73
+ conditions.
74
+
75
+
76
+ Section 1 -- Definitions.
77
+
78
+ a. Adapted Material means material subject to Copyright and Similar
79
+ Rights that is derived from or based upon the Licensed Material
80
+ and in which the Licensed Material is translated, altered,
81
+ arranged, transformed, or otherwise modified in a manner requiring
82
+ permission under the Copyright and Similar Rights held by the
83
+ Licensor. For purposes of this Public License, where the Licensed
84
+ Material is a musical work, performance, or sound recording,
85
+ Adapted Material is always produced where the Licensed Material is
86
+ synched in timed relation with a moving image.
87
+
88
+ b. Adapter's License means the license You apply to Your Copyright
89
+ and Similar Rights in Your contributions to Adapted Material in
90
+ accordance with the terms and conditions of this Public License.
91
+
92
+ c. Copyright and Similar Rights means copyright and/or similar rights
93
+ closely related to copyright including, without limitation,
94
+ performance, broadcast, sound recording, and Sui Generis Database
95
+ Rights, without regard to how the rights are labeled or
96
+ categorized. For purposes of this Public License, the rights
97
+ specified in Section 2(b)(1)-(2) are not Copyright and Similar
98
+ Rights.
99
+ d. Effective Technological Measures means those measures that, in the
100
+ absence of proper authority, may not be circumvented under laws
101
+ fulfilling obligations under Article 11 of the WIPO Copyright
102
+ Treaty adopted on December 20, 1996, and/or similar international
103
+ agreements.
104
+
105
+ e. Exceptions and Limitations means fair use, fair dealing, and/or
106
+ any other exception or limitation to Copyright and Similar Rights
107
+ that applies to Your use of the Licensed Material.
108
+
109
+ f. Licensed Material means the artistic or literary work, database,
110
+ or other material to which the Licensor applied this Public
111
+ License.
112
+
113
+ g. Licensed Rights means the rights granted to You subject to the
114
+ terms and conditions of this Public License, which are limited to
115
+ all Copyright and Similar Rights that apply to Your use of the
116
+ Licensed Material and that the Licensor has authority to license.
117
+
118
+ h. Licensor means the individual(s) or entity(ies) granting rights
119
+ under this Public License.
120
+
121
+ i. NonCommercial means not primarily intended for or directed towards
122
+ commercial advantage or monetary compensation. For purposes of
123
+ this Public License, the exchange of the Licensed Material for
124
+ other material subject to Copyright and Similar Rights by digital
125
+ file-sharing or similar means is NonCommercial provided there is
126
+ no payment of monetary compensation in connection with the
127
+ exchange.
128
+
129
+ j. Share means to provide material to the public by any means or
130
+ process that requires permission under the Licensed Rights, such
131
+ as reproduction, public display, public performance, distribution,
132
+ dissemination, communication, or importation, and to make material
133
+ available to the public including in ways that members of the
134
+ public may access the material from a place and at a time
135
+ individually chosen by them.
136
+
137
+ k. Sui Generis Database Rights means rights other than copyright
138
+ resulting from Directive 96/9/EC of the European Parliament and of
139
+ the Council of 11 March 1996 on the legal protection of databases,
140
+ as amended and/or succeeded, as well as other essentially
141
+ equivalent rights anywhere in the world.
142
+
143
+ l. You means the individual or entity exercising the Licensed Rights
144
+ under this Public License. Your has a corresponding meaning.
145
+
146
+
147
+ Section 2 -- Scope.
148
+
149
+ a. License grant.
150
+
151
+ 1. Subject to the terms and conditions of this Public License,
152
+ the Licensor hereby grants You a worldwide, royalty-free,
153
+ non-sublicensable, non-exclusive, irrevocable license to
154
+ exercise the Licensed Rights in the Licensed Material to:
155
+
156
+ a. reproduce and Share the Licensed Material, in whole or
157
+ in part, for NonCommercial purposes only; and
158
+
159
+ b. produce, reproduce, and Share Adapted Material for
160
+ NonCommercial purposes only.
161
+
162
+ 2. Exceptions and Limitations. For the avoidance of doubt, where
163
+ Exceptions and Limitations apply to Your use, this Public
164
+ License does not apply, and You do not need to comply with
165
+ its terms and conditions.
166
+
167
+ 3. Term. The term of this Public License is specified in Section
168
+ 6(a).
169
+
170
+ 4. Media and formats; technical modifications allowed. The
171
+ Licensor authorizes You to exercise the Licensed Rights in
172
+ all media and formats whether now known or hereafter created,
173
+ and to make technical modifications necessary to do so. The
174
+ Licensor waives and/or agrees not to assert any right or
175
+ authority to forbid You from making technical modifications
176
+ necessary to exercise the Licensed Rights, including
177
+ technical modifications necessary to circumvent Effective
178
+ Technological Measures. For purposes of this Public License,
179
+ simply making modifications authorized by this Section 2(a)
180
+ (4) never produces Adapted Material.
181
+
182
+ 5. Downstream recipients.
183
+
184
+ a. Offer from the Licensor -- Licensed Material. Every
185
+ recipient of the Licensed Material automatically
186
+ receives an offer from the Licensor to exercise the
187
+ Licensed Rights under the terms and conditions of this
188
+ Public License.
189
+
190
+ b. No downstream restrictions. You may not offer or impose
191
+ any additional or different terms or conditions on, or
192
+ apply any Effective Technological Measures to, the
193
+ Licensed Material if doing so restricts exercise of the
194
+ Licensed Rights by any recipient of the Licensed
195
+ Material.
196
+
197
+ 6. No endorsement. Nothing in this Public License constitutes or
198
+ may be construed as permission to assert or imply that You
199
+ are, or that Your use of the Licensed Material is, connected
200
+ with, or sponsored, endorsed, or granted official status by,
201
+ the Licensor or others designated to receive attribution as
202
+ provided in Section 3(a)(1)(A)(i).
203
+
204
+ b. Other rights.
205
+
206
+ 1. Moral rights, such as the right of integrity, are not
207
+ licensed under this Public License, nor are publicity,
208
+ privacy, and/or other similar personality rights; however, to
209
+ the extent possible, the Licensor waives and/or agrees not to
210
+ assert any such rights held by the Licensor to the limited
211
+ extent necessary to allow You to exercise the Licensed
212
+ Rights, but not otherwise.
213
+
214
+ 2. Patent and trademark rights are not licensed under this
215
+ Public License.
216
+
217
+ 3. To the extent possible, the Licensor waives any right to
218
+ collect royalties from You for the exercise of the Licensed
219
+ Rights, whether directly or through a collecting society
220
+ under any voluntary or waivable statutory or compulsory
221
+ licensing scheme. In all other cases the Licensor expressly
222
+ reserves any right to collect such royalties, including when
223
+ the Licensed Material is used other than for NonCommercial
224
+ purposes.
225
+
226
+
227
+ Section 3 -- License Conditions.
228
+
229
+ Your exercise of the Licensed Rights is expressly made subject to the
230
+ following conditions.
231
+
232
+ a. Attribution.
233
+
234
+ 1. If You Share the Licensed Material (including in modified
235
+ form), You must:
236
+
237
+ a. retain the following if it is supplied by the Licensor
238
+ with the Licensed Material:
239
+
240
+ i. identification of the creator(s) of the Licensed
241
+ Material and any others designated to receive
242
+ attribution, in any reasonable manner requested by
243
+ the Licensor (including by pseudonym if
244
+ designated);
245
+
246
+ ii. a copyright notice;
247
+
248
+ iii. a notice that refers to this Public License;
249
+
250
+ iv. a notice that refers to the disclaimer of
251
+ warranties;
252
+
253
+ v. a URI or hyperlink to the Licensed Material to the
254
+ extent reasonably practicable;
255
+
256
+ b. indicate if You modified the Licensed Material and
257
+ retain an indication of any previous modifications; and
258
+
259
+ c. indicate the Licensed Material is licensed under this
260
+ Public License, and include the text of, or the URI or
261
+ hyperlink to, this Public License.
262
+
263
+ 2. You may satisfy the conditions in Section 3(a)(1) in any
264
+ reasonable manner based on the medium, means, and context in
265
+ which You Share the Licensed Material. For example, it may be
266
+ reasonable to satisfy the conditions by providing a URI or
267
+ hyperlink to a resource that includes the required
268
+ information.
269
+
270
+ 3. If requested by the Licensor, You must remove any of the
271
+ information required by Section 3(a)(1)(A) to the extent
272
+ reasonably practicable.
273
+
274
+ 4. If You Share Adapted Material You produce, the Adapter's
275
+ License You apply must not prevent recipients of the Adapted
276
+ Material from complying with this Public License.
277
+
278
+
279
+ Section 4 -- Sui Generis Database Rights.
280
+
281
+ Where the Licensed Rights include Sui Generis Database Rights that
282
+ apply to Your use of the Licensed Material:
283
+
284
+ a. for the avoidance of doubt, Section 2(a)(1) grants You the right
285
+ to extract, reuse, reproduce, and Share all or a substantial
286
+ portion of the contents of the database for NonCommercial purposes
287
+ only;
288
+
289
+ b. if You include all or a substantial portion of the database
290
+ contents in a database in which You have Sui Generis Database
291
+ Rights, then the database in which You have Sui Generis Database
292
+ Rights (but not its individual contents) is Adapted Material; and
293
+
294
+ c. You must comply with the conditions in Section 3(a) if You Share
295
+ all or a substantial portion of the contents of the database.
296
+
297
+ For the avoidance of doubt, this Section 4 supplements and does not
298
+ replace Your obligations under this Public License where the Licensed
299
+ Rights include other Copyright and Similar Rights.
300
+
301
+
302
+ Section 5 -- Disclaimer of Warranties and Limitation of Liability.
303
+
304
+ a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
305
+ EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
306
+ AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
307
+ ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
308
+ IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
309
+ WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
310
+ PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
311
+ ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
312
+ KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
313
+ ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
314
+
315
+ b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
316
+ TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
317
+ NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
318
+ INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
319
+ COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
320
+ USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
321
+ ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
322
+ DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
323
+ IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
324
+
325
+ c. The disclaimer of warranties and limitation of liability provided
326
+ above shall be interpreted in a manner that, to the extent
327
+ possible, most closely approximates an absolute disclaimer and
328
+ waiver of all liability.
329
+
330
+
331
+ Section 6 -- Term and Termination.
332
+
333
+ a. This Public License applies for the term of the Copyright and
334
+ Similar Rights licensed here. However, if You fail to comply with
335
+ this Public License, then Your rights under this Public License
336
+ terminate automatically.
337
+
338
+ b. Where Your right to use the Licensed Material has terminated under
339
+ Section 6(a), it reinstates:
340
+
341
+ 1. automatically as of the date the violation is cured, provided
342
+ it is cured within 30 days of Your discovery of the
343
+ violation; or
344
+
345
+ 2. upon express reinstatement by the Licensor.
346
+
347
+ For the avoidance of doubt, this Section 6(b) does not affect any
348
+ right the Licensor may have to seek remedies for Your violations
349
+ of this Public License.
350
+
351
+ c. For the avoidance of doubt, the Licensor may also offer the
352
+ Licensed Material under separate terms or conditions or stop
353
+ distributing the Licensed Material at any time; however, doing so
354
+ will not terminate this Public License.
355
+
356
+ d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
357
+ License.
358
+
359
+
360
+ Section 7 -- Other Terms and Conditions.
361
+
362
+ a. The Licensor shall not be bound by any additional or different
363
+ terms or conditions communicated by You unless expressly agreed.
364
+
365
+ b. Any arrangements, understandings, or agreements regarding the
366
+ Licensed Material not stated herein are separate from and
367
+ independent of the terms and conditions of this Public License.
368
+
369
+
370
+ Section 8 -- Interpretation.
371
+
372
+ a. For the avoidance of doubt, this Public License does not, and
373
+ shall not be interpreted to, reduce, limit, restrict, or impose
374
+ conditions on any use of the Licensed Material that could lawfully
375
+ be made without permission under this Public License.
376
+
377
+ b. To the extent possible, if any provision of this Public License is
378
+ deemed unenforceable, it shall be automatically reformed to the
379
+ minimum extent necessary to make it enforceable. If the provision
380
+ cannot be reformed, it shall be severed from this Public License
381
+ without affecting the enforceability of the remaining terms and
382
+ conditions.
383
+
384
+ c. No term or condition of this Public License will be waived and no
385
+ failure to comply consented to unless expressly agreed to by the
386
+ Licensor.
387
+
388
+ d. Nothing in this Public License constitutes or may be interpreted
389
+ as a limitation upon, or waiver of, any privileges and immunities
390
+ that apply to the Licensor or You, including from the legal
391
+ processes of any jurisdiction or authority.
392
+
393
+ =======================================================================
394
+
395
+ Creative Commons is not a party to its public
396
+ licenses. Notwithstanding, Creative Commons may elect to apply one of
397
+ its public licenses to material it publishes and in those instances
398
+ will be considered the “Licensor.” The text of the Creative Commons
399
+ public licenses is dedicated to the public domain under the CC0 Public
400
+ Domain Dedication. Except for the limited purpose of indicating that
401
+ material is shared under a Creative Commons public license or as
402
+ otherwise permitted by the Creative Commons policies published at
403
+ creativecommons.org/policies, Creative Commons does not authorize the
404
+ use of the trademark "Creative Commons" or any other trademark or logo
405
+ of Creative Commons without its prior written consent including,
406
+ without limitation, in connection with any unauthorized modifications
407
+ to any of its public licenses or any other arrangements,
408
+ understandings, or agreements concerning use of licensed material. For
409
+ the avoidance of doubt, this paragraph does not form part of the
410
+ public licenses.
411
+
412
+ Creative Commons may be contacted at creativecommons.org.
413
+
414
+ -----------------
415
+
416
+
417
+ Apache License
418
+ Version 2.0, January 2004
419
+ http://www.apache.org/licenses/
420
+
421
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
422
+
423
+ 1. Definitions.
424
+
425
+ "License" shall mean the terms and conditions for use, reproduction,
426
+ and distribution as defined by Sections 1 through 9 of this document.
427
+
428
+ "Licensor" shall mean the copyright owner or entity authorized by
429
+ the copyright owner that is granting the License.
430
+
431
+ "Legal Entity" shall mean the union of the acting entity and all
432
+ other entities that control, are controlled by, or are under common
433
+ control with that entity. For the purposes of this definition,
434
+ "control" means (i) the power, direct or indirect, to cause the
435
+ direction or management of such entity, whether by contract or
436
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
437
+ outstanding shares, or (iii) beneficial ownership of such entity.
438
+
439
+ "You" (or "Your") shall mean an individual or Legal Entity
440
+ exercising permissions granted by this License.
441
+
442
+ "Source" form shall mean the preferred form for making modifications,
443
+ including but not limited to software source code, documentation
444
+ source, and configuration files.
445
+
446
+ "Object" form shall mean any form resulting from mechanical
447
+ transformation or translation of a Source form, including but
448
+ not limited to compiled object code, generated documentation,
449
+ and conversions to other media types.
450
+
451
+ "Work" shall mean the work of authorship, whether in Source or
452
+ Object form, made available under the License, as indicated by a
453
+ copyright notice that is included in or attached to the work
454
+ (an example is provided in the Appendix below).
455
+
456
+ "Derivative Works" shall mean any work, whether in Source or Object
457
+ form, that is based on (or derived from) the Work and for which the
458
+ editorial revisions, annotations, elaborations, or other modifications
459
+ represent, as a whole, an original work of authorship. For the purposes
460
+ of this License, Derivative Works shall not include works that remain
461
+ separable from, or merely link (or bind by name) to the interfaces of,
462
+ the Work and Derivative Works thereof.
463
+
464
+ "Contribution" shall mean any work of authorship, including
465
+ the original version of the Work and any modifications or additions
466
+ to that Work or Derivative Works thereof, that is intentionally
467
+ submitted to Licensor for inclusion in the Work by the copyright owner
468
+ or by an individual or Legal Entity authorized to submit on behalf of
469
+ the copyright owner. For the purposes of this definition, "submitted"
470
+ means any form of electronic, verbal, or written communication sent
471
+ to the Licensor or its representatives, including but not limited to
472
+ communication on electronic mailing lists, source code control systems,
473
+ and issue tracking systems that are managed by, or on behalf of, the
474
+ Licensor for the purpose of discussing and improving the Work, but
475
+ excluding communication that is conspicuously marked or otherwise
476
+ designated in writing by the copyright owner as "Not a Contribution."
477
+
478
+ "Contributor" shall mean Licensor and any individual or Legal Entity
479
+ on behalf of whom a Contribution has been received by Licensor and
480
+ subsequently incorporated within the Work.
481
+
482
+ 2. Grant of Copyright License. Subject to the terms and conditions of
483
+ this License, each Contributor hereby grants to You a perpetual,
484
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
485
+ copyright license to reproduce, prepare Derivative Works of,
486
+ publicly display, publicly perform, sublicense, and distribute the
487
+ Work and such Derivative Works in Source or Object form.
488
+
489
+ 3. Grant of Patent License. Subject to the terms and conditions of
490
+ this License, each Contributor hereby grants to You a perpetual,
491
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
492
+ (except as stated in this section) patent license to make, have made,
493
+ use, offer to sell, sell, import, and otherwise transfer the Work,
494
+ where such license applies only to those patent claims licensable
495
+ by such Contributor that are necessarily infringed by their
496
+ Contribution(s) alone or by combination of their Contribution(s)
497
+ with the Work to which such Contribution(s) was submitted. If You
498
+ institute patent litigation against any entity (including a
499
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
500
+ or a Contribution incorporated within the Work constitutes direct
501
+ or contributory patent infringement, then any patent licenses
502
+ granted to You under this License for that Work shall terminate
503
+ as of the date such litigation is filed.
504
+
505
+ 4. Redistribution. You may reproduce and distribute copies of the
506
+ Work or Derivative Works thereof in any medium, with or without
507
+ modifications, and in Source or Object form, provided that You
508
+ meet the following conditions:
509
+
510
+ (a) You must give any other recipients of the Work or
511
+ Derivative Works a copy of this License; and
512
+
513
+ (b) You must cause any modified files to carry prominent notices
514
+ stating that You changed the files; and
515
+
516
+ (c) You must retain, in the Source form of any Derivative Works
517
+ that You distribute, all copyright, patent, trademark, and
518
+ attribution notices from the Source form of the Work,
519
+ excluding those notices that do not pertain to any part of
520
+ the Derivative Works; and
521
+
522
+ (d) If the Work includes a "NOTICE" text file as part of its
523
+ distribution, then any Derivative Works that You distribute must
524
+ include a readable copy of the attribution notices contained
525
+ within such NOTICE file, excluding those notices that do not
526
+ pertain to any part of the Derivative Works, in at least one
527
+ of the following places: within a NOTICE text file distributed
528
+ as part of the Derivative Works; within the Source form or
529
+ documentation, if provided along with the Derivative Works; or,
530
+ within a display generated by the Derivative Works, if and
531
+ wherever such third-party notices normally appear. The contents
532
+ of the NOTICE file are for informational purposes only and
533
+ do not modify the License. You may add Your own attribution
534
+ notices within Derivative Works that You distribute, alongside
535
+ or as an addendum to the NOTICE text from the Work, provided
536
+ that such additional attribution notices cannot be construed
537
+ as modifying the License.
538
+
539
+ You may add Your own copyright statement to Your modifications and
540
+ may provide additional or different license terms and conditions
541
+ for use, reproduction, or distribution of Your modifications, or
542
+ for any such Derivative Works as a whole, provided Your use,
543
+ reproduction, and distribution of the Work otherwise complies with
544
+ the conditions stated in this License.
545
+
546
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
547
+ any Contribution intentionally submitted for inclusion in the Work
548
+ by You to the Licensor shall be under the terms and conditions of
549
+ this License, without any additional terms or conditions.
550
+ Notwithstanding the above, nothing herein shall supersede or modify
551
+ the terms of any separate license agreement you may have executed
552
+ with Licensor regarding such Contributions.
553
+
554
+ 6. Trademarks. This License does not grant permission to use the trade
555
+ names, trademarks, service marks, or product names of the Licensor,
556
+ except as required for reasonable and customary use in describing the
557
+ origin of the Work and reproducing the content of the NOTICE file.
558
+
559
+ 7. Disclaimer of Warranty. Unless required by applicable law or
560
+ agreed to in writing, Licensor provides the Work (and each
561
+ Contributor provides its Contributions) on an "AS IS" BASIS,
562
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
563
+ implied, including, without limitation, any warranties or conditions
564
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
565
+ PARTICULAR PURPOSE. You are solely responsible for determining the
566
+ appropriateness of using or redistributing the Work and assume any
567
+ risks associated with Your exercise of permissions under this License.
568
+
569
+ 8. Limitation of Liability. In no event and under no legal theory,
570
+ whether in tort (including negligence), contract, or otherwise,
571
+ unless required by applicable law (such as deliberate and grossly
572
+ negligent acts) or agreed to in writing, shall any Contributor be
573
+ liable to You for damages, including any direct, indirect, special,
574
+ incidental, or consequential damages of any character arising as a
575
+ result of this License or out of the use or inability to use the
576
+ Work (including but not limited to damages for loss of goodwill,
577
+ work stoppage, computer failure or malfunction, or any and all
578
+ other commercial damages or losses), even if such Contributor
579
+ has been advised of the possibility of such damages.
580
+
581
+ 9. Accepting Warranty or Additional Liability. While redistributing
582
+ the Work or Derivative Works thereof, You may choose to offer,
583
+ and charge a fee for, acceptance of support, warranty, indemnity,
584
+ or other liability obligations and/or rights consistent with this
585
+ License. However, in accepting such obligations, You may act only
586
+ on Your own behalf and on Your sole responsibility, not on behalf
587
+ of any other Contributor, and only if You agree to indemnify,
588
+ defend, and hold each Contributor harmless for any liability
589
+ incurred by, or claims asserted against, such Contributor by reason
590
+ of your accepting any such warranty or additional liability.
591
+
592
+ END OF TERMS AND CONDITIONS
593
+
594
+ APPENDIX: How to apply the Apache License to your work.
595
+
596
+ To apply the Apache License to your work, attach the following
597
+ boilerplate notice, with the fields enclosed by brackets "[]"
598
+ replaced with your own identifying information. (Don't include
599
+ the brackets!) The text should be enclosed in the appropriate
600
+ comment syntax for the file format. We also recommend that a
601
+ file or class name and description of purpose be included on the
602
+ same "printed page" as the copyright notice for easier
603
+ identification within third-party archives.
604
+
605
+ Copyright [yyyy] [name of copyright owner]
606
+
607
+ Licensed under the Apache License, Version 2.0 (the "License");
608
+ you may not use this file except in compliance with the License.
609
+ You may obtain a copy of the License at
610
+
611
+ http://www.apache.org/licenses/LICENSE-2.0
612
+
613
+ Unless required by applicable law or agreed to in writing, software
614
+ distributed under the License is distributed on an "AS IS" BASIS,
615
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
616
+ See the License for the specific language governing permissions and
617
+ limitations under the License.
618
+
619
+
NOTICE.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ Meta Llama 3.2 is licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  license: cc-by-nc-4.0
3
- library_name: peft
4
  base_model: nvidia/llama-nv-embed-reasoning-3b
 
5
  pipeline_tag: feature-extraction
6
  tags:
7
  - sentence-transformers
@@ -10,7 +10,6 @@ tags:
10
  - oblique-retrieval
11
  - authorship
12
  - stylometry
13
- - lora
14
  language:
15
  - en
16
  ---
@@ -18,38 +17,19 @@ language:
18
  # OBLIQ-IR-3B
19
 
20
  A 3B single-vector dense retriever for **oblique retrieval** — queries whose relevance is decided by a
21
- latent attribute (implicit stance, abstract proof strategy, authorial style, a lossy recollection of a
22
- rhetorical exchange) that has little or no surface expression in the document.
23
 
24
- This is the model from *OBLIQ-IR: Training a Dense Retriever for Oblique Queries* (EMNLP 2026).
25
 
26
- > ⚠️ **This repository contains a LoRA adapter, not merged weights.** You need the base model
27
- > [`nvidia/llama-nv-embed-reasoning-3b`](https://huggingface.co/nvidia/llama-nv-embed-reasoning-3b)
28
- > as well. Loading with `SentenceTransformer` handles this automatically (see below).
29
 
30
- - 📊 Data, training mixture and all run files: [`DataScience-UIBK/OBLIQ-IR-Data`](https://huggingface.co/datasets/DataScience-UIBK/OBLIQ-IR-Data)
31
- - 🧪 Benchmark: [`dianetc/OBLIQ-Bench`](https://huggingface.co/datasets/dianetc/OBLIQ-Bench)
32
-
33
- ## Results
34
-
35
- NDCG@10 (Gold) on OBLIQ-Bench. **Bold** = best non-oracle.
36
-
37
- | Pipeline | Writing | Math | Twitter | Congress |
38
- |---|---|---|---|---|
39
- | BM25 | .077 | .022 | .000 | .000 |
40
- | Qwen3-Embed-4B | .033 | .095 | .032 | .040 |
41
- | Gemini-2-Embedding | .164 | .144 | .068 | .059 |
42
- | GPT-5.2 Multi-Hop Agent | .061 | .161 | .141 | .183 |
43
- | OBLIQ-IR (no distillation) | .096 | .148 | .158 | .196 |
44
- | OBLIQ-IR (dense, this model) | **.211** | .140 | .151 | .187 |
45
- | **OBLIQ-IR (full = dense + reranker)** | **.211** | **.171** | **.177** | **.281** |
46
- | *Oracle GPT-5.2 Tournament (not attainable)* | *.515* | *.279* | *.331* | *.913* |
47
-
48
- "Full" applies TourRank (Y=5, Qwen3.6-27B) on Math/Twitter/Congress; for Writing the selected policy is
49
- the identity, so **the numbers above are what this checkpoint produces on its own for Writing**.
50
-
51
- Gains are large and significant on Writing (+0.116) and Congress (+0.085, both *p*<0.001), and marginal
52
- on Math (*p*=0.08) and Twitter (*p*=0.06). We do not claim the fifth OBLIQ-Bench task, WildChat-Errors.
53
 
54
  ## Usage
55
 
@@ -58,83 +38,101 @@ from sentence_transformers import SentenceTransformer
58
 
59
  model = SentenceTransformer("DataScience-UIBK/OBLIQ-IR-3B", trust_remote_code=True)
60
 
61
- queries = ["A passage written with the same restless, aphoristic voice as this one."]
62
- documents = ["...your corpus documents..."]
63
-
64
- q = model.encode(["query: " + x for x in queries])
65
- d = model.encode(["passage: " + x for x in documents])
66
-
67
  scores = model.similarity(q, d)
68
  ```
69
 
70
- **The `query: ` and `passage: ` prefixes are required** — the model was trained with them, and dropping
71
- them changes the embedding. Queries take no other prefix.
72
 
73
- > **Note on task instructions.** The training file carries a per-task instruction string alongside each
74
- > query, but the training script consumes only the query text, and the evaluation that produced every
75
- > number in the paper encodes the raw benchmark query with the `query: ` prefix alone. **Do not prepend a
76
- > task instruction** the released checkpoint was neither trained nor evaluated with one, so adding one
77
- > moves you off-distribution. Task identity is carried by the mixture the model was trained on, not by a
78
- > runtime prefix.
79
 
80
- Truncate documents at 1024 tokens and queries at 256 to match the paper's evaluation:
81
 
82
  ```python
83
- model.max_seq_length = 1024 # documents
84
- d = model.encode(["passage: " + x for x in documents])
85
- model.max_seq_length = 256 # queries
86
- q = model.encode(["query: " + x for x in queries])
87
  ```
88
 
89
- ## How it was trained
90
 
91
- Two training-time signals, mixed into one encoder shared by all tasks:
 
92
 
93
- 1. **Per-mechanism synthetic queries.** A document is read by an instruction LM (Qwen3.6-27B) which first
94
- writes a description of the task's latent attribute, then writes a query through one of three lenses
95
- matched to the OBLIQ query mechanisms *descriptive* (Twitter), *analogue* (Math, Writing), or
96
- *tip-of-the-tongue* (Congress). Four BM25 hard negatives are mined per query.
97
- 2. **kNN-graph distillation from a frozen authorship encoder** (writing task only). A frozen authorship
98
- encoder ([Rivera-Soto et al., 2021](https://huggingface.co/rrivera1849/LUAR-MUD)) defines a k=3 nearest-neighbour
99
- graph over the writing corpus; its **edges** become anchor–positive training pairs. Only the topology is
100
- used the teacher's similarity *scores* are discarded. This transfers a style-versus-topic inductive
101
- bias that no topical lens can express, lifting Writing from .096 to .211.
 
 
 
 
102
 
103
- Neither stage touches the OBLIQ qrels.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
 
105
  | Setting | Value |
106
  |---|---|
107
- | Base model | `nvidia/llama-nv-embed-reasoning-3b` (Llama-3.2-3B bi-encoder) |
108
- | Adapter | LoRA r=16, α=32, dropout 0.05, on q/k/v/o/gate/up/down (~24M params) |
109
- | Pooling | mean over token embeddings, then L2 normalise → 3072-dim |
110
- | Loss | CachedMultipleNegativesRankingLoss (Sentence-Transformers) |
111
- | Temperature | τ = 1/0.02 |
112
- | LR / schedule | 2e-4, 3% warmup, **linear** decay |
113
- | Batch | 4 per device × 40 GPUs × 2 grad-accum = **320** anchors |
114
- | Seq length | 1024 (queries truncated to 256 when encoding for evaluation) |
115
- | Epochs / hardware | 1 epoch, 40× H100, ~50 min |
116
- | Training rows | 154,361 (= 149,361 synthetic + 5,000 authorship kNN pairs) |
117
-
118
- ## Reproducibility
119
-
120
- Every number in the paper is regenerable from this checkpoint. On identical hardware the short-document
121
- tasks reproduce **bit-for-bit** (Math and Twitter: identical top-100 documents, identical order, identical
122
- cosine scores); the long-document tasks reproduce to within 0.0006 NDCG@10 (Writing 0.00015, Congress
123
- 0.00056), because sequences at the 1024-token cap accumulate more bf16 rounding across GPU generations.
124
- Expect third-decimal movement on Writing and Congress on different hardware.
125
 
126
  ## Licence and intended use
127
 
128
- The adapter weights are released under **CC BY-NC 4.0**. Use also requires accepting the terms of the base
129
- model `nvidia/llama-nv-embed-reasoning-3b` and of Llama-3.2, which it derives from. Note that NVIDIA's
130
- repository is internally inconsistent about its licence (its `LICENSE` file states CC BY-NC 4.0 while its
131
- README front-matter states CC BY 4.0); check with NVIDIA before any commercial use.
 
 
 
132
 
133
- **Intended use** is research on latent-attribute retrieval. Please note that retrieval by authorial
134
- fingerprint can be used to link texts to an author across topics and venues, and therefore to deanonymise
135
- writers who rely on pseudonymity. The authorship encoder we distil is trained on Reddit data, and because
136
- style correlates with demography and dialect, the distilled neighbourhood structure may carry those
137
- correlates. We did not audit for this. Any deployment over people's writing should.
138
 
139
  ## Citation
140
 
 
1
  ---
2
  license: cc-by-nc-4.0
 
3
  base_model: nvidia/llama-nv-embed-reasoning-3b
4
+ library_name: sentence-transformers
5
  pipeline_tag: feature-extraction
6
  tags:
7
  - sentence-transformers
 
10
  - oblique-retrieval
11
  - authorship
12
  - stylometry
 
13
  language:
14
  - en
15
  ---
 
17
  # OBLIQ-IR-3B
18
 
19
  A 3B single-vector dense retriever for **oblique retrieval** — queries whose relevance is decided by a
20
+ latent attribute (an implicit stance, an abstract proof strategy, an authorial style, a lossy recollection
21
+ of a rhetorical exchange) that has little or no surface expression in the document.
22
 
23
+ From *OBLIQ-IR: Training a Dense Retriever for Oblique Queries* (EMNLP 2026).
24
 
25
+ **This repository holds the merged full-weight model** load it and go, no base download, no PEFT.
 
 
26
 
27
+ | | |
28
+ |---|---|
29
+ | 🧩 LoRA adapter only (48 MB) | [`DataScience-UIBK/OBLIQ-IR-3B-LoRA`](https://huggingface.co/DataScience-UIBK/OBLIQ-IR-3B-LoRA) |
30
+ | 📊 Training data, runs, results | [`DataScience-UIBK/OBLIQ-IR-Data`](https://huggingface.co/datasets/DataScience-UIBK/OBLIQ-IR-Data) |
31
+ | 💻 Code | [github.com/DataScience-UIBK/obliq-ir](https://github.com/DataScience-UIBK/obliq-ir) |
32
+ | 🧪 Benchmark | [`dianetc/OBLIQ-Bench`](https://huggingface.co/datasets/dianetc/OBLIQ-Bench) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
  ## Usage
35
 
 
38
 
39
  model = SentenceTransformer("DataScience-UIBK/OBLIQ-IR-3B", trust_remote_code=True)
40
 
41
+ q = model.encode(["query: " + "A passage in the same restless, aphoristic voice as this one."])
42
+ d = model.encode(["passage: " + doc for doc in corpus])
 
 
 
 
43
  scores = model.similarity(q, d)
44
  ```
45
 
46
+ **The `query: ` and `passage: ` prefixes are required** — the model was trained with them.
 
47
 
48
+ > **Do not prepend a task instruction.** The training file carries a per-task instruction string next to each
49
+ > query, but the training script consumes only the query text, and the evaluation behind every number in the
50
+ > paper encodes the raw query with the `query: ` prefix alone. Adding an instruction moves you
51
+ > off-distribution. Task identity comes from the training mixture, not a runtime prefix.
 
 
52
 
53
+ To match the paper's evaluation, encode documents at 1024 tokens and queries at 256:
54
 
55
  ```python
56
+ model.max_seq_length = 1024
57
+ d = model.encode(["passage: " + x for x in corpus])
58
+ model.max_seq_length = 256
59
+ q = model.encode(["query: " + x for x in queries])
60
  ```
61
 
62
+ ## Results
63
 
64
+ NDCG@10 Gold on OBLIQ-Bench. "Full" adds TourRank (Y=5) on Math/Twitter/Congress; on Writing the selected
65
+ policy is the identity, so this model alone *is* the full system there.
66
 
67
+ | Pipeline | Writing | Math | Twitter | Congress |
68
+ |---|---|---|---|---|
69
+ | BM25 | .077 | .022 | .000 | .000 |
70
+ | Qwen3-Embed-4B | .033 | .095 | .032 | .040 |
71
+ | Gemini-2-Embedding | .164 | .144 | .068 | .059 |
72
+ | GPT-5.2 Multi-Hop Agent | .061 | .161 | .141 | .183 |
73
+ | OBLIQ-IR (no distillation) | .096 | .148 | .158 | .196 |
74
+ | OBLIQ-IR dense | **.211** | .140 | .151 | .187 |
75
+ | **OBLIQ-IR full** | **.211** | **.171** | **.177** | **.281** |
76
+ | *Oracle GPT-5.2 Tournament (not attainable)* | *.515* | *.279* | *.331* | *.913* |
77
+
78
+ Gains are large and significant on Writing (+0.116) and Congress (+0.085, both *p*<0.001) and marginal on
79
+ Math (*p*=0.08) and Twitter (*p*=0.06). We do not claim the fifth task, WildChat-Errors.
80
 
81
+ ## Merged vs. adapter: a small numerical caveat
82
+
83
+ The published numbers were produced by the **adapter** path (base weights + LoRA computed separately). This
84
+ repository stores `W + BA·scaling` folded into bf16 weights. Because bf16 carries only about three
85
+ significant digits, folding is slightly lossy, so this model does not reproduce the adapter bit-for-bit:
86
+
87
+ | Task | Metric | This merged model | Adapter (paper) | Δ |
88
+ |---|---|---|---|---|
89
+ | Math | NDCG@10 | 0.1407 | 0.1403 | +0.0004 |
90
+ | Twitter | NDCG@10 | 0.1516 | 0.1508 | +0.0008 |
91
+
92
+ Top-1 results agree on 151/151 math queries and 275/281 twitter queries; deeper in the ranking the order
93
+ shifts slightly. The differences are far below run-to-run variance, but they can move the **third decimal**.
94
+
95
+ **For exact reproduction of the paper's tables, use
96
+ [`OBLIQ-IR-3B-LoRA`](https://huggingface.co/DataScience-UIBK/OBLIQ-IR-3B-LoRA)**, which regenerates the
97
+ published rankings bit-for-bit. Use this repository when you want convenience.
98
+
99
+ ## Training
100
+
101
+ Two signals mixed into one encoder shared by all tasks. **(1) Per-mechanism synthetic queries:** an
102
+ instruction LM describes a document's latent attribute, then writes a query through one of three lenses
103
+ matched to the OBLIQ query mechanisms — descriptive, analogue, tip-of-the-tongue — with four BM25 hard
104
+ negatives each. **(2) kNN-graph distillation from a frozen authorship encoder** (writing only): the k=3
105
+ nearest-neighbour graph of [Rivera-Soto et al. (2021)](https://huggingface.co/rrivera1849/LUAR-MUD) over the
106
+ writing corpus supplies anchor–positive pairs. Only the graph's *topology* is used; the teacher's similarity
107
+ scores are discarded. That lifts Writing from .096 to .211. Neither stage touches the OBLIQ qrels.
108
 
109
  | Setting | Value |
110
  |---|---|
111
+ | Base | `nvidia/llama-nv-embed-reasoning-3b` (Llama-3.2-3B bi-encoder) |
112
+ | Adapter (before merging) | LoRA r=16, α=32, dropout 0.05 on q/k/v/o/gate/up/down (~24M params) |
113
+ | Pooling | mean over tokens, then L2 normalise → 3072-dim |
114
+ | Loss | CachedMultipleNegativesRankingLoss, τ = 1/0.02 |
115
+ | LR | 2e-4, 3% warmup, linear decay |
116
+ | Batch | 4 per device × 40 GPUs × 2 grad-accum = 320 anchors |
117
+ | Seq length | 1024 (queries truncated to 256 at evaluation) |
118
+ | Schedule | 1 epoch, 40× H100, ~50 min |
119
+ | Training rows | 154,361 (149,361 synthetic + 5,000 authorship kNN pairs) |
 
 
 
 
 
 
 
 
 
120
 
121
  ## Licence and intended use
122
 
123
+ Released under **CC BY-NC 4.0**. This repository redistributes weights derived from
124
+ `nvidia/llama-nv-embed-reasoning-3b`, which NVIDIA releases for **non-commercial / research use only**; their
125
+ licence is included as `LICENSE_nvidia_base_model.txt` and their attribution notice as `NOTICE.txt`.
126
+
127
+ **Built with Llama.** The base model derives from `meta-llama/Llama-3.2-3B` and the
128
+ [Llama 3.2 Community License](https://huggingface.co/meta-llama/Llama-3.2-1B/blob/main/LICENSE.txt) applies
129
+ in addition to the terms above.
130
 
131
+ **Intended use** is research on latent-attribute retrieval. Retrieval by authorial fingerprint can link texts
132
+ to an author across topics and venues, and so deanonymise writers who rely on pseudonymity. The authorship
133
+ encoder we distil is trained on Reddit data, and because style correlates with demography and dialect, the
134
+ distilled neighbourhood structure may carry those correlates. We did not audit for this; any deployment over
135
+ people's writing should.
136
 
137
  ## Citation
138
 
adapter_config.json DELETED
@@ -1,48 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": null,
6
- "base_model_name_or_path": "nvidia/llama-nv-embed-reasoning-3b",
7
- "bias": "none",
8
- "corda_config": null,
9
- "ensure_weight_tying": false,
10
- "eva_config": null,
11
- "exclude_modules": null,
12
- "fan_in_fan_out": false,
13
- "inference_mode": false,
14
- "init_lora_weights": true,
15
- "layer_replication": null,
16
- "layers_pattern": null,
17
- "layers_to_transform": null,
18
- "loftq_config": {},
19
- "lora_alpha": 32,
20
- "lora_bias": false,
21
- "lora_dropout": 0.05,
22
- "lora_ga_config": null,
23
- "megatron_config": null,
24
- "megatron_core": "megatron.core",
25
- "modules_to_save": null,
26
- "peft_type": "LORA",
27
- "peft_version": "0.19.1",
28
- "qalora_group_size": 16,
29
- "r": 16,
30
- "rank_pattern": {},
31
- "revision": null,
32
- "target_modules": [
33
- "k_proj",
34
- "o_proj",
35
- "up_proj",
36
- "gate_proj",
37
- "down_proj",
38
- "v_proj",
39
- "q_proj"
40
- ],
41
- "target_parameters": null,
42
- "task_type": "FEATURE_EXTRACTION",
43
- "trainable_token_indices": null,
44
- "use_bdlora": null,
45
- "use_dora": false,
46
- "use_qalora": false,
47
- "use_rslora": false
48
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LlamaBidirectionalModel"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "auto_map": {
8
+ "AutoConfig": "llama_bidirectional_model.LlamaBidirectionalConfig",
9
+ "AutoModel": "llama_bidirectional_model.LlamaBidirectionalModel"
10
+ },
11
+ "bos_token_id": 128000,
12
+ "dtype": "bfloat16",
13
+ "eos_token_id": 128001,
14
+ "head_dim": 128,
15
+ "hidden_act": "silu",
16
+ "hidden_size": 3072,
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 8192,
19
+ "max_position_embeddings": 131072,
20
+ "mlp_bias": false,
21
+ "model_type": "llama_bidirec",
22
+ "num_attention_heads": 24,
23
+ "num_hidden_layers": 28,
24
+ "num_key_value_heads": 8,
25
+ "pooling": "avg",
26
+ "pretraining_tp": 1,
27
+ "rms_norm_eps": 1e-05,
28
+ "rope_scaling": {
29
+ "factor": 32.0,
30
+ "high_freq_factor": 4.0,
31
+ "low_freq_factor": 1.0,
32
+ "original_max_position_embeddings": 8192,
33
+ "rope_type": "llama3"
34
+ },
35
+ "rope_theta": 500000.0,
36
+ "tie_word_embeddings": true,
37
+ "transformers_version": "4.57.6",
38
+ "use_cache": true,
39
+ "vocab_size": 128256
40
+ }
adapter_model.safetensors → model-00001-of-00002.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e3c3ea234570f1c06fc5053c50685bc7b8ae06bee8598795269be16b9794f79a
3
- size 48677784
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7610b73f373235697ea1b8628d8d4efc1dee90f0c83759935500cd31f878eec2
3
+ size 4965797976
model-00002-of-00002.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7574b7842f7d83ee063dac764927b5df1d0386fa5f503fab97edc946c2290d12
3
+ size 1459729544
model.safetensors.index.json ADDED
@@ -0,0 +1,262 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_parameters": 3212749824,
4
+ "total_size": 6425499648
5
+ },
6
+ "weight_map": {
7
+ "embed_tokens.weight": "model-00001-of-00002.safetensors",
8
+ "layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
9
+ "layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
10
+ "layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
11
+ "layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
12
+ "layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
13
+ "layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
14
+ "layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
15
+ "layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
16
+ "layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
17
+ "layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
18
+ "layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
19
+ "layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
20
+ "layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
21
+ "layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
22
+ "layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
23
+ "layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
24
+ "layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
25
+ "layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
26
+ "layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
27
+ "layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
28
+ "layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
29
+ "layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
30
+ "layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
31
+ "layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
32
+ "layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
33
+ "layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
34
+ "layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
35
+ "layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
36
+ "layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
37
+ "layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
38
+ "layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
39
+ "layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
40
+ "layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
41
+ "layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
42
+ "layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
43
+ "layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
44
+ "layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
45
+ "layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
46
+ "layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
47
+ "layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
48
+ "layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
49
+ "layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
50
+ "layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
51
+ "layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
52
+ "layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
53
+ "layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
54
+ "layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
55
+ "layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
56
+ "layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
57
+ "layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
58
+ "layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
59
+ "layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
60
+ "layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
61
+ "layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
62
+ "layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
63
+ "layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
64
+ "layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
65
+ "layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
66
+ "layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
67
+ "layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
68
+ "layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
69
+ "layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
70
+ "layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
71
+ "layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
72
+ "layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
73
+ "layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
74
+ "layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
75
+ "layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
76
+ "layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
77
+ "layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
78
+ "layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
79
+ "layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
80
+ "layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
81
+ "layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
82
+ "layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
83
+ "layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
84
+ "layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
85
+ "layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
86
+ "layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
87
+ "layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
88
+ "layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
89
+ "layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
90
+ "layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
91
+ "layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
92
+ "layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
93
+ "layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
94
+ "layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
95
+ "layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
96
+ "layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
97
+ "layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
98
+ "layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
99
+ "layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
100
+ "layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
101
+ "layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
102
+ "layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
103
+ "layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
104
+ "layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
105
+ "layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
106
+ "layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
107
+ "layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
108
+ "layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
109
+ "layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
110
+ "layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
111
+ "layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
112
+ "layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
113
+ "layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
114
+ "layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
115
+ "layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
116
+ "layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
117
+ "layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
118
+ "layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
119
+ "layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
120
+ "layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
121
+ "layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
122
+ "layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
123
+ "layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
124
+ "layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
125
+ "layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
126
+ "layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
127
+ "layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
128
+ "layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
129
+ "layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
130
+ "layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
131
+ "layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
132
+ "layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
133
+ "layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
134
+ "layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
135
+ "layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
136
+ "layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
137
+ "layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
138
+ "layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
139
+ "layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
140
+ "layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
141
+ "layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
142
+ "layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
143
+ "layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
144
+ "layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
145
+ "layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
146
+ "layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
147
+ "layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
148
+ "layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
149
+ "layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
150
+ "layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
151
+ "layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
152
+ "layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
153
+ "layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
154
+ "layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
155
+ "layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
156
+ "layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
157
+ "layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
158
+ "layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
159
+ "layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
160
+ "layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
161
+ "layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
162
+ "layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
163
+ "layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
164
+ "layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
165
+ "layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
166
+ "layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
167
+ "layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
168
+ "layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
169
+ "layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
170
+ "layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
171
+ "layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
172
+ "layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
173
+ "layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
174
+ "layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
175
+ "layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
176
+ "layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
177
+ "layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
178
+ "layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
179
+ "layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
180
+ "layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
181
+ "layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
182
+ "layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
183
+ "layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
184
+ "layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
185
+ "layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
186
+ "layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
187
+ "layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
188
+ "layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
189
+ "layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
190
+ "layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
191
+ "layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
192
+ "layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
193
+ "layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
194
+ "layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
195
+ "layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
196
+ "layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
197
+ "layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
198
+ "layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
199
+ "layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
200
+ "layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
201
+ "layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
202
+ "layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
203
+ "layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
204
+ "layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
205
+ "layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
206
+ "layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
207
+ "layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
208
+ "layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
209
+ "layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
210
+ "layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
211
+ "layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
212
+ "layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
213
+ "layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
214
+ "layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
215
+ "layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
216
+ "layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
217
+ "layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
218
+ "layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
219
+ "layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
220
+ "layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
221
+ "layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
222
+ "layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
223
+ "layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
224
+ "layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
225
+ "layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
226
+ "layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
227
+ "layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
228
+ "layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
229
+ "layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
230
+ "layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
231
+ "layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
232
+ "layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
233
+ "layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
234
+ "layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
235
+ "layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
236
+ "layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
237
+ "layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
238
+ "layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
239
+ "layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
240
+ "layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
241
+ "layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
242
+ "layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
243
+ "layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
244
+ "layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
245
+ "layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
246
+ "layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
247
+ "layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
248
+ "layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
249
+ "layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
250
+ "layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
251
+ "layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
252
+ "layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
253
+ "layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
254
+ "layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
255
+ "layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
256
+ "layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
257
+ "layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
258
+ "layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
259
+ "layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
260
+ "norm.weight": "model-00002-of-00002.safetensors"
261
+ }
262
+ }
tokenizer.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:881912e4e7f25194ad8e82fd5f12292ba4a70376303f608aa270fb4bde3bc9b7
3
- size 17210189
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
3
+ size 17209920
tokenizer_config.json CHANGED
@@ -2057,7 +2057,7 @@
2057
  "input_ids",
2058
  "attention_mask"
2059
  ],
2060
- "model_max_length": 1024,
2061
  "pad_token": "<|end_of_text|>",
2062
  "tokenizer_class": "PreTrainedTokenizerFast"
2063
  }
 
2057
  "input_ids",
2058
  "attention_mask"
2059
  ],
2060
+ "model_max_length": 131072,
2061
  "pad_token": "<|end_of_text|>",
2062
  "tokenizer_class": "PreTrainedTokenizerFast"
2063
  }