Hugging Face's logo

PepPixie
/
DuskiBlipQuantized

PEFT
Safetensors
Model card Files
xet
Community
Isabelle Cudlitz commited on
Commit
a237df4
·
1 Parent(s): 0f39ee6

Upload model

Browse files
Files changed (2) hide show
  1. README.md +0 -2850
  2. adapter_model.safetensors +1 -1
README.md CHANGED
@@ -217,2854 +217,4 @@ The following `bitsandbytes` quantization config was used during training:
217
  ### Framework versions
218
 
219
 
220
- - PEFT 0.6.2
221
- ## Training procedure
222
-
223
-
224
- The following `bitsandbytes` quantization config was used during training:
225
- - quant_method: bitsandbytes
226
- - load_in_8bit: True
227
- - load_in_4bit: False
228
- - llm_int8_threshold: 6.0
229
- - llm_int8_skip_modules: None
230
- - llm_int8_enable_fp32_cpu_offload: False
231
- - llm_int8_has_fp16_weight: False
232
- - bnb_4bit_quant_type: fp4
233
- - bnb_4bit_use_double_quant: False
234
- - bnb_4bit_compute_dtype: float32
235
-
236
- ### Framework versions
237
-
238
-
239
- - PEFT 0.6.2
240
- ## Training procedure
241
-
242
-
243
- The following `bitsandbytes` quantization config was used during training:
244
- - quant_method: bitsandbytes
245
- - load_in_8bit: True
246
- - load_in_4bit: False
247
- - llm_int8_threshold: 6.0
248
- - llm_int8_skip_modules: None
249
- - llm_int8_enable_fp32_cpu_offload: False
250
- - llm_int8_has_fp16_weight: False
251
- - bnb_4bit_quant_type: fp4
252
- - bnb_4bit_use_double_quant: False
253
- - bnb_4bit_compute_dtype: float32
254
-
255
- ### Framework versions
256
-
257
-
258
- - PEFT 0.6.2
259
- ## Training procedure
260
-
261
-
262
- The following `bitsandbytes` quantization config was used during training:
263
- - quant_method: bitsandbytes
264
- - load_in_8bit: True
265
- - load_in_4bit: False
266
- - llm_int8_threshold: 6.0
267
- - llm_int8_skip_modules: None
268
- - llm_int8_enable_fp32_cpu_offload: False
269
- - llm_int8_has_fp16_weight: False
270
- - bnb_4bit_quant_type: fp4
271
- - bnb_4bit_use_double_quant: False
272
- - bnb_4bit_compute_dtype: float32
273
-
274
- ### Framework versions
275
-
276
-
277
- - PEFT 0.6.2
278
- ## Training procedure
279
-
280
-
281
- The following `bitsandbytes` quantization config was used during training:
282
- - quant_method: bitsandbytes
283
- - load_in_8bit: True
284
- - load_in_4bit: False
285
- - llm_int8_threshold: 6.0
286
- - llm_int8_skip_modules: None
287
- - llm_int8_enable_fp32_cpu_offload: False
288
- - llm_int8_has_fp16_weight: False
289
- - bnb_4bit_quant_type: fp4
290
- - bnb_4bit_use_double_quant: False
291
- - bnb_4bit_compute_dtype: float32
292
-
293
- ### Framework versions
294
-
295
-
296
- - PEFT 0.6.2
297
- ## Training procedure
298
-
299
-
300
- The following `bitsandbytes` quantization config was used during training:
301
- - quant_method: bitsandbytes
302
- - load_in_8bit: True
303
- - load_in_4bit: False
304
- - llm_int8_threshold: 6.0
305
- - llm_int8_skip_modules: None
306
- - llm_int8_enable_fp32_cpu_offload: False
307
- - llm_int8_has_fp16_weight: False
308
- - bnb_4bit_quant_type: fp4
309
- - bnb_4bit_use_double_quant: False
310
- - bnb_4bit_compute_dtype: float32
311
-
312
- ### Framework versions
313
-
314
-
315
- - PEFT 0.6.2
316
- ## Training procedure
317
-
318
-
319
- The following `bitsandbytes` quantization config was used during training:
320
- - quant_method: bitsandbytes
321
- - load_in_8bit: True
322
- - load_in_4bit: False
323
- - llm_int8_threshold: 6.0
324
- - llm_int8_skip_modules: None
325
- - llm_int8_enable_fp32_cpu_offload: False
326
- - llm_int8_has_fp16_weight: False
327
- - bnb_4bit_quant_type: fp4
328
- - bnb_4bit_use_double_quant: False
329
- - bnb_4bit_compute_dtype: float32
330
-
331
- ### Framework versions
332
-
333
-
334
- - PEFT 0.6.2
335
- ## Training procedure
336
-
337
-
338
- The following `bitsandbytes` quantization config was used during training:
339
- - quant_method: bitsandbytes
340
- - load_in_8bit: True
341
- - load_in_4bit: False
342
- - llm_int8_threshold: 6.0
343
- - llm_int8_skip_modules: None
344
- - llm_int8_enable_fp32_cpu_offload: False
345
- - llm_int8_has_fp16_weight: False
346
- - bnb_4bit_quant_type: fp4
347
- - bnb_4bit_use_double_quant: False
348
- - bnb_4bit_compute_dtype: float32
349
-
350
- ### Framework versions
351
-
352
-
353
- - PEFT 0.6.2
354
- ## Training procedure
355
-
356
-
357
- The following `bitsandbytes` quantization config was used during training:
358
- - quant_method: bitsandbytes
359
- - load_in_8bit: True
360
- - load_in_4bit: False
361
- - llm_int8_threshold: 6.0
362
- - llm_int8_skip_modules: None
363
- - llm_int8_enable_fp32_cpu_offload: False
364
- - llm_int8_has_fp16_weight: False
365
- - bnb_4bit_quant_type: fp4
366
- - bnb_4bit_use_double_quant: False
367
- - bnb_4bit_compute_dtype: float32
368
-
369
- ### Framework versions
370
-
371
-
372
- - PEFT 0.6.2
373
- ## Training procedure
374
-
375
-
376
- The following `bitsandbytes` quantization config was used during training:
377
- - quant_method: bitsandbytes
378
- - load_in_8bit: True
379
- - load_in_4bit: False
380
- - llm_int8_threshold: 6.0
381
- - llm_int8_skip_modules: None
382
- - llm_int8_enable_fp32_cpu_offload: False
383
- - llm_int8_has_fp16_weight: False
384
- - bnb_4bit_quant_type: fp4
385
- - bnb_4bit_use_double_quant: False
386
- - bnb_4bit_compute_dtype: float32
387
-
388
- ### Framework versions
389
-
390
-
391
- - PEFT 0.6.2
392
- ## Training procedure
393
-
394
-
395
- The following `bitsandbytes` quantization config was used during training:
396
- - quant_method: bitsandbytes
397
- - load_in_8bit: True
398
- - load_in_4bit: False
399
- - llm_int8_threshold: 6.0
400
- - llm_int8_skip_modules: None
401
- - llm_int8_enable_fp32_cpu_offload: False
402
- - llm_int8_has_fp16_weight: False
403
- - bnb_4bit_quant_type: fp4
404
- - bnb_4bit_use_double_quant: False
405
- - bnb_4bit_compute_dtype: float32
406
-
407
- ### Framework versions
408
-
409
-
410
- - PEFT 0.6.2
411
- ## Training procedure
412
-
413
-
414
- The following `bitsandbytes` quantization config was used during training:
415
- - quant_method: bitsandbytes
416
- - load_in_8bit: True
417
- - load_in_4bit: False
418
- - llm_int8_threshold: 6.0
419
- - llm_int8_skip_modules: None
420
- - llm_int8_enable_fp32_cpu_offload: False
421
- - llm_int8_has_fp16_weight: False
422
- - bnb_4bit_quant_type: fp4
423
- - bnb_4bit_use_double_quant: False
424
- - bnb_4bit_compute_dtype: float32
425
-
426
- ### Framework versions
427
-
428
-
429
- - PEFT 0.6.2
430
- ## Training procedure
431
-
432
-
433
- The following `bitsandbytes` quantization config was used during training:
434
- - quant_method: bitsandbytes
435
- - load_in_8bit: True
436
- - load_in_4bit: False
437
- - llm_int8_threshold: 6.0
438
- - llm_int8_skip_modules: None
439
- - llm_int8_enable_fp32_cpu_offload: False
440
- - llm_int8_has_fp16_weight: False
441
- - bnb_4bit_quant_type: fp4
442
- - bnb_4bit_use_double_quant: False
443
- - bnb_4bit_compute_dtype: float32
444
-
445
- ### Framework versions
446
-
447
-
448
- - PEFT 0.6.2
449
- ## Training procedure
450
-
451
-
452
- The following `bitsandbytes` quantization config was used during training:
453
- - quant_method: bitsandbytes
454
- - load_in_8bit: True
455
- - load_in_4bit: False
456
- - llm_int8_threshold: 6.0
457
- - llm_int8_skip_modules: None
458
- - llm_int8_enable_fp32_cpu_offload: False
459
- - llm_int8_has_fp16_weight: False
460
- - bnb_4bit_quant_type: fp4
461
- - bnb_4bit_use_double_quant: False
462
- - bnb_4bit_compute_dtype: float32
463
-
464
- ### Framework versions
465
-
466
-
467
- - PEFT 0.6.2
468
- ## Training procedure
469
-
470
-
471
- The following `bitsandbytes` quantization config was used during training:
472
- - quant_method: bitsandbytes
473
- - load_in_8bit: True
474
- - load_in_4bit: False
475
- - llm_int8_threshold: 6.0
476
- - llm_int8_skip_modules: None
477
- - llm_int8_enable_fp32_cpu_offload: False
478
- - llm_int8_has_fp16_weight: False
479
- - bnb_4bit_quant_type: fp4
480
- - bnb_4bit_use_double_quant: False
481
- - bnb_4bit_compute_dtype: float32
482
-
483
- ### Framework versions
484
-
485
-
486
- - PEFT 0.6.2
487
- ## Training procedure
488
-
489
-
490
- The following `bitsandbytes` quantization config was used during training:
491
- - quant_method: bitsandbytes
492
- - load_in_8bit: True
493
- - load_in_4bit: False
494
- - llm_int8_threshold: 6.0
495
- - llm_int8_skip_modules: None
496
- - llm_int8_enable_fp32_cpu_offload: False
497
- - llm_int8_has_fp16_weight: False
498
- - bnb_4bit_quant_type: fp4
499
- - bnb_4bit_use_double_quant: False
500
- - bnb_4bit_compute_dtype: float32
501
-
502
- ### Framework versions
503
-
504
-
505
- - PEFT 0.6.2
506
- ## Training procedure
507
-
508
-
509
- The following `bitsandbytes` quantization config was used during training:
510
- - quant_method: bitsandbytes
511
- - load_in_8bit: True
512
- - load_in_4bit: False
513
- - llm_int8_threshold: 6.0
514
- - llm_int8_skip_modules: None
515
- - llm_int8_enable_fp32_cpu_offload: False
516
- - llm_int8_has_fp16_weight: False
517
- - bnb_4bit_quant_type: fp4
518
- - bnb_4bit_use_double_quant: False
519
- - bnb_4bit_compute_dtype: float32
520
-
521
- ### Framework versions
522
-
523
-
524
- - PEFT 0.6.2
525
- ## Training procedure
526
-
527
-
528
- The following `bitsandbytes` quantization config was used during training:
529
- - quant_method: bitsandbytes
530
- - load_in_8bit: True
531
- - load_in_4bit: False
532
- - llm_int8_threshold: 6.0
533
- - llm_int8_skip_modules: None
534
- - llm_int8_enable_fp32_cpu_offload: False
535
- - llm_int8_has_fp16_weight: False
536
- - bnb_4bit_quant_type: fp4
537
- - bnb_4bit_use_double_quant: False
538
- - bnb_4bit_compute_dtype: float32
539
-
540
- ### Framework versions
541
-
542
-
543
- - PEFT 0.6.2
544
- ## Training procedure
545
-
546
-
547
- The following `bitsandbytes` quantization config was used during training:
548
- - quant_method: bitsandbytes
549
- - load_in_8bit: True
550
- - load_in_4bit: False
551
- - llm_int8_threshold: 6.0
552
- - llm_int8_skip_modules: None
553
- - llm_int8_enable_fp32_cpu_offload: False
554
- - llm_int8_has_fp16_weight: False
555
- - bnb_4bit_quant_type: fp4
556
- - bnb_4bit_use_double_quant: False
557
- - bnb_4bit_compute_dtype: float32
558
-
559
- ### Framework versions
560
-
561
-
562
- - PEFT 0.6.2
563
- ## Training procedure
564
-
565
-
566
- The following `bitsandbytes` quantization config was used during training:
567
- - quant_method: bitsandbytes
568
- - load_in_8bit: True
569
- - load_in_4bit: False
570
- - llm_int8_threshold: 6.0
571
- - llm_int8_skip_modules: None
572
- - llm_int8_enable_fp32_cpu_offload: False
573
- - llm_int8_has_fp16_weight: False
574
- - bnb_4bit_quant_type: fp4
575
- - bnb_4bit_use_double_quant: False
576
- - bnb_4bit_compute_dtype: float32
577
-
578
- ### Framework versions
579
-
580
-
581
- - PEFT 0.6.2
582
- ## Training procedure
583
-
584
-
585
- The following `bitsandbytes` quantization config was used during training:
586
- - quant_method: bitsandbytes
587
- - load_in_8bit: True
588
- - load_in_4bit: False
589
- - llm_int8_threshold: 6.0
590
- - llm_int8_skip_modules: None
591
- - llm_int8_enable_fp32_cpu_offload: False
592
- - llm_int8_has_fp16_weight: False
593
- - bnb_4bit_quant_type: fp4
594
- - bnb_4bit_use_double_quant: False
595
- - bnb_4bit_compute_dtype: float32
596
-
597
- ### Framework versions
598
-
599
-
600
- - PEFT 0.6.2
601
- ## Training procedure
602
-
603
-
604
- The following `bitsandbytes` quantization config was used during training:
605
- - quant_method: bitsandbytes
606
- - load_in_8bit: True
607
- - load_in_4bit: False
608
- - llm_int8_threshold: 6.0
609
- - llm_int8_skip_modules: None
610
- - llm_int8_enable_fp32_cpu_offload: False
611
- - llm_int8_has_fp16_weight: False
612
- - bnb_4bit_quant_type: fp4
613
- - bnb_4bit_use_double_quant: False
614
- - bnb_4bit_compute_dtype: float32
615
-
616
- ### Framework versions
617
-
618
-
619
- - PEFT 0.6.2
620
- ## Training procedure
621
-
622
-
623
- The following `bitsandbytes` quantization config was used during training:
624
- - quant_method: bitsandbytes
625
- - load_in_8bit: True
626
- - load_in_4bit: False
627
- - llm_int8_threshold: 6.0
628
- - llm_int8_skip_modules: None
629
- - llm_int8_enable_fp32_cpu_offload: False
630
- - llm_int8_has_fp16_weight: False
631
- - bnb_4bit_quant_type: fp4
632
- - bnb_4bit_use_double_quant: False
633
- - bnb_4bit_compute_dtype: float32
634
-
635
- ### Framework versions
636
-
637
-
638
- - PEFT 0.6.2
639
- ## Training procedure
640
-
641
-
642
- The following `bitsandbytes` quantization config was used during training:
643
- - quant_method: bitsandbytes
644
- - load_in_8bit: True
645
- - load_in_4bit: False
646
- - llm_int8_threshold: 6.0
647
- - llm_int8_skip_modules: None
648
- - llm_int8_enable_fp32_cpu_offload: False
649
- - llm_int8_has_fp16_weight: False
650
- - bnb_4bit_quant_type: fp4
651
- - bnb_4bit_use_double_quant: False
652
- - bnb_4bit_compute_dtype: float32
653
-
654
- ### Framework versions
655
-
656
-
657
- - PEFT 0.6.2
658
- ## Training procedure
659
-
660
-
661
- The following `bitsandbytes` quantization config was used during training:
662
- - quant_method: bitsandbytes
663
- - load_in_8bit: True
664
- - load_in_4bit: False
665
- - llm_int8_threshold: 6.0
666
- - llm_int8_skip_modules: None
667
- - llm_int8_enable_fp32_cpu_offload: False
668
- - llm_int8_has_fp16_weight: False
669
- - bnb_4bit_quant_type: fp4
670
- - bnb_4bit_use_double_quant: False
671
- - bnb_4bit_compute_dtype: float32
672
-
673
- ### Framework versions
674
-
675
-
676
- - PEFT 0.6.2
677
- ## Training procedure
678
-
679
-
680
- The following `bitsandbytes` quantization config was used during training:
681
- - quant_method: bitsandbytes
682
- - load_in_8bit: True
683
- - load_in_4bit: False
684
- - llm_int8_threshold: 6.0
685
- - llm_int8_skip_modules: None
686
- - llm_int8_enable_fp32_cpu_offload: False
687
- - llm_int8_has_fp16_weight: False
688
- - bnb_4bit_quant_type: fp4
689
- - bnb_4bit_use_double_quant: False
690
- - bnb_4bit_compute_dtype: float32
691
-
692
- ### Framework versions
693
-
694
-
695
- - PEFT 0.6.2
696
- ## Training procedure
697
-
698
-
699
- The following `bitsandbytes` quantization config was used during training:
700
- - quant_method: bitsandbytes
701
- - load_in_8bit: True
702
- - load_in_4bit: False
703
- - llm_int8_threshold: 6.0
704
- - llm_int8_skip_modules: None
705
- - llm_int8_enable_fp32_cpu_offload: False
706
- - llm_int8_has_fp16_weight: False
707
- - bnb_4bit_quant_type: fp4
708
- - bnb_4bit_use_double_quant: False
709
- - bnb_4bit_compute_dtype: float32
710
-
711
- ### Framework versions
712
-
713
-
714
- - PEFT 0.6.2
715
- ## Training procedure
716
-
717
-
718
- The following `bitsandbytes` quantization config was used during training:
719
- - quant_method: bitsandbytes
720
- - load_in_8bit: True
721
- - load_in_4bit: False
722
- - llm_int8_threshold: 6.0
723
- - llm_int8_skip_modules: None
724
- - llm_int8_enable_fp32_cpu_offload: False
725
- - llm_int8_has_fp16_weight: False
726
- - bnb_4bit_quant_type: fp4
727
- - bnb_4bit_use_double_quant: False
728
- - bnb_4bit_compute_dtype: float32
729
-
730
- ### Framework versions
731
-
732
-
733
- - PEFT 0.6.2
734
- ## Training procedure
735
-
736
-
737
- The following `bitsandbytes` quantization config was used during training:
738
- - quant_method: bitsandbytes
739
- - load_in_8bit: True
740
- - load_in_4bit: False
741
- - llm_int8_threshold: 6.0
742
- - llm_int8_skip_modules: None
743
- - llm_int8_enable_fp32_cpu_offload: False
744
- - llm_int8_has_fp16_weight: False
745
- - bnb_4bit_quant_type: fp4
746
- - bnb_4bit_use_double_quant: False
747
- - bnb_4bit_compute_dtype: float32
748
-
749
- ### Framework versions
750
-
751
-
752
- - PEFT 0.6.2
753
- ## Training procedure
754
-
755
-
756
- The following `bitsandbytes` quantization config was used during training:
757
- - quant_method: bitsandbytes
758
- - load_in_8bit: True
759
- - load_in_4bit: False
760
- - llm_int8_threshold: 6.0
761
- - llm_int8_skip_modules: None
762
- - llm_int8_enable_fp32_cpu_offload: False
763
- - llm_int8_has_fp16_weight: False
764
- - bnb_4bit_quant_type: fp4
765
- - bnb_4bit_use_double_quant: False
766
- - bnb_4bit_compute_dtype: float32
767
-
768
- ### Framework versions
769
-
770
-
771
- - PEFT 0.6.2
772
- ## Training procedure
773
-
774
-
775
- The following `bitsandbytes` quantization config was used during training:
776
- - quant_method: bitsandbytes
777
- - load_in_8bit: True
778
- - load_in_4bit: False
779
- - llm_int8_threshold: 6.0
780
- - llm_int8_skip_modules: None
781
- - llm_int8_enable_fp32_cpu_offload: False
782
- - llm_int8_has_fp16_weight: False
783
- - bnb_4bit_quant_type: fp4
784
- - bnb_4bit_use_double_quant: False
785
- - bnb_4bit_compute_dtype: float32
786
-
787
- ### Framework versions
788
-
789
-
790
- - PEFT 0.6.2
791
- ## Training procedure
792
-
793
-
794
- The following `bitsandbytes` quantization config was used during training:
795
- - quant_method: bitsandbytes
796
- - load_in_8bit: True
797
- - load_in_4bit: False
798
- - llm_int8_threshold: 6.0
799
- - llm_int8_skip_modules: None
800
- - llm_int8_enable_fp32_cpu_offload: False
801
- - llm_int8_has_fp16_weight: False
802
- - bnb_4bit_quant_type: fp4
803
- - bnb_4bit_use_double_quant: False
804
- - bnb_4bit_compute_dtype: float32
805
-
806
- ### Framework versions
807
-
808
-
809
- - PEFT 0.6.2
810
- ## Training procedure
811
-
812
-
813
- The following `bitsandbytes` quantization config was used during training:
814
- - quant_method: bitsandbytes
815
- - load_in_8bit: True
816
- - load_in_4bit: False
817
- - llm_int8_threshold: 6.0
818
- - llm_int8_skip_modules: None
819
- - llm_int8_enable_fp32_cpu_offload: False
820
- - llm_int8_has_fp16_weight: False
821
- - bnb_4bit_quant_type: fp4
822
- - bnb_4bit_use_double_quant: False
823
- - bnb_4bit_compute_dtype: float32
824
-
825
- ### Framework versions
826
-
827
-
828
- - PEFT 0.6.2
829
- ## Training procedure
830
-
831
-
832
- The following `bitsandbytes` quantization config was used during training:
833
- - quant_method: bitsandbytes
834
- - load_in_8bit: True
835
- - load_in_4bit: False
836
- - llm_int8_threshold: 6.0
837
- - llm_int8_skip_modules: None
838
- - llm_int8_enable_fp32_cpu_offload: False
839
- - llm_int8_has_fp16_weight: False
840
- - bnb_4bit_quant_type: fp4
841
- - bnb_4bit_use_double_quant: False
842
- - bnb_4bit_compute_dtype: float32
843
-
844
- ### Framework versions
845
-
846
-
847
- - PEFT 0.6.2
848
- ## Training procedure
849
-
850
-
851
- The following `bitsandbytes` quantization config was used during training:
852
- - quant_method: bitsandbytes
853
- - load_in_8bit: True
854
- - load_in_4bit: False
855
- - llm_int8_threshold: 6.0
856
- - llm_int8_skip_modules: None
857
- - llm_int8_enable_fp32_cpu_offload: False
858
- - llm_int8_has_fp16_weight: False
859
- - bnb_4bit_quant_type: fp4
860
- - bnb_4bit_use_double_quant: False
861
- - bnb_4bit_compute_dtype: float32
862
-
863
- ### Framework versions
864
-
865
-
866
- - PEFT 0.6.2
867
- ## Training procedure
868
-
869
-
870
- The following `bitsandbytes` quantization config was used during training:
871
- - quant_method: bitsandbytes
872
- - load_in_8bit: True
873
- - load_in_4bit: False
874
- - llm_int8_threshold: 6.0
875
- - llm_int8_skip_modules: None
876
- - llm_int8_enable_fp32_cpu_offload: False
877
- - llm_int8_has_fp16_weight: False
878
- - bnb_4bit_quant_type: fp4
879
- - bnb_4bit_use_double_quant: False
880
- - bnb_4bit_compute_dtype: float32
881
-
882
- ### Framework versions
883
-
884
-
885
- - PEFT 0.6.2
886
- ## Training procedure
887
-
888
-
889
- The following `bitsandbytes` quantization config was used during training:
890
- - quant_method: bitsandbytes
891
- - load_in_8bit: True
892
- - load_in_4bit: False
893
- - llm_int8_threshold: 6.0
894
- - llm_int8_skip_modules: None
895
- - llm_int8_enable_fp32_cpu_offload: False
896
- - llm_int8_has_fp16_weight: False
897
- - bnb_4bit_quant_type: fp4
898
- - bnb_4bit_use_double_quant: False
899
- - bnb_4bit_compute_dtype: float32
900
-
901
- ### Framework versions
902
-
903
-
904
- - PEFT 0.6.2
905
- ## Training procedure
906
-
907
-
908
- The following `bitsandbytes` quantization config was used during training:
909
- - quant_method: bitsandbytes
910
- - load_in_8bit: True
911
- - load_in_4bit: False
912
- - llm_int8_threshold: 6.0
913
- - llm_int8_skip_modules: None
914
- - llm_int8_enable_fp32_cpu_offload: False
915
- - llm_int8_has_fp16_weight: False
916
- - bnb_4bit_quant_type: fp4
917
- - bnb_4bit_use_double_quant: False
918
- - bnb_4bit_compute_dtype: float32
919
-
920
- ### Framework versions
921
-
922
-
923
- - PEFT 0.6.2
924
- ## Training procedure
925
-
926
-
927
- The following `bitsandbytes` quantization config was used during training:
928
- - quant_method: bitsandbytes
929
- - load_in_8bit: True
930
- - load_in_4bit: False
931
- - llm_int8_threshold: 6.0
932
- - llm_int8_skip_modules: None
933
- - llm_int8_enable_fp32_cpu_offload: False
934
- - llm_int8_has_fp16_weight: False
935
- - bnb_4bit_quant_type: fp4
936
- - bnb_4bit_use_double_quant: False
937
- - bnb_4bit_compute_dtype: float32
938
-
939
- ### Framework versions
940
-
941
-
942
- - PEFT 0.6.2
943
- ## Training procedure
944
-
945
-
946
- The following `bitsandbytes` quantization config was used during training:
947
- - quant_method: bitsandbytes
948
- - load_in_8bit: True
949
- - load_in_4bit: False
950
- - llm_int8_threshold: 6.0
951
- - llm_int8_skip_modules: None
952
- - llm_int8_enable_fp32_cpu_offload: False
953
- - llm_int8_has_fp16_weight: False
954
- - bnb_4bit_quant_type: fp4
955
- - bnb_4bit_use_double_quant: False
956
- - bnb_4bit_compute_dtype: float32
957
-
958
- ### Framework versions
959
-
960
-
961
- - PEFT 0.6.2
962
- ## Training procedure
963
-
964
-
965
- The following `bitsandbytes` quantization config was used during training:
966
- - quant_method: bitsandbytes
967
- - load_in_8bit: True
968
- - load_in_4bit: False
969
- - llm_int8_threshold: 6.0
970
- - llm_int8_skip_modules: None
971
- - llm_int8_enable_fp32_cpu_offload: False
972
- - llm_int8_has_fp16_weight: False
973
- - bnb_4bit_quant_type: fp4
974
- - bnb_4bit_use_double_quant: False
975
- - bnb_4bit_compute_dtype: float32
976
-
977
- ### Framework versions
978
-
979
-
980
- - PEFT 0.6.2
981
- ## Training procedure
982
-
983
-
984
- The following `bitsandbytes` quantization config was used during training:
985
- - quant_method: bitsandbytes
986
- - load_in_8bit: True
987
- - load_in_4bit: False
988
- - llm_int8_threshold: 6.0
989
- - llm_int8_skip_modules: None
990
- - llm_int8_enable_fp32_cpu_offload: False
991
- - llm_int8_has_fp16_weight: False
992
- - bnb_4bit_quant_type: fp4
993
- - bnb_4bit_use_double_quant: False
994
- - bnb_4bit_compute_dtype: float32
995
-
996
- ### Framework versions
997
-
998
-
999
- - PEFT 0.6.2
1000
- ## Training procedure
1001
-
1002
-
1003
- The following `bitsandbytes` quantization config was used during training:
1004
- - quant_method: bitsandbytes
1005
- - load_in_8bit: True
1006
- - load_in_4bit: False
1007
- - llm_int8_threshold: 6.0
1008
- - llm_int8_skip_modules: None
1009
- - llm_int8_enable_fp32_cpu_offload: False
1010
- - llm_int8_has_fp16_weight: False
1011
- - bnb_4bit_quant_type: fp4
1012
- - bnb_4bit_use_double_quant: False
1013
- - bnb_4bit_compute_dtype: float32
1014
-
1015
- ### Framework versions
1016
-
1017
-
1018
- - PEFT 0.6.2
1019
- ## Training procedure
1020
-
1021
-
1022
- The following `bitsandbytes` quantization config was used during training:
1023
- - quant_method: bitsandbytes
1024
- - load_in_8bit: True
1025
- - load_in_4bit: False
1026
- - llm_int8_threshold: 6.0
1027
- - llm_int8_skip_modules: None
1028
- - llm_int8_enable_fp32_cpu_offload: False
1029
- - llm_int8_has_fp16_weight: False
1030
- - bnb_4bit_quant_type: fp4
1031
- - bnb_4bit_use_double_quant: False
1032
- - bnb_4bit_compute_dtype: float32
1033
-
1034
- ### Framework versions
1035
-
1036
-
1037
- - PEFT 0.6.2
1038
- ## Training procedure
1039
-
1040
-
1041
- The following `bitsandbytes` quantization config was used during training:
1042
- - quant_method: bitsandbytes
1043
- - load_in_8bit: True
1044
- - load_in_4bit: False
1045
- - llm_int8_threshold: 6.0
1046
- - llm_int8_skip_modules: None
1047
- - llm_int8_enable_fp32_cpu_offload: False
1048
- - llm_int8_has_fp16_weight: False
1049
- - bnb_4bit_quant_type: fp4
1050
- - bnb_4bit_use_double_quant: False
1051
- - bnb_4bit_compute_dtype: float32
1052
-
1053
- ### Framework versions
1054
-
1055
-
1056
- - PEFT 0.6.2
1057
- ## Training procedure
1058
-
1059
-
1060
- The following `bitsandbytes` quantization config was used during training:
1061
- - quant_method: bitsandbytes
1062
- - load_in_8bit: True
1063
- - load_in_4bit: False
1064
- - llm_int8_threshold: 6.0
1065
- - llm_int8_skip_modules: None
1066
- - llm_int8_enable_fp32_cpu_offload: False
1067
- - llm_int8_has_fp16_weight: False
1068
- - bnb_4bit_quant_type: fp4
1069
- - bnb_4bit_use_double_quant: False
1070
- - bnb_4bit_compute_dtype: float32
1071
-
1072
- ### Framework versions
1073
-
1074
-
1075
- - PEFT 0.6.2
1076
- ## Training procedure
1077
-
1078
-
1079
- The following `bitsandbytes` quantization config was used during training:
1080
- - quant_method: bitsandbytes
1081
- - load_in_8bit: True
1082
- - load_in_4bit: False
1083
- - llm_int8_threshold: 6.0
1084
- - llm_int8_skip_modules: None
1085
- - llm_int8_enable_fp32_cpu_offload: False
1086
- - llm_int8_has_fp16_weight: False
1087
- - bnb_4bit_quant_type: fp4
1088
- - bnb_4bit_use_double_quant: False
1089
- - bnb_4bit_compute_dtype: float32
1090
-
1091
- ### Framework versions
1092
-
1093
-
1094
- - PEFT 0.6.2
1095
- ## Training procedure
1096
-
1097
-
1098
- The following `bitsandbytes` quantization config was used during training:
1099
- - quant_method: bitsandbytes
1100
- - load_in_8bit: True
1101
- - load_in_4bit: False
1102
- - llm_int8_threshold: 6.0
1103
- - llm_int8_skip_modules: None
1104
- - llm_int8_enable_fp32_cpu_offload: False
1105
- - llm_int8_has_fp16_weight: False
1106
- - bnb_4bit_quant_type: fp4
1107
- - bnb_4bit_use_double_quant: False
1108
- - bnb_4bit_compute_dtype: float32
1109
-
1110
- ### Framework versions
1111
-
1112
-
1113
- - PEFT 0.6.2
1114
- ## Training procedure
1115
-
1116
-
1117
- The following `bitsandbytes` quantization config was used during training:
1118
- - quant_method: bitsandbytes
1119
- - load_in_8bit: True
1120
- - load_in_4bit: False
1121
- - llm_int8_threshold: 6.0
1122
- - llm_int8_skip_modules: None
1123
- - llm_int8_enable_fp32_cpu_offload: False
1124
- - llm_int8_has_fp16_weight: False
1125
- - bnb_4bit_quant_type: fp4
1126
- - bnb_4bit_use_double_quant: False
1127
- - bnb_4bit_compute_dtype: float32
1128
-
1129
- ### Framework versions
1130
-
1131
-
1132
- - PEFT 0.6.2
1133
- ## Training procedure
1134
-
1135
-
1136
- The following `bitsandbytes` quantization config was used during training:
1137
- - quant_method: bitsandbytes
1138
- - load_in_8bit: True
1139
- - load_in_4bit: False
1140
- - llm_int8_threshold: 6.0
1141
- - llm_int8_skip_modules: None
1142
- - llm_int8_enable_fp32_cpu_offload: False
1143
- - llm_int8_has_fp16_weight: False
1144
- - bnb_4bit_quant_type: fp4
1145
- - bnb_4bit_use_double_quant: False
1146
- - bnb_4bit_compute_dtype: float32
1147
-
1148
- ### Framework versions
1149
-
1150
-
1151
- - PEFT 0.6.2
1152
- ## Training procedure
1153
-
1154
-
1155
- The following `bitsandbytes` quantization config was used during training:
1156
- - quant_method: bitsandbytes
1157
- - load_in_8bit: True
1158
- - load_in_4bit: False
1159
- - llm_int8_threshold: 6.0
1160
- - llm_int8_skip_modules: None
1161
- - llm_int8_enable_fp32_cpu_offload: False
1162
- - llm_int8_has_fp16_weight: False
1163
- - bnb_4bit_quant_type: fp4
1164
- - bnb_4bit_use_double_quant: False
1165
- - bnb_4bit_compute_dtype: float32
1166
-
1167
- ### Framework versions
1168
-
1169
-
1170
- - PEFT 0.6.2
1171
- ## Training procedure
1172
-
1173
-
1174
- The following `bitsandbytes` quantization config was used during training:
1175
- - quant_method: bitsandbytes
1176
- - load_in_8bit: True
1177
- - load_in_4bit: False
1178
- - llm_int8_threshold: 6.0
1179
- - llm_int8_skip_modules: None
1180
- - llm_int8_enable_fp32_cpu_offload: False
1181
- - llm_int8_has_fp16_weight: False
1182
- - bnb_4bit_quant_type: fp4
1183
- - bnb_4bit_use_double_quant: False
1184
- - bnb_4bit_compute_dtype: float32
1185
-
1186
- ### Framework versions
1187
-
1188
-
1189
- - PEFT 0.6.2
1190
- ## Training procedure
1191
-
1192
-
1193
- The following `bitsandbytes` quantization config was used during training:
1194
- - quant_method: bitsandbytes
1195
- - load_in_8bit: True
1196
- - load_in_4bit: False
1197
- - llm_int8_threshold: 6.0
1198
- - llm_int8_skip_modules: None
1199
- - llm_int8_enable_fp32_cpu_offload: False
1200
- - llm_int8_has_fp16_weight: False
1201
- - bnb_4bit_quant_type: fp4
1202
- - bnb_4bit_use_double_quant: False
1203
- - bnb_4bit_compute_dtype: float32
1204
-
1205
- ### Framework versions
1206
-
1207
-
1208
- - PEFT 0.6.2
1209
- ## Training procedure
1210
-
1211
-
1212
- The following `bitsandbytes` quantization config was used during training:
1213
- - quant_method: bitsandbytes
1214
- - load_in_8bit: True
1215
- - load_in_4bit: False
1216
- - llm_int8_threshold: 6.0
1217
- - llm_int8_skip_modules: None
1218
- - llm_int8_enable_fp32_cpu_offload: False
1219
- - llm_int8_has_fp16_weight: False
1220
- - bnb_4bit_quant_type: fp4
1221
- - bnb_4bit_use_double_quant: False
1222
- - bnb_4bit_compute_dtype: float32
1223
-
1224
- ### Framework versions
1225
-
1226
-
1227
- - PEFT 0.6.2
1228
- ## Training procedure
1229
-
1230
-
1231
- The following `bitsandbytes` quantization config was used during training:
1232
- - quant_method: bitsandbytes
1233
- - load_in_8bit: True
1234
- - load_in_4bit: False
1235
- - llm_int8_threshold: 6.0
1236
- - llm_int8_skip_modules: None
1237
- - llm_int8_enable_fp32_cpu_offload: False
1238
- - llm_int8_has_fp16_weight: False
1239
- - bnb_4bit_quant_type: fp4
1240
- - bnb_4bit_use_double_quant: False
1241
- - bnb_4bit_compute_dtype: float32
1242
-
1243
- ### Framework versions
1244
-
1245
-
1246
- - PEFT 0.6.2
1247
- ## Training procedure
1248
-
1249
-
1250
- The following `bitsandbytes` quantization config was used during training:
1251
- - quant_method: bitsandbytes
1252
- - load_in_8bit: True
1253
- - load_in_4bit: False
1254
- - llm_int8_threshold: 6.0
1255
- - llm_int8_skip_modules: None
1256
- - llm_int8_enable_fp32_cpu_offload: False
1257
- - llm_int8_has_fp16_weight: False
1258
- - bnb_4bit_quant_type: fp4
1259
- - bnb_4bit_use_double_quant: False
1260
- - bnb_4bit_compute_dtype: float32
1261
-
1262
- ### Framework versions
1263
-
1264
-
1265
- - PEFT 0.6.2
1266
- ## Training procedure
1267
-
1268
-
1269
- The following `bitsandbytes` quantization config was used during training:
1270
- - quant_method: bitsandbytes
1271
- - load_in_8bit: True
1272
- - load_in_4bit: False
1273
- - llm_int8_threshold: 6.0
1274
- - llm_int8_skip_modules: None
1275
- - llm_int8_enable_fp32_cpu_offload: False
1276
- - llm_int8_has_fp16_weight: False
1277
- - bnb_4bit_quant_type: fp4
1278
- - bnb_4bit_use_double_quant: False
1279
- - bnb_4bit_compute_dtype: float32
1280
-
1281
- ### Framework versions
1282
-
1283
-
1284
- - PEFT 0.6.2
1285
- ## Training procedure
1286
-
1287
-
1288
- The following `bitsandbytes` quantization config was used during training:
1289
- - quant_method: bitsandbytes
1290
- - load_in_8bit: True
1291
- - load_in_4bit: False
1292
- - llm_int8_threshold: 6.0
1293
- - llm_int8_skip_modules: None
1294
- - llm_int8_enable_fp32_cpu_offload: False
1295
- - llm_int8_has_fp16_weight: False
1296
- - bnb_4bit_quant_type: fp4
1297
- - bnb_4bit_use_double_quant: False
1298
- - bnb_4bit_compute_dtype: float32
1299
-
1300
- ### Framework versions
1301
-
1302
-
1303
- - PEFT 0.6.2
1304
- ## Training procedure
1305
-
1306
-
1307
- The following `bitsandbytes` quantization config was used during training:
1308
- - quant_method: bitsandbytes
1309
- - load_in_8bit: True
1310
- - load_in_4bit: False
1311
- - llm_int8_threshold: 6.0
1312
- - llm_int8_skip_modules: None
1313
- - llm_int8_enable_fp32_cpu_offload: False
1314
- - llm_int8_has_fp16_weight: False
1315
- - bnb_4bit_quant_type: fp4
1316
- - bnb_4bit_use_double_quant: False
1317
- - bnb_4bit_compute_dtype: float32
1318
-
1319
- ### Framework versions
1320
-
1321
-
1322
- - PEFT 0.6.2
1323
- ## Training procedure
1324
-
1325
-
1326
- The following `bitsandbytes` quantization config was used during training:
1327
- - quant_method: bitsandbytes
1328
- - load_in_8bit: True
1329
- - load_in_4bit: False
1330
- - llm_int8_threshold: 6.0
1331
- - llm_int8_skip_modules: None
1332
- - llm_int8_enable_fp32_cpu_offload: False
1333
- - llm_int8_has_fp16_weight: False
1334
- - bnb_4bit_quant_type: fp4
1335
- - bnb_4bit_use_double_quant: False
1336
- - bnb_4bit_compute_dtype: float32
1337
-
1338
- ### Framework versions
1339
-
1340
-
1341
- - PEFT 0.6.2
1342
- ## Training procedure
1343
-
1344
-
1345
- The following `bitsandbytes` quantization config was used during training:
1346
- - quant_method: bitsandbytes
1347
- - load_in_8bit: True
1348
- - load_in_4bit: False
1349
- - llm_int8_threshold: 6.0
1350
- - llm_int8_skip_modules: None
1351
- - llm_int8_enable_fp32_cpu_offload: False
1352
- - llm_int8_has_fp16_weight: False
1353
- - bnb_4bit_quant_type: fp4
1354
- - bnb_4bit_use_double_quant: False
1355
- - bnb_4bit_compute_dtype: float32
1356
-
1357
- ### Framework versions
1358
-
1359
-
1360
- - PEFT 0.6.2
1361
- ## Training procedure
1362
-
1363
-
1364
- The following `bitsandbytes` quantization config was used during training:
1365
- - quant_method: bitsandbytes
1366
- - load_in_8bit: True
1367
- - load_in_4bit: False
1368
- - llm_int8_threshold: 6.0
1369
- - llm_int8_skip_modules: None
1370
- - llm_int8_enable_fp32_cpu_offload: False
1371
- - llm_int8_has_fp16_weight: False
1372
- - bnb_4bit_quant_type: fp4
1373
- - bnb_4bit_use_double_quant: False
1374
- - bnb_4bit_compute_dtype: float32
1375
-
1376
- ### Framework versions
1377
-
1378
-
1379
- - PEFT 0.6.2
1380
- ## Training procedure
1381
-
1382
-
1383
- The following `bitsandbytes` quantization config was used during training:
1384
- - quant_method: bitsandbytes
1385
- - load_in_8bit: True
1386
- - load_in_4bit: False
1387
- - llm_int8_threshold: 6.0
1388
- - llm_int8_skip_modules: None
1389
- - llm_int8_enable_fp32_cpu_offload: False
1390
- - llm_int8_has_fp16_weight: False
1391
- - bnb_4bit_quant_type: fp4
1392
- - bnb_4bit_use_double_quant: False
1393
- - bnb_4bit_compute_dtype: float32
1394
-
1395
- ### Framework versions
1396
-
1397
-
1398
- - PEFT 0.6.2
1399
- ## Training procedure
1400
-
1401
-
1402
- The following `bitsandbytes` quantization config was used during training:
1403
- - quant_method: bitsandbytes
1404
- - load_in_8bit: True
1405
- - load_in_4bit: False
1406
- - llm_int8_threshold: 6.0
1407
- - llm_int8_skip_modules: None
1408
- - llm_int8_enable_fp32_cpu_offload: False
1409
- - llm_int8_has_fp16_weight: False
1410
- - bnb_4bit_quant_type: fp4
1411
- - bnb_4bit_use_double_quant: False
1412
- - bnb_4bit_compute_dtype: float32
1413
-
1414
- ### Framework versions
1415
-
1416
-
1417
- - PEFT 0.6.2
1418
- ## Training procedure
1419
-
1420
-
1421
- The following `bitsandbytes` quantization config was used during training:
1422
- - quant_method: bitsandbytes
1423
- - load_in_8bit: True
1424
- - load_in_4bit: False
1425
- - llm_int8_threshold: 6.0
1426
- - llm_int8_skip_modules: None
1427
- - llm_int8_enable_fp32_cpu_offload: False
1428
- - llm_int8_has_fp16_weight: False
1429
- - bnb_4bit_quant_type: fp4
1430
- - bnb_4bit_use_double_quant: False
1431
- - bnb_4bit_compute_dtype: float32
1432
-
1433
- ### Framework versions
1434
-
1435
-
1436
- - PEFT 0.6.2
1437
- ## Training procedure
1438
-
1439
-
1440
- The following `bitsandbytes` quantization config was used during training:
1441
- - quant_method: bitsandbytes
1442
- - load_in_8bit: True
1443
- - load_in_4bit: False
1444
- - llm_int8_threshold: 6.0
1445
- - llm_int8_skip_modules: None
1446
- - llm_int8_enable_fp32_cpu_offload: False
1447
- - llm_int8_has_fp16_weight: False
1448
- - bnb_4bit_quant_type: fp4
1449
- - bnb_4bit_use_double_quant: False
1450
- - bnb_4bit_compute_dtype: float32
1451
-
1452
- ### Framework versions
1453
-
1454
-
1455
- - PEFT 0.6.2
1456
- ## Training procedure
1457
-
1458
-
1459
- The following `bitsandbytes` quantization config was used during training:
1460
- - quant_method: bitsandbytes
1461
- - load_in_8bit: True
1462
- - load_in_4bit: False
1463
- - llm_int8_threshold: 6.0
1464
- - llm_int8_skip_modules: None
1465
- - llm_int8_enable_fp32_cpu_offload: False
1466
- - llm_int8_has_fp16_weight: False
1467
- - bnb_4bit_quant_type: fp4
1468
- - bnb_4bit_use_double_quant: False
1469
- - bnb_4bit_compute_dtype: float32
1470
-
1471
- ### Framework versions
1472
-
1473
-
1474
- - PEFT 0.6.2
1475
- ## Training procedure
1476
-
1477
-
1478
- The following `bitsandbytes` quantization config was used during training:
1479
- - quant_method: bitsandbytes
1480
- - load_in_8bit: True
1481
- - load_in_4bit: False
1482
- - llm_int8_threshold: 6.0
1483
- - llm_int8_skip_modules: None
1484
- - llm_int8_enable_fp32_cpu_offload: False
1485
- - llm_int8_has_fp16_weight: False
1486
- - bnb_4bit_quant_type: fp4
1487
- - bnb_4bit_use_double_quant: False
1488
- - bnb_4bit_compute_dtype: float32
1489
-
1490
- ### Framework versions
1491
-
1492
-
1493
- - PEFT 0.6.2
1494
- ## Training procedure
1495
-
1496
-
1497
- The following `bitsandbytes` quantization config was used during training:
1498
- - quant_method: bitsandbytes
1499
- - load_in_8bit: True
1500
- - load_in_4bit: False
1501
- - llm_int8_threshold: 6.0
1502
- - llm_int8_skip_modules: None
1503
- - llm_int8_enable_fp32_cpu_offload: False
1504
- - llm_int8_has_fp16_weight: False
1505
- - bnb_4bit_quant_type: fp4
1506
- - bnb_4bit_use_double_quant: False
1507
- - bnb_4bit_compute_dtype: float32
1508
-
1509
- ### Framework versions
1510
-
1511
-
1512
- - PEFT 0.6.2
1513
- ## Training procedure
1514
-
1515
-
1516
- The following `bitsandbytes` quantization config was used during training:
1517
- - quant_method: bitsandbytes
1518
- - load_in_8bit: True
1519
- - load_in_4bit: False
1520
- - llm_int8_threshold: 6.0
1521
- - llm_int8_skip_modules: None
1522
- - llm_int8_enable_fp32_cpu_offload: False
1523
- - llm_int8_has_fp16_weight: False
1524
- - bnb_4bit_quant_type: fp4
1525
- - bnb_4bit_use_double_quant: False
1526
- - bnb_4bit_compute_dtype: float32
1527
-
1528
- ### Framework versions
1529
-
1530
-
1531
- - PEFT 0.6.2
1532
- ## Training procedure
1533
-
1534
-
1535
- The following `bitsandbytes` quantization config was used during training:
1536
- - quant_method: bitsandbytes
1537
- - load_in_8bit: True
1538
- - load_in_4bit: False
1539
- - llm_int8_threshold: 6.0
1540
- - llm_int8_skip_modules: None
1541
- - llm_int8_enable_fp32_cpu_offload: False
1542
- - llm_int8_has_fp16_weight: False
1543
- - bnb_4bit_quant_type: fp4
1544
- - bnb_4bit_use_double_quant: False
1545
- - bnb_4bit_compute_dtype: float32
1546
-
1547
- ### Framework versions
1548
-
1549
-
1550
- - PEFT 0.6.2
1551
- ## Training procedure
1552
-
1553
-
1554
- The following `bitsandbytes` quantization config was used during training:
1555
- - quant_method: bitsandbytes
1556
- - load_in_8bit: True
1557
- - load_in_4bit: False
1558
- - llm_int8_threshold: 6.0
1559
- - llm_int8_skip_modules: None
1560
- - llm_int8_enable_fp32_cpu_offload: False
1561
- - llm_int8_has_fp16_weight: False
1562
- - bnb_4bit_quant_type: fp4
1563
- - bnb_4bit_use_double_quant: False
1564
- - bnb_4bit_compute_dtype: float32
1565
-
1566
- ### Framework versions
1567
-
1568
-
1569
- - PEFT 0.6.2
1570
- ## Training procedure
1571
-
1572
-
1573
- The following `bitsandbytes` quantization config was used during training:
1574
- - quant_method: bitsandbytes
1575
- - load_in_8bit: True
1576
- - load_in_4bit: False
1577
- - llm_int8_threshold: 6.0
1578
- - llm_int8_skip_modules: None
1579
- - llm_int8_enable_fp32_cpu_offload: False
1580
- - llm_int8_has_fp16_weight: False
1581
- - bnb_4bit_quant_type: fp4
1582
- - bnb_4bit_use_double_quant: False
1583
- - bnb_4bit_compute_dtype: float32
1584
-
1585
- ### Framework versions
1586
-
1587
-
1588
- - PEFT 0.6.2
1589
- ## Training procedure
1590
-
1591
-
1592
- The following `bitsandbytes` quantization config was used during training:
1593
- - quant_method: bitsandbytes
1594
- - load_in_8bit: True
1595
- - load_in_4bit: False
1596
- - llm_int8_threshold: 6.0
1597
- - llm_int8_skip_modules: None
1598
- - llm_int8_enable_fp32_cpu_offload: False
1599
- - llm_int8_has_fp16_weight: False
1600
- - bnb_4bit_quant_type: fp4
1601
- - bnb_4bit_use_double_quant: False
1602
- - bnb_4bit_compute_dtype: float32
1603
-
1604
- ### Framework versions
1605
-
1606
-
1607
- - PEFT 0.6.2
1608
- ## Training procedure
1609
-
1610
-
1611
- The following `bitsandbytes` quantization config was used during training:
1612
- - quant_method: bitsandbytes
1613
- - load_in_8bit: True
1614
- - load_in_4bit: False
1615
- - llm_int8_threshold: 6.0
1616
- - llm_int8_skip_modules: None
1617
- - llm_int8_enable_fp32_cpu_offload: False
1618
- - llm_int8_has_fp16_weight: False
1619
- - bnb_4bit_quant_type: fp4
1620
- - bnb_4bit_use_double_quant: False
1621
- - bnb_4bit_compute_dtype: float32
1622
-
1623
- ### Framework versions
1624
-
1625
-
1626
- - PEFT 0.6.2
1627
- ## Training procedure
1628
-
1629
-
1630
- The following `bitsandbytes` quantization config was used during training:
1631
- - quant_method: bitsandbytes
1632
- - load_in_8bit: True
1633
- - load_in_4bit: False
1634
- - llm_int8_threshold: 6.0
1635
- - llm_int8_skip_modules: None
1636
- - llm_int8_enable_fp32_cpu_offload: False
1637
- - llm_int8_has_fp16_weight: False
1638
- - bnb_4bit_quant_type: fp4
1639
- - bnb_4bit_use_double_quant: False
1640
- - bnb_4bit_compute_dtype: float32
1641
-
1642
- ### Framework versions
1643
-
1644
-
1645
- - PEFT 0.6.2
1646
- ## Training procedure
1647
-
1648
-
1649
- The following `bitsandbytes` quantization config was used during training:
1650
- - quant_method: bitsandbytes
1651
- - load_in_8bit: True
1652
- - load_in_4bit: False
1653
- - llm_int8_threshold: 6.0
1654
- - llm_int8_skip_modules: None
1655
- - llm_int8_enable_fp32_cpu_offload: False
1656
- - llm_int8_has_fp16_weight: False
1657
- - bnb_4bit_quant_type: fp4
1658
- - bnb_4bit_use_double_quant: False
1659
- - bnb_4bit_compute_dtype: float32
1660
-
1661
- ### Framework versions
1662
-
1663
-
1664
- - PEFT 0.6.2
1665
- ## Training procedure
1666
-
1667
-
1668
- The following `bitsandbytes` quantization config was used during training:
1669
- - quant_method: bitsandbytes
1670
- - load_in_8bit: True
1671
- - load_in_4bit: False
1672
- - llm_int8_threshold: 6.0
1673
- - llm_int8_skip_modules: None
1674
- - llm_int8_enable_fp32_cpu_offload: False
1675
- - llm_int8_has_fp16_weight: False
1676
- - bnb_4bit_quant_type: fp4
1677
- - bnb_4bit_use_double_quant: False
1678
- - bnb_4bit_compute_dtype: float32
1679
-
1680
- ### Framework versions
1681
-
1682
-
1683
- - PEFT 0.6.2
1684
- ## Training procedure
1685
-
1686
-
1687
- The following `bitsandbytes` quantization config was used during training:
1688
- - quant_method: bitsandbytes
1689
- - load_in_8bit: True
1690
- - load_in_4bit: False
1691
- - llm_int8_threshold: 6.0
1692
- - llm_int8_skip_modules: None
1693
- - llm_int8_enable_fp32_cpu_offload: False
1694
- - llm_int8_has_fp16_weight: False
1695
- - bnb_4bit_quant_type: fp4
1696
- - bnb_4bit_use_double_quant: False
1697
- - bnb_4bit_compute_dtype: float32
1698
-
1699
- ### Framework versions
1700
-
1701
-
1702
- - PEFT 0.6.2
1703
- ## Training procedure
1704
-
1705
-
1706
- The following `bitsandbytes` quantization config was used during training:
1707
- - quant_method: bitsandbytes
1708
- - load_in_8bit: True
1709
- - load_in_4bit: False
1710
- - llm_int8_threshold: 6.0
1711
- - llm_int8_skip_modules: None
1712
- - llm_int8_enable_fp32_cpu_offload: False
1713
- - llm_int8_has_fp16_weight: False
1714
- - bnb_4bit_quant_type: fp4
1715
- - bnb_4bit_use_double_quant: False
1716
- - bnb_4bit_compute_dtype: float32
1717
-
1718
- ### Framework versions
1719
-
1720
-
1721
- - PEFT 0.6.2
1722
- ## Training procedure
1723
-
1724
-
1725
- The following `bitsandbytes` quantization config was used during training:
1726
- - quant_method: bitsandbytes
1727
- - load_in_8bit: True
1728
- - load_in_4bit: False
1729
- - llm_int8_threshold: 6.0
1730
- - llm_int8_skip_modules: None
1731
- - llm_int8_enable_fp32_cpu_offload: False
1732
- - llm_int8_has_fp16_weight: False
1733
- - bnb_4bit_quant_type: fp4
1734
- - bnb_4bit_use_double_quant: False
1735
- - bnb_4bit_compute_dtype: float32
1736
-
1737
- ### Framework versions
1738
-
1739
-
1740
- - PEFT 0.6.2
1741
- ## Training procedure
1742
-
1743
-
1744
- The following `bitsandbytes` quantization config was used during training:
1745
- - quant_method: bitsandbytes
1746
- - load_in_8bit: True
1747
- - load_in_4bit: False
1748
- - llm_int8_threshold: 6.0
1749
- - llm_int8_skip_modules: None
1750
- - llm_int8_enable_fp32_cpu_offload: False
1751
- - llm_int8_has_fp16_weight: False
1752
- - bnb_4bit_quant_type: fp4
1753
- - bnb_4bit_use_double_quant: False
1754
- - bnb_4bit_compute_dtype: float32
1755
-
1756
- ### Framework versions
1757
-
1758
-
1759
- - PEFT 0.6.2
1760
- ## Training procedure
1761
-
1762
-
1763
- The following `bitsandbytes` quantization config was used during training:
1764
- - quant_method: bitsandbytes
1765
- - load_in_8bit: True
1766
- - load_in_4bit: False
1767
- - llm_int8_threshold: 6.0
1768
- - llm_int8_skip_modules: None
1769
- - llm_int8_enable_fp32_cpu_offload: False
1770
- - llm_int8_has_fp16_weight: False
1771
- - bnb_4bit_quant_type: fp4
1772
- - bnb_4bit_use_double_quant: False
1773
- - bnb_4bit_compute_dtype: float32
1774
-
1775
- ### Framework versions
1776
-
1777
-
1778
- - PEFT 0.6.2
1779
- ## Training procedure
1780
-
1781
-
1782
- The following `bitsandbytes` quantization config was used during training:
1783
- - quant_method: bitsandbytes
1784
- - load_in_8bit: True
1785
- - load_in_4bit: False
1786
- - llm_int8_threshold: 6.0
1787
- - llm_int8_skip_modules: None
1788
- - llm_int8_enable_fp32_cpu_offload: False
1789
- - llm_int8_has_fp16_weight: False
1790
- - bnb_4bit_quant_type: fp4
1791
- - bnb_4bit_use_double_quant: False
1792
- - bnb_4bit_compute_dtype: float32
1793
-
1794
- ### Framework versions
1795
-
1796
-
1797
- - PEFT 0.6.2
1798
- ## Training procedure
1799
-
1800
-
1801
- The following `bitsandbytes` quantization config was used during training:
1802
- - quant_method: bitsandbytes
1803
- - load_in_8bit: True
1804
- - load_in_4bit: False
1805
- - llm_int8_threshold: 6.0
1806
- - llm_int8_skip_modules: None
1807
- - llm_int8_enable_fp32_cpu_offload: False
1808
- - llm_int8_has_fp16_weight: False
1809
- - bnb_4bit_quant_type: fp4
1810
- - bnb_4bit_use_double_quant: False
1811
- - bnb_4bit_compute_dtype: float32
1812
-
1813
- ### Framework versions
1814
-
1815
-
1816
- - PEFT 0.6.2
1817
- ## Training procedure
1818
-
1819
-
1820
- The following `bitsandbytes` quantization config was used during training:
1821
- - quant_method: bitsandbytes
1822
- - load_in_8bit: True
1823
- - load_in_4bit: False
1824
- - llm_int8_threshold: 6.0
1825
- - llm_int8_skip_modules: None
1826
- - llm_int8_enable_fp32_cpu_offload: False
1827
- - llm_int8_has_fp16_weight: False
1828
- - bnb_4bit_quant_type: fp4
1829
- - bnb_4bit_use_double_quant: False
1830
- - bnb_4bit_compute_dtype: float32
1831
-
1832
- ### Framework versions
1833
-
1834
-
1835
- - PEFT 0.6.2
1836
- ## Training procedure
1837
-
1838
-
1839
- The following `bitsandbytes` quantization config was used during training:
1840
- - quant_method: bitsandbytes
1841
- - load_in_8bit: True
1842
- - load_in_4bit: False
1843
- - llm_int8_threshold: 6.0
1844
- - llm_int8_skip_modules: None
1845
- - llm_int8_enable_fp32_cpu_offload: False
1846
- - llm_int8_has_fp16_weight: False
1847
- - bnb_4bit_quant_type: fp4
1848
- - bnb_4bit_use_double_quant: False
1849
- - bnb_4bit_compute_dtype: float32
1850
-
1851
- ### Framework versions
1852
-
1853
-
1854
- - PEFT 0.6.2
1855
- ## Training procedure
1856
-
1857
-
1858
- The following `bitsandbytes` quantization config was used during training:
1859
- - quant_method: bitsandbytes
1860
- - load_in_8bit: True
1861
- - load_in_4bit: False
1862
- - llm_int8_threshold: 6.0
1863
- - llm_int8_skip_modules: None
1864
- - llm_int8_enable_fp32_cpu_offload: False
1865
- - llm_int8_has_fp16_weight: False
1866
- - bnb_4bit_quant_type: fp4
1867
- - bnb_4bit_use_double_quant: False
1868
- - bnb_4bit_compute_dtype: float32
1869
-
1870
- ### Framework versions
1871
-
1872
-
1873
- - PEFT 0.6.2
1874
- ## Training procedure
1875
-
1876
-
1877
- The following `bitsandbytes` quantization config was used during training:
1878
- - quant_method: bitsandbytes
1879
- - load_in_8bit: True
1880
- - load_in_4bit: False
1881
- - llm_int8_threshold: 6.0
1882
- - llm_int8_skip_modules: None
1883
- - llm_int8_enable_fp32_cpu_offload: False
1884
- - llm_int8_has_fp16_weight: False
1885
- - bnb_4bit_quant_type: fp4
1886
- - bnb_4bit_use_double_quant: False
1887
- - bnb_4bit_compute_dtype: float32
1888
-
1889
- ### Framework versions
1890
-
1891
-
1892
- - PEFT 0.6.2
1893
- ## Training procedure
1894
-
1895
-
1896
- The following `bitsandbytes` quantization config was used during training:
1897
- - quant_method: bitsandbytes
1898
- - load_in_8bit: True
1899
- - load_in_4bit: False
1900
- - llm_int8_threshold: 6.0
1901
- - llm_int8_skip_modules: None
1902
- - llm_int8_enable_fp32_cpu_offload: False
1903
- - llm_int8_has_fp16_weight: False
1904
- - bnb_4bit_quant_type: fp4
1905
- - bnb_4bit_use_double_quant: False
1906
- - bnb_4bit_compute_dtype: float32
1907
-
1908
- ### Framework versions
1909
-
1910
-
1911
- - PEFT 0.6.2
1912
- ## Training procedure
1913
-
1914
-
1915
- The following `bitsandbytes` quantization config was used during training:
1916
- - quant_method: bitsandbytes
1917
- - load_in_8bit: True
1918
- - load_in_4bit: False
1919
- - llm_int8_threshold: 6.0
1920
- - llm_int8_skip_modules: None
1921
- - llm_int8_enable_fp32_cpu_offload: False
1922
- - llm_int8_has_fp16_weight: False
1923
- - bnb_4bit_quant_type: fp4
1924
- - bnb_4bit_use_double_quant: False
1925
- - bnb_4bit_compute_dtype: float32
1926
-
1927
- ### Framework versions
1928
-
1929
-
1930
- - PEFT 0.6.2
1931
- ## Training procedure
1932
-
1933
-
1934
- The following `bitsandbytes` quantization config was used during training:
1935
- - quant_method: bitsandbytes
1936
- - load_in_8bit: True
1937
- - load_in_4bit: False
1938
- - llm_int8_threshold: 6.0
1939
- - llm_int8_skip_modules: None
1940
- - llm_int8_enable_fp32_cpu_offload: False
1941
- - llm_int8_has_fp16_weight: False
1942
- - bnb_4bit_quant_type: fp4
1943
- - bnb_4bit_use_double_quant: False
1944
- - bnb_4bit_compute_dtype: float32
1945
-
1946
- ### Framework versions
1947
-
1948
-
1949
- - PEFT 0.6.2
1950
- ## Training procedure
1951
-
1952
-
1953
- The following `bitsandbytes` quantization config was used during training:
1954
- - quant_method: bitsandbytes
1955
- - load_in_8bit: True
1956
- - load_in_4bit: False
1957
- - llm_int8_threshold: 6.0
1958
- - llm_int8_skip_modules: None
1959
- - llm_int8_enable_fp32_cpu_offload: False
1960
- - llm_int8_has_fp16_weight: False
1961
- - bnb_4bit_quant_type: fp4
1962
- - bnb_4bit_use_double_quant: False
1963
- - bnb_4bit_compute_dtype: float32
1964
-
1965
- ### Framework versions
1966
-
1967
-
1968
- - PEFT 0.6.2
1969
- ## Training procedure
1970
-
1971
-
1972
- The following `bitsandbytes` quantization config was used during training:
1973
- - quant_method: bitsandbytes
1974
- - load_in_8bit: True
1975
- - load_in_4bit: False
1976
- - llm_int8_threshold: 6.0
1977
- - llm_int8_skip_modules: None
1978
- - llm_int8_enable_fp32_cpu_offload: False
1979
- - llm_int8_has_fp16_weight: False
1980
- - bnb_4bit_quant_type: fp4
1981
- - bnb_4bit_use_double_quant: False
1982
- - bnb_4bit_compute_dtype: float32
1983
-
1984
- ### Framework versions
1985
-
1986
-
1987
- - PEFT 0.6.2
1988
- ## Training procedure
1989
-
1990
-
1991
- The following `bitsandbytes` quantization config was used during training:
1992
- - quant_method: bitsandbytes
1993
- - load_in_8bit: True
1994
- - load_in_4bit: False
1995
- - llm_int8_threshold: 6.0
1996
- - llm_int8_skip_modules: None
1997
- - llm_int8_enable_fp32_cpu_offload: False
1998
- - llm_int8_has_fp16_weight: False
1999
- - bnb_4bit_quant_type: fp4
2000
- - bnb_4bit_use_double_quant: False
2001
- - bnb_4bit_compute_dtype: float32
2002
-
2003
- ### Framework versions
2004
-
2005
-
2006
- - PEFT 0.6.2
2007
- ## Training procedure
2008
-
2009
-
2010
- The following `bitsandbytes` quantization config was used during training:
2011
- - quant_method: bitsandbytes
2012
- - load_in_8bit: True
2013
- - load_in_4bit: False
2014
- - llm_int8_threshold: 6.0
2015
- - llm_int8_skip_modules: None
2016
- - llm_int8_enable_fp32_cpu_offload: False
2017
- - llm_int8_has_fp16_weight: False
2018
- - bnb_4bit_quant_type: fp4
2019
- - bnb_4bit_use_double_quant: False
2020
- - bnb_4bit_compute_dtype: float32
2021
-
2022
- ### Framework versions
2023
-
2024
-
2025
- - PEFT 0.6.2
2026
- ## Training procedure
2027
-
2028
-
2029
- The following `bitsandbytes` quantization config was used during training:
2030
- - quant_method: bitsandbytes
2031
- - load_in_8bit: True
2032
- - load_in_4bit: False
2033
- - llm_int8_threshold: 6.0
2034
- - llm_int8_skip_modules: None
2035
- - llm_int8_enable_fp32_cpu_offload: False
2036
- - llm_int8_has_fp16_weight: False
2037
- - bnb_4bit_quant_type: fp4
2038
- - bnb_4bit_use_double_quant: False
2039
- - bnb_4bit_compute_dtype: float32
2040
-
2041
- ### Framework versions
2042
-
2043
-
2044
- - PEFT 0.6.2
2045
- ## Training procedure
2046
-
2047
-
2048
- The following `bitsandbytes` quantization config was used during training:
2049
- - quant_method: bitsandbytes
2050
- - load_in_8bit: True
2051
- - load_in_4bit: False
2052
- - llm_int8_threshold: 6.0
2053
- - llm_int8_skip_modules: None
2054
- - llm_int8_enable_fp32_cpu_offload: False
2055
- - llm_int8_has_fp16_weight: False
2056
- - bnb_4bit_quant_type: fp4
2057
- - bnb_4bit_use_double_quant: False
2058
- - bnb_4bit_compute_dtype: float32
2059
-
2060
- ### Framework versions
2061
-
2062
-
2063
- - PEFT 0.6.2
2064
- ## Training procedure
2065
-
2066
-
2067
- The following `bitsandbytes` quantization config was used during training:
2068
- - quant_method: bitsandbytes
2069
- - load_in_8bit: True
2070
- - load_in_4bit: False
2071
- - llm_int8_threshold: 6.0
2072
- - llm_int8_skip_modules: None
2073
- - llm_int8_enable_fp32_cpu_offload: False
2074
- - llm_int8_has_fp16_weight: False
2075
- - bnb_4bit_quant_type: fp4
2076
- - bnb_4bit_use_double_quant: False
2077
- - bnb_4bit_compute_dtype: float32
2078
-
2079
- ### Framework versions
2080
-
2081
-
2082
- - PEFT 0.6.2
2083
- ## Training procedure
2084
-
2085
-
2086
- The following `bitsandbytes` quantization config was used during training:
2087
- - quant_method: bitsandbytes
2088
- - load_in_8bit: True
2089
- - load_in_4bit: False
2090
- - llm_int8_threshold: 6.0
2091
- - llm_int8_skip_modules: None
2092
- - llm_int8_enable_fp32_cpu_offload: False
2093
- - llm_int8_has_fp16_weight: False
2094
- - bnb_4bit_quant_type: fp4
2095
- - bnb_4bit_use_double_quant: False
2096
- - bnb_4bit_compute_dtype: float32
2097
-
2098
- ### Framework versions
2099
-
2100
-
2101
- - PEFT 0.6.2
2102
- ## Training procedure
2103
-
2104
-
2105
- The following `bitsandbytes` quantization config was used during training:
2106
- - quant_method: bitsandbytes
2107
- - load_in_8bit: True
2108
- - load_in_4bit: False
2109
- - llm_int8_threshold: 6.0
2110
- - llm_int8_skip_modules: None
2111
- - llm_int8_enable_fp32_cpu_offload: False
2112
- - llm_int8_has_fp16_weight: False
2113
- - bnb_4bit_quant_type: fp4
2114
- - bnb_4bit_use_double_quant: False
2115
- - bnb_4bit_compute_dtype: float32
2116
-
2117
- ### Framework versions
2118
-
2119
-
2120
- - PEFT 0.6.2
2121
- ## Training procedure
2122
-
2123
-
2124
- The following `bitsandbytes` quantization config was used during training:
2125
- - quant_method: bitsandbytes
2126
- - load_in_8bit: True
2127
- - load_in_4bit: False
2128
- - llm_int8_threshold: 6.0
2129
- - llm_int8_skip_modules: None
2130
- - llm_int8_enable_fp32_cpu_offload: False
2131
- - llm_int8_has_fp16_weight: False
2132
- - bnb_4bit_quant_type: fp4
2133
- - bnb_4bit_use_double_quant: False
2134
- - bnb_4bit_compute_dtype: float32
2135
-
2136
- ### Framework versions
2137
-
2138
-
2139
- - PEFT 0.6.2
2140
- ## Training procedure
2141
-
2142
-
2143
- The following `bitsandbytes` quantization config was used during training:
2144
- - quant_method: bitsandbytes
2145
- - load_in_8bit: True
2146
- - load_in_4bit: False
2147
- - llm_int8_threshold: 6.0
2148
- - llm_int8_skip_modules: None
2149
- - llm_int8_enable_fp32_cpu_offload: False
2150
- - llm_int8_has_fp16_weight: False
2151
- - bnb_4bit_quant_type: fp4
2152
- - bnb_4bit_use_double_quant: False
2153
- - bnb_4bit_compute_dtype: float32
2154
-
2155
- ### Framework versions
2156
-
2157
-
2158
- - PEFT 0.6.2
2159
- ## Training procedure
2160
-
2161
-
2162
- The following `bitsandbytes` quantization config was used during training:
2163
- - quant_method: bitsandbytes
2164
- - load_in_8bit: True
2165
- - load_in_4bit: False
2166
- - llm_int8_threshold: 6.0
2167
- - llm_int8_skip_modules: None
2168
- - llm_int8_enable_fp32_cpu_offload: False
2169
- - llm_int8_has_fp16_weight: False
2170
- - bnb_4bit_quant_type: fp4
2171
- - bnb_4bit_use_double_quant: False
2172
- - bnb_4bit_compute_dtype: float32
2173
-
2174
- ### Framework versions
2175
-
2176
-
2177
- - PEFT 0.6.2
2178
- ## Training procedure
2179
-
2180
-
2181
- The following `bitsandbytes` quantization config was used during training:
2182
- - quant_method: bitsandbytes
2183
- - load_in_8bit: True
2184
- - load_in_4bit: False
2185
- - llm_int8_threshold: 6.0
2186
- - llm_int8_skip_modules: None
2187
- - llm_int8_enable_fp32_cpu_offload: False
2188
- - llm_int8_has_fp16_weight: False
2189
- - bnb_4bit_quant_type: fp4
2190
- - bnb_4bit_use_double_quant: False
2191
- - bnb_4bit_compute_dtype: float32
2192
-
2193
- ### Framework versions
2194
-
2195
-
2196
- - PEFT 0.6.2
2197
- ## Training procedure
2198
-
2199
-
2200
- The following `bitsandbytes` quantization config was used during training:
2201
- - quant_method: bitsandbytes
2202
- - load_in_8bit: True
2203
- - load_in_4bit: False
2204
- - llm_int8_threshold: 6.0
2205
- - llm_int8_skip_modules: None
2206
- - llm_int8_enable_fp32_cpu_offload: False
2207
- - llm_int8_has_fp16_weight: False
2208
- - bnb_4bit_quant_type: fp4
2209
- - bnb_4bit_use_double_quant: False
2210
- - bnb_4bit_compute_dtype: float32
2211
-
2212
- ### Framework versions
2213
-
2214
-
2215
- - PEFT 0.6.2
2216
- ## Training procedure
2217
-
2218
-
2219
- The following `bitsandbytes` quantization config was used during training:
2220
- - quant_method: bitsandbytes
2221
- - load_in_8bit: True
2222
- - load_in_4bit: False
2223
- - llm_int8_threshold: 6.0
2224
- - llm_int8_skip_modules: None
2225
- - llm_int8_enable_fp32_cpu_offload: False
2226
- - llm_int8_has_fp16_weight: False
2227
- - bnb_4bit_quant_type: fp4
2228
- - bnb_4bit_use_double_quant: False
2229
- - bnb_4bit_compute_dtype: float32
2230
-
2231
- ### Framework versions
2232
-
2233
-
2234
- - PEFT 0.6.2
2235
- ## Training procedure
2236
-
2237
-
2238
- The following `bitsandbytes` quantization config was used during training:
2239
- - quant_method: bitsandbytes
2240
- - load_in_8bit: True
2241
- - load_in_4bit: False
2242
- - llm_int8_threshold: 6.0
2243
- - llm_int8_skip_modules: None
2244
- - llm_int8_enable_fp32_cpu_offload: False
2245
- - llm_int8_has_fp16_weight: False
2246
- - bnb_4bit_quant_type: fp4
2247
- - bnb_4bit_use_double_quant: False
2248
- - bnb_4bit_compute_dtype: float32
2249
-
2250
- ### Framework versions
2251
-
2252
-
2253
- - PEFT 0.6.2
2254
- ## Training procedure
2255
-
2256
-
2257
- The following `bitsandbytes` quantization config was used during training:
2258
- - quant_method: bitsandbytes
2259
- - load_in_8bit: True
2260
- - load_in_4bit: False
2261
- - llm_int8_threshold: 6.0
2262
- - llm_int8_skip_modules: None
2263
- - llm_int8_enable_fp32_cpu_offload: False
2264
- - llm_int8_has_fp16_weight: False
2265
- - bnb_4bit_quant_type: fp4
2266
- - bnb_4bit_use_double_quant: False
2267
- - bnb_4bit_compute_dtype: float32
2268
-
2269
- ### Framework versions
2270
-
2271
-
2272
- - PEFT 0.6.2
2273
- ## Training procedure
2274
-
2275
-
2276
- The following `bitsandbytes` quantization config was used during training:
2277
- - quant_method: bitsandbytes
2278
- - load_in_8bit: True
2279
- - load_in_4bit: False
2280
- - llm_int8_threshold: 6.0
2281
- - llm_int8_skip_modules: None
2282
- - llm_int8_enable_fp32_cpu_offload: False
2283
- - llm_int8_has_fp16_weight: False
2284
- - bnb_4bit_quant_type: fp4
2285
- - bnb_4bit_use_double_quant: False
2286
- - bnb_4bit_compute_dtype: float32
2287
-
2288
- ### Framework versions
2289
-
2290
-
2291
- - PEFT 0.6.2
2292
- ## Training procedure
2293
-
2294
-
2295
- The following `bitsandbytes` quantization config was used during training:
2296
- - quant_method: bitsandbytes
2297
- - load_in_8bit: True
2298
- - load_in_4bit: False
2299
- - llm_int8_threshold: 6.0
2300
- - llm_int8_skip_modules: None
2301
- - llm_int8_enable_fp32_cpu_offload: False
2302
- - llm_int8_has_fp16_weight: False
2303
- - bnb_4bit_quant_type: fp4
2304
- - bnb_4bit_use_double_quant: False
2305
- - bnb_4bit_compute_dtype: float32
2306
-
2307
- ### Framework versions
2308
-
2309
-
2310
- - PEFT 0.6.2
2311
- ## Training procedure
2312
-
2313
-
2314
- The following `bitsandbytes` quantization config was used during training:
2315
- - quant_method: bitsandbytes
2316
- - load_in_8bit: True
2317
- - load_in_4bit: False
2318
- - llm_int8_threshold: 6.0
2319
- - llm_int8_skip_modules: None
2320
- - llm_int8_enable_fp32_cpu_offload: False
2321
- - llm_int8_has_fp16_weight: False
2322
- - bnb_4bit_quant_type: fp4
2323
- - bnb_4bit_use_double_quant: False
2324
- - bnb_4bit_compute_dtype: float32
2325
-
2326
- ### Framework versions
2327
-
2328
-
2329
- - PEFT 0.6.2
2330
- ## Training procedure
2331
-
2332
-
2333
- The following `bitsandbytes` quantization config was used during training:
2334
- - quant_method: bitsandbytes
2335
- - load_in_8bit: True
2336
- - load_in_4bit: False
2337
- - llm_int8_threshold: 6.0
2338
- - llm_int8_skip_modules: None
2339
- - llm_int8_enable_fp32_cpu_offload: False
2340
- - llm_int8_has_fp16_weight: False
2341
- - bnb_4bit_quant_type: fp4
2342
- - bnb_4bit_use_double_quant: False
2343
- - bnb_4bit_compute_dtype: float32
2344
-
2345
- ### Framework versions
2346
-
2347
-
2348
- - PEFT 0.6.2
2349
- ## Training procedure
2350
-
2351
-
2352
- The following `bitsandbytes` quantization config was used during training:
2353
- - quant_method: bitsandbytes
2354
- - load_in_8bit: True
2355
- - load_in_4bit: False
2356
- - llm_int8_threshold: 6.0
2357
- - llm_int8_skip_modules: None
2358
- - llm_int8_enable_fp32_cpu_offload: False
2359
- - llm_int8_has_fp16_weight: False
2360
- - bnb_4bit_quant_type: fp4
2361
- - bnb_4bit_use_double_quant: False
2362
- - bnb_4bit_compute_dtype: float32
2363
-
2364
- ### Framework versions
2365
-
2366
-
2367
- - PEFT 0.6.2
2368
- ## Training procedure
2369
-
2370
-
2371
- The following `bitsandbytes` quantization config was used during training:
2372
- - quant_method: bitsandbytes
2373
- - load_in_8bit: True
2374
- - load_in_4bit: False
2375
- - llm_int8_threshold: 6.0
2376
- - llm_int8_skip_modules: None
2377
- - llm_int8_enable_fp32_cpu_offload: False
2378
- - llm_int8_has_fp16_weight: False
2379
- - bnb_4bit_quant_type: fp4
2380
- - bnb_4bit_use_double_quant: False
2381
- - bnb_4bit_compute_dtype: float32
2382
-
2383
- ### Framework versions
2384
-
2385
-
2386
- - PEFT 0.6.2
2387
- ## Training procedure
2388
-
2389
-
2390
- The following `bitsandbytes` quantization config was used during training:
2391
- - quant_method: bitsandbytes
2392
- - load_in_8bit: True
2393
- - load_in_4bit: False
2394
- - llm_int8_threshold: 6.0
2395
- - llm_int8_skip_modules: None
2396
- - llm_int8_enable_fp32_cpu_offload: False
2397
- - llm_int8_has_fp16_weight: False
2398
- - bnb_4bit_quant_type: fp4
2399
- - bnb_4bit_use_double_quant: False
2400
- - bnb_4bit_compute_dtype: float32
2401
-
2402
- ### Framework versions
2403
-
2404
-
2405
- - PEFT 0.6.2
2406
- ## Training procedure
2407
-
2408
-
2409
- The following `bitsandbytes` quantization config was used during training:
2410
- - quant_method: bitsandbytes
2411
- - load_in_8bit: True
2412
- - load_in_4bit: False
2413
- - llm_int8_threshold: 6.0
2414
- - llm_int8_skip_modules: None
2415
- - llm_int8_enable_fp32_cpu_offload: False
2416
- - llm_int8_has_fp16_weight: False
2417
- - bnb_4bit_quant_type: fp4
2418
- - bnb_4bit_use_double_quant: False
2419
- - bnb_4bit_compute_dtype: float32
2420
-
2421
- ### Framework versions
2422
-
2423
-
2424
- - PEFT 0.6.2
2425
- ## Training procedure
2426
-
2427
-
2428
- The following `bitsandbytes` quantization config was used during training:
2429
- - quant_method: bitsandbytes
2430
- - load_in_8bit: True
2431
- - load_in_4bit: False
2432
- - llm_int8_threshold: 6.0
2433
- - llm_int8_skip_modules: None
2434
- - llm_int8_enable_fp32_cpu_offload: False
2435
- - llm_int8_has_fp16_weight: False
2436
- - bnb_4bit_quant_type: fp4
2437
- - bnb_4bit_use_double_quant: False
2438
- - bnb_4bit_compute_dtype: float32
2439
-
2440
- ### Framework versions
2441
-
2442
-
2443
- - PEFT 0.6.2
2444
- ## Training procedure
2445
-
2446
-
2447
- The following `bitsandbytes` quantization config was used during training:
2448
- - quant_method: bitsandbytes
2449
- - load_in_8bit: True
2450
- - load_in_4bit: False
2451
- - llm_int8_threshold: 6.0
2452
- - llm_int8_skip_modules: None
2453
- - llm_int8_enable_fp32_cpu_offload: False
2454
- - llm_int8_has_fp16_weight: False
2455
- - bnb_4bit_quant_type: fp4
2456
- - bnb_4bit_use_double_quant: False
2457
- - bnb_4bit_compute_dtype: float32
2458
-
2459
- ### Framework versions
2460
-
2461
-
2462
- - PEFT 0.6.2
2463
- ## Training procedure
2464
-
2465
-
2466
- The following `bitsandbytes` quantization config was used during training:
2467
- - quant_method: bitsandbytes
2468
- - load_in_8bit: True
2469
- - load_in_4bit: False
2470
- - llm_int8_threshold: 6.0
2471
- - llm_int8_skip_modules: None
2472
- - llm_int8_enable_fp32_cpu_offload: False
2473
- - llm_int8_has_fp16_weight: False
2474
- - bnb_4bit_quant_type: fp4
2475
- - bnb_4bit_use_double_quant: False
2476
- - bnb_4bit_compute_dtype: float32
2477
-
2478
- ### Framework versions
2479
-
2480
-
2481
- - PEFT 0.6.2
2482
- ## Training procedure
2483
-
2484
-
2485
- The following `bitsandbytes` quantization config was used during training:
2486
- - quant_method: bitsandbytes
2487
- - load_in_8bit: True
2488
- - load_in_4bit: False
2489
- - llm_int8_threshold: 6.0
2490
- - llm_int8_skip_modules: None
2491
- - llm_int8_enable_fp32_cpu_offload: False
2492
- - llm_int8_has_fp16_weight: False
2493
- - bnb_4bit_quant_type: fp4
2494
- - bnb_4bit_use_double_quant: False
2495
- - bnb_4bit_compute_dtype: float32
2496
-
2497
- ### Framework versions
2498
-
2499
-
2500
- - PEFT 0.6.2
2501
- ## Training procedure
2502
-
2503
-
2504
- The following `bitsandbytes` quantization config was used during training:
2505
- - quant_method: bitsandbytes
2506
- - load_in_8bit: True
2507
- - load_in_4bit: False
2508
- - llm_int8_threshold: 6.0
2509
- - llm_int8_skip_modules: None
2510
- - llm_int8_enable_fp32_cpu_offload: False
2511
- - llm_int8_has_fp16_weight: False
2512
- - bnb_4bit_quant_type: fp4
2513
- - bnb_4bit_use_double_quant: False
2514
- - bnb_4bit_compute_dtype: float32
2515
-
2516
- ### Framework versions
2517
-
2518
-
2519
- - PEFT 0.6.2
2520
- ## Training procedure
2521
-
2522
-
2523
- The following `bitsandbytes` quantization config was used during training:
2524
- - quant_method: bitsandbytes
2525
- - load_in_8bit: True
2526
- - load_in_4bit: False
2527
- - llm_int8_threshold: 6.0
2528
- - llm_int8_skip_modules: None
2529
- - llm_int8_enable_fp32_cpu_offload: False
2530
- - llm_int8_has_fp16_weight: False
2531
- - bnb_4bit_quant_type: fp4
2532
- - bnb_4bit_use_double_quant: False
2533
- - bnb_4bit_compute_dtype: float32
2534
-
2535
- ### Framework versions
2536
-
2537
-
2538
- - PEFT 0.6.2
2539
- ## Training procedure
2540
-
2541
-
2542
- The following `bitsandbytes` quantization config was used during training:
2543
- - quant_method: bitsandbytes
2544
- - load_in_8bit: True
2545
- - load_in_4bit: False
2546
- - llm_int8_threshold: 6.0
2547
- - llm_int8_skip_modules: None
2548
- - llm_int8_enable_fp32_cpu_offload: False
2549
- - llm_int8_has_fp16_weight: False
2550
- - bnb_4bit_quant_type: fp4
2551
- - bnb_4bit_use_double_quant: False
2552
- - bnb_4bit_compute_dtype: float32
2553
-
2554
- ### Framework versions
2555
-
2556
-
2557
- - PEFT 0.6.2
2558
- ## Training procedure
2559
-
2560
-
2561
- The following `bitsandbytes` quantization config was used during training:
2562
- - quant_method: bitsandbytes
2563
- - load_in_8bit: True
2564
- - load_in_4bit: False
2565
- - llm_int8_threshold: 6.0
2566
- - llm_int8_skip_modules: None
2567
- - llm_int8_enable_fp32_cpu_offload: False
2568
- - llm_int8_has_fp16_weight: False
2569
- - bnb_4bit_quant_type: fp4
2570
- - bnb_4bit_use_double_quant: False
2571
- - bnb_4bit_compute_dtype: float32
2572
-
2573
- ### Framework versions
2574
-
2575
-
2576
- - PEFT 0.6.2
2577
- ## Training procedure
2578
-
2579
-
2580
- The following `bitsandbytes` quantization config was used during training:
2581
- - quant_method: bitsandbytes
2582
- - load_in_8bit: True
2583
- - load_in_4bit: False
2584
- - llm_int8_threshold: 6.0
2585
- - llm_int8_skip_modules: None
2586
- - llm_int8_enable_fp32_cpu_offload: False
2587
- - llm_int8_has_fp16_weight: False
2588
- - bnb_4bit_quant_type: fp4
2589
- - bnb_4bit_use_double_quant: False
2590
- - bnb_4bit_compute_dtype: float32
2591
-
2592
- ### Framework versions
2593
-
2594
-
2595
- - PEFT 0.6.2
2596
- ## Training procedure
2597
-
2598
-
2599
- The following `bitsandbytes` quantization config was used during training:
2600
- - quant_method: bitsandbytes
2601
- - load_in_8bit: True
2602
- - load_in_4bit: False
2603
- - llm_int8_threshold: 6.0
2604
- - llm_int8_skip_modules: None
2605
- - llm_int8_enable_fp32_cpu_offload: False
2606
- - llm_int8_has_fp16_weight: False
2607
- - bnb_4bit_quant_type: fp4
2608
- - bnb_4bit_use_double_quant: False
2609
- - bnb_4bit_compute_dtype: float32
2610
-
2611
- ### Framework versions
2612
-
2613
-
2614
- - PEFT 0.6.2
2615
- ## Training procedure
2616
-
2617
-
2618
- The following `bitsandbytes` quantization config was used during training:
2619
- - quant_method: bitsandbytes
2620
- - load_in_8bit: True
2621
- - load_in_4bit: False
2622
- - llm_int8_threshold: 6.0
2623
- - llm_int8_skip_modules: None
2624
- - llm_int8_enable_fp32_cpu_offload: False
2625
- - llm_int8_has_fp16_weight: False
2626
- - bnb_4bit_quant_type: fp4
2627
- - bnb_4bit_use_double_quant: False
2628
- - bnb_4bit_compute_dtype: float32
2629
-
2630
- ### Framework versions
2631
-
2632
-
2633
- - PEFT 0.6.2
2634
- ## Training procedure
2635
-
2636
-
2637
- The following `bitsandbytes` quantization config was used during training:
2638
- - quant_method: bitsandbytes
2639
- - load_in_8bit: True
2640
- - load_in_4bit: False
2641
- - llm_int8_threshold: 6.0
2642
- - llm_int8_skip_modules: None
2643
- - llm_int8_enable_fp32_cpu_offload: False
2644
- - llm_int8_has_fp16_weight: False
2645
- - bnb_4bit_quant_type: fp4
2646
- - bnb_4bit_use_double_quant: False
2647
- - bnb_4bit_compute_dtype: float32
2648
-
2649
- ### Framework versions
2650
-
2651
-
2652
- - PEFT 0.6.2
2653
- ## Training procedure
2654
-
2655
-
2656
- The following `bitsandbytes` quantization config was used during training:
2657
- - quant_method: bitsandbytes
2658
- - load_in_8bit: True
2659
- - load_in_4bit: False
2660
- - llm_int8_threshold: 6.0
2661
- - llm_int8_skip_modules: None
2662
- - llm_int8_enable_fp32_cpu_offload: False
2663
- - llm_int8_has_fp16_weight: False
2664
- - bnb_4bit_quant_type: fp4
2665
- - bnb_4bit_use_double_quant: False
2666
- - bnb_4bit_compute_dtype: float32
2667
-
2668
- ### Framework versions
2669
-
2670
-
2671
- - PEFT 0.6.2
2672
- ## Training procedure
2673
-
2674
-
2675
- The following `bitsandbytes` quantization config was used during training:
2676
- - quant_method: bitsandbytes
2677
- - load_in_8bit: True
2678
- - load_in_4bit: False
2679
- - llm_int8_threshold: 6.0
2680
- - llm_int8_skip_modules: None
2681
- - llm_int8_enable_fp32_cpu_offload: False
2682
- - llm_int8_has_fp16_weight: False
2683
- - bnb_4bit_quant_type: fp4
2684
- - bnb_4bit_use_double_quant: False
2685
- - bnb_4bit_compute_dtype: float32
2686
-
2687
- ### Framework versions
2688
-
2689
-
2690
- - PEFT 0.6.2
2691
- ## Training procedure
2692
-
2693
-
2694
- The following `bitsandbytes` quantization config was used during training:
2695
- - quant_method: bitsandbytes
2696
- - load_in_8bit: True
2697
- - load_in_4bit: False
2698
- - llm_int8_threshold: 6.0
2699
- - llm_int8_skip_modules: None
2700
- - llm_int8_enable_fp32_cpu_offload: False
2701
- - llm_int8_has_fp16_weight: False
2702
- - bnb_4bit_quant_type: fp4
2703
- - bnb_4bit_use_double_quant: False
2704
- - bnb_4bit_compute_dtype: float32
2705
-
2706
- ### Framework versions
2707
-
2708
-
2709
- - PEFT 0.6.2
2710
- ## Training procedure
2711
-
2712
-
2713
- The following `bitsandbytes` quantization config was used during training:
2714
- - quant_method: bitsandbytes
2715
- - load_in_8bit: True
2716
- - load_in_4bit: False
2717
- - llm_int8_threshold: 6.0
2718
- - llm_int8_skip_modules: None
2719
- - llm_int8_enable_fp32_cpu_offload: False
2720
- - llm_int8_has_fp16_weight: False
2721
- - bnb_4bit_quant_type: fp4
2722
- - bnb_4bit_use_double_quant: False
2723
- - bnb_4bit_compute_dtype: float32
2724
-
2725
- ### Framework versions
2726
-
2727
-
2728
- - PEFT 0.6.2
2729
- ## Training procedure
2730
-
2731
-
2732
- The following `bitsandbytes` quantization config was used during training:
2733
- - quant_method: bitsandbytes
2734
- - load_in_8bit: True
2735
- - load_in_4bit: False
2736
- - llm_int8_threshold: 6.0
2737
- - llm_int8_skip_modules: None
2738
- - llm_int8_enable_fp32_cpu_offload: False
2739
- - llm_int8_has_fp16_weight: False
2740
- - bnb_4bit_quant_type: fp4
2741
- - bnb_4bit_use_double_quant: False
2742
- - bnb_4bit_compute_dtype: float32
2743
-
2744
- ### Framework versions
2745
-
2746
-
2747
- - PEFT 0.6.2
2748
- ## Training procedure
2749
-
2750
-
2751
- The following `bitsandbytes` quantization config was used during training:
2752
- - quant_method: bitsandbytes
2753
- - load_in_8bit: True
2754
- - load_in_4bit: False
2755
- - llm_int8_threshold: 6.0
2756
- - llm_int8_skip_modules: None
2757
- - llm_int8_enable_fp32_cpu_offload: False
2758
- - llm_int8_has_fp16_weight: False
2759
- - bnb_4bit_quant_type: fp4
2760
- - bnb_4bit_use_double_quant: False
2761
- - bnb_4bit_compute_dtype: float32
2762
-
2763
- ### Framework versions
2764
-
2765
-
2766
- - PEFT 0.6.2
2767
- ## Training procedure
2768
-
2769
-
2770
- The following `bitsandbytes` quantization config was used during training:
2771
- - quant_method: bitsandbytes
2772
- - load_in_8bit: True
2773
- - load_in_4bit: False
2774
- - llm_int8_threshold: 6.0
2775
- - llm_int8_skip_modules: None
2776
- - llm_int8_enable_fp32_cpu_offload: False
2777
- - llm_int8_has_fp16_weight: False
2778
- - bnb_4bit_quant_type: fp4
2779
- - bnb_4bit_use_double_quant: False
2780
- - bnb_4bit_compute_dtype: float32
2781
-
2782
- ### Framework versions
2783
-
2784
-
2785
- - PEFT 0.6.2
2786
- ## Training procedure
2787
-
2788
-
2789
- The following `bitsandbytes` quantization config was used during training:
2790
- - quant_method: bitsandbytes
2791
- - load_in_8bit: True
2792
- - load_in_4bit: False
2793
- - llm_int8_threshold: 6.0
2794
- - llm_int8_skip_modules: None
2795
- - llm_int8_enable_fp32_cpu_offload: False
2796
- - llm_int8_has_fp16_weight: False
2797
- - bnb_4bit_quant_type: fp4
2798
- - bnb_4bit_use_double_quant: False
2799
- - bnb_4bit_compute_dtype: float32
2800
-
2801
- ### Framework versions
2802
-
2803
-
2804
- - PEFT 0.6.2
2805
- ## Training procedure
2806
-
2807
-
2808
- The following `bitsandbytes` quantization config was used during training:
2809
- - quant_method: bitsandbytes
2810
- - load_in_8bit: True
2811
- - load_in_4bit: False
2812
- - llm_int8_threshold: 6.0
2813
- - llm_int8_skip_modules: None
2814
- - llm_int8_enable_fp32_cpu_offload: False
2815
- - llm_int8_has_fp16_weight: False
2816
- - bnb_4bit_quant_type: fp4
2817
- - bnb_4bit_use_double_quant: False
2818
- - bnb_4bit_compute_dtype: float32
2819
-
2820
- ### Framework versions
2821
-
2822
-
2823
- - PEFT 0.6.2
2824
- ## Training procedure
2825
-
2826
-
2827
- The following `bitsandbytes` quantization config was used during training:
2828
- - quant_method: bitsandbytes
2829
- - load_in_8bit: True
2830
- - load_in_4bit: False
2831
- - llm_int8_threshold: 6.0
2832
- - llm_int8_skip_modules: None
2833
- - llm_int8_enable_fp32_cpu_offload: False
2834
- - llm_int8_has_fp16_weight: False
2835
- - bnb_4bit_quant_type: fp4
2836
- - bnb_4bit_use_double_quant: False
2837
- - bnb_4bit_compute_dtype: float32
2838
-
2839
- ### Framework versions
2840
-
2841
-
2842
- - PEFT 0.6.2
2843
- ## Training procedure
2844
-
2845
-
2846
- The following `bitsandbytes` quantization config was used during training:
2847
- - quant_method: bitsandbytes
2848
- - load_in_8bit: True
2849
- - load_in_4bit: False
2850
- - llm_int8_threshold: 6.0
2851
- - llm_int8_skip_modules: None
2852
- - llm_int8_enable_fp32_cpu_offload: False
2853
- - llm_int8_has_fp16_weight: False
2854
- - bnb_4bit_quant_type: fp4
2855
- - bnb_4bit_use_double_quant: False
2856
- - bnb_4bit_compute_dtype: float32
2857
-
2858
- ### Framework versions
2859
-
2860
-
2861
- - PEFT 0.6.2
2862
- ## Training procedure
2863
-
2864
-
2865
- The following `bitsandbytes` quantization config was used during training:
2866
- - quant_method: bitsandbytes
2867
- - load_in_8bit: True
2868
- - load_in_4bit: False
2869
- - llm_int8_threshold: 6.0
2870
- - llm_int8_skip_modules: None
2871
- - llm_int8_enable_fp32_cpu_offload: False
2872
- - llm_int8_has_fp16_weight: False
2873
- - bnb_4bit_quant_type: fp4
2874
- - bnb_4bit_use_double_quant: False
2875
- - bnb_4bit_compute_dtype: float32
2876
-
2877
- ### Framework versions
2878
-
2879
-
2880
- - PEFT 0.6.2
2881
- ## Training procedure
2882
-
2883
-
2884
- The following `bitsandbytes` quantization config was used during training:
2885
- - quant_method: bitsandbytes
2886
- - load_in_8bit: True
2887
- - load_in_4bit: False
2888
- - llm_int8_threshold: 6.0
2889
- - llm_int8_skip_modules: None
2890
- - llm_int8_enable_fp32_cpu_offload: False
2891
- - llm_int8_has_fp16_weight: False
2892
- - bnb_4bit_quant_type: fp4
2893
- - bnb_4bit_use_double_quant: False
2894
- - bnb_4bit_compute_dtype: float32
2895
-
2896
- ### Framework versions
2897
-
2898
-
2899
- - PEFT 0.6.2
2900
- ## Training procedure
2901
-
2902
-
2903
- The following `bitsandbytes` quantization config was used during training:
2904
- - quant_method: bitsandbytes
2905
- - load_in_8bit: True
2906
- - load_in_4bit: False
2907
- - llm_int8_threshold: 6.0
2908
- - llm_int8_skip_modules: None
2909
- - llm_int8_enable_fp32_cpu_offload: False
2910
- - llm_int8_has_fp16_weight: False
2911
- - bnb_4bit_quant_type: fp4
2912
- - bnb_4bit_use_double_quant: False
2913
- - bnb_4bit_compute_dtype: float32
2914
-
2915
- ### Framework versions
2916
-
2917
-
2918
- - PEFT 0.6.2
2919
- ## Training procedure
2920
-
2921
-
2922
- The following `bitsandbytes` quantization config was used during training:
2923
- - quant_method: bitsandbytes
2924
- - load_in_8bit: True
2925
- - load_in_4bit: False
2926
- - llm_int8_threshold: 6.0
2927
- - llm_int8_skip_modules: None
2928
- - llm_int8_enable_fp32_cpu_offload: False
2929
- - llm_int8_has_fp16_weight: False
2930
- - bnb_4bit_quant_type: fp4
2931
- - bnb_4bit_use_double_quant: False
2932
- - bnb_4bit_compute_dtype: float32
2933
-
2934
- ### Framework versions
2935
-
2936
-
2937
- - PEFT 0.6.2
2938
- ## Training procedure
2939
-
2940
-
2941
- The following `bitsandbytes` quantization config was used during training:
2942
- - quant_method: bitsandbytes
2943
- - load_in_8bit: True
2944
- - load_in_4bit: False
2945
- - llm_int8_threshold: 6.0
2946
- - llm_int8_skip_modules: None
2947
- - llm_int8_enable_fp32_cpu_offload: False
2948
- - llm_int8_has_fp16_weight: False
2949
- - bnb_4bit_quant_type: fp4
2950
- - bnb_4bit_use_double_quant: False
2951
- - bnb_4bit_compute_dtype: float32
2952
-
2953
- ### Framework versions
2954
-
2955
-
2956
- - PEFT 0.6.2
2957
- ## Training procedure
2958
-
2959
-
2960
- The following `bitsandbytes` quantization config was used during training:
2961
- - quant_method: bitsandbytes
2962
- - load_in_8bit: True
2963
- - load_in_4bit: False
2964
- - llm_int8_threshold: 6.0
2965
- - llm_int8_skip_modules: None
2966
- - llm_int8_enable_fp32_cpu_offload: False
2967
- - llm_int8_has_fp16_weight: False
2968
- - bnb_4bit_quant_type: fp4
2969
- - bnb_4bit_use_double_quant: False
2970
- - bnb_4bit_compute_dtype: float32
2971
-
2972
- ### Framework versions
2973
-
2974
-
2975
- - PEFT 0.6.2
2976
- ## Training procedure
2977
-
2978
-
2979
- The following `bitsandbytes` quantization config was used during training:
2980
- - quant_method: bitsandbytes
2981
- - load_in_8bit: True
2982
- - load_in_4bit: False
2983
- - llm_int8_threshold: 6.0
2984
- - llm_int8_skip_modules: None
2985
- - llm_int8_enable_fp32_cpu_offload: False
2986
- - llm_int8_has_fp16_weight: False
2987
- - bnb_4bit_quant_type: fp4
2988
- - bnb_4bit_use_double_quant: False
2989
- - bnb_4bit_compute_dtype: float32
2990
-
2991
- ### Framework versions
2992
-
2993
-
2994
- - PEFT 0.6.2
2995
- ## Training procedure
2996
-
2997
-
2998
- The following `bitsandbytes` quantization config was used during training:
2999
- - quant_method: bitsandbytes
3000
- - load_in_8bit: True
3001
- - load_in_4bit: False
3002
- - llm_int8_threshold: 6.0
3003
- - llm_int8_skip_modules: None
3004
- - llm_int8_enable_fp32_cpu_offload: False
3005
- - llm_int8_has_fp16_weight: False
3006
- - bnb_4bit_quant_type: fp4
3007
- - bnb_4bit_use_double_quant: False
3008
- - bnb_4bit_compute_dtype: float32
3009
-
3010
- ### Framework versions
3011
-
3012
-
3013
- - PEFT 0.6.2
3014
- ## Training procedure
3015
-
3016
-
3017
- The following `bitsandbytes` quantization config was used during training:
3018
- - quant_method: bitsandbytes
3019
- - load_in_8bit: True
3020
- - load_in_4bit: False
3021
- - llm_int8_threshold: 6.0
3022
- - llm_int8_skip_modules: None
3023
- - llm_int8_enable_fp32_cpu_offload: False
3024
- - llm_int8_has_fp16_weight: False
3025
- - bnb_4bit_quant_type: fp4
3026
- - bnb_4bit_use_double_quant: False
3027
- - bnb_4bit_compute_dtype: float32
3028
-
3029
- ### Framework versions
3030
-
3031
-
3032
- - PEFT 0.6.2
3033
- ## Training procedure
3034
-
3035
-
3036
- The following `bitsandbytes` quantization config was used during training:
3037
- - quant_method: bitsandbytes
3038
- - load_in_8bit: True
3039
- - load_in_4bit: False
3040
- - llm_int8_threshold: 6.0
3041
- - llm_int8_skip_modules: None
3042
- - llm_int8_enable_fp32_cpu_offload: False
3043
- - llm_int8_has_fp16_weight: False
3044
- - bnb_4bit_quant_type: fp4
3045
- - bnb_4bit_use_double_quant: False
3046
- - bnb_4bit_compute_dtype: float32
3047
-
3048
- ### Framework versions
3049
-
3050
-
3051
- - PEFT 0.6.2
3052
- ## Training procedure
3053
-
3054
-
3055
- The following `bitsandbytes` quantization config was used during training:
3056
- - quant_method: bitsandbytes
3057
- - load_in_8bit: True
3058
- - load_in_4bit: False
3059
- - llm_int8_threshold: 6.0
3060
- - llm_int8_skip_modules: None
3061
- - llm_int8_enable_fp32_cpu_offload: False
3062
- - llm_int8_has_fp16_weight: False
3063
- - bnb_4bit_quant_type: fp4
3064
- - bnb_4bit_use_double_quant: False
3065
- - bnb_4bit_compute_dtype: float32
3066
-
3067
- ### Framework versions
3068
-
3069
-
3070
  - PEFT 0.6.2
 
217
  ### Framework versions
218
 
219
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
220
  - PEFT 0.6.2
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5b5720726fa177a677a36861414d4dce2b67578dc472567c018f50f26d8d2305
3
  size 20991608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aac8bddc305a9bef54fe4cbdc58adcb22056174e088f75189c171c497fe6a5b6
3
  size 20991608