|
|
--- |
|
|
base_model: |
|
|
- mistralai/Mistral-Large-Instruct-2411 |
|
|
--- |
|
|
|
|
|
Aggressive deep layer compression of Mistral Large 2411 |
|
|
|
|
|
``` |
|
|
Detected 88 source layers. |
|
|
Surgery Plan: 63 operations. |
|
|
Merging Block 47-72 into 5 layers... |
|
|
> Compressing 26 layers -> 5 layers |
|
|
Copying Embeddings, Norms, and Head... |
|
|
Saving Index... |
|
|
Updating Config... |
|
|
|
|
|
DONE! New Model Depth: 67 Layers |
|
|
``` |
|
|
|
|
|
|
|
|
``` |
|
|
Loading Embeddings... |
|
|
Layer | BI Score | Weight Sim | Verdict |
|
|
-------------------------------------------------- |
|
|
0 | 0.5022 | N/A | |
|
|
1 | 0.9008 | 0.5000 | Active Layer |
|
|
2 | 0.8713 | 0.7422 | Active Layer |
|
|
3 | 0.9583 | 0.7109 | Active Layer |
|
|
4 | 0.9647 | 0.8984 | Active Layer |
|
|
5 | 0.9697 | 0.9062 | Active Layer |
|
|
6 | 0.9531 | 0.9102 | Active Layer |
|
|
7 | 0.9682 | 0.9219 | Active Layer |
|
|
8 | 0.9694 | 0.9219 | Active Layer |
|
|
9 | 0.9449 | 0.9219 | Active Layer |
|
|
10 | 0.9621 | 0.9141 | Active Layer |
|
|
11 | 0.9635 | 0.9141 | Active Layer |
|
|
12 | 0.9480 | 0.9336 | Active Layer |
|
|
13 | 0.9689 | 0.9375 | Active Layer |
|
|
14 | 0.9666 | 0.9375 | Active Layer |
|
|
15 | 0.9658 | 0.9336 | Active Layer |
|
|
16 | 0.9609 | 0.9336 | Active Layer |
|
|
17 | 0.9651 | 0.9414 | Active Layer |
|
|
18 | 0.9645 | 0.9414 | Active Layer |
|
|
19 | 0.9633 | 0.9375 | Active Layer |
|
|
20 | 0.9644 | 0.9414 | Active Layer |
|
|
21 | 0.9571 | 0.9453 | Active Layer |
|
|
22 | 0.9557 | 0.9492 | Active Layer |
|
|
23 | 0.9326 | 0.9492 | Active Layer |
|
|
24 | 0.9488 | 0.9492 | Active Layer |
|
|
25 | 0.9479 | 0.9531 | Active Layer |
|
|
26 | 0.9613 | 0.9492 | Active Layer |
|
|
27 | 0.9586 | 0.9492 | Active Layer |
|
|
28 | 0.9398 | 0.9492 | Active Layer |
|
|
29 | 0.9459 | 0.9453 | Active Layer |
|
|
30 | 0.9445 | 0.9492 | Active Layer |
|
|
31 | 0.9378 | 0.9531 | Active Layer |
|
|
32 | 0.9465 | 0.9531 | Active Layer |
|
|
33 | 0.9487 | 0.9531 | Active Layer |
|
|
34 | 0.9458 | 0.9570 | Active Layer |
|
|
35 | 0.9502 | 0.9609 | Active Layer |
|
|
36 | 0.9221 | 0.9609 | Active Layer |
|
|
37 | 0.9402 | 0.9609 | Active Layer |
|
|
38 | 0.9406 | 0.9609 | Active Layer |
|
|
39 | 0.9464 | 0.9648 | Active Layer |
|
|
40 | 0.9593 | 0.9648 | Active Layer |
|
|
41 | 0.9596 | 0.9648 | Active Layer |
|
|
42 | 0.9559 | 0.9648 | Active Layer |
|
|
43 | 0.9710 | 0.9688 | Active Layer |
|
|
44 | 0.9792 | 0.9688 | Active Layer |
|
|
45 | 0.9820 | 0.9727 | Possible Merge |
|
|
46 | 0.9580 | 0.9727 | Active Layer |
|
|
47 | 0.9874 | 0.9727 | Possible Merge |
|
|
48 | 0.9876 | 0.9688 | Possible Merge |
|
|
49 | 0.9894 | 0.9688 | Possible Merge |
|
|
50 | 0.9877 | 0.9648 | Possible Merge |
|
|
51 | 0.9883 | 0.9688 | Possible Merge |
|
|
52 | 0.9897 | 0.9727 | Possible Merge |
|
|
53 | 0.9886 | 0.9727 | Possible Merge |
|
|
54 | 0.9882 | 0.9688 | Possible Merge |
|
|
55 | 0.9864 | 0.9688 | Possible Merge |
|
|
56 | 0.9885 | 0.9727 | Possible Merge |
|
|
57 | 0.9868 | 0.9688 | Possible Merge |
|
|
58 | 0.9899 | 0.9688 | Possible Merge |
|
|
59 | 0.9871 | 0.9688 | Possible Merge |
|
|
60 | 0.9903 | 0.9727 | MERGE CANDIDATE |
|
|
61 | 0.9888 | 0.9766 | Possible Merge |
|
|
62 | 0.9878 | 0.9727 | Possible Merge |
|
|
63 | 0.9899 | 0.9727 | Possible Merge |
|
|
64 | 0.9885 | 0.9727 | Possible Merge |
|
|
65 | 0.9854 | 0.9727 | Possible Merge |
|
|
66 | 0.9886 | 0.9727 | Possible Merge |
|
|
67 | 0.9837 | 0.9688 | Possible Merge |
|
|
68 | 0.9894 | 0.9727 | Possible Merge |
|
|
69 | 0.9879 | 0.9766 | Possible Merge |
|
|
70 | 0.9872 | 0.9688 | Possible Merge |
|
|
71 | 0.9809 | 0.9727 | Possible Merge |
|
|
72 | 0.9827 | 0.9766 | Possible Merge |
|
|
73 | 0.9789 | 0.9766 | Active Layer |
|
|
74 | 0.9790 | 0.9766 | Active Layer |
|
|
75 | 0.9692 | 0.9844 | Active Layer |
|
|
76 | 0.9611 | 0.9766 | Active Layer |
|
|
77 | 0.9762 | 0.9727 | Active Layer |
|
|
78 | 0.9757 | 0.9727 | Active Layer |
|
|
79 | 0.9729 | 0.9766 | Active Layer |
|
|
80 | 0.9744 | 0.9766 | Active Layer |
|
|
81 | 0.9677 | 0.9727 | Active Layer |
|
|
82 | 0.9665 | 0.9766 | Active Layer |
|
|
83 | 0.9643 | 0.9766 | Active Layer |
|
|
84 | 0.9571 | 0.9727 | Active Layer |
|
|
85 | 0.9287 | 0.9727 | Active Layer |
|
|
86 | 0.8072 | 0.9766 | Active Layer |
|
|
87 | 0.6603 | 0.9609 | Active Layer |
|
|
``` |
