Add MiniMax-M3 OSCAR KV rotations
Browse files
MiniMax-M3/README.md
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# MiniMax-M3 OSCAR KV rotations
|
| 2 |
+
|
| 3 |
+
Data-free Hadamard, 60 layers, `[128, 128]`. Calibration was measured to add
|
| 4 |
+
nothing on this model -- the shared rotation is already saturated.
|
MiniMax-M3/k_rotation_hadamard.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f2425507755468f72b4bc0278776ec9504c4bdb35160d0da8fdc865d3789b871
|
| 3 |
+
size 69253
|
MiniMax-M3/v_rotation_hadamard.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6a4fae7c4cdffff5699e79ecd658fc41dd62ceb150781706597589e56929e3d2
|
| 3 |
+
size 69253
|