Add cpu-amx: torch-free C++/AMX engines (T5Gemma bf16, medium DiT int8, SAME-S/L bf16+int8 decoders, SAME-S/L encoders) + weight blobs + README
Browse files- .gitattributes +8 -0
- cpu-amx/README.md +85 -0
- cpu-amx/dit_medium.so +3 -0
- cpu-amx/dit_medium_core.bin +3 -0
- cpu-amx/dit_medium_core_manifest.txt +627 -0
- cpu-amx/dit_medium_kernels.tar.gz +3 -0
- cpu-amx/same_l_decoder_bf16.so +3 -0
- cpu-amx/same_l_decoder_bf16_weights.bin +3 -0
- cpu-amx/same_l_decoder_bf16_weights_manifest.txt +222 -0
- cpu-amx/same_l_decoder_int8.so +3 -0
- cpu-amx/same_l_decoder_int8_weights.bin +3 -0
- cpu-amx/same_l_decoder_int8_weights_manifest.txt +272 -0
- cpu-amx/same_l_encoder.so +3 -0
- cpu-amx/same_l_encoder_weights.bin +3 -0
- cpu-amx/same_l_encoder_weights_f32.bin +3 -0
- cpu-amx/same_l_encoder_weights_f32_manifest.txt +50 -0
- cpu-amx/same_l_encoder_weights_manifest.txt +224 -0
- cpu-amx/same_s_decoder_bf16.so +3 -0
- cpu-amx/same_s_decoder_bf16_weights.bin +3 -0
- cpu-amx/same_s_decoder_bf16_weights_manifest.txt +114 -0
- cpu-amx/same_s_decoder_int8.so +3 -0
- cpu-amx/same_s_decoder_int8_weights.bin +3 -0
- cpu-amx/same_s_decoder_int8_weights_manifest.txt +140 -0
- cpu-amx/same_s_encoder.so +3 -0
- cpu-amx/same_s_encoder_weights.bin +3 -0
- cpu-amx/same_s_encoder_weights_manifest.txt +116 -0
- cpu-amx/t5gemma.so +3 -0
- cpu-amx/t5gemma_weights.bin +3 -0
- cpu-amx/t5gemma_weights_manifest.txt +135 -0
.gitattributes
CHANGED
|
@@ -39,3 +39,11 @@ tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
| 39 |
|
| 40 |
# ONNX external-data blobs (for >2GB models)
|
| 41 |
*.onnx.data filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
# ONNX external-data blobs (for >2GB models)
|
| 41 |
*.onnx.data filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
cpu-amx/dit_medium.so filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
cpu-amx/same_l_decoder_bf16.so filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
cpu-amx/same_l_decoder_int8.so filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
cpu-amx/same_l_encoder.so filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
cpu-amx/same_s_decoder_bf16.so filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
cpu-amx/same_s_decoder_int8.so filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
cpu-amx/same_s_encoder.so filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
cpu-amx/t5gemma.so filter=lfs diff=lfs merge=lfs -text
|
cpu-amx/README.md
ADDED
|
@@ -0,0 +1,85 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
tags:
|
| 4 |
+
- stable-audio
|
| 5 |
+
- text-to-audio
|
| 6 |
+
- cpu
|
| 7 |
+
- amx
|
| 8 |
+
- int8
|
| 9 |
+
- bf16
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Stable Audio 3 — cpu-amx engines
|
| 13 |
+
|
| 14 |
+
Torch-free **C++ / Intel-AMX** inference engines for **Stable Audio 3 medium**, running the whole
|
| 15 |
+
pipeline on CPU with no PyTorch, MLX, TFLite, or stable-audio-tools at runtime — just the `.so`s + numpy.
|
| 16 |
+
|
| 17 |
+
```
|
| 18 |
+
prompt ─▶ T5Gemma (C++ AMX) ─▶ DiT pingpong (C++ AMX int8) ─▶ SAME-S/L decoder (C++ AMX) ─▶ WAV
|
| 19 |
+
audio ─▶ SAME-S/L encoder (C++ AMX) ─▶ latent ─▶ … (audio-to-audio / inpainting)
|
| 20 |
+
```
|
| 21 |
+
|
| 22 |
+
The runtime code (CLI, gradio, sampler) lives in the GitHub repo under
|
| 23 |
+
`optimized/cpu-amx/`; **this HF folder holds the compiled engines + their weight blobs**, which the
|
| 24 |
+
repo downloads on first use.
|
| 25 |
+
|
| 26 |
+
## ⚠ Target hardware
|
| 27 |
+
|
| 28 |
+
These `.so`s are prebuilt for **AMX-equipped Intel Xeon** — 4th/5th-gen Sapphire Rapids / Emerald
|
| 29 |
+
Rapids (the CPU must report `amx_tile` + `amx_int8` + `amx_bf16`, i.e. `cpu_isa_avx10_1_512_amx`). The
|
| 30 |
+
headline speed comes entirely from the AMX matrix unit: **on a non-AMX CPU these engines are not faster
|
| 31 |
+
than fp32** — rebuild from source (`optimized/cpu-amx/build/`) for other ISAs. Linux x86-64, glibc ≥ 2.31.
|
| 32 |
+
|
| 33 |
+
## Model classes & how each was quantized
|
| 34 |
+
|
| 35 |
+
| file(s) | model | precision / method | notes |
|
| 36 |
+
|---|---|---|---|
|
| 37 |
+
| `t5gemma.*` | T5Gemma-b-b-ul2 encoder (text conditioner) | **bf16** AMX GEMMs, fp32 islands (RMSNorm/softmax/softcap/RoPE) | standard softmax attention → bf16-safe |
|
| 38 |
+
| `dit_medium.*` + `dit_medium_kernels.tar.gz` | medium DiT (24-block rectified-flow) | **int8** — fully-fused all-integer, AOT-compiled Triton kernels + oneDNN | naive RTN (weight-PTQ *hurts* the DiT — see below) |
|
| 39 |
+
| `same_{s,l}_decoder_bf16.*` | SAME-S (50M) / SAME-L (426M) decoder | **bf16** AMX GEMMs, fp32 differential-attention islands | highest-fidelity CPU decoders |
|
| 40 |
+
| `same_{s,l}_decoder_int8.*` | SAME-S / SAME-L decoder | **int8 (w8a8, fused)** via **SmoothQuant α0.9 → GPTQ** ("improved" grid) | ½ size; SAME-S +1.2 dB over naive int8 on real music |
|
| 41 |
+
| `same_{s,l}_encoder.*` | SAME-S / SAME-L encoder (audio→latent) | **bf16** AMX GEMMs, fp32 differential-attention islands | for audio-to-audio / inpainting |
|
| 42 |
+
| `same_l_encoder_weights_f32.bin` | SAME-L encoder (optional) | **fp32** refinement mode | max-fidelity; bf16 is the default |
|
| 43 |
+
|
| 44 |
+
**Quantization findings baked into these choices** (full write-up in the repo's `LESSONS.md`):
|
| 45 |
+
- The medium **DiT stays naive-int8** — GPTQ/SmoothQuant *lower* its accuracy (its adaLN-modulated qkv +
|
| 46 |
+
the chaotic 8-step sampler make a calibration-averaged Hessian overfit and generalize worse than RTN).
|
| 47 |
+
- The **decoders are activation-limited**: SmoothQuant (activation-outlier migration) does the work,
|
| 48 |
+
GPTQ just lets α go higher. The gain **transfers to real music for SAME-S (+1.2 dB), not SAME-L**.
|
| 49 |
+
- **bf16** is used where attention is either standard-softmax (T5Gemma) or where the fp32 differential-
|
| 50 |
+
attention island absorbs the rounding (decoders/encoders); RoPE stays fp32 (a bf16-RoPE bug clips
|
| 51 |
+
long renders).
|
| 52 |
+
|
| 53 |
+
## Benchmarks (single AMX Xeon socket, this build)
|
| 54 |
+
|
| 55 |
+
**Per-stage** (the encoder/DiT run once per generation; the decoder once):
|
| 56 |
+
|
| 57 |
+
| stage | precision | time | ×realtime | vs alternative |
|
| 58 |
+
|---|---|---|---|---|
|
| 59 |
+
| T5Gemma encode (256 tok) | bf16 | **28 ms** | ~1000× | 2.5× faster than TFLite fp16 |
|
| 60 |
+
| SAME-L decode (20 s clip) | bf16 / int8 | 1028 / **618 ms** | 20× / **32×** | int8 1.66× faster than bf16 |
|
| 61 |
+
| SAME-S decode (20 s clip) | bf16 / int8 | 162 / **124 ms** | 124× / **161×** | int8 1.30× faster |
|
| 62 |
+
| SAME-S encode (30 s clip) | bf16 | **208 ms** | 143× | (a2a/inpaint) |
|
| 63 |
+
| SAME-L encode (30 s clip) | bf16 | 1046 ms | 28× | fp32 mode 4169 ms / 7× |
|
| 64 |
+
|
| 65 |
+
**End-to-end** (medium DiT, 8-step): a 3 s clip generates in ~6 s (t2a, CFG, and — with the C++
|
| 66 |
+
encoders — audio-to-audio & inpainting alike). All 14 CLI configs (t2a / a2a / inpaint / CFG / negative
|
| 67 |
+
prompt / APG / step counts / unconditional) pass the release self-test.
|
| 68 |
+
|
| 69 |
+
**Quality** (vs the fp32/reference of each stage): T5Gemma cos 0.9997 (61–67 dB); bf16 decoders ~62 dB
|
| 70 |
+
vs the torch port; int8 decoders ~40–49 dB; encoders cos 0.9995 (SAME-S) / mean-cos 0.99996 (SAME-L
|
| 71 |
+
bf16, 107–116 dB in fp32 mode). Audio quality is ear-verified, not dB-gated (PSNR is a harsh proxy).
|
| 72 |
+
|
| 73 |
+
## Usage
|
| 74 |
+
|
| 75 |
+
Use through the GitHub repo (`Stability-AI/stable-audio-3` → `optimized/cpu-amx/`), which pulls these
|
| 76 |
+
files on first run:
|
| 77 |
+
|
| 78 |
+
```bash
|
| 79 |
+
optimized/cpu-amx/sa3 --prompt "warm analog synthwave, 120 bpm" --seconds 10
|
| 80 |
+
optimized/cpu-amx/sa3-gradio # web UI
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
Files are flat and self-describing: each engine is `<name>.so` + `<name>_weights.bin` (+ a
|
| 84 |
+
`_manifest.txt` giving the mmap layout the loader reads). The DiT additionally needs
|
| 85 |
+
`dit_medium_core.bin` (int8 block weights) and `dit_medium_kernels.tar.gz` (its AOT Triton kernels).
|
cpu-amx/dit_medium.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:79a56abbb7217d7947b69dcf4d570acf7b2d33b9596ec149ad20edd96998352e
|
| 3 |
+
size 66968384
|
cpu-amx/dit_medium_core.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13dc7d122d88fa705ee9d6407b5c0e9b3088676c720cc90bae0cdb3fafe0bd31
|
| 3 |
+
size 1475244036
|
cpu-amx/dit_medium_core_manifest.txt
ADDED
|
@@ -0,0 +1,627 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Wpre.wt f32 0 65536 256 256
|
| 2 |
+
Wpost.wt f32 262144 65536 256 256
|
| 3 |
+
cond0.wt f32 524288 1179648 768 1536
|
| 4 |
+
cond2.wt f32 5242880 2359296 1536 1536
|
| 5 |
+
gl0.wt f32 14680064 1179648 768 1536
|
| 6 |
+
gl2.wt f32 19398656 2359296 1536 1536
|
| 7 |
+
ts0.wt f32 28835840 393216 256 1536
|
| 8 |
+
ts0.bias f32 30408704 1536 1536
|
| 9 |
+
ts2.wt f32 30414848 2359296 1536 1536
|
| 10 |
+
ts2.bias f32 39852032 1536 1536
|
| 11 |
+
gce0.wt f32 39858176 2359296 1536 1536
|
| 12 |
+
gce0.bias f32 49295360 1536 1536
|
| 13 |
+
gce2.wt f32 49301504 14155776 1536 9216
|
| 14 |
+
gce2.bias f32 105924608 9216 9216
|
| 15 |
+
pin.q i8 105961472 393216 256 1536
|
| 16 |
+
pin.scale f32 106354688 1536 1536
|
| 17 |
+
pout.q i8 106360832 393216 1536 256
|
| 18 |
+
pout.scale f32 106754048 256 256
|
| 19 |
+
mem f32 106755072 98304 64 1536
|
| 20 |
+
freqs f32 107148288 128 128
|
| 21 |
+
rope_cos f32 107148800 12288 384 32
|
| 22 |
+
rope_sin f32 107197952 12288 384 32
|
| 23 |
+
b0.pre_g f32 107247104 1536 1536
|
| 24 |
+
b0.ff_g f32 107253248 1536 1536
|
| 25 |
+
b0.cx_g f32 107259392 1536 1536
|
| 26 |
+
b0.s_qn f32 107265536 64 64
|
| 27 |
+
b0.s_kn f32 107265792 64 64
|
| 28 |
+
b0.c_qn f32 107266048 64 64
|
| 29 |
+
b0.c_kn f32 107266304 64 64
|
| 30 |
+
b0.ssg f32 107266560 9216 9216
|
| 31 |
+
b0.local f32 107303424 1536 1536
|
| 32 |
+
b0.s_qkv.q i8 107309568 11796480 1536 7680
|
| 33 |
+
b0.s_qkv.scale f32 119106048 7680 7680
|
| 34 |
+
b0.s_out.q i8 119136768 2359296 1536 1536
|
| 35 |
+
b0.s_out.scale f32 121496064 1536 1536
|
| 36 |
+
b0.c_q.q i8 121502208 4718592 1536 3072
|
| 37 |
+
b0.c_q.scale f32 126220800 3072 3072
|
| 38 |
+
b0.c_kv.q i8 126233088 7077888 1536 4608
|
| 39 |
+
b0.c_kv.scale f32 133310976 4608 4608
|
| 40 |
+
b0.c_out.q i8 133329408 2359296 1536 1536
|
| 41 |
+
b0.c_out.scale f32 135688704 1536 1536
|
| 42 |
+
b0.ff0.q i8 135694848 18874368 1536 12288
|
| 43 |
+
b0.ff0.scale f32 154569216 12288 12288
|
| 44 |
+
b0.ff0.bias f32 154618368 12288 12288
|
| 45 |
+
b0.ff2.q i8 154667520 9437184 6144 1536
|
| 46 |
+
b0.ff2.scale f32 164104704 1536 1536
|
| 47 |
+
b0.ff2.bias f32 164110848 1536 1536
|
| 48 |
+
b1.pre_g f32 164116992 1536 1536
|
| 49 |
+
b1.ff_g f32 164123136 1536 1536
|
| 50 |
+
b1.cx_g f32 164129280 1536 1536
|
| 51 |
+
b1.s_qn f32 164135424 64 64
|
| 52 |
+
b1.s_kn f32 164135680 64 64
|
| 53 |
+
b1.c_qn f32 164135936 64 64
|
| 54 |
+
b1.c_kn f32 164136192 64 64
|
| 55 |
+
b1.ssg f32 164136448 9216 9216
|
| 56 |
+
b1.local f32 164173312 1536 1536
|
| 57 |
+
b1.s_qkv.q i8 164179456 11796480 1536 7680
|
| 58 |
+
b1.s_qkv.scale f32 175975936 7680 7680
|
| 59 |
+
b1.s_out.q i8 176006656 2359296 1536 1536
|
| 60 |
+
b1.s_out.scale f32 178365952 1536 1536
|
| 61 |
+
b1.c_q.q i8 178372096 4718592 1536 3072
|
| 62 |
+
b1.c_q.scale f32 183090688 3072 3072
|
| 63 |
+
b1.c_kv.q i8 183102976 7077888 1536 4608
|
| 64 |
+
b1.c_kv.scale f32 190180864 4608 4608
|
| 65 |
+
b1.c_out.q i8 190199296 2359296 1536 1536
|
| 66 |
+
b1.c_out.scale f32 192558592 1536 1536
|
| 67 |
+
b1.ff0.q i8 192564736 18874368 1536 12288
|
| 68 |
+
b1.ff0.scale f32 211439104 12288 12288
|
| 69 |
+
b1.ff0.bias f32 211488256 12288 12288
|
| 70 |
+
b1.ff2.q i8 211537408 9437184 6144 1536
|
| 71 |
+
b1.ff2.scale f32 220974592 1536 1536
|
| 72 |
+
b1.ff2.bias f32 220980736 1536 1536
|
| 73 |
+
b2.pre_g f32 220986880 1536 1536
|
| 74 |
+
b2.ff_g f32 220993024 1536 1536
|
| 75 |
+
b2.cx_g f32 220999168 1536 1536
|
| 76 |
+
b2.s_qn f32 221005312 64 64
|
| 77 |
+
b2.s_kn f32 221005568 64 64
|
| 78 |
+
b2.c_qn f32 221005824 64 64
|
| 79 |
+
b2.c_kn f32 221006080 64 64
|
| 80 |
+
b2.ssg f32 221006336 9216 9216
|
| 81 |
+
b2.local f32 221043200 1536 1536
|
| 82 |
+
b2.s_qkv.q i8 221049344 11796480 1536 7680
|
| 83 |
+
b2.s_qkv.scale f32 232845824 7680 7680
|
| 84 |
+
b2.s_out.q i8 232876544 2359296 1536 1536
|
| 85 |
+
b2.s_out.scale f32 235235840 1536 1536
|
| 86 |
+
b2.c_q.q i8 235241984 4718592 1536 3072
|
| 87 |
+
b2.c_q.scale f32 239960576 3072 3072
|
| 88 |
+
b2.c_kv.q i8 239972864 7077888 1536 4608
|
| 89 |
+
b2.c_kv.scale f32 247050752 4608 4608
|
| 90 |
+
b2.c_out.q i8 247069184 2359296 1536 1536
|
| 91 |
+
b2.c_out.scale f32 249428480 1536 1536
|
| 92 |
+
b2.ff0.q i8 249434624 18874368 1536 12288
|
| 93 |
+
b2.ff0.scale f32 268308992 12288 12288
|
| 94 |
+
b2.ff0.bias f32 268358144 12288 12288
|
| 95 |
+
b2.ff2.q i8 268407296 9437184 6144 1536
|
| 96 |
+
b2.ff2.scale f32 277844480 1536 1536
|
| 97 |
+
b2.ff2.bias f32 277850624 1536 1536
|
| 98 |
+
b3.pre_g f32 277856768 1536 1536
|
| 99 |
+
b3.ff_g f32 277862912 1536 1536
|
| 100 |
+
b3.cx_g f32 277869056 1536 1536
|
| 101 |
+
b3.s_qn f32 277875200 64 64
|
| 102 |
+
b3.s_kn f32 277875456 64 64
|
| 103 |
+
b3.c_qn f32 277875712 64 64
|
| 104 |
+
b3.c_kn f32 277875968 64 64
|
| 105 |
+
b3.ssg f32 277876224 9216 9216
|
| 106 |
+
b3.local f32 277913088 1536 1536
|
| 107 |
+
b3.s_qkv.q i8 277919232 11796480 1536 7680
|
| 108 |
+
b3.s_qkv.scale f32 289715712 7680 7680
|
| 109 |
+
b3.s_out.q i8 289746432 2359296 1536 1536
|
| 110 |
+
b3.s_out.scale f32 292105728 1536 1536
|
| 111 |
+
b3.c_q.q i8 292111872 4718592 1536 3072
|
| 112 |
+
b3.c_q.scale f32 296830464 3072 3072
|
| 113 |
+
b3.c_kv.q i8 296842752 7077888 1536 4608
|
| 114 |
+
b3.c_kv.scale f32 303920640 4608 4608
|
| 115 |
+
b3.c_out.q i8 303939072 2359296 1536 1536
|
| 116 |
+
b3.c_out.scale f32 306298368 1536 1536
|
| 117 |
+
b3.ff0.q i8 306304512 18874368 1536 12288
|
| 118 |
+
b3.ff0.scale f32 325178880 12288 12288
|
| 119 |
+
b3.ff0.bias f32 325228032 12288 12288
|
| 120 |
+
b3.ff2.q i8 325277184 9437184 6144 1536
|
| 121 |
+
b3.ff2.scale f32 334714368 1536 1536
|
| 122 |
+
b3.ff2.bias f32 334720512 1536 1536
|
| 123 |
+
b4.pre_g f32 334726656 1536 1536
|
| 124 |
+
b4.ff_g f32 334732800 1536 1536
|
| 125 |
+
b4.cx_g f32 334738944 1536 1536
|
| 126 |
+
b4.s_qn f32 334745088 64 64
|
| 127 |
+
b4.s_kn f32 334745344 64 64
|
| 128 |
+
b4.c_qn f32 334745600 64 64
|
| 129 |
+
b4.c_kn f32 334745856 64 64
|
| 130 |
+
b4.ssg f32 334746112 9216 9216
|
| 131 |
+
b4.local f32 334782976 1536 1536
|
| 132 |
+
b4.s_qkv.q i8 334789120 11796480 1536 7680
|
| 133 |
+
b4.s_qkv.scale f32 346585600 7680 7680
|
| 134 |
+
b4.s_out.q i8 346616320 2359296 1536 1536
|
| 135 |
+
b4.s_out.scale f32 348975616 1536 1536
|
| 136 |
+
b4.c_q.q i8 348981760 4718592 1536 3072
|
| 137 |
+
b4.c_q.scale f32 353700352 3072 3072
|
| 138 |
+
b4.c_kv.q i8 353712640 7077888 1536 4608
|
| 139 |
+
b4.c_kv.scale f32 360790528 4608 4608
|
| 140 |
+
b4.c_out.q i8 360808960 2359296 1536 1536
|
| 141 |
+
b4.c_out.scale f32 363168256 1536 1536
|
| 142 |
+
b4.ff0.q i8 363174400 18874368 1536 12288
|
| 143 |
+
b4.ff0.scale f32 382048768 12288 12288
|
| 144 |
+
b4.ff0.bias f32 382097920 12288 12288
|
| 145 |
+
b4.ff2.q i8 382147072 9437184 6144 1536
|
| 146 |
+
b4.ff2.scale f32 391584256 1536 1536
|
| 147 |
+
b4.ff2.bias f32 391590400 1536 1536
|
| 148 |
+
b5.pre_g f32 391596544 1536 1536
|
| 149 |
+
b5.ff_g f32 391602688 1536 1536
|
| 150 |
+
b5.cx_g f32 391608832 1536 1536
|
| 151 |
+
b5.s_qn f32 391614976 64 64
|
| 152 |
+
b5.s_kn f32 391615232 64 64
|
| 153 |
+
b5.c_qn f32 391615488 64 64
|
| 154 |
+
b5.c_kn f32 391615744 64 64
|
| 155 |
+
b5.ssg f32 391616000 9216 9216
|
| 156 |
+
b5.local f32 391652864 1536 1536
|
| 157 |
+
b5.s_qkv.q i8 391659008 11796480 1536 7680
|
| 158 |
+
b5.s_qkv.scale f32 403455488 7680 7680
|
| 159 |
+
b5.s_out.q i8 403486208 2359296 1536 1536
|
| 160 |
+
b5.s_out.scale f32 405845504 1536 1536
|
| 161 |
+
b5.c_q.q i8 405851648 4718592 1536 3072
|
| 162 |
+
b5.c_q.scale f32 410570240 3072 3072
|
| 163 |
+
b5.c_kv.q i8 410582528 7077888 1536 4608
|
| 164 |
+
b5.c_kv.scale f32 417660416 4608 4608
|
| 165 |
+
b5.c_out.q i8 417678848 2359296 1536 1536
|
| 166 |
+
b5.c_out.scale f32 420038144 1536 1536
|
| 167 |
+
b5.ff0.q i8 420044288 18874368 1536 12288
|
| 168 |
+
b5.ff0.scale f32 438918656 12288 12288
|
| 169 |
+
b5.ff0.bias f32 438967808 12288 12288
|
| 170 |
+
b5.ff2.q i8 439016960 9437184 6144 1536
|
| 171 |
+
b5.ff2.scale f32 448454144 1536 1536
|
| 172 |
+
b5.ff2.bias f32 448460288 1536 1536
|
| 173 |
+
b6.pre_g f32 448466432 1536 1536
|
| 174 |
+
b6.ff_g f32 448472576 1536 1536
|
| 175 |
+
b6.cx_g f32 448478720 1536 1536
|
| 176 |
+
b6.s_qn f32 448484864 64 64
|
| 177 |
+
b6.s_kn f32 448485120 64 64
|
| 178 |
+
b6.c_qn f32 448485376 64 64
|
| 179 |
+
b6.c_kn f32 448485632 64 64
|
| 180 |
+
b6.ssg f32 448485888 9216 9216
|
| 181 |
+
b6.local f32 448522752 1536 1536
|
| 182 |
+
b6.s_qkv.q i8 448528896 11796480 1536 7680
|
| 183 |
+
b6.s_qkv.scale f32 460325376 7680 7680
|
| 184 |
+
b6.s_out.q i8 460356096 2359296 1536 1536
|
| 185 |
+
b6.s_out.scale f32 462715392 1536 1536
|
| 186 |
+
b6.c_q.q i8 462721536 4718592 1536 3072
|
| 187 |
+
b6.c_q.scale f32 467440128 3072 3072
|
| 188 |
+
b6.c_kv.q i8 467452416 7077888 1536 4608
|
| 189 |
+
b6.c_kv.scale f32 474530304 4608 4608
|
| 190 |
+
b6.c_out.q i8 474548736 2359296 1536 1536
|
| 191 |
+
b6.c_out.scale f32 476908032 1536 1536
|
| 192 |
+
b6.ff0.q i8 476914176 18874368 1536 12288
|
| 193 |
+
b6.ff0.scale f32 495788544 12288 12288
|
| 194 |
+
b6.ff0.bias f32 495837696 12288 12288
|
| 195 |
+
b6.ff2.q i8 495886848 9437184 6144 1536
|
| 196 |
+
b6.ff2.scale f32 505324032 1536 1536
|
| 197 |
+
b6.ff2.bias f32 505330176 1536 1536
|
| 198 |
+
b7.pre_g f32 505336320 1536 1536
|
| 199 |
+
b7.ff_g f32 505342464 1536 1536
|
| 200 |
+
b7.cx_g f32 505348608 1536 1536
|
| 201 |
+
b7.s_qn f32 505354752 64 64
|
| 202 |
+
b7.s_kn f32 505355008 64 64
|
| 203 |
+
b7.c_qn f32 505355264 64 64
|
| 204 |
+
b7.c_kn f32 505355520 64 64
|
| 205 |
+
b7.ssg f32 505355776 9216 9216
|
| 206 |
+
b7.local f32 505392640 1536 1536
|
| 207 |
+
b7.s_qkv.q i8 505398784 11796480 1536 7680
|
| 208 |
+
b7.s_qkv.scale f32 517195264 7680 7680
|
| 209 |
+
b7.s_out.q i8 517225984 2359296 1536 1536
|
| 210 |
+
b7.s_out.scale f32 519585280 1536 1536
|
| 211 |
+
b7.c_q.q i8 519591424 4718592 1536 3072
|
| 212 |
+
b7.c_q.scale f32 524310016 3072 3072
|
| 213 |
+
b7.c_kv.q i8 524322304 7077888 1536 4608
|
| 214 |
+
b7.c_kv.scale f32 531400192 4608 4608
|
| 215 |
+
b7.c_out.q i8 531418624 2359296 1536 1536
|
| 216 |
+
b7.c_out.scale f32 533777920 1536 1536
|
| 217 |
+
b7.ff0.q i8 533784064 18874368 1536 12288
|
| 218 |
+
b7.ff0.scale f32 552658432 12288 12288
|
| 219 |
+
b7.ff0.bias f32 552707584 12288 12288
|
| 220 |
+
b7.ff2.q i8 552756736 9437184 6144 1536
|
| 221 |
+
b7.ff2.scale f32 562193920 1536 1536
|
| 222 |
+
b7.ff2.bias f32 562200064 1536 1536
|
| 223 |
+
b8.pre_g f32 562206208 1536 1536
|
| 224 |
+
b8.ff_g f32 562212352 1536 1536
|
| 225 |
+
b8.cx_g f32 562218496 1536 1536
|
| 226 |
+
b8.s_qn f32 562224640 64 64
|
| 227 |
+
b8.s_kn f32 562224896 64 64
|
| 228 |
+
b8.c_qn f32 562225152 64 64
|
| 229 |
+
b8.c_kn f32 562225408 64 64
|
| 230 |
+
b8.ssg f32 562225664 9216 9216
|
| 231 |
+
b8.local f32 562262528 1536 1536
|
| 232 |
+
b8.s_qkv.q i8 562268672 11796480 1536 7680
|
| 233 |
+
b8.s_qkv.scale f32 574065152 7680 7680
|
| 234 |
+
b8.s_out.q i8 574095872 2359296 1536 1536
|
| 235 |
+
b8.s_out.scale f32 576455168 1536 1536
|
| 236 |
+
b8.c_q.q i8 576461312 4718592 1536 3072
|
| 237 |
+
b8.c_q.scale f32 581179904 3072 3072
|
| 238 |
+
b8.c_kv.q i8 581192192 7077888 1536 4608
|
| 239 |
+
b8.c_kv.scale f32 588270080 4608 4608
|
| 240 |
+
b8.c_out.q i8 588288512 2359296 1536 1536
|
| 241 |
+
b8.c_out.scale f32 590647808 1536 1536
|
| 242 |
+
b8.ff0.q i8 590653952 18874368 1536 12288
|
| 243 |
+
b8.ff0.scale f32 609528320 12288 12288
|
| 244 |
+
b8.ff0.bias f32 609577472 12288 12288
|
| 245 |
+
b8.ff2.q i8 609626624 9437184 6144 1536
|
| 246 |
+
b8.ff2.scale f32 619063808 1536 1536
|
| 247 |
+
b8.ff2.bias f32 619069952 1536 1536
|
| 248 |
+
b9.pre_g f32 619076096 1536 1536
|
| 249 |
+
b9.ff_g f32 619082240 1536 1536
|
| 250 |
+
b9.cx_g f32 619088384 1536 1536
|
| 251 |
+
b9.s_qn f32 619094528 64 64
|
| 252 |
+
b9.s_kn f32 619094784 64 64
|
| 253 |
+
b9.c_qn f32 619095040 64 64
|
| 254 |
+
b9.c_kn f32 619095296 64 64
|
| 255 |
+
b9.ssg f32 619095552 9216 9216
|
| 256 |
+
b9.local f32 619132416 1536 1536
|
| 257 |
+
b9.s_qkv.q i8 619138560 11796480 1536 7680
|
| 258 |
+
b9.s_qkv.scale f32 630935040 7680 7680
|
| 259 |
+
b9.s_out.q i8 630965760 2359296 1536 1536
|
| 260 |
+
b9.s_out.scale f32 633325056 1536 1536
|
| 261 |
+
b9.c_q.q i8 633331200 4718592 1536 3072
|
| 262 |
+
b9.c_q.scale f32 638049792 3072 3072
|
| 263 |
+
b9.c_kv.q i8 638062080 7077888 1536 4608
|
| 264 |
+
b9.c_kv.scale f32 645139968 4608 4608
|
| 265 |
+
b9.c_out.q i8 645158400 2359296 1536 1536
|
| 266 |
+
b9.c_out.scale f32 647517696 1536 1536
|
| 267 |
+
b9.ff0.q i8 647523840 18874368 1536 12288
|
| 268 |
+
b9.ff0.scale f32 666398208 12288 12288
|
| 269 |
+
b9.ff0.bias f32 666447360 12288 12288
|
| 270 |
+
b9.ff2.q i8 666496512 9437184 6144 1536
|
| 271 |
+
b9.ff2.scale f32 675933696 1536 1536
|
| 272 |
+
b9.ff2.bias f32 675939840 1536 1536
|
| 273 |
+
b10.pre_g f32 675945984 1536 1536
|
| 274 |
+
b10.ff_g f32 675952128 1536 1536
|
| 275 |
+
b10.cx_g f32 675958272 1536 1536
|
| 276 |
+
b10.s_qn f32 675964416 64 64
|
| 277 |
+
b10.s_kn f32 675964672 64 64
|
| 278 |
+
b10.c_qn f32 675964928 64 64
|
| 279 |
+
b10.c_kn f32 675965184 64 64
|
| 280 |
+
b10.ssg f32 675965440 9216 9216
|
| 281 |
+
b10.local f32 676002304 1536 1536
|
| 282 |
+
b10.s_qkv.q i8 676008448 11796480 1536 7680
|
| 283 |
+
b10.s_qkv.scale f32 687804928 7680 7680
|
| 284 |
+
b10.s_out.q i8 687835648 2359296 1536 1536
|
| 285 |
+
b10.s_out.scale f32 690194944 1536 1536
|
| 286 |
+
b10.c_q.q i8 690201088 4718592 1536 3072
|
| 287 |
+
b10.c_q.scale f32 694919680 3072 3072
|
| 288 |
+
b10.c_kv.q i8 694931968 7077888 1536 4608
|
| 289 |
+
b10.c_kv.scale f32 702009856 4608 4608
|
| 290 |
+
b10.c_out.q i8 702028288 2359296 1536 1536
|
| 291 |
+
b10.c_out.scale f32 704387584 1536 1536
|
| 292 |
+
b10.ff0.q i8 704393728 18874368 1536 12288
|
| 293 |
+
b10.ff0.scale f32 723268096 12288 12288
|
| 294 |
+
b10.ff0.bias f32 723317248 12288 12288
|
| 295 |
+
b10.ff2.q i8 723366400 9437184 6144 1536
|
| 296 |
+
b10.ff2.scale f32 732803584 1536 1536
|
| 297 |
+
b10.ff2.bias f32 732809728 1536 1536
|
| 298 |
+
b11.pre_g f32 732815872 1536 1536
|
| 299 |
+
b11.ff_g f32 732822016 1536 1536
|
| 300 |
+
b11.cx_g f32 732828160 1536 1536
|
| 301 |
+
b11.s_qn f32 732834304 64 64
|
| 302 |
+
b11.s_kn f32 732834560 64 64
|
| 303 |
+
b11.c_qn f32 732834816 64 64
|
| 304 |
+
b11.c_kn f32 732835072 64 64
|
| 305 |
+
b11.ssg f32 732835328 9216 9216
|
| 306 |
+
b11.local f32 732872192 1536 1536
|
| 307 |
+
b11.s_qkv.q i8 732878336 11796480 1536 7680
|
| 308 |
+
b11.s_qkv.scale f32 744674816 7680 7680
|
| 309 |
+
b11.s_out.q i8 744705536 2359296 1536 1536
|
| 310 |
+
b11.s_out.scale f32 747064832 1536 1536
|
| 311 |
+
b11.c_q.q i8 747070976 4718592 1536 3072
|
| 312 |
+
b11.c_q.scale f32 751789568 3072 3072
|
| 313 |
+
b11.c_kv.q i8 751801856 7077888 1536 4608
|
| 314 |
+
b11.c_kv.scale f32 758879744 4608 4608
|
| 315 |
+
b11.c_out.q i8 758898176 2359296 1536 1536
|
| 316 |
+
b11.c_out.scale f32 761257472 1536 1536
|
| 317 |
+
b11.ff0.q i8 761263616 18874368 1536 12288
|
| 318 |
+
b11.ff0.scale f32 780137984 12288 12288
|
| 319 |
+
b11.ff0.bias f32 780187136 12288 12288
|
| 320 |
+
b11.ff2.q i8 780236288 9437184 6144 1536
|
| 321 |
+
b11.ff2.scale f32 789673472 1536 1536
|
| 322 |
+
b11.ff2.bias f32 789679616 1536 1536
|
| 323 |
+
b12.pre_g f32 789685760 1536 1536
|
| 324 |
+
b12.ff_g f32 789691904 1536 1536
|
| 325 |
+
b12.cx_g f32 789698048 1536 1536
|
| 326 |
+
b12.s_qn f32 789704192 64 64
|
| 327 |
+
b12.s_kn f32 789704448 64 64
|
| 328 |
+
b12.c_qn f32 789704704 64 64
|
| 329 |
+
b12.c_kn f32 789704960 64 64
|
| 330 |
+
b12.ssg f32 789705216 9216 9216
|
| 331 |
+
b12.local f32 789742080 1536 1536
|
| 332 |
+
b12.s_qkv.q i8 789748224 11796480 1536 7680
|
| 333 |
+
b12.s_qkv.scale f32 801544704 7680 7680
|
| 334 |
+
b12.s_out.q i8 801575424 2359296 1536 1536
|
| 335 |
+
b12.s_out.scale f32 803934720 1536 1536
|
| 336 |
+
b12.c_q.q i8 803940864 4718592 1536 3072
|
| 337 |
+
b12.c_q.scale f32 808659456 3072 3072
|
| 338 |
+
b12.c_kv.q i8 808671744 7077888 1536 4608
|
| 339 |
+
b12.c_kv.scale f32 815749632 4608 4608
|
| 340 |
+
b12.c_out.q i8 815768064 2359296 1536 1536
|
| 341 |
+
b12.c_out.scale f32 818127360 1536 1536
|
| 342 |
+
b12.ff0.q i8 818133504 18874368 1536 12288
|
| 343 |
+
b12.ff0.scale f32 837007872 12288 12288
|
| 344 |
+
b12.ff0.bias f32 837057024 12288 12288
|
| 345 |
+
b12.ff2.q i8 837106176 9437184 6144 1536
|
| 346 |
+
b12.ff2.scale f32 846543360 1536 1536
|
| 347 |
+
b12.ff2.bias f32 846549504 1536 1536
|
| 348 |
+
b13.pre_g f32 846555648 1536 1536
|
| 349 |
+
b13.ff_g f32 846561792 1536 1536
|
| 350 |
+
b13.cx_g f32 846567936 1536 1536
|
| 351 |
+
b13.s_qn f32 846574080 64 64
|
| 352 |
+
b13.s_kn f32 846574336 64 64
|
| 353 |
+
b13.c_qn f32 846574592 64 64
|
| 354 |
+
b13.c_kn f32 846574848 64 64
|
| 355 |
+
b13.ssg f32 846575104 9216 9216
|
| 356 |
+
b13.local f32 846611968 1536 1536
|
| 357 |
+
b13.s_qkv.q i8 846618112 11796480 1536 7680
|
| 358 |
+
b13.s_qkv.scale f32 858414592 7680 7680
|
| 359 |
+
b13.s_out.q i8 858445312 2359296 1536 1536
|
| 360 |
+
b13.s_out.scale f32 860804608 1536 1536
|
| 361 |
+
b13.c_q.q i8 860810752 4718592 1536 3072
|
| 362 |
+
b13.c_q.scale f32 865529344 3072 3072
|
| 363 |
+
b13.c_kv.q i8 865541632 7077888 1536 4608
|
| 364 |
+
b13.c_kv.scale f32 872619520 4608 4608
|
| 365 |
+
b13.c_out.q i8 872637952 2359296 1536 1536
|
| 366 |
+
b13.c_out.scale f32 874997248 1536 1536
|
| 367 |
+
b13.ff0.q i8 875003392 18874368 1536 12288
|
| 368 |
+
b13.ff0.scale f32 893877760 12288 12288
|
| 369 |
+
b13.ff0.bias f32 893926912 12288 12288
|
| 370 |
+
b13.ff2.q i8 893976064 9437184 6144 1536
|
| 371 |
+
b13.ff2.scale f32 903413248 1536 1536
|
| 372 |
+
b13.ff2.bias f32 903419392 1536 1536
|
| 373 |
+
b14.pre_g f32 903425536 1536 1536
|
| 374 |
+
b14.ff_g f32 903431680 1536 1536
|
| 375 |
+
b14.cx_g f32 903437824 1536 1536
|
| 376 |
+
b14.s_qn f32 903443968 64 64
|
| 377 |
+
b14.s_kn f32 903444224 64 64
|
| 378 |
+
b14.c_qn f32 903444480 64 64
|
| 379 |
+
b14.c_kn f32 903444736 64 64
|
| 380 |
+
b14.ssg f32 903444992 9216 9216
|
| 381 |
+
b14.local f32 903481856 1536 1536
|
| 382 |
+
b14.s_qkv.q i8 903488000 11796480 1536 7680
|
| 383 |
+
b14.s_qkv.scale f32 915284480 7680 7680
|
| 384 |
+
b14.s_out.q i8 915315200 2359296 1536 1536
|
| 385 |
+
b14.s_out.scale f32 917674496 1536 1536
|
| 386 |
+
b14.c_q.q i8 917680640 4718592 1536 3072
|
| 387 |
+
b14.c_q.scale f32 922399232 3072 3072
|
| 388 |
+
b14.c_kv.q i8 922411520 7077888 1536 4608
|
| 389 |
+
b14.c_kv.scale f32 929489408 4608 4608
|
| 390 |
+
b14.c_out.q i8 929507840 2359296 1536 1536
|
| 391 |
+
b14.c_out.scale f32 931867136 1536 1536
|
| 392 |
+
b14.ff0.q i8 931873280 18874368 1536 12288
|
| 393 |
+
b14.ff0.scale f32 950747648 12288 12288
|
| 394 |
+
b14.ff0.bias f32 950796800 12288 12288
|
| 395 |
+
b14.ff2.q i8 950845952 9437184 6144 1536
|
| 396 |
+
b14.ff2.scale f32 960283136 1536 1536
|
| 397 |
+
b14.ff2.bias f32 960289280 1536 1536
|
| 398 |
+
b15.pre_g f32 960295424 1536 1536
|
| 399 |
+
b15.ff_g f32 960301568 1536 1536
|
| 400 |
+
b15.cx_g f32 960307712 1536 1536
|
| 401 |
+
b15.s_qn f32 960313856 64 64
|
| 402 |
+
b15.s_kn f32 960314112 64 64
|
| 403 |
+
b15.c_qn f32 960314368 64 64
|
| 404 |
+
b15.c_kn f32 960314624 64 64
|
| 405 |
+
b15.ssg f32 960314880 9216 9216
|
| 406 |
+
b15.local f32 960351744 1536 1536
|
| 407 |
+
b15.s_qkv.q i8 960357888 11796480 1536 7680
|
| 408 |
+
b15.s_qkv.scale f32 972154368 7680 7680
|
| 409 |
+
b15.s_out.q i8 972185088 2359296 1536 1536
|
| 410 |
+
b15.s_out.scale f32 974544384 1536 1536
|
| 411 |
+
b15.c_q.q i8 974550528 4718592 1536 3072
|
| 412 |
+
b15.c_q.scale f32 979269120 3072 3072
|
| 413 |
+
b15.c_kv.q i8 979281408 7077888 1536 4608
|
| 414 |
+
b15.c_kv.scale f32 986359296 4608 4608
|
| 415 |
+
b15.c_out.q i8 986377728 2359296 1536 1536
|
| 416 |
+
b15.c_out.scale f32 988737024 1536 1536
|
| 417 |
+
b15.ff0.q i8 988743168 18874368 1536 12288
|
| 418 |
+
b15.ff0.scale f32 1007617536 12288 12288
|
| 419 |
+
b15.ff0.bias f32 1007666688 12288 12288
|
| 420 |
+
b15.ff2.q i8 1007715840 9437184 6144 1536
|
| 421 |
+
b15.ff2.scale f32 1017153024 1536 1536
|
| 422 |
+
b15.ff2.bias f32 1017159168 1536 1536
|
| 423 |
+
b16.pre_g f32 1017165312 1536 1536
|
| 424 |
+
b16.ff_g f32 1017171456 1536 1536
|
| 425 |
+
b16.cx_g f32 1017177600 1536 1536
|
| 426 |
+
b16.s_qn f32 1017183744 64 64
|
| 427 |
+
b16.s_kn f32 1017184000 64 64
|
| 428 |
+
b16.c_qn f32 1017184256 64 64
|
| 429 |
+
b16.c_kn f32 1017184512 64 64
|
| 430 |
+
b16.ssg f32 1017184768 9216 9216
|
| 431 |
+
b16.local f32 1017221632 1536 1536
|
| 432 |
+
b16.s_qkv.q i8 1017227776 11796480 1536 7680
|
| 433 |
+
b16.s_qkv.scale f32 1029024256 7680 7680
|
| 434 |
+
b16.s_out.q i8 1029054976 2359296 1536 1536
|
| 435 |
+
b16.s_out.scale f32 1031414272 1536 1536
|
| 436 |
+
b16.c_q.q i8 1031420416 4718592 1536 3072
|
| 437 |
+
b16.c_q.scale f32 1036139008 3072 3072
|
| 438 |
+
b16.c_kv.q i8 1036151296 7077888 1536 4608
|
| 439 |
+
b16.c_kv.scale f32 1043229184 4608 4608
|
| 440 |
+
b16.c_out.q i8 1043247616 2359296 1536 1536
|
| 441 |
+
b16.c_out.scale f32 1045606912 1536 1536
|
| 442 |
+
b16.ff0.q i8 1045613056 18874368 1536 12288
|
| 443 |
+
b16.ff0.scale f32 1064487424 12288 12288
|
| 444 |
+
b16.ff0.bias f32 1064536576 12288 12288
|
| 445 |
+
b16.ff2.q i8 1064585728 9437184 6144 1536
|
| 446 |
+
b16.ff2.scale f32 1074022912 1536 1536
|
| 447 |
+
b16.ff2.bias f32 1074029056 1536 1536
|
| 448 |
+
b17.pre_g f32 1074035200 1536 1536
|
| 449 |
+
b17.ff_g f32 1074041344 1536 1536
|
| 450 |
+
b17.cx_g f32 1074047488 1536 1536
|
| 451 |
+
b17.s_qn f32 1074053632 64 64
|
| 452 |
+
b17.s_kn f32 1074053888 64 64
|
| 453 |
+
b17.c_qn f32 1074054144 64 64
|
| 454 |
+
b17.c_kn f32 1074054400 64 64
|
| 455 |
+
b17.ssg f32 1074054656 9216 9216
|
| 456 |
+
b17.local f32 1074091520 1536 1536
|
| 457 |
+
b17.s_qkv.q i8 1074097664 11796480 1536 7680
|
| 458 |
+
b17.s_qkv.scale f32 1085894144 7680 7680
|
| 459 |
+
b17.s_out.q i8 1085924864 2359296 1536 1536
|
| 460 |
+
b17.s_out.scale f32 1088284160 1536 1536
|
| 461 |
+
b17.c_q.q i8 1088290304 4718592 1536 3072
|
| 462 |
+
b17.c_q.scale f32 1093008896 3072 3072
|
| 463 |
+
b17.c_kv.q i8 1093021184 7077888 1536 4608
|
| 464 |
+
b17.c_kv.scale f32 1100099072 4608 4608
|
| 465 |
+
b17.c_out.q i8 1100117504 2359296 1536 1536
|
| 466 |
+
b17.c_out.scale f32 1102476800 1536 1536
|
| 467 |
+
b17.ff0.q i8 1102482944 18874368 1536 12288
|
| 468 |
+
b17.ff0.scale f32 1121357312 12288 12288
|
| 469 |
+
b17.ff0.bias f32 1121406464 12288 12288
|
| 470 |
+
b17.ff2.q i8 1121455616 9437184 6144 1536
|
| 471 |
+
b17.ff2.scale f32 1130892800 1536 1536
|
| 472 |
+
b17.ff2.bias f32 1130898944 1536 1536
|
| 473 |
+
b18.pre_g f32 1130905088 1536 1536
|
| 474 |
+
b18.ff_g f32 1130911232 1536 1536
|
| 475 |
+
b18.cx_g f32 1130917376 1536 1536
|
| 476 |
+
b18.s_qn f32 1130923520 64 64
|
| 477 |
+
b18.s_kn f32 1130923776 64 64
|
| 478 |
+
b18.c_qn f32 1130924032 64 64
|
| 479 |
+
b18.c_kn f32 1130924288 64 64
|
| 480 |
+
b18.ssg f32 1130924544 9216 9216
|
| 481 |
+
b18.local f32 1130961408 1536 1536
|
| 482 |
+
b18.s_qkv.q i8 1130967552 11796480 1536 7680
|
| 483 |
+
b18.s_qkv.scale f32 1142764032 7680 7680
|
| 484 |
+
b18.s_out.q i8 1142794752 2359296 1536 1536
|
| 485 |
+
b18.s_out.scale f32 1145154048 1536 1536
|
| 486 |
+
b18.c_q.q i8 1145160192 4718592 1536 3072
|
| 487 |
+
b18.c_q.scale f32 1149878784 3072 3072
|
| 488 |
+
b18.c_kv.q i8 1149891072 7077888 1536 4608
|
| 489 |
+
b18.c_kv.scale f32 1156968960 4608 4608
|
| 490 |
+
b18.c_out.q i8 1156987392 2359296 1536 1536
|
| 491 |
+
b18.c_out.scale f32 1159346688 1536 1536
|
| 492 |
+
b18.ff0.q i8 1159352832 18874368 1536 12288
|
| 493 |
+
b18.ff0.scale f32 1178227200 12288 12288
|
| 494 |
+
b18.ff0.bias f32 1178276352 12288 12288
|
| 495 |
+
b18.ff2.q i8 1178325504 9437184 6144 1536
|
| 496 |
+
b18.ff2.scale f32 1187762688 1536 1536
|
| 497 |
+
b18.ff2.bias f32 1187768832 1536 1536
|
| 498 |
+
b19.pre_g f32 1187774976 1536 1536
|
| 499 |
+
b19.ff_g f32 1187781120 1536 1536
|
| 500 |
+
b19.cx_g f32 1187787264 1536 1536
|
| 501 |
+
b19.s_qn f32 1187793408 64 64
|
| 502 |
+
b19.s_kn f32 1187793664 64 64
|
| 503 |
+
b19.c_qn f32 1187793920 64 64
|
| 504 |
+
b19.c_kn f32 1187794176 64 64
|
| 505 |
+
b19.ssg f32 1187794432 9216 9216
|
| 506 |
+
b19.local f32 1187831296 1536 1536
|
| 507 |
+
b19.s_qkv.q i8 1187837440 11796480 1536 7680
|
| 508 |
+
b19.s_qkv.scale f32 1199633920 7680 7680
|
| 509 |
+
b19.s_out.q i8 1199664640 2359296 1536 1536
|
| 510 |
+
b19.s_out.scale f32 1202023936 1536 1536
|
| 511 |
+
b19.c_q.q i8 1202030080 4718592 1536 3072
|
| 512 |
+
b19.c_q.scale f32 1206748672 3072 3072
|
| 513 |
+
b19.c_kv.q i8 1206760960 7077888 1536 4608
|
| 514 |
+
b19.c_kv.scale f32 1213838848 4608 4608
|
| 515 |
+
b19.c_out.q i8 1213857280 2359296 1536 1536
|
| 516 |
+
b19.c_out.scale f32 1216216576 1536 1536
|
| 517 |
+
b19.ff0.q i8 1216222720 18874368 1536 12288
|
| 518 |
+
b19.ff0.scale f32 1235097088 12288 12288
|
| 519 |
+
b19.ff0.bias f32 1235146240 12288 12288
|
| 520 |
+
b19.ff2.q i8 1235195392 9437184 6144 1536
|
| 521 |
+
b19.ff2.scale f32 1244632576 1536 1536
|
| 522 |
+
b19.ff2.bias f32 1244638720 1536 1536
|
| 523 |
+
b20.pre_g f32 1244644864 1536 1536
|
| 524 |
+
b20.ff_g f32 1244651008 1536 1536
|
| 525 |
+
b20.cx_g f32 1244657152 1536 1536
|
| 526 |
+
b20.s_qn f32 1244663296 64 64
|
| 527 |
+
b20.s_kn f32 1244663552 64 64
|
| 528 |
+
b20.c_qn f32 1244663808 64 64
|
| 529 |
+
b20.c_kn f32 1244664064 64 64
|
| 530 |
+
b20.ssg f32 1244664320 9216 9216
|
| 531 |
+
b20.local f32 1244701184 1536 1536
|
| 532 |
+
b20.s_qkv.q i8 1244707328 11796480 1536 7680
|
| 533 |
+
b20.s_qkv.scale f32 1256503808 7680 7680
|
| 534 |
+
b20.s_out.q i8 1256534528 2359296 1536 1536
|
| 535 |
+
b20.s_out.scale f32 1258893824 1536 1536
|
| 536 |
+
b20.c_q.q i8 1258899968 4718592 1536 3072
|
| 537 |
+
b20.c_q.scale f32 1263618560 3072 3072
|
| 538 |
+
b20.c_kv.q i8 1263630848 7077888 1536 4608
|
| 539 |
+
b20.c_kv.scale f32 1270708736 4608 4608
|
| 540 |
+
b20.c_out.q i8 1270727168 2359296 1536 1536
|
| 541 |
+
b20.c_out.scale f32 1273086464 1536 1536
|
| 542 |
+
b20.ff0.q i8 1273092608 18874368 1536 12288
|
| 543 |
+
b20.ff0.scale f32 1291966976 12288 12288
|
| 544 |
+
b20.ff0.bias f32 1292016128 12288 12288
|
| 545 |
+
b20.ff2.q i8 1292065280 9437184 6144 1536
|
| 546 |
+
b20.ff2.scale f32 1301502464 1536 1536
|
| 547 |
+
b20.ff2.bias f32 1301508608 1536 1536
|
| 548 |
+
b21.pre_g f32 1301514752 1536 1536
|
| 549 |
+
b21.ff_g f32 1301520896 1536 1536
|
| 550 |
+
b21.cx_g f32 1301527040 1536 1536
|
| 551 |
+
b21.s_qn f32 1301533184 64 64
|
| 552 |
+
b21.s_kn f32 1301533440 64 64
|
| 553 |
+
b21.c_qn f32 1301533696 64 64
|
| 554 |
+
b21.c_kn f32 1301533952 64 64
|
| 555 |
+
b21.ssg f32 1301534208 9216 9216
|
| 556 |
+
b21.local f32 1301571072 1536 1536
|
| 557 |
+
b21.s_qkv.q i8 1301577216 11796480 1536 7680
|
| 558 |
+
b21.s_qkv.scale f32 1313373696 7680 7680
|
| 559 |
+
b21.s_out.q i8 1313404416 2359296 1536 1536
|
| 560 |
+
b21.s_out.scale f32 1315763712 1536 1536
|
| 561 |
+
b21.c_q.q i8 1315769856 4718592 1536 3072
|
| 562 |
+
b21.c_q.scale f32 1320488448 3072 3072
|
| 563 |
+
b21.c_kv.q i8 1320500736 7077888 1536 4608
|
| 564 |
+
b21.c_kv.scale f32 1327578624 4608 4608
|
| 565 |
+
b21.c_out.q i8 1327597056 2359296 1536 1536
|
| 566 |
+
b21.c_out.scale f32 1329956352 1536 1536
|
| 567 |
+
b21.ff0.q i8 1329962496 18874368 1536 12288
|
| 568 |
+
b21.ff0.scale f32 1348836864 12288 12288
|
| 569 |
+
b21.ff0.bias f32 1348886016 12288 12288
|
| 570 |
+
b21.ff2.q i8 1348935168 9437184 6144 1536
|
| 571 |
+
b21.ff2.scale f32 1358372352 1536 1536
|
| 572 |
+
b21.ff2.bias f32 1358378496 1536 1536
|
| 573 |
+
b22.pre_g f32 1358384640 1536 1536
|
| 574 |
+
b22.ff_g f32 1358390784 1536 1536
|
| 575 |
+
b22.cx_g f32 1358396928 1536 1536
|
| 576 |
+
b22.s_qn f32 1358403072 64 64
|
| 577 |
+
b22.s_kn f32 1358403328 64 64
|
| 578 |
+
b22.c_qn f32 1358403584 64 64
|
| 579 |
+
b22.c_kn f32 1358403840 64 64
|
| 580 |
+
b22.ssg f32 1358404096 9216 9216
|
| 581 |
+
b22.local f32 1358440960 1536 1536
|
| 582 |
+
b22.s_qkv.q i8 1358447104 11796480 1536 7680
|
| 583 |
+
b22.s_qkv.scale f32 1370243584 7680 7680
|
| 584 |
+
b22.s_out.q i8 1370274304 2359296 1536 1536
|
| 585 |
+
b22.s_out.scale f32 1372633600 1536 1536
|
| 586 |
+
b22.c_q.q i8 1372639744 4718592 1536 3072
|
| 587 |
+
b22.c_q.scale f32 1377358336 3072 3072
|
| 588 |
+
b22.c_kv.q i8 1377370624 7077888 1536 4608
|
| 589 |
+
b22.c_kv.scale f32 1384448512 4608 4608
|
| 590 |
+
b22.c_out.q i8 1384466944 2359296 1536 1536
|
| 591 |
+
b22.c_out.scale f32 1386826240 1536 1536
|
| 592 |
+
b22.ff0.q i8 1386832384 18874368 1536 12288
|
| 593 |
+
b22.ff0.scale f32 1405706752 12288 12288
|
| 594 |
+
b22.ff0.bias f32 1405755904 12288 12288
|
| 595 |
+
b22.ff2.q i8 1405805056 9437184 6144 1536
|
| 596 |
+
b22.ff2.scale f32 1415242240 1536 1536
|
| 597 |
+
b22.ff2.bias f32 1415248384 1536 1536
|
| 598 |
+
b23.pre_g f32 1415254528 1536 1536
|
| 599 |
+
b23.ff_g f32 1415260672 1536 1536
|
| 600 |
+
b23.cx_g f32 1415266816 1536 1536
|
| 601 |
+
b23.s_qn f32 1415272960 64 64
|
| 602 |
+
b23.s_kn f32 1415273216 64 64
|
| 603 |
+
b23.c_qn f32 1415273472 64 64
|
| 604 |
+
b23.c_kn f32 1415273728 64 64
|
| 605 |
+
b23.ssg f32 1415273984 9216 9216
|
| 606 |
+
b23.local f32 1415310848 1536 1536
|
| 607 |
+
b23.s_qkv.q i8 1415316992 11796480 1536 7680
|
| 608 |
+
b23.s_qkv.scale f32 1427113472 7680 7680
|
| 609 |
+
b23.s_out.q i8 1427144192 2359296 1536 1536
|
| 610 |
+
b23.s_out.scale f32 1429503488 1536 1536
|
| 611 |
+
b23.c_q.q i8 1429509632 4718592 1536 3072
|
| 612 |
+
b23.c_q.scale f32 1434228224 3072 3072
|
| 613 |
+
b23.c_kv.q i8 1434240512 7077888 1536 4608
|
| 614 |
+
b23.c_kv.scale f32 1441318400 4608 4608
|
| 615 |
+
b23.c_out.q i8 1441336832 2359296 1536 1536
|
| 616 |
+
b23.c_out.scale f32 1443696128 1536 1536
|
| 617 |
+
b23.ff0.q i8 1443702272 18874368 1536 12288
|
| 618 |
+
b23.ff0.scale f32 1462576640 12288 12288
|
| 619 |
+
b23.ff0.bias f32 1462625792 12288 12288
|
| 620 |
+
b23.ff2.q i8 1462674944 9437184 6144 1536
|
| 621 |
+
b23.ff2.scale f32 1472112128 1536 1536
|
| 622 |
+
b23.ff2.bias f32 1472118272 1536 1536
|
| 623 |
+
x_init f32 1472124416 589824 384 1536
|
| 624 |
+
ctx_i8 i8 1474483712 394752 257 1536
|
| 625 |
+
ctx_s f32 1474878464 257 257
|
| 626 |
+
gc f32 1474879492 9216 9216
|
| 627 |
+
out f32 1474916356 81920 1 256 320
|
cpu-amx/dit_medium_kernels.tar.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9669f5396d0c60986189c0816a53105a7ba2993a755d7f5bf20eb0f0e3716b26
|
| 3 |
+
size 1164636
|
cpu-amx/same_l_decoder_bf16.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6890c90ba2f50506cc62554b42697b333aac7c83ab5373d879fe949648e348be
|
| 3 |
+
size 66987192
|
cpu-amx/same_l_decoder_bf16_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8945034f5c2221fcecdcb0b9f21d521c0206589209c513e7bcba7245142ab8d6
|
| 3 |
+
size 852543684
|
cpu-amx/same_l_decoder_bf16_weights_manifest.txt
ADDED
|
@@ -0,0 +1,222 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
running_std f32 0 1 1
|
| 2 |
+
project_in.w bf16 4 393216 256 1536
|
| 3 |
+
project_in.b f32 786436 1536 1536
|
| 4 |
+
new_tokens f32 792580 1536 1536
|
| 5 |
+
b0.pre.alpha f32 798724 1 1
|
| 6 |
+
b0.pre.gamma f32 798728 1536 1536
|
| 7 |
+
b0.pre.beta f32 804872 1536 1536
|
| 8 |
+
b0.qkv.w bf16 811016 11796480 1536 7680
|
| 9 |
+
b0.qn.alpha f32 24403976 1 1
|
| 10 |
+
b0.qn.gamma f32 24403980 64 64
|
| 11 |
+
b0.qn.beta f32 24404236 64 64
|
| 12 |
+
b0.kn.alpha f32 24404492 1 1
|
| 13 |
+
b0.kn.gamma f32 24404496 64 64
|
| 14 |
+
b0.kn.beta f32 24404752 64 64
|
| 15 |
+
b0.out.w bf16 24405008 2359296 1536 1536
|
| 16 |
+
b0.ff.alpha f32 29123600 1 1
|
| 17 |
+
b0.ff.gamma f32 29123604 1536 1536
|
| 18 |
+
b0.ff.beta f32 29129748 1536 1536
|
| 19 |
+
b0.glu.w bf16 29135892 14155776 1536 9216
|
| 20 |
+
b0.glu.b f32 57447444 9216 9216
|
| 21 |
+
b0.proj.w bf16 57484308 7077888 4608 1536
|
| 22 |
+
b0.proj.b f32 71640084 1536 1536
|
| 23 |
+
b1.pre.alpha f32 71646228 1 1
|
| 24 |
+
b1.pre.gamma f32 71646232 1536 1536
|
| 25 |
+
b1.pre.beta f32 71652376 1536 1536
|
| 26 |
+
b1.qkv.w bf16 71658520 11796480 1536 7680
|
| 27 |
+
b1.qn.alpha f32 95251480 1 1
|
| 28 |
+
b1.qn.gamma f32 95251484 64 64
|
| 29 |
+
b1.qn.beta f32 95251740 64 64
|
| 30 |
+
b1.kn.alpha f32 95251996 1 1
|
| 31 |
+
b1.kn.gamma f32 95252000 64 64
|
| 32 |
+
b1.kn.beta f32 95252256 64 64
|
| 33 |
+
b1.out.w bf16 95252512 2359296 1536 1536
|
| 34 |
+
b1.ff.alpha f32 99971104 1 1
|
| 35 |
+
b1.ff.gamma f32 99971108 1536 1536
|
| 36 |
+
b1.ff.beta f32 99977252 1536 1536
|
| 37 |
+
b1.glu.w bf16 99983396 14155776 1536 9216
|
| 38 |
+
b1.glu.b f32 128294948 9216 9216
|
| 39 |
+
b1.proj.w bf16 128331812 7077888 4608 1536
|
| 40 |
+
b1.proj.b f32 142487588 1536 1536
|
| 41 |
+
b2.pre.alpha f32 142493732 1 1
|
| 42 |
+
b2.pre.gamma f32 142493736 1536 1536
|
| 43 |
+
b2.pre.beta f32 142499880 1536 1536
|
| 44 |
+
b2.qkv.w bf16 142506024 11796480 1536 7680
|
| 45 |
+
b2.qn.alpha f32 166098984 1 1
|
| 46 |
+
b2.qn.gamma f32 166098988 64 64
|
| 47 |
+
b2.qn.beta f32 166099244 64 64
|
| 48 |
+
b2.kn.alpha f32 166099500 1 1
|
| 49 |
+
b2.kn.gamma f32 166099504 64 64
|
| 50 |
+
b2.kn.beta f32 166099760 64 64
|
| 51 |
+
b2.out.w bf16 166100016 2359296 1536 1536
|
| 52 |
+
b2.ff.alpha f32 170818608 1 1
|
| 53 |
+
b2.ff.gamma f32 170818612 1536 1536
|
| 54 |
+
b2.ff.beta f32 170824756 1536 1536
|
| 55 |
+
b2.glu.w bf16 170830900 14155776 1536 9216
|
| 56 |
+
b2.glu.b f32 199142452 9216 9216
|
| 57 |
+
b2.proj.w bf16 199179316 7077888 4608 1536
|
| 58 |
+
b2.proj.b f32 213335092 1536 1536
|
| 59 |
+
b3.pre.alpha f32 213341236 1 1
|
| 60 |
+
b3.pre.gamma f32 213341240 1536 1536
|
| 61 |
+
b3.pre.beta f32 213347384 1536 1536
|
| 62 |
+
b3.qkv.w bf16 213353528 11796480 1536 7680
|
| 63 |
+
b3.qn.alpha f32 236946488 1 1
|
| 64 |
+
b3.qn.gamma f32 236946492 64 64
|
| 65 |
+
b3.qn.beta f32 236946748 64 64
|
| 66 |
+
b3.kn.alpha f32 236947004 1 1
|
| 67 |
+
b3.kn.gamma f32 236947008 64 64
|
| 68 |
+
b3.kn.beta f32 236947264 64 64
|
| 69 |
+
b3.out.w bf16 236947520 2359296 1536 1536
|
| 70 |
+
b3.ff.alpha f32 241666112 1 1
|
| 71 |
+
b3.ff.gamma f32 241666116 1536 1536
|
| 72 |
+
b3.ff.beta f32 241672260 1536 1536
|
| 73 |
+
b3.glu.w bf16 241678404 14155776 1536 9216
|
| 74 |
+
b3.glu.b f32 269989956 9216 9216
|
| 75 |
+
b3.proj.w bf16 270026820 7077888 4608 1536
|
| 76 |
+
b3.proj.b f32 284182596 1536 1536
|
| 77 |
+
b4.pre.alpha f32 284188740 1 1
|
| 78 |
+
b4.pre.gamma f32 284188744 1536 1536
|
| 79 |
+
b4.pre.beta f32 284194888 1536 1536
|
| 80 |
+
b4.qkv.w bf16 284201032 11796480 1536 7680
|
| 81 |
+
b4.qn.alpha f32 307793992 1 1
|
| 82 |
+
b4.qn.gamma f32 307793996 64 64
|
| 83 |
+
b4.qn.beta f32 307794252 64 64
|
| 84 |
+
b4.kn.alpha f32 307794508 1 1
|
| 85 |
+
b4.kn.gamma f32 307794512 64 64
|
| 86 |
+
b4.kn.beta f32 307794768 64 64
|
| 87 |
+
b4.out.w bf16 307795024 2359296 1536 1536
|
| 88 |
+
b4.ff.alpha f32 312513616 1 1
|
| 89 |
+
b4.ff.gamma f32 312513620 1536 1536
|
| 90 |
+
b4.ff.beta f32 312519764 1536 1536
|
| 91 |
+
b4.glu.w bf16 312525908 14155776 1536 9216
|
| 92 |
+
b4.glu.b f32 340837460 9216 9216
|
| 93 |
+
b4.proj.w bf16 340874324 7077888 4608 1536
|
| 94 |
+
b4.proj.b f32 355030100 1536 1536
|
| 95 |
+
b5.pre.alpha f32 355036244 1 1
|
| 96 |
+
b5.pre.gamma f32 355036248 1536 1536
|
| 97 |
+
b5.pre.beta f32 355042392 1536 1536
|
| 98 |
+
b5.qkv.w bf16 355048536 11796480 1536 7680
|
| 99 |
+
b5.qn.alpha f32 378641496 1 1
|
| 100 |
+
b5.qn.gamma f32 378641500 64 64
|
| 101 |
+
b5.qn.beta f32 378641756 64 64
|
| 102 |
+
b5.kn.alpha f32 378642012 1 1
|
| 103 |
+
b5.kn.gamma f32 378642016 64 64
|
| 104 |
+
b5.kn.beta f32 378642272 64 64
|
| 105 |
+
b5.out.w bf16 378642528 2359296 1536 1536
|
| 106 |
+
b5.ff.alpha f32 383361120 1 1
|
| 107 |
+
b5.ff.gamma f32 383361124 1536 1536
|
| 108 |
+
b5.ff.beta f32 383367268 1536 1536
|
| 109 |
+
b5.glu.w bf16 383373412 14155776 1536 9216
|
| 110 |
+
b5.glu.b f32 411684964 9216 9216
|
| 111 |
+
b5.proj.w bf16 411721828 7077888 4608 1536
|
| 112 |
+
b5.proj.b f32 425877604 1536 1536
|
| 113 |
+
b6.pre.alpha f32 425883748 1 1
|
| 114 |
+
b6.pre.gamma f32 425883752 1536 1536
|
| 115 |
+
b6.pre.beta f32 425889896 1536 1536
|
| 116 |
+
b6.qkv.w bf16 425896040 11796480 1536 7680
|
| 117 |
+
b6.qn.alpha f32 449489000 1 1
|
| 118 |
+
b6.qn.gamma f32 449489004 64 64
|
| 119 |
+
b6.qn.beta f32 449489260 64 64
|
| 120 |
+
b6.kn.alpha f32 449489516 1 1
|
| 121 |
+
b6.kn.gamma f32 449489520 64 64
|
| 122 |
+
b6.kn.beta f32 449489776 64 64
|
| 123 |
+
b6.out.w bf16 449490032 2359296 1536 1536
|
| 124 |
+
b6.ff.alpha f32 454208624 1 1
|
| 125 |
+
b6.ff.gamma f32 454208628 1536 1536
|
| 126 |
+
b6.ff.beta f32 454214772 1536 1536
|
| 127 |
+
b6.glu.w bf16 454220916 14155776 1536 9216
|
| 128 |
+
b6.glu.b f32 482532468 9216 9216
|
| 129 |
+
b6.proj.w bf16 482569332 7077888 4608 1536
|
| 130 |
+
b6.proj.b f32 496725108 1536 1536
|
| 131 |
+
b7.pre.alpha f32 496731252 1 1
|
| 132 |
+
b7.pre.gamma f32 496731256 1536 1536
|
| 133 |
+
b7.pre.beta f32 496737400 1536 1536
|
| 134 |
+
b7.qkv.w bf16 496743544 11796480 1536 7680
|
| 135 |
+
b7.qn.alpha f32 520336504 1 1
|
| 136 |
+
b7.qn.gamma f32 520336508 64 64
|
| 137 |
+
b7.qn.beta f32 520336764 64 64
|
| 138 |
+
b7.kn.alpha f32 520337020 1 1
|
| 139 |
+
b7.kn.gamma f32 520337024 64 64
|
| 140 |
+
b7.kn.beta f32 520337280 64 64
|
| 141 |
+
b7.out.w bf16 520337536 2359296 1536 1536
|
| 142 |
+
b7.ff.alpha f32 525056128 1 1
|
| 143 |
+
b7.ff.gamma f32 525056132 1536 1536
|
| 144 |
+
b7.ff.beta f32 525062276 1536 1536
|
| 145 |
+
b7.glu.w bf16 525068420 14155776 1536 9216
|
| 146 |
+
b7.glu.b f32 553379972 9216 9216
|
| 147 |
+
b7.proj.w bf16 553416836 7077888 4608 1536
|
| 148 |
+
b7.proj.b f32 567572612 1536 1536
|
| 149 |
+
b8.pre.alpha f32 567578756 1 1
|
| 150 |
+
b8.pre.gamma f32 567578760 1536 1536
|
| 151 |
+
b8.pre.beta f32 567584904 1536 1536
|
| 152 |
+
b8.qkv.w bf16 567591048 11796480 1536 7680
|
| 153 |
+
b8.qn.alpha f32 591184008 1 1
|
| 154 |
+
b8.qn.gamma f32 591184012 64 64
|
| 155 |
+
b8.qn.beta f32 591184268 64 64
|
| 156 |
+
b8.kn.alpha f32 591184524 1 1
|
| 157 |
+
b8.kn.gamma f32 591184528 64 64
|
| 158 |
+
b8.kn.beta f32 591184784 64 64
|
| 159 |
+
b8.out.w bf16 591185040 2359296 1536 1536
|
| 160 |
+
b8.ff.alpha f32 595903632 1 1
|
| 161 |
+
b8.ff.gamma f32 595903636 1536 1536
|
| 162 |
+
b8.ff.beta f32 595909780 1536 1536
|
| 163 |
+
b8.glu.w bf16 595915924 14155776 1536 9216
|
| 164 |
+
b8.glu.b f32 624227476 9216 9216
|
| 165 |
+
b8.proj.w bf16 624264340 7077888 4608 1536
|
| 166 |
+
b8.proj.b f32 638420116 1536 1536
|
| 167 |
+
b9.pre.alpha f32 638426260 1 1
|
| 168 |
+
b9.pre.gamma f32 638426264 1536 1536
|
| 169 |
+
b9.pre.beta f32 638432408 1536 1536
|
| 170 |
+
b9.qkv.w bf16 638438552 11796480 1536 7680
|
| 171 |
+
b9.qn.alpha f32 662031512 1 1
|
| 172 |
+
b9.qn.gamma f32 662031516 64 64
|
| 173 |
+
b9.qn.beta f32 662031772 64 64
|
| 174 |
+
b9.kn.alpha f32 662032028 1 1
|
| 175 |
+
b9.kn.gamma f32 662032032 64 64
|
| 176 |
+
b9.kn.beta f32 662032288 64 64
|
| 177 |
+
b9.out.w bf16 662032544 2359296 1536 1536
|
| 178 |
+
b9.ff.alpha f32 666751136 1 1
|
| 179 |
+
b9.ff.gamma f32 666751140 1536 1536
|
| 180 |
+
b9.ff.beta f32 666757284 1536 1536
|
| 181 |
+
b9.glu.w bf16 666763428 14155776 1536 9216
|
| 182 |
+
b9.glu.b f32 695074980 9216 9216
|
| 183 |
+
b9.proj.w bf16 695111844 7077888 4608 1536
|
| 184 |
+
b9.proj.b f32 709267620 1536 1536
|
| 185 |
+
b10.pre.alpha f32 709273764 1 1
|
| 186 |
+
b10.pre.gamma f32 709273768 1536 1536
|
| 187 |
+
b10.pre.beta f32 709279912 1536 1536
|
| 188 |
+
b10.qkv.w bf16 709286056 11796480 1536 7680
|
| 189 |
+
b10.qn.alpha f32 732879016 1 1
|
| 190 |
+
b10.qn.gamma f32 732879020 64 64
|
| 191 |
+
b10.qn.beta f32 732879276 64 64
|
| 192 |
+
b10.kn.alpha f32 732879532 1 1
|
| 193 |
+
b10.kn.gamma f32 732879536 64 64
|
| 194 |
+
b10.kn.beta f32 732879792 64 64
|
| 195 |
+
b10.out.w bf16 732880048 2359296 1536 1536
|
| 196 |
+
b10.ff.alpha f32 737598640 1 1
|
| 197 |
+
b10.ff.gamma f32 737598644 1536 1536
|
| 198 |
+
b10.ff.beta f32 737604788 1536 1536
|
| 199 |
+
b10.glu.w bf16 737610932 14155776 1536 9216
|
| 200 |
+
b10.glu.b f32 765922484 9216 9216
|
| 201 |
+
b10.proj.w bf16 765959348 7077888 4608 1536
|
| 202 |
+
b10.proj.b f32 780115124 1536 1536
|
| 203 |
+
b11.pre.alpha f32 780121268 1 1
|
| 204 |
+
b11.pre.gamma f32 780121272 1536 1536
|
| 205 |
+
b11.pre.beta f32 780127416 1536 1536
|
| 206 |
+
b11.qkv.w bf16 780133560 11796480 1536 7680
|
| 207 |
+
b11.qn.alpha f32 803726520 1 1
|
| 208 |
+
b11.qn.gamma f32 803726524 64 64
|
| 209 |
+
b11.qn.beta f32 803726780 64 64
|
| 210 |
+
b11.kn.alpha f32 803727036 1 1
|
| 211 |
+
b11.kn.gamma f32 803727040 64 64
|
| 212 |
+
b11.kn.beta f32 803727296 64 64
|
| 213 |
+
b11.out.w bf16 803727552 2359296 1536 1536
|
| 214 |
+
b11.ff.alpha f32 808446144 1 1
|
| 215 |
+
b11.ff.gamma f32 808446148 1536 1536
|
| 216 |
+
b11.ff.beta f32 808452292 1536 1536
|
| 217 |
+
b11.glu.w bf16 808458436 14155776 1536 9216
|
| 218 |
+
b11.glu.b f32 836769988 9216 9216
|
| 219 |
+
b11.proj.w bf16 836806852 7077888 4608 1536
|
| 220 |
+
b11.proj.b f32 850962628 1536 1536
|
| 221 |
+
map.w bf16 850968772 786432 1536 512
|
| 222 |
+
map.b f32 852541636 512 512
|
cpu-amx/same_l_decoder_int8.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b8b9ad2805d7d1625580363b0ccfa9b82e12c690f6c37eed92b784474d1632a4
|
| 3 |
+
size 29358736
|
cpu-amx/same_l_decoder_int8_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3c1026f5c8270b247dc0d3c6c9d86814a54c3273a8af5d0327d6e38d7aeae442
|
| 3 |
+
size 427657412
|
cpu-amx/same_l_decoder_int8_weights_manifest.txt
ADDED
|
@@ -0,0 +1,272 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
running_std f32 0 1 1
|
| 2 |
+
project_in.q i8 4 393216 256 1536
|
| 3 |
+
project_in.scale f32 393220 1536 1536
|
| 4 |
+
project_in.b f32 399364 1536 1536
|
| 5 |
+
new_tokens f32 405508 1536 1536
|
| 6 |
+
b0.pre.alpha f32 411652 1 1
|
| 7 |
+
b0.pre.gamma f32 411656 1536 1536
|
| 8 |
+
b0.pre.beta f32 417800 1536 1536
|
| 9 |
+
b0.qkv.q i8 423944 11796480 1536 7680
|
| 10 |
+
b0.qkv.scale f32 12220424 7680 7680
|
| 11 |
+
b0.qn.alpha f32 12251144 1 1
|
| 12 |
+
b0.qn.gamma f32 12251148 64 64
|
| 13 |
+
b0.qn.beta f32 12251404 64 64
|
| 14 |
+
b0.kn.alpha f32 12251660 1 1
|
| 15 |
+
b0.kn.gamma f32 12251664 64 64
|
| 16 |
+
b0.kn.beta f32 12251920 64 64
|
| 17 |
+
b0.out.q i8 12252176 2359296 1536 1536
|
| 18 |
+
b0.out.scale f32 14611472 1536 1536
|
| 19 |
+
b0.ff.alpha f32 14617616 1 1
|
| 20 |
+
b0.ff.gamma f32 14617620 1536 1536
|
| 21 |
+
b0.ff.beta f32 14623764 1536 1536
|
| 22 |
+
b0.glu.q i8 14629908 14155776 1536 9216
|
| 23 |
+
b0.glu.scale f32 28785684 9216 9216
|
| 24 |
+
b0.glu.b f32 28822548 9216 9216
|
| 25 |
+
b0.proj.q i8 28859412 7077888 4608 1536
|
| 26 |
+
b0.proj.scale f32 35937300 1536 1536
|
| 27 |
+
b0.proj.b f32 35943444 1536 1536
|
| 28 |
+
b1.pre.alpha f32 35949588 1 1
|
| 29 |
+
b1.pre.gamma f32 35949592 1536 1536
|
| 30 |
+
b1.pre.beta f32 35955736 1536 1536
|
| 31 |
+
b1.qkv.q i8 35961880 11796480 1536 7680
|
| 32 |
+
b1.qkv.scale f32 47758360 7680 7680
|
| 33 |
+
b1.qn.alpha f32 47789080 1 1
|
| 34 |
+
b1.qn.gamma f32 47789084 64 64
|
| 35 |
+
b1.qn.beta f32 47789340 64 64
|
| 36 |
+
b1.kn.alpha f32 47789596 1 1
|
| 37 |
+
b1.kn.gamma f32 47789600 64 64
|
| 38 |
+
b1.kn.beta f32 47789856 64 64
|
| 39 |
+
b1.out.q i8 47790112 2359296 1536 1536
|
| 40 |
+
b1.out.scale f32 50149408 1536 1536
|
| 41 |
+
b1.ff.alpha f32 50155552 1 1
|
| 42 |
+
b1.ff.gamma f32 50155556 1536 1536
|
| 43 |
+
b1.ff.beta f32 50161700 1536 1536
|
| 44 |
+
b1.glu.q i8 50167844 14155776 1536 9216
|
| 45 |
+
b1.glu.scale f32 64323620 9216 9216
|
| 46 |
+
b1.glu.b f32 64360484 9216 9216
|
| 47 |
+
b1.proj.q i8 64397348 7077888 4608 1536
|
| 48 |
+
b1.proj.scale f32 71475236 1536 1536
|
| 49 |
+
b1.proj.b f32 71481380 1536 1536
|
| 50 |
+
b2.pre.alpha f32 71487524 1 1
|
| 51 |
+
b2.pre.gamma f32 71487528 1536 1536
|
| 52 |
+
b2.pre.beta f32 71493672 1536 1536
|
| 53 |
+
b2.qkv.q i8 71499816 11796480 1536 7680
|
| 54 |
+
b2.qkv.scale f32 83296296 7680 7680
|
| 55 |
+
b2.qn.alpha f32 83327016 1 1
|
| 56 |
+
b2.qn.gamma f32 83327020 64 64
|
| 57 |
+
b2.qn.beta f32 83327276 64 64
|
| 58 |
+
b2.kn.alpha f32 83327532 1 1
|
| 59 |
+
b2.kn.gamma f32 83327536 64 64
|
| 60 |
+
b2.kn.beta f32 83327792 64 64
|
| 61 |
+
b2.out.q i8 83328048 2359296 1536 1536
|
| 62 |
+
b2.out.scale f32 85687344 1536 1536
|
| 63 |
+
b2.ff.alpha f32 85693488 1 1
|
| 64 |
+
b2.ff.gamma f32 85693492 1536 1536
|
| 65 |
+
b2.ff.beta f32 85699636 1536 1536
|
| 66 |
+
b2.glu.q i8 85705780 14155776 1536 9216
|
| 67 |
+
b2.glu.scale f32 99861556 9216 9216
|
| 68 |
+
b2.glu.b f32 99898420 9216 9216
|
| 69 |
+
b2.proj.q i8 99935284 7077888 4608 1536
|
| 70 |
+
b2.proj.scale f32 107013172 1536 1536
|
| 71 |
+
b2.proj.b f32 107019316 1536 1536
|
| 72 |
+
b3.pre.alpha f32 107025460 1 1
|
| 73 |
+
b3.pre.gamma f32 107025464 1536 1536
|
| 74 |
+
b3.pre.beta f32 107031608 1536 1536
|
| 75 |
+
b3.qkv.q i8 107037752 11796480 1536 7680
|
| 76 |
+
b3.qkv.scale f32 118834232 7680 7680
|
| 77 |
+
b3.qn.alpha f32 118864952 1 1
|
| 78 |
+
b3.qn.gamma f32 118864956 64 64
|
| 79 |
+
b3.qn.beta f32 118865212 64 64
|
| 80 |
+
b3.kn.alpha f32 118865468 1 1
|
| 81 |
+
b3.kn.gamma f32 118865472 64 64
|
| 82 |
+
b3.kn.beta f32 118865728 64 64
|
| 83 |
+
b3.out.q i8 118865984 2359296 1536 1536
|
| 84 |
+
b3.out.scale f32 121225280 1536 1536
|
| 85 |
+
b3.ff.alpha f32 121231424 1 1
|
| 86 |
+
b3.ff.gamma f32 121231428 1536 1536
|
| 87 |
+
b3.ff.beta f32 121237572 1536 1536
|
| 88 |
+
b3.glu.q i8 121243716 14155776 1536 9216
|
| 89 |
+
b3.glu.scale f32 135399492 9216 9216
|
| 90 |
+
b3.glu.b f32 135436356 9216 9216
|
| 91 |
+
b3.proj.q i8 135473220 7077888 4608 1536
|
| 92 |
+
b3.proj.scale f32 142551108 1536 1536
|
| 93 |
+
b3.proj.b f32 142557252 1536 1536
|
| 94 |
+
b4.pre.alpha f32 142563396 1 1
|
| 95 |
+
b4.pre.gamma f32 142563400 1536 1536
|
| 96 |
+
b4.pre.beta f32 142569544 1536 1536
|
| 97 |
+
b4.qkv.q i8 142575688 11796480 1536 7680
|
| 98 |
+
b4.qkv.scale f32 154372168 7680 7680
|
| 99 |
+
b4.qn.alpha f32 154402888 1 1
|
| 100 |
+
b4.qn.gamma f32 154402892 64 64
|
| 101 |
+
b4.qn.beta f32 154403148 64 64
|
| 102 |
+
b4.kn.alpha f32 154403404 1 1
|
| 103 |
+
b4.kn.gamma f32 154403408 64 64
|
| 104 |
+
b4.kn.beta f32 154403664 64 64
|
| 105 |
+
b4.out.q i8 154403920 2359296 1536 1536
|
| 106 |
+
b4.out.scale f32 156763216 1536 1536
|
| 107 |
+
b4.ff.alpha f32 156769360 1 1
|
| 108 |
+
b4.ff.gamma f32 156769364 1536 1536
|
| 109 |
+
b4.ff.beta f32 156775508 1536 1536
|
| 110 |
+
b4.glu.q i8 156781652 14155776 1536 9216
|
| 111 |
+
b4.glu.scale f32 170937428 9216 9216
|
| 112 |
+
b4.glu.b f32 170974292 9216 9216
|
| 113 |
+
b4.proj.q i8 171011156 7077888 4608 1536
|
| 114 |
+
b4.proj.scale f32 178089044 1536 1536
|
| 115 |
+
b4.proj.b f32 178095188 1536 1536
|
| 116 |
+
b5.pre.alpha f32 178101332 1 1
|
| 117 |
+
b5.pre.gamma f32 178101336 1536 1536
|
| 118 |
+
b5.pre.beta f32 178107480 1536 1536
|
| 119 |
+
b5.qkv.q i8 178113624 11796480 1536 7680
|
| 120 |
+
b5.qkv.scale f32 189910104 7680 7680
|
| 121 |
+
b5.qn.alpha f32 189940824 1 1
|
| 122 |
+
b5.qn.gamma f32 189940828 64 64
|
| 123 |
+
b5.qn.beta f32 189941084 64 64
|
| 124 |
+
b5.kn.alpha f32 189941340 1 1
|
| 125 |
+
b5.kn.gamma f32 189941344 64 64
|
| 126 |
+
b5.kn.beta f32 189941600 64 64
|
| 127 |
+
b5.out.q i8 189941856 2359296 1536 1536
|
| 128 |
+
b5.out.scale f32 192301152 1536 1536
|
| 129 |
+
b5.ff.alpha f32 192307296 1 1
|
| 130 |
+
b5.ff.gamma f32 192307300 1536 1536
|
| 131 |
+
b5.ff.beta f32 192313444 1536 1536
|
| 132 |
+
b5.glu.q i8 192319588 14155776 1536 9216
|
| 133 |
+
b5.glu.scale f32 206475364 9216 9216
|
| 134 |
+
b5.glu.b f32 206512228 9216 9216
|
| 135 |
+
b5.proj.q i8 206549092 7077888 4608 1536
|
| 136 |
+
b5.proj.scale f32 213626980 1536 1536
|
| 137 |
+
b5.proj.b f32 213633124 1536 1536
|
| 138 |
+
b6.pre.alpha f32 213639268 1 1
|
| 139 |
+
b6.pre.gamma f32 213639272 1536 1536
|
| 140 |
+
b6.pre.beta f32 213645416 1536 1536
|
| 141 |
+
b6.qkv.q i8 213651560 11796480 1536 7680
|
| 142 |
+
b6.qkv.scale f32 225448040 7680 7680
|
| 143 |
+
b6.qn.alpha f32 225478760 1 1
|
| 144 |
+
b6.qn.gamma f32 225478764 64 64
|
| 145 |
+
b6.qn.beta f32 225479020 64 64
|
| 146 |
+
b6.kn.alpha f32 225479276 1 1
|
| 147 |
+
b6.kn.gamma f32 225479280 64 64
|
| 148 |
+
b6.kn.beta f32 225479536 64 64
|
| 149 |
+
b6.out.q i8 225479792 2359296 1536 1536
|
| 150 |
+
b6.out.scale f32 227839088 1536 1536
|
| 151 |
+
b6.ff.alpha f32 227845232 1 1
|
| 152 |
+
b6.ff.gamma f32 227845236 1536 1536
|
| 153 |
+
b6.ff.beta f32 227851380 1536 1536
|
| 154 |
+
b6.glu.q i8 227857524 14155776 1536 9216
|
| 155 |
+
b6.glu.scale f32 242013300 9216 9216
|
| 156 |
+
b6.glu.b f32 242050164 9216 9216
|
| 157 |
+
b6.proj.q i8 242087028 7077888 4608 1536
|
| 158 |
+
b6.proj.scale f32 249164916 1536 1536
|
| 159 |
+
b6.proj.b f32 249171060 1536 1536
|
| 160 |
+
b7.pre.alpha f32 249177204 1 1
|
| 161 |
+
b7.pre.gamma f32 249177208 1536 1536
|
| 162 |
+
b7.pre.beta f32 249183352 1536 1536
|
| 163 |
+
b7.qkv.q i8 249189496 11796480 1536 7680
|
| 164 |
+
b7.qkv.scale f32 260985976 7680 7680
|
| 165 |
+
b7.qn.alpha f32 261016696 1 1
|
| 166 |
+
b7.qn.gamma f32 261016700 64 64
|
| 167 |
+
b7.qn.beta f32 261016956 64 64
|
| 168 |
+
b7.kn.alpha f32 261017212 1 1
|
| 169 |
+
b7.kn.gamma f32 261017216 64 64
|
| 170 |
+
b7.kn.beta f32 261017472 64 64
|
| 171 |
+
b7.out.q i8 261017728 2359296 1536 1536
|
| 172 |
+
b7.out.scale f32 263377024 1536 1536
|
| 173 |
+
b7.ff.alpha f32 263383168 1 1
|
| 174 |
+
b7.ff.gamma f32 263383172 1536 1536
|
| 175 |
+
b7.ff.beta f32 263389316 1536 1536
|
| 176 |
+
b7.glu.q i8 263395460 14155776 1536 9216
|
| 177 |
+
b7.glu.scale f32 277551236 9216 9216
|
| 178 |
+
b7.glu.b f32 277588100 9216 9216
|
| 179 |
+
b7.proj.q i8 277624964 7077888 4608 1536
|
| 180 |
+
b7.proj.scale f32 284702852 1536 1536
|
| 181 |
+
b7.proj.b f32 284708996 1536 1536
|
| 182 |
+
b8.pre.alpha f32 284715140 1 1
|
| 183 |
+
b8.pre.gamma f32 284715144 1536 1536
|
| 184 |
+
b8.pre.beta f32 284721288 1536 1536
|
| 185 |
+
b8.qkv.q i8 284727432 11796480 1536 7680
|
| 186 |
+
b8.qkv.scale f32 296523912 7680 7680
|
| 187 |
+
b8.qn.alpha f32 296554632 1 1
|
| 188 |
+
b8.qn.gamma f32 296554636 64 64
|
| 189 |
+
b8.qn.beta f32 296554892 64 64
|
| 190 |
+
b8.kn.alpha f32 296555148 1 1
|
| 191 |
+
b8.kn.gamma f32 296555152 64 64
|
| 192 |
+
b8.kn.beta f32 296555408 64 64
|
| 193 |
+
b8.out.q i8 296555664 2359296 1536 1536
|
| 194 |
+
b8.out.scale f32 298914960 1536 1536
|
| 195 |
+
b8.ff.alpha f32 298921104 1 1
|
| 196 |
+
b8.ff.gamma f32 298921108 1536 1536
|
| 197 |
+
b8.ff.beta f32 298927252 1536 1536
|
| 198 |
+
b8.glu.q i8 298933396 14155776 1536 9216
|
| 199 |
+
b8.glu.scale f32 313089172 9216 9216
|
| 200 |
+
b8.glu.b f32 313126036 9216 9216
|
| 201 |
+
b8.proj.q i8 313162900 7077888 4608 1536
|
| 202 |
+
b8.proj.scale f32 320240788 1536 1536
|
| 203 |
+
b8.proj.b f32 320246932 1536 1536
|
| 204 |
+
b9.pre.alpha f32 320253076 1 1
|
| 205 |
+
b9.pre.gamma f32 320253080 1536 1536
|
| 206 |
+
b9.pre.beta f32 320259224 1536 1536
|
| 207 |
+
b9.qkv.q i8 320265368 11796480 1536 7680
|
| 208 |
+
b9.qkv.scale f32 332061848 7680 7680
|
| 209 |
+
b9.qn.alpha f32 332092568 1 1
|
| 210 |
+
b9.qn.gamma f32 332092572 64 64
|
| 211 |
+
b9.qn.beta f32 332092828 64 64
|
| 212 |
+
b9.kn.alpha f32 332093084 1 1
|
| 213 |
+
b9.kn.gamma f32 332093088 64 64
|
| 214 |
+
b9.kn.beta f32 332093344 64 64
|
| 215 |
+
b9.out.q i8 332093600 2359296 1536 1536
|
| 216 |
+
b9.out.scale f32 334452896 1536 1536
|
| 217 |
+
b9.ff.alpha f32 334459040 1 1
|
| 218 |
+
b9.ff.gamma f32 334459044 1536 1536
|
| 219 |
+
b9.ff.beta f32 334465188 1536 1536
|
| 220 |
+
b9.glu.q i8 334471332 14155776 1536 9216
|
| 221 |
+
b9.glu.scale f32 348627108 9216 9216
|
| 222 |
+
b9.glu.b f32 348663972 9216 9216
|
| 223 |
+
b9.proj.q i8 348700836 7077888 4608 1536
|
| 224 |
+
b9.proj.scale f32 355778724 1536 1536
|
| 225 |
+
b9.proj.b f32 355784868 1536 1536
|
| 226 |
+
b10.pre.alpha f32 355791012 1 1
|
| 227 |
+
b10.pre.gamma f32 355791016 1536 1536
|
| 228 |
+
b10.pre.beta f32 355797160 1536 1536
|
| 229 |
+
b10.qkv.q i8 355803304 11796480 1536 7680
|
| 230 |
+
b10.qkv.scale f32 367599784 7680 7680
|
| 231 |
+
b10.qn.alpha f32 367630504 1 1
|
| 232 |
+
b10.qn.gamma f32 367630508 64 64
|
| 233 |
+
b10.qn.beta f32 367630764 64 64
|
| 234 |
+
b10.kn.alpha f32 367631020 1 1
|
| 235 |
+
b10.kn.gamma f32 367631024 64 64
|
| 236 |
+
b10.kn.beta f32 367631280 64 64
|
| 237 |
+
b10.out.q i8 367631536 2359296 1536 1536
|
| 238 |
+
b10.out.scale f32 369990832 1536 1536
|
| 239 |
+
b10.ff.alpha f32 369996976 1 1
|
| 240 |
+
b10.ff.gamma f32 369996980 1536 1536
|
| 241 |
+
b10.ff.beta f32 370003124 1536 1536
|
| 242 |
+
b10.glu.q i8 370009268 14155776 1536 9216
|
| 243 |
+
b10.glu.scale f32 384165044 9216 9216
|
| 244 |
+
b10.glu.b f32 384201908 9216 9216
|
| 245 |
+
b10.proj.q i8 384238772 7077888 4608 1536
|
| 246 |
+
b10.proj.scale f32 391316660 1536 1536
|
| 247 |
+
b10.proj.b f32 391322804 1536 1536
|
| 248 |
+
b11.pre.alpha f32 391328948 1 1
|
| 249 |
+
b11.pre.gamma f32 391328952 1536 1536
|
| 250 |
+
b11.pre.beta f32 391335096 1536 1536
|
| 251 |
+
b11.qkv.q i8 391341240 11796480 1536 7680
|
| 252 |
+
b11.qkv.scale f32 403137720 7680 7680
|
| 253 |
+
b11.qn.alpha f32 403168440 1 1
|
| 254 |
+
b11.qn.gamma f32 403168444 64 64
|
| 255 |
+
b11.qn.beta f32 403168700 64 64
|
| 256 |
+
b11.kn.alpha f32 403168956 1 1
|
| 257 |
+
b11.kn.gamma f32 403168960 64 64
|
| 258 |
+
b11.kn.beta f32 403169216 64 64
|
| 259 |
+
b11.out.q i8 403169472 2359296 1536 1536
|
| 260 |
+
b11.out.scale f32 405528768 1536 1536
|
| 261 |
+
b11.ff.alpha f32 405534912 1 1
|
| 262 |
+
b11.ff.gamma f32 405534916 1536 1536
|
| 263 |
+
b11.ff.beta f32 405541060 1536 1536
|
| 264 |
+
b11.glu.q i8 405547204 14155776 1536 9216
|
| 265 |
+
b11.glu.scale f32 419702980 9216 9216
|
| 266 |
+
b11.glu.b f32 419739844 9216 9216
|
| 267 |
+
b11.proj.q i8 419776708 7077888 4608 1536
|
| 268 |
+
b11.proj.scale f32 426854596 1536 1536
|
| 269 |
+
b11.proj.b f32 426860740 1536 1536
|
| 270 |
+
map.q i8 426866884 786432 1536 512
|
| 271 |
+
map.scale f32 427653316 512 512
|
| 272 |
+
map.b f32 427655364 512 512
|
cpu-amx/same_l_encoder.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:21c77188d4a644bdbac852bdd847f7bf0d12ddf2f524f504cdeb58e2cc01a3f6
|
| 3 |
+
size 29355520
|
cpu-amx/same_l_encoder_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:49482e2a8f916d1fff73385ff9b5c577bea03129fb9f00751fa837582a4d2fba
|
| 3 |
+
size 852544708
|
cpu-amx/same_l_encoder_weights_f32.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:297aa69ee0c1e8219bb80bd16036e8f8bda0b8a15064c58092f4dc17e8f72b35
|
| 3 |
+
size 1703411712
|
cpu-amx/same_l_encoder_weights_f32_manifest.txt
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
map_in.w f32 0 786432 512 1536
|
| 2 |
+
b0.qkv.w f32 3145728 11796480 1536 7680
|
| 3 |
+
b0.out.w f32 50331648 2359296 1536 1536
|
| 4 |
+
b0.glu.w f32 59768832 14155776 1536 9216
|
| 5 |
+
b0.proj.w f32 116391936 7077888 4608 1536
|
| 6 |
+
b1.qkv.w f32 144703488 11796480 1536 7680
|
| 7 |
+
b1.out.w f32 191889408 2359296 1536 1536
|
| 8 |
+
b1.glu.w f32 201326592 14155776 1536 9216
|
| 9 |
+
b1.proj.w f32 257949696 7077888 4608 1536
|
| 10 |
+
b2.qkv.w f32 286261248 11796480 1536 7680
|
| 11 |
+
b2.out.w f32 333447168 2359296 1536 1536
|
| 12 |
+
b2.glu.w f32 342884352 14155776 1536 9216
|
| 13 |
+
b2.proj.w f32 399507456 7077888 4608 1536
|
| 14 |
+
b3.qkv.w f32 427819008 11796480 1536 7680
|
| 15 |
+
b3.out.w f32 475004928 2359296 1536 1536
|
| 16 |
+
b3.glu.w f32 484442112 14155776 1536 9216
|
| 17 |
+
b3.proj.w f32 541065216 7077888 4608 1536
|
| 18 |
+
b4.qkv.w f32 569376768 11796480 1536 7680
|
| 19 |
+
b4.out.w f32 616562688 2359296 1536 1536
|
| 20 |
+
b4.glu.w f32 625999872 14155776 1536 9216
|
| 21 |
+
b4.proj.w f32 682622976 7077888 4608 1536
|
| 22 |
+
b5.qkv.w f32 710934528 11796480 1536 7680
|
| 23 |
+
b5.out.w f32 758120448 2359296 1536 1536
|
| 24 |
+
b5.glu.w f32 767557632 14155776 1536 9216
|
| 25 |
+
b5.proj.w f32 824180736 7077888 4608 1536
|
| 26 |
+
b6.qkv.w f32 852492288 11796480 1536 7680
|
| 27 |
+
b6.out.w f32 899678208 2359296 1536 1536
|
| 28 |
+
b6.glu.w f32 909115392 14155776 1536 9216
|
| 29 |
+
b6.proj.w f32 965738496 7077888 4608 1536
|
| 30 |
+
b7.qkv.w f32 994050048 11796480 1536 7680
|
| 31 |
+
b7.out.w f32 1041235968 2359296 1536 1536
|
| 32 |
+
b7.glu.w f32 1050673152 14155776 1536 9216
|
| 33 |
+
b7.proj.w f32 1107296256 7077888 4608 1536
|
| 34 |
+
b8.qkv.w f32 1135607808 11796480 1536 7680
|
| 35 |
+
b8.out.w f32 1182793728 2359296 1536 1536
|
| 36 |
+
b8.glu.w f32 1192230912 14155776 1536 9216
|
| 37 |
+
b8.proj.w f32 1248854016 7077888 4608 1536
|
| 38 |
+
b9.qkv.w f32 1277165568 11796480 1536 7680
|
| 39 |
+
b9.out.w f32 1324351488 2359296 1536 1536
|
| 40 |
+
b9.glu.w f32 1333788672 14155776 1536 9216
|
| 41 |
+
b9.proj.w f32 1390411776 7077888 4608 1536
|
| 42 |
+
b10.qkv.w f32 1418723328 11796480 1536 7680
|
| 43 |
+
b10.out.w f32 1465909248 2359296 1536 1536
|
| 44 |
+
b10.glu.w f32 1475346432 14155776 1536 9216
|
| 45 |
+
b10.proj.w f32 1531969536 7077888 4608 1536
|
| 46 |
+
b11.qkv.w f32 1560281088 11796480 1536 7680
|
| 47 |
+
b11.out.w f32 1607467008 2359296 1536 1536
|
| 48 |
+
b11.glu.w f32 1616904192 14155776 1536 9216
|
| 49 |
+
b11.proj.w f32 1673527296 7077888 4608 1536
|
| 50 |
+
proj_out.w f32 1701838848 393216 1536 256
|
cpu-amx/same_l_encoder_weights_manifest.txt
ADDED
|
@@ -0,0 +1,224 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
map_in.w bf16 0 786432 512 1536
|
| 2 |
+
map_in.b f32 1572864 1536 1536
|
| 3 |
+
new_tokens f32 1579008 1536 1536
|
| 4 |
+
b0.pre.alpha f32 1585152 1 1
|
| 5 |
+
b0.pre.gamma f32 1585156 1536 1536
|
| 6 |
+
b0.pre.beta f32 1591300 1536 1536
|
| 7 |
+
b0.qkv.w bf16 1597444 11796480 1536 7680
|
| 8 |
+
b0.qn.alpha f32 25190404 1 1
|
| 9 |
+
b0.qn.gamma f32 25190408 64 64
|
| 10 |
+
b0.qn.beta f32 25190664 64 64
|
| 11 |
+
b0.kn.alpha f32 25190920 1 1
|
| 12 |
+
b0.kn.gamma f32 25190924 64 64
|
| 13 |
+
b0.kn.beta f32 25191180 64 64
|
| 14 |
+
b0.out.w bf16 25191436 2359296 1536 1536
|
| 15 |
+
b0.ff.alpha f32 29910028 1 1
|
| 16 |
+
b0.ff.gamma f32 29910032 1536 1536
|
| 17 |
+
b0.ff.beta f32 29916176 1536 1536
|
| 18 |
+
b0.glu.w bf16 29922320 14155776 1536 9216
|
| 19 |
+
b0.glu.b f32 58233872 9216 9216
|
| 20 |
+
b0.proj.w bf16 58270736 7077888 4608 1536
|
| 21 |
+
b0.proj.b f32 72426512 1536 1536
|
| 22 |
+
b1.pre.alpha f32 72432656 1 1
|
| 23 |
+
b1.pre.gamma f32 72432660 1536 1536
|
| 24 |
+
b1.pre.beta f32 72438804 1536 1536
|
| 25 |
+
b1.qkv.w bf16 72444948 11796480 1536 7680
|
| 26 |
+
b1.qn.alpha f32 96037908 1 1
|
| 27 |
+
b1.qn.gamma f32 96037912 64 64
|
| 28 |
+
b1.qn.beta f32 96038168 64 64
|
| 29 |
+
b1.kn.alpha f32 96038424 1 1
|
| 30 |
+
b1.kn.gamma f32 96038428 64 64
|
| 31 |
+
b1.kn.beta f32 96038684 64 64
|
| 32 |
+
b1.out.w bf16 96038940 2359296 1536 1536
|
| 33 |
+
b1.ff.alpha f32 100757532 1 1
|
| 34 |
+
b1.ff.gamma f32 100757536 1536 1536
|
| 35 |
+
b1.ff.beta f32 100763680 1536 1536
|
| 36 |
+
b1.glu.w bf16 100769824 14155776 1536 9216
|
| 37 |
+
b1.glu.b f32 129081376 9216 9216
|
| 38 |
+
b1.proj.w bf16 129118240 7077888 4608 1536
|
| 39 |
+
b1.proj.b f32 143274016 1536 1536
|
| 40 |
+
b2.pre.alpha f32 143280160 1 1
|
| 41 |
+
b2.pre.gamma f32 143280164 1536 1536
|
| 42 |
+
b2.pre.beta f32 143286308 1536 1536
|
| 43 |
+
b2.qkv.w bf16 143292452 11796480 1536 7680
|
| 44 |
+
b2.qn.alpha f32 166885412 1 1
|
| 45 |
+
b2.qn.gamma f32 166885416 64 64
|
| 46 |
+
b2.qn.beta f32 166885672 64 64
|
| 47 |
+
b2.kn.alpha f32 166885928 1 1
|
| 48 |
+
b2.kn.gamma f32 166885932 64 64
|
| 49 |
+
b2.kn.beta f32 166886188 64 64
|
| 50 |
+
b2.out.w bf16 166886444 2359296 1536 1536
|
| 51 |
+
b2.ff.alpha f32 171605036 1 1
|
| 52 |
+
b2.ff.gamma f32 171605040 1536 1536
|
| 53 |
+
b2.ff.beta f32 171611184 1536 1536
|
| 54 |
+
b2.glu.w bf16 171617328 14155776 1536 9216
|
| 55 |
+
b2.glu.b f32 199928880 9216 9216
|
| 56 |
+
b2.proj.w bf16 199965744 7077888 4608 1536
|
| 57 |
+
b2.proj.b f32 214121520 1536 1536
|
| 58 |
+
b3.pre.alpha f32 214127664 1 1
|
| 59 |
+
b3.pre.gamma f32 214127668 1536 1536
|
| 60 |
+
b3.pre.beta f32 214133812 1536 1536
|
| 61 |
+
b3.qkv.w bf16 214139956 11796480 1536 7680
|
| 62 |
+
b3.qn.alpha f32 237732916 1 1
|
| 63 |
+
b3.qn.gamma f32 237732920 64 64
|
| 64 |
+
b3.qn.beta f32 237733176 64 64
|
| 65 |
+
b3.kn.alpha f32 237733432 1 1
|
| 66 |
+
b3.kn.gamma f32 237733436 64 64
|
| 67 |
+
b3.kn.beta f32 237733692 64 64
|
| 68 |
+
b3.out.w bf16 237733948 2359296 1536 1536
|
| 69 |
+
b3.ff.alpha f32 242452540 1 1
|
| 70 |
+
b3.ff.gamma f32 242452544 1536 1536
|
| 71 |
+
b3.ff.beta f32 242458688 1536 1536
|
| 72 |
+
b3.glu.w bf16 242464832 14155776 1536 9216
|
| 73 |
+
b3.glu.b f32 270776384 9216 9216
|
| 74 |
+
b3.proj.w bf16 270813248 7077888 4608 1536
|
| 75 |
+
b3.proj.b f32 284969024 1536 1536
|
| 76 |
+
b4.pre.alpha f32 284975168 1 1
|
| 77 |
+
b4.pre.gamma f32 284975172 1536 1536
|
| 78 |
+
b4.pre.beta f32 284981316 1536 1536
|
| 79 |
+
b4.qkv.w bf16 284987460 11796480 1536 7680
|
| 80 |
+
b4.qn.alpha f32 308580420 1 1
|
| 81 |
+
b4.qn.gamma f32 308580424 64 64
|
| 82 |
+
b4.qn.beta f32 308580680 64 64
|
| 83 |
+
b4.kn.alpha f32 308580936 1 1
|
| 84 |
+
b4.kn.gamma f32 308580940 64 64
|
| 85 |
+
b4.kn.beta f32 308581196 64 64
|
| 86 |
+
b4.out.w bf16 308581452 2359296 1536 1536
|
| 87 |
+
b4.ff.alpha f32 313300044 1 1
|
| 88 |
+
b4.ff.gamma f32 313300048 1536 1536
|
| 89 |
+
b4.ff.beta f32 313306192 1536 1536
|
| 90 |
+
b4.glu.w bf16 313312336 14155776 1536 9216
|
| 91 |
+
b4.glu.b f32 341623888 9216 9216
|
| 92 |
+
b4.proj.w bf16 341660752 7077888 4608 1536
|
| 93 |
+
b4.proj.b f32 355816528 1536 1536
|
| 94 |
+
b5.pre.alpha f32 355822672 1 1
|
| 95 |
+
b5.pre.gamma f32 355822676 1536 1536
|
| 96 |
+
b5.pre.beta f32 355828820 1536 1536
|
| 97 |
+
b5.qkv.w bf16 355834964 11796480 1536 7680
|
| 98 |
+
b5.qn.alpha f32 379427924 1 1
|
| 99 |
+
b5.qn.gamma f32 379427928 64 64
|
| 100 |
+
b5.qn.beta f32 379428184 64 64
|
| 101 |
+
b5.kn.alpha f32 379428440 1 1
|
| 102 |
+
b5.kn.gamma f32 379428444 64 64
|
| 103 |
+
b5.kn.beta f32 379428700 64 64
|
| 104 |
+
b5.out.w bf16 379428956 2359296 1536 1536
|
| 105 |
+
b5.ff.alpha f32 384147548 1 1
|
| 106 |
+
b5.ff.gamma f32 384147552 1536 1536
|
| 107 |
+
b5.ff.beta f32 384153696 1536 1536
|
| 108 |
+
b5.glu.w bf16 384159840 14155776 1536 9216
|
| 109 |
+
b5.glu.b f32 412471392 9216 9216
|
| 110 |
+
b5.proj.w bf16 412508256 7077888 4608 1536
|
| 111 |
+
b5.proj.b f32 426664032 1536 1536
|
| 112 |
+
b6.pre.alpha f32 426670176 1 1
|
| 113 |
+
b6.pre.gamma f32 426670180 1536 1536
|
| 114 |
+
b6.pre.beta f32 426676324 1536 1536
|
| 115 |
+
b6.qkv.w bf16 426682468 11796480 1536 7680
|
| 116 |
+
b6.qn.alpha f32 450275428 1 1
|
| 117 |
+
b6.qn.gamma f32 450275432 64 64
|
| 118 |
+
b6.qn.beta f32 450275688 64 64
|
| 119 |
+
b6.kn.alpha f32 450275944 1 1
|
| 120 |
+
b6.kn.gamma f32 450275948 64 64
|
| 121 |
+
b6.kn.beta f32 450276204 64 64
|
| 122 |
+
b6.out.w bf16 450276460 2359296 1536 1536
|
| 123 |
+
b6.ff.alpha f32 454995052 1 1
|
| 124 |
+
b6.ff.gamma f32 454995056 1536 1536
|
| 125 |
+
b6.ff.beta f32 455001200 1536 1536
|
| 126 |
+
b6.glu.w bf16 455007344 14155776 1536 9216
|
| 127 |
+
b6.glu.b f32 483318896 9216 9216
|
| 128 |
+
b6.proj.w bf16 483355760 7077888 4608 1536
|
| 129 |
+
b6.proj.b f32 497511536 1536 1536
|
| 130 |
+
b7.pre.alpha f32 497517680 1 1
|
| 131 |
+
b7.pre.gamma f32 497517684 1536 1536
|
| 132 |
+
b7.pre.beta f32 497523828 1536 1536
|
| 133 |
+
b7.qkv.w bf16 497529972 11796480 1536 7680
|
| 134 |
+
b7.qn.alpha f32 521122932 1 1
|
| 135 |
+
b7.qn.gamma f32 521122936 64 64
|
| 136 |
+
b7.qn.beta f32 521123192 64 64
|
| 137 |
+
b7.kn.alpha f32 521123448 1 1
|
| 138 |
+
b7.kn.gamma f32 521123452 64 64
|
| 139 |
+
b7.kn.beta f32 521123708 64 64
|
| 140 |
+
b7.out.w bf16 521123964 2359296 1536 1536
|
| 141 |
+
b7.ff.alpha f32 525842556 1 1
|
| 142 |
+
b7.ff.gamma f32 525842560 1536 1536
|
| 143 |
+
b7.ff.beta f32 525848704 1536 1536
|
| 144 |
+
b7.glu.w bf16 525854848 14155776 1536 9216
|
| 145 |
+
b7.glu.b f32 554166400 9216 9216
|
| 146 |
+
b7.proj.w bf16 554203264 7077888 4608 1536
|
| 147 |
+
b7.proj.b f32 568359040 1536 1536
|
| 148 |
+
b8.pre.alpha f32 568365184 1 1
|
| 149 |
+
b8.pre.gamma f32 568365188 1536 1536
|
| 150 |
+
b8.pre.beta f32 568371332 1536 1536
|
| 151 |
+
b8.qkv.w bf16 568377476 11796480 1536 7680
|
| 152 |
+
b8.qn.alpha f32 591970436 1 1
|
| 153 |
+
b8.qn.gamma f32 591970440 64 64
|
| 154 |
+
b8.qn.beta f32 591970696 64 64
|
| 155 |
+
b8.kn.alpha f32 591970952 1 1
|
| 156 |
+
b8.kn.gamma f32 591970956 64 64
|
| 157 |
+
b8.kn.beta f32 591971212 64 64
|
| 158 |
+
b8.out.w bf16 591971468 2359296 1536 1536
|
| 159 |
+
b8.ff.alpha f32 596690060 1 1
|
| 160 |
+
b8.ff.gamma f32 596690064 1536 1536
|
| 161 |
+
b8.ff.beta f32 596696208 1536 1536
|
| 162 |
+
b8.glu.w bf16 596702352 14155776 1536 9216
|
| 163 |
+
b8.glu.b f32 625013904 9216 9216
|
| 164 |
+
b8.proj.w bf16 625050768 7077888 4608 1536
|
| 165 |
+
b8.proj.b f32 639206544 1536 1536
|
| 166 |
+
b9.pre.alpha f32 639212688 1 1
|
| 167 |
+
b9.pre.gamma f32 639212692 1536 1536
|
| 168 |
+
b9.pre.beta f32 639218836 1536 1536
|
| 169 |
+
b9.qkv.w bf16 639224980 11796480 1536 7680
|
| 170 |
+
b9.qn.alpha f32 662817940 1 1
|
| 171 |
+
b9.qn.gamma f32 662817944 64 64
|
| 172 |
+
b9.qn.beta f32 662818200 64 64
|
| 173 |
+
b9.kn.alpha f32 662818456 1 1
|
| 174 |
+
b9.kn.gamma f32 662818460 64 64
|
| 175 |
+
b9.kn.beta f32 662818716 64 64
|
| 176 |
+
b9.out.w bf16 662818972 2359296 1536 1536
|
| 177 |
+
b9.ff.alpha f32 667537564 1 1
|
| 178 |
+
b9.ff.gamma f32 667537568 1536 1536
|
| 179 |
+
b9.ff.beta f32 667543712 1536 1536
|
| 180 |
+
b9.glu.w bf16 667549856 14155776 1536 9216
|
| 181 |
+
b9.glu.b f32 695861408 9216 9216
|
| 182 |
+
b9.proj.w bf16 695898272 7077888 4608 1536
|
| 183 |
+
b9.proj.b f32 710054048 1536 1536
|
| 184 |
+
b10.pre.alpha f32 710060192 1 1
|
| 185 |
+
b10.pre.gamma f32 710060196 1536 1536
|
| 186 |
+
b10.pre.beta f32 710066340 1536 1536
|
| 187 |
+
b10.qkv.w bf16 710072484 11796480 1536 7680
|
| 188 |
+
b10.qn.alpha f32 733665444 1 1
|
| 189 |
+
b10.qn.gamma f32 733665448 64 64
|
| 190 |
+
b10.qn.beta f32 733665704 64 64
|
| 191 |
+
b10.kn.alpha f32 733665960 1 1
|
| 192 |
+
b10.kn.gamma f32 733665964 64 64
|
| 193 |
+
b10.kn.beta f32 733666220 64 64
|
| 194 |
+
b10.out.w bf16 733666476 2359296 1536 1536
|
| 195 |
+
b10.ff.alpha f32 738385068 1 1
|
| 196 |
+
b10.ff.gamma f32 738385072 1536 1536
|
| 197 |
+
b10.ff.beta f32 738391216 1536 1536
|
| 198 |
+
b10.glu.w bf16 738397360 14155776 1536 9216
|
| 199 |
+
b10.glu.b f32 766708912 9216 9216
|
| 200 |
+
b10.proj.w bf16 766745776 7077888 4608 1536
|
| 201 |
+
b10.proj.b f32 780901552 1536 1536
|
| 202 |
+
b11.pre.alpha f32 780907696 1 1
|
| 203 |
+
b11.pre.gamma f32 780907700 1536 1536
|
| 204 |
+
b11.pre.beta f32 780913844 1536 1536
|
| 205 |
+
b11.qkv.w bf16 780919988 11796480 1536 7680
|
| 206 |
+
b11.qn.alpha f32 804512948 1 1
|
| 207 |
+
b11.qn.gamma f32 804512952 64 64
|
| 208 |
+
b11.qn.beta f32 804513208 64 64
|
| 209 |
+
b11.kn.alpha f32 804513464 1 1
|
| 210 |
+
b11.kn.gamma f32 804513468 64 64
|
| 211 |
+
b11.kn.beta f32 804513724 64 64
|
| 212 |
+
b11.out.w bf16 804513980 2359296 1536 1536
|
| 213 |
+
b11.ff.alpha f32 809232572 1 1
|
| 214 |
+
b11.ff.gamma f32 809232576 1536 1536
|
| 215 |
+
b11.ff.beta f32 809238720 1536 1536
|
| 216 |
+
b11.glu.w bf16 809244864 14155776 1536 9216
|
| 217 |
+
b11.glu.b f32 837556416 9216 9216
|
| 218 |
+
b11.proj.w bf16 837593280 7077888 4608 1536
|
| 219 |
+
b11.proj.b f32 851749056 1536 1536
|
| 220 |
+
proj_out.w bf16 851755200 393216 1536 256
|
| 221 |
+
proj_out.b f32 852541632 256 256
|
| 222 |
+
scaling_factor f32 852542656 256 256
|
| 223 |
+
sn_bias f32 852543680 256 256
|
| 224 |
+
running_std f32 852544704 1 1
|
cpu-amx/same_s_decoder_bf16.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:840396c782a7417bc4779c877be546c29b811f0341d1589d41f21947b147fdd3
|
| 3 |
+
size 66974616
|
cpu-amx/same_s_decoder_bf16_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2a1b60e9e718a4967ac53fecd91e9fe24757de049b5e36f9de7857dcaa23d986
|
| 3 |
+
size 109138020
|
cpu-amx/same_s_decoder_bf16_weights_manifest.txt
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
running_std f32 0 1 1
|
| 2 |
+
project_in.w bf16 4 196608 256 768
|
| 3 |
+
project_in.b f32 393220 768 768
|
| 4 |
+
new_tokens f32 396292 768 768
|
| 5 |
+
b0.pre.alpha f32 399364 1 1
|
| 6 |
+
b0.pre.gamma f32 399368 768 768
|
| 7 |
+
b0.pre.beta f32 402440 768 768
|
| 8 |
+
b0.qkv.w bf16 405512 2949120 768 3840
|
| 9 |
+
b0.qn.alpha f32 6303752 1 1
|
| 10 |
+
b0.qn.gamma f32 6303756 64 64
|
| 11 |
+
b0.qn.beta f32 6304012 64 64
|
| 12 |
+
b0.kn.alpha f32 6304268 1 1
|
| 13 |
+
b0.kn.gamma f32 6304272 64 64
|
| 14 |
+
b0.kn.beta f32 6304528 64 64
|
| 15 |
+
b0.out.w bf16 6304784 589824 768 768
|
| 16 |
+
b0.ff.alpha f32 7484432 1 1
|
| 17 |
+
b0.ff.gamma f32 7484436 768 768
|
| 18 |
+
b0.ff.beta f32 7487508 768 768
|
| 19 |
+
b0.glu.w bf16 7490580 3538944 768 4608
|
| 20 |
+
b0.glu.b f32 14568468 4608 4608
|
| 21 |
+
b0.proj.w bf16 14586900 1769472 2304 768
|
| 22 |
+
b0.proj.b f32 18125844 768 768
|
| 23 |
+
b1.pre.alpha f32 18128916 1 1
|
| 24 |
+
b1.pre.gamma f32 18128920 768 768
|
| 25 |
+
b1.pre.beta f32 18131992 768 768
|
| 26 |
+
b1.qkv.w bf16 18135064 2949120 768 3840
|
| 27 |
+
b1.qn.alpha f32 24033304 1 1
|
| 28 |
+
b1.qn.gamma f32 24033308 64 64
|
| 29 |
+
b1.qn.beta f32 24033564 64 64
|
| 30 |
+
b1.kn.alpha f32 24033820 1 1
|
| 31 |
+
b1.kn.gamma f32 24033824 64 64
|
| 32 |
+
b1.kn.beta f32 24034080 64 64
|
| 33 |
+
b1.out.w bf16 24034336 589824 768 768
|
| 34 |
+
b1.ff.alpha f32 25213984 1 1
|
| 35 |
+
b1.ff.gamma f32 25213988 768 768
|
| 36 |
+
b1.ff.beta f32 25217060 768 768
|
| 37 |
+
b1.glu.w bf16 25220132 3538944 768 4608
|
| 38 |
+
b1.glu.b f32 32298020 4608 4608
|
| 39 |
+
b1.proj.w bf16 32316452 1769472 2304 768
|
| 40 |
+
b1.proj.b f32 35855396 768 768
|
| 41 |
+
b2.pre.alpha f32 35858468 1 1
|
| 42 |
+
b2.pre.gamma f32 35858472 768 768
|
| 43 |
+
b2.pre.beta f32 35861544 768 768
|
| 44 |
+
b2.qkv.w bf16 35864616 2949120 768 3840
|
| 45 |
+
b2.qn.alpha f32 41762856 1 1
|
| 46 |
+
b2.qn.gamma f32 41762860 64 64
|
| 47 |
+
b2.qn.beta f32 41763116 64 64
|
| 48 |
+
b2.kn.alpha f32 41763372 1 1
|
| 49 |
+
b2.kn.gamma f32 41763376 64 64
|
| 50 |
+
b2.kn.beta f32 41763632 64 64
|
| 51 |
+
b2.out.w bf16 41763888 589824 768 768
|
| 52 |
+
b2.ff.alpha f32 42943536 1 1
|
| 53 |
+
b2.ff.gamma f32 42943540 768 768
|
| 54 |
+
b2.ff.beta f32 42946612 768 768
|
| 55 |
+
b2.glu.w bf16 42949684 3538944 768 4608
|
| 56 |
+
b2.glu.b f32 50027572 4608 4608
|
| 57 |
+
b2.proj.w bf16 50046004 1769472 2304 768
|
| 58 |
+
b2.proj.b f32 53584948 768 768
|
| 59 |
+
b3.pre.alpha f32 53588020 1 1
|
| 60 |
+
b3.pre.gamma f32 53588024 768 768
|
| 61 |
+
b3.pre.beta f32 53591096 768 768
|
| 62 |
+
b3.qkv.w bf16 53594168 2949120 768 3840
|
| 63 |
+
b3.qn.alpha f32 59492408 1 1
|
| 64 |
+
b3.qn.gamma f32 59492412 64 64
|
| 65 |
+
b3.qn.beta f32 59492668 64 64
|
| 66 |
+
b3.kn.alpha f32 59492924 1 1
|
| 67 |
+
b3.kn.gamma f32 59492928 64 64
|
| 68 |
+
b3.kn.beta f32 59493184 64 64
|
| 69 |
+
b3.out.w bf16 59493440 589824 768 768
|
| 70 |
+
b3.ff.alpha f32 60673088 1 1
|
| 71 |
+
b3.ff.gamma f32 60673092 768 768
|
| 72 |
+
b3.ff.beta f32 60676164 768 768
|
| 73 |
+
b3.glu.w bf16 60679236 3538944 768 4608
|
| 74 |
+
b3.glu.b f32 67757124 4608 4608
|
| 75 |
+
b3.proj.w bf16 67775556 1769472 2304 768
|
| 76 |
+
b3.proj.b f32 71314500 768 768
|
| 77 |
+
b4.pre.alpha f32 71317572 1 1
|
| 78 |
+
b4.pre.gamma f32 71317576 768 768
|
| 79 |
+
b4.pre.beta f32 71320648 768 768
|
| 80 |
+
b4.qkv.w bf16 71323720 2949120 768 3840
|
| 81 |
+
b4.qn.alpha f32 77221960 1 1
|
| 82 |
+
b4.qn.gamma f32 77221964 64 64
|
| 83 |
+
b4.qn.beta f32 77222220 64 64
|
| 84 |
+
b4.kn.alpha f32 77222476 1 1
|
| 85 |
+
b4.kn.gamma f32 77222480 64 64
|
| 86 |
+
b4.kn.beta f32 77222736 64 64
|
| 87 |
+
b4.out.w bf16 77222992 589824 768 768
|
| 88 |
+
b4.ff.alpha f32 78402640 1 1
|
| 89 |
+
b4.ff.gamma f32 78402644 768 768
|
| 90 |
+
b4.ff.beta f32 78405716 768 768
|
| 91 |
+
b4.glu.w bf16 78408788 3538944 768 4608
|
| 92 |
+
b4.glu.b f32 85486676 4608 4608
|
| 93 |
+
b4.proj.w bf16 85505108 1769472 2304 768
|
| 94 |
+
b4.proj.b f32 89044052 768 768
|
| 95 |
+
b5.pre.alpha f32 89047124 1 1
|
| 96 |
+
b5.pre.gamma f32 89047128 768 768
|
| 97 |
+
b5.pre.beta f32 89050200 768 768
|
| 98 |
+
b5.qkv.w bf16 89053272 2949120 768 3840
|
| 99 |
+
b5.qn.alpha f32 94951512 1 1
|
| 100 |
+
b5.qn.gamma f32 94951516 64 64
|
| 101 |
+
b5.qn.beta f32 94951772 64 64
|
| 102 |
+
b5.kn.alpha f32 94952028 1 1
|
| 103 |
+
b5.kn.gamma f32 94952032 64 64
|
| 104 |
+
b5.kn.beta f32 94952288 64 64
|
| 105 |
+
b5.out.w bf16 94952544 589824 768 768
|
| 106 |
+
b5.ff.alpha f32 96132192 1 1
|
| 107 |
+
b5.ff.gamma f32 96132196 768 768
|
| 108 |
+
b5.ff.beta f32 96135268 768 768
|
| 109 |
+
b5.glu.w bf16 96138340 3538944 768 4608
|
| 110 |
+
b5.glu.b f32 103216228 4608 4608
|
| 111 |
+
b5.proj.w bf16 103234660 1769472 2304 768
|
| 112 |
+
b5.proj.b f32 106773604 768 768
|
| 113 |
+
conv.w bf16 106776676 1179648 2304 512
|
| 114 |
+
conv.b f32 109135972 512 512
|
cpu-amx/same_s_decoder_int8.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e66dfee72eaacedb2dc7e89d9915777773bb93086a453a4e64a329792b946f35
|
| 3 |
+
size 29345968
|
cpu-amx/same_s_decoder_int8_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:af4667431a379274bf3d920f9a62f7e10dcad3b9282d7c201c3006d8ec3e86e1
|
| 3 |
+
size 54922340
|
cpu-amx/same_s_decoder_int8_weights_manifest.txt
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
running_std f32 0 1 1
|
| 2 |
+
project_in.q i8 4 196608 256 768
|
| 3 |
+
project_in.scale f32 196612 768 768
|
| 4 |
+
project_in.b f32 199684 768 768
|
| 5 |
+
new_tokens f32 202756 768 768
|
| 6 |
+
b0.pre.alpha f32 205828 1 1
|
| 7 |
+
b0.pre.gamma f32 205832 768 768
|
| 8 |
+
b0.pre.beta f32 208904 768 768
|
| 9 |
+
b0.qkv.q i8 211976 2949120 768 3840
|
| 10 |
+
b0.qkv.scale f32 3161096 3840 3840
|
| 11 |
+
b0.qn.alpha f32 3176456 1 1
|
| 12 |
+
b0.qn.gamma f32 3176460 64 64
|
| 13 |
+
b0.qn.beta f32 3176716 64 64
|
| 14 |
+
b0.kn.alpha f32 3176972 1 1
|
| 15 |
+
b0.kn.gamma f32 3176976 64 64
|
| 16 |
+
b0.kn.beta f32 3177232 64 64
|
| 17 |
+
b0.out.q i8 3177488 589824 768 768
|
| 18 |
+
b0.out.scale f32 3767312 768 768
|
| 19 |
+
b0.ff.alpha f32 3770384 1 1
|
| 20 |
+
b0.ff.gamma f32 3770388 768 768
|
| 21 |
+
b0.ff.beta f32 3773460 768 768
|
| 22 |
+
b0.glu.q i8 3776532 3538944 768 4608
|
| 23 |
+
b0.glu.scale f32 7315476 4608 4608
|
| 24 |
+
b0.glu.b f32 7333908 4608 4608
|
| 25 |
+
b0.proj.q i8 7352340 1769472 2304 768
|
| 26 |
+
b0.proj.scale f32 9121812 768 768
|
| 27 |
+
b0.proj.b f32 9124884 768 768
|
| 28 |
+
b1.pre.alpha f32 9127956 1 1
|
| 29 |
+
b1.pre.gamma f32 9127960 768 768
|
| 30 |
+
b1.pre.beta f32 9131032 768 768
|
| 31 |
+
b1.qkv.q i8 9134104 2949120 768 3840
|
| 32 |
+
b1.qkv.scale f32 12083224 3840 3840
|
| 33 |
+
b1.qn.alpha f32 12098584 1 1
|
| 34 |
+
b1.qn.gamma f32 12098588 64 64
|
| 35 |
+
b1.qn.beta f32 12098844 64 64
|
| 36 |
+
b1.kn.alpha f32 12099100 1 1
|
| 37 |
+
b1.kn.gamma f32 12099104 64 64
|
| 38 |
+
b1.kn.beta f32 12099360 64 64
|
| 39 |
+
b1.out.q i8 12099616 589824 768 768
|
| 40 |
+
b1.out.scale f32 12689440 768 768
|
| 41 |
+
b1.ff.alpha f32 12692512 1 1
|
| 42 |
+
b1.ff.gamma f32 12692516 768 768
|
| 43 |
+
b1.ff.beta f32 12695588 768 768
|
| 44 |
+
b1.glu.q i8 12698660 3538944 768 4608
|
| 45 |
+
b1.glu.scale f32 16237604 4608 4608
|
| 46 |
+
b1.glu.b f32 16256036 4608 4608
|
| 47 |
+
b1.proj.q i8 16274468 1769472 2304 768
|
| 48 |
+
b1.proj.scale f32 18043940 768 768
|
| 49 |
+
b1.proj.b f32 18047012 768 768
|
| 50 |
+
b2.pre.alpha f32 18050084 1 1
|
| 51 |
+
b2.pre.gamma f32 18050088 768 768
|
| 52 |
+
b2.pre.beta f32 18053160 768 768
|
| 53 |
+
b2.qkv.q i8 18056232 2949120 768 3840
|
| 54 |
+
b2.qkv.scale f32 21005352 3840 3840
|
| 55 |
+
b2.qn.alpha f32 21020712 1 1
|
| 56 |
+
b2.qn.gamma f32 21020716 64 64
|
| 57 |
+
b2.qn.beta f32 21020972 64 64
|
| 58 |
+
b2.kn.alpha f32 21021228 1 1
|
| 59 |
+
b2.kn.gamma f32 21021232 64 64
|
| 60 |
+
b2.kn.beta f32 21021488 64 64
|
| 61 |
+
b2.out.q i8 21021744 589824 768 768
|
| 62 |
+
b2.out.scale f32 21611568 768 768
|
| 63 |
+
b2.ff.alpha f32 21614640 1 1
|
| 64 |
+
b2.ff.gamma f32 21614644 768 768
|
| 65 |
+
b2.ff.beta f32 21617716 768 768
|
| 66 |
+
b2.glu.q i8 21620788 3538944 768 4608
|
| 67 |
+
b2.glu.scale f32 25159732 4608 4608
|
| 68 |
+
b2.glu.b f32 25178164 4608 4608
|
| 69 |
+
b2.proj.q i8 25196596 1769472 2304 768
|
| 70 |
+
b2.proj.scale f32 26966068 768 768
|
| 71 |
+
b2.proj.b f32 26969140 768 768
|
| 72 |
+
b3.pre.alpha f32 26972212 1 1
|
| 73 |
+
b3.pre.gamma f32 26972216 768 768
|
| 74 |
+
b3.pre.beta f32 26975288 768 768
|
| 75 |
+
b3.qkv.q i8 26978360 2949120 768 3840
|
| 76 |
+
b3.qkv.scale f32 29927480 3840 3840
|
| 77 |
+
b3.qn.alpha f32 29942840 1 1
|
| 78 |
+
b3.qn.gamma f32 29942844 64 64
|
| 79 |
+
b3.qn.beta f32 29943100 64 64
|
| 80 |
+
b3.kn.alpha f32 29943356 1 1
|
| 81 |
+
b3.kn.gamma f32 29943360 64 64
|
| 82 |
+
b3.kn.beta f32 29943616 64 64
|
| 83 |
+
b3.out.q i8 29943872 589824 768 768
|
| 84 |
+
b3.out.scale f32 30533696 768 768
|
| 85 |
+
b3.ff.alpha f32 30536768 1 1
|
| 86 |
+
b3.ff.gamma f32 30536772 768 768
|
| 87 |
+
b3.ff.beta f32 30539844 768 768
|
| 88 |
+
b3.glu.q i8 30542916 3538944 768 4608
|
| 89 |
+
b3.glu.scale f32 34081860 4608 4608
|
| 90 |
+
b3.glu.b f32 34100292 4608 4608
|
| 91 |
+
b3.proj.q i8 34118724 1769472 2304 768
|
| 92 |
+
b3.proj.scale f32 35888196 768 768
|
| 93 |
+
b3.proj.b f32 35891268 768 768
|
| 94 |
+
b4.pre.alpha f32 35894340 1 1
|
| 95 |
+
b4.pre.gamma f32 35894344 768 768
|
| 96 |
+
b4.pre.beta f32 35897416 768 768
|
| 97 |
+
b4.qkv.q i8 35900488 2949120 768 3840
|
| 98 |
+
b4.qkv.scale f32 38849608 3840 3840
|
| 99 |
+
b4.qn.alpha f32 38864968 1 1
|
| 100 |
+
b4.qn.gamma f32 38864972 64 64
|
| 101 |
+
b4.qn.beta f32 38865228 64 64
|
| 102 |
+
b4.kn.alpha f32 38865484 1 1
|
| 103 |
+
b4.kn.gamma f32 38865488 64 64
|
| 104 |
+
b4.kn.beta f32 38865744 64 64
|
| 105 |
+
b4.out.q i8 38866000 589824 768 768
|
| 106 |
+
b4.out.scale f32 39455824 768 768
|
| 107 |
+
b4.ff.alpha f32 39458896 1 1
|
| 108 |
+
b4.ff.gamma f32 39458900 768 768
|
| 109 |
+
b4.ff.beta f32 39461972 768 768
|
| 110 |
+
b4.glu.q i8 39465044 3538944 768 4608
|
| 111 |
+
b4.glu.scale f32 43003988 4608 4608
|
| 112 |
+
b4.glu.b f32 43022420 4608 4608
|
| 113 |
+
b4.proj.q i8 43040852 1769472 2304 768
|
| 114 |
+
b4.proj.scale f32 44810324 768 768
|
| 115 |
+
b4.proj.b f32 44813396 768 768
|
| 116 |
+
b5.pre.alpha f32 44816468 1 1
|
| 117 |
+
b5.pre.gamma f32 44816472 768 768
|
| 118 |
+
b5.pre.beta f32 44819544 768 768
|
| 119 |
+
b5.qkv.q i8 44822616 2949120 768 3840
|
| 120 |
+
b5.qkv.scale f32 47771736 3840 3840
|
| 121 |
+
b5.qn.alpha f32 47787096 1 1
|
| 122 |
+
b5.qn.gamma f32 47787100 64 64
|
| 123 |
+
b5.qn.beta f32 47787356 64 64
|
| 124 |
+
b5.kn.alpha f32 47787612 1 1
|
| 125 |
+
b5.kn.gamma f32 47787616 64 64
|
| 126 |
+
b5.kn.beta f32 47787872 64 64
|
| 127 |
+
b5.out.q i8 47788128 589824 768 768
|
| 128 |
+
b5.out.scale f32 48377952 768 768
|
| 129 |
+
b5.ff.alpha f32 48381024 1 1
|
| 130 |
+
b5.ff.gamma f32 48381028 768 768
|
| 131 |
+
b5.ff.beta f32 48384100 768 768
|
| 132 |
+
b5.glu.q i8 48387172 3538944 768 4608
|
| 133 |
+
b5.glu.scale f32 51926116 4608 4608
|
| 134 |
+
b5.glu.b f32 51944548 4608 4608
|
| 135 |
+
b5.proj.q i8 51962980 1769472 2304 768
|
| 136 |
+
b5.proj.scale f32 53732452 768 768
|
| 137 |
+
b5.proj.b f32 53735524 768 768
|
| 138 |
+
conv.q i8 53738596 1179648 2304 512
|
| 139 |
+
conv.scale f32 54918244 512 512
|
| 140 |
+
conv.b f32 54920292 512 512
|
cpu-amx/same_s_encoder.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7ad21c5359e2fefe971dbb121d2c374557ad8cdca2a36d1d1f9b8db677ff9558
|
| 3 |
+
size 29336432
|
cpu-amx/same_s_encoder_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e0983f9041f396d8f1bfe4ff397bac5369c82b9cc046b3d99dbc7e9fa6091e80
|
| 3 |
+
size 107566180
|
cpu-amx/same_s_encoder_weights_manifest.txt
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
map_in.w bf16 0 393216 512 768
|
| 2 |
+
map_in.b f32 786432 768 768
|
| 3 |
+
new_tokens f32 789504 768 768
|
| 4 |
+
b0.pre.alpha f32 792576 1 1
|
| 5 |
+
b0.pre.gamma f32 792580 768 768
|
| 6 |
+
b0.pre.beta f32 795652 768 768
|
| 7 |
+
b0.qkv.w bf16 798724 2949120 768 3840
|
| 8 |
+
b0.qn.alpha f32 6696964 1 1
|
| 9 |
+
b0.qn.gamma f32 6696968 64 64
|
| 10 |
+
b0.qn.beta f32 6697224 64 64
|
| 11 |
+
b0.kn.alpha f32 6697480 1 1
|
| 12 |
+
b0.kn.gamma f32 6697484 64 64
|
| 13 |
+
b0.kn.beta f32 6697740 64 64
|
| 14 |
+
b0.out.w bf16 6697996 589824 768 768
|
| 15 |
+
b0.ff.alpha f32 7877644 1 1
|
| 16 |
+
b0.ff.gamma f32 7877648 768 768
|
| 17 |
+
b0.ff.beta f32 7880720 768 768
|
| 18 |
+
b0.glu.w bf16 7883792 3538944 768 4608
|
| 19 |
+
b0.glu.b f32 14961680 4608 4608
|
| 20 |
+
b0.proj.w bf16 14980112 1769472 2304 768
|
| 21 |
+
b0.proj.b f32 18519056 768 768
|
| 22 |
+
b1.pre.alpha f32 18522128 1 1
|
| 23 |
+
b1.pre.gamma f32 18522132 768 768
|
| 24 |
+
b1.pre.beta f32 18525204 768 768
|
| 25 |
+
b1.qkv.w bf16 18528276 2949120 768 3840
|
| 26 |
+
b1.qn.alpha f32 24426516 1 1
|
| 27 |
+
b1.qn.gamma f32 24426520 64 64
|
| 28 |
+
b1.qn.beta f32 24426776 64 64
|
| 29 |
+
b1.kn.alpha f32 24427032 1 1
|
| 30 |
+
b1.kn.gamma f32 24427036 64 64
|
| 31 |
+
b1.kn.beta f32 24427292 64 64
|
| 32 |
+
b1.out.w bf16 24427548 589824 768 768
|
| 33 |
+
b1.ff.alpha f32 25607196 1 1
|
| 34 |
+
b1.ff.gamma f32 25607200 768 768
|
| 35 |
+
b1.ff.beta f32 25610272 768 768
|
| 36 |
+
b1.glu.w bf16 25613344 3538944 768 4608
|
| 37 |
+
b1.glu.b f32 32691232 4608 4608
|
| 38 |
+
b1.proj.w bf16 32709664 1769472 2304 768
|
| 39 |
+
b1.proj.b f32 36248608 768 768
|
| 40 |
+
b2.pre.alpha f32 36251680 1 1
|
| 41 |
+
b2.pre.gamma f32 36251684 768 768
|
| 42 |
+
b2.pre.beta f32 36254756 768 768
|
| 43 |
+
b2.qkv.w bf16 36257828 2949120 768 3840
|
| 44 |
+
b2.qn.alpha f32 42156068 1 1
|
| 45 |
+
b2.qn.gamma f32 42156072 64 64
|
| 46 |
+
b2.qn.beta f32 42156328 64 64
|
| 47 |
+
b2.kn.alpha f32 42156584 1 1
|
| 48 |
+
b2.kn.gamma f32 42156588 64 64
|
| 49 |
+
b2.kn.beta f32 42156844 64 64
|
| 50 |
+
b2.out.w bf16 42157100 589824 768 768
|
| 51 |
+
b2.ff.alpha f32 43336748 1 1
|
| 52 |
+
b2.ff.gamma f32 43336752 768 768
|
| 53 |
+
b2.ff.beta f32 43339824 768 768
|
| 54 |
+
b2.glu.w bf16 43342896 3538944 768 4608
|
| 55 |
+
b2.glu.b f32 50420784 4608 4608
|
| 56 |
+
b2.proj.w bf16 50439216 1769472 2304 768
|
| 57 |
+
b2.proj.b f32 53978160 768 768
|
| 58 |
+
b3.pre.alpha f32 53981232 1 1
|
| 59 |
+
b3.pre.gamma f32 53981236 768 768
|
| 60 |
+
b3.pre.beta f32 53984308 768 768
|
| 61 |
+
b3.qkv.w bf16 53987380 2949120 768 3840
|
| 62 |
+
b3.qn.alpha f32 59885620 1 1
|
| 63 |
+
b3.qn.gamma f32 59885624 64 64
|
| 64 |
+
b3.qn.beta f32 59885880 64 64
|
| 65 |
+
b3.kn.alpha f32 59886136 1 1
|
| 66 |
+
b3.kn.gamma f32 59886140 64 64
|
| 67 |
+
b3.kn.beta f32 59886396 64 64
|
| 68 |
+
b3.out.w bf16 59886652 589824 768 768
|
| 69 |
+
b3.ff.alpha f32 61066300 1 1
|
| 70 |
+
b3.ff.gamma f32 61066304 768 768
|
| 71 |
+
b3.ff.beta f32 61069376 768 768
|
| 72 |
+
b3.glu.w bf16 61072448 3538944 768 4608
|
| 73 |
+
b3.glu.b f32 68150336 4608 4608
|
| 74 |
+
b3.proj.w bf16 68168768 1769472 2304 768
|
| 75 |
+
b3.proj.b f32 71707712 768 768
|
| 76 |
+
b4.pre.alpha f32 71710784 1 1
|
| 77 |
+
b4.pre.gamma f32 71710788 768 768
|
| 78 |
+
b4.pre.beta f32 71713860 768 768
|
| 79 |
+
b4.qkv.w bf16 71716932 2949120 768 3840
|
| 80 |
+
b4.qn.alpha f32 77615172 1 1
|
| 81 |
+
b4.qn.gamma f32 77615176 64 64
|
| 82 |
+
b4.qn.beta f32 77615432 64 64
|
| 83 |
+
b4.kn.alpha f32 77615688 1 1
|
| 84 |
+
b4.kn.gamma f32 77615692 64 64
|
| 85 |
+
b4.kn.beta f32 77615948 64 64
|
| 86 |
+
b4.out.w bf16 77616204 589824 768 768
|
| 87 |
+
b4.ff.alpha f32 78795852 1 1
|
| 88 |
+
b4.ff.gamma f32 78795856 768 768
|
| 89 |
+
b4.ff.beta f32 78798928 768 768
|
| 90 |
+
b4.glu.w bf16 78802000 3538944 768 4608
|
| 91 |
+
b4.glu.b f32 85879888 4608 4608
|
| 92 |
+
b4.proj.w bf16 85898320 1769472 2304 768
|
| 93 |
+
b4.proj.b f32 89437264 768 768
|
| 94 |
+
b5.pre.alpha f32 89440336 1 1
|
| 95 |
+
b5.pre.gamma f32 89440340 768 768
|
| 96 |
+
b5.pre.beta f32 89443412 768 768
|
| 97 |
+
b5.qkv.w bf16 89446484 2949120 768 3840
|
| 98 |
+
b5.qn.alpha f32 95344724 1 1
|
| 99 |
+
b5.qn.gamma f32 95344728 64 64
|
| 100 |
+
b5.qn.beta f32 95344984 64 64
|
| 101 |
+
b5.kn.alpha f32 95345240 1 1
|
| 102 |
+
b5.kn.gamma f32 95345244 64 64
|
| 103 |
+
b5.kn.beta f32 95345500 64 64
|
| 104 |
+
b5.out.w bf16 95345756 589824 768 768
|
| 105 |
+
b5.ff.alpha f32 96525404 1 1
|
| 106 |
+
b5.ff.gamma f32 96525408 768 768
|
| 107 |
+
b5.ff.beta f32 96528480 768 768
|
| 108 |
+
b5.glu.w bf16 96531552 3538944 768 4608
|
| 109 |
+
b5.glu.b f32 103609440 4608 4608
|
| 110 |
+
b5.proj.w bf16 103627872 1769472 2304 768
|
| 111 |
+
b5.proj.b f32 107166816 768 768
|
| 112 |
+
proj_out.w bf16 107169888 196608 768 256
|
| 113 |
+
proj_out.b f32 107563104 256 256
|
| 114 |
+
scaling_factor f32 107564128 256 256
|
| 115 |
+
sn_bias f32 107565152 256 256
|
| 116 |
+
running_std f32 107566176 1 1
|
cpu-amx/t5gemma.so
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:953d56365b4487e37c910fc8b294610f06e3e1c4c7046fba43adede54b5f9075
|
| 3 |
+
size 29321696
|
cpu-amx/t5gemma_weights.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f8302b5b785fd8c5774988f2c07558b33a3d8674f4a77ace7e60376cf5d2a41d
|
| 3 |
+
size 563235968
|
cpu-amx/t5gemma_weights_manifest.txt
ADDED
|
@@ -0,0 +1,135 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
embed bf16 0 196608000 256000 768
|
| 2 |
+
norm f32 393216000 768 768
|
| 3 |
+
rope_inv f32 393219072 32 32
|
| 4 |
+
L0.pre_a f32 393219200 768 768
|
| 5 |
+
L0.post_a f32 393222272 768 768
|
| 6 |
+
L0.pre_f f32 393225344 768 768
|
| 7 |
+
L0.post_f f32 393228416 768 768
|
| 8 |
+
L0.q bf16 393231488 589824 768 768
|
| 9 |
+
L0.k bf16 394411136 589824 768 768
|
| 10 |
+
L0.v bf16 395590784 589824 768 768
|
| 11 |
+
L0.o bf16 396770432 589824 768 768
|
| 12 |
+
L0.gate bf16 397950080 1572864 768 2048
|
| 13 |
+
L0.up bf16 401095808 1572864 768 2048
|
| 14 |
+
L0.down bf16 404241536 1572864 2048 768
|
| 15 |
+
L1.pre_a f32 407387264 768 768
|
| 16 |
+
L1.post_a f32 407390336 768 768
|
| 17 |
+
L1.pre_f f32 407393408 768 768
|
| 18 |
+
L1.post_f f32 407396480 768 768
|
| 19 |
+
L1.q bf16 407399552 589824 768 768
|
| 20 |
+
L1.k bf16 408579200 589824 768 768
|
| 21 |
+
L1.v bf16 409758848 589824 768 768
|
| 22 |
+
L1.o bf16 410938496 589824 768 768
|
| 23 |
+
L1.gate bf16 412118144 1572864 768 2048
|
| 24 |
+
L1.up bf16 415263872 1572864 768 2048
|
| 25 |
+
L1.down bf16 418409600 1572864 2048 768
|
| 26 |
+
L2.pre_a f32 421555328 768 768
|
| 27 |
+
L2.post_a f32 421558400 768 768
|
| 28 |
+
L2.pre_f f32 421561472 768 768
|
| 29 |
+
L2.post_f f32 421564544 768 768
|
| 30 |
+
L2.q bf16 421567616 589824 768 768
|
| 31 |
+
L2.k bf16 422747264 589824 768 768
|
| 32 |
+
L2.v bf16 423926912 589824 768 768
|
| 33 |
+
L2.o bf16 425106560 589824 768 768
|
| 34 |
+
L2.gate bf16 426286208 1572864 768 2048
|
| 35 |
+
L2.up bf16 429431936 1572864 768 2048
|
| 36 |
+
L2.down bf16 432577664 1572864 2048 768
|
| 37 |
+
L3.pre_a f32 435723392 768 768
|
| 38 |
+
L3.post_a f32 435726464 768 768
|
| 39 |
+
L3.pre_f f32 435729536 768 768
|
| 40 |
+
L3.post_f f32 435732608 768 768
|
| 41 |
+
L3.q bf16 435735680 589824 768 768
|
| 42 |
+
L3.k bf16 436915328 589824 768 768
|
| 43 |
+
L3.v bf16 438094976 589824 768 768
|
| 44 |
+
L3.o bf16 439274624 589824 768 768
|
| 45 |
+
L3.gate bf16 440454272 1572864 768 2048
|
| 46 |
+
L3.up bf16 443600000 1572864 768 2048
|
| 47 |
+
L3.down bf16 446745728 1572864 2048 768
|
| 48 |
+
L4.pre_a f32 449891456 768 768
|
| 49 |
+
L4.post_a f32 449894528 768 768
|
| 50 |
+
L4.pre_f f32 449897600 768 768
|
| 51 |
+
L4.post_f f32 449900672 768 768
|
| 52 |
+
L4.q bf16 449903744 589824 768 768
|
| 53 |
+
L4.k bf16 451083392 589824 768 768
|
| 54 |
+
L4.v bf16 452263040 589824 768 768
|
| 55 |
+
L4.o bf16 453442688 589824 768 768
|
| 56 |
+
L4.gate bf16 454622336 1572864 768 2048
|
| 57 |
+
L4.up bf16 457768064 1572864 768 2048
|
| 58 |
+
L4.down bf16 460913792 1572864 2048 768
|
| 59 |
+
L5.pre_a f32 464059520 768 768
|
| 60 |
+
L5.post_a f32 464062592 768 768
|
| 61 |
+
L5.pre_f f32 464065664 768 768
|
| 62 |
+
L5.post_f f32 464068736 768 768
|
| 63 |
+
L5.q bf16 464071808 589824 768 768
|
| 64 |
+
L5.k bf16 465251456 589824 768 768
|
| 65 |
+
L5.v bf16 466431104 589824 768 768
|
| 66 |
+
L5.o bf16 467610752 589824 768 768
|
| 67 |
+
L5.gate bf16 468790400 1572864 768 2048
|
| 68 |
+
L5.up bf16 471936128 1572864 768 2048
|
| 69 |
+
L5.down bf16 475081856 1572864 2048 768
|
| 70 |
+
L6.pre_a f32 478227584 768 768
|
| 71 |
+
L6.post_a f32 478230656 768 768
|
| 72 |
+
L6.pre_f f32 478233728 768 768
|
| 73 |
+
L6.post_f f32 478236800 768 768
|
| 74 |
+
L6.q bf16 478239872 589824 768 768
|
| 75 |
+
L6.k bf16 479419520 589824 768 768
|
| 76 |
+
L6.v bf16 480599168 589824 768 768
|
| 77 |
+
L6.o bf16 481778816 589824 768 768
|
| 78 |
+
L6.gate bf16 482958464 1572864 768 2048
|
| 79 |
+
L6.up bf16 486104192 1572864 768 2048
|
| 80 |
+
L6.down bf16 489249920 1572864 2048 768
|
| 81 |
+
L7.pre_a f32 492395648 768 768
|
| 82 |
+
L7.post_a f32 492398720 768 768
|
| 83 |
+
L7.pre_f f32 492401792 768 768
|
| 84 |
+
L7.post_f f32 492404864 768 768
|
| 85 |
+
L7.q bf16 492407936 589824 768 768
|
| 86 |
+
L7.k bf16 493587584 589824 768 768
|
| 87 |
+
L7.v bf16 494767232 589824 768 768
|
| 88 |
+
L7.o bf16 495946880 589824 768 768
|
| 89 |
+
L7.gate bf16 497126528 1572864 768 2048
|
| 90 |
+
L7.up bf16 500272256 1572864 768 2048
|
| 91 |
+
L7.down bf16 503417984 1572864 2048 768
|
| 92 |
+
L8.pre_a f32 506563712 768 768
|
| 93 |
+
L8.post_a f32 506566784 768 768
|
| 94 |
+
L8.pre_f f32 506569856 768 768
|
| 95 |
+
L8.post_f f32 506572928 768 768
|
| 96 |
+
L8.q bf16 506576000 589824 768 768
|
| 97 |
+
L8.k bf16 507755648 589824 768 768
|
| 98 |
+
L8.v bf16 508935296 589824 768 768
|
| 99 |
+
L8.o bf16 510114944 589824 768 768
|
| 100 |
+
L8.gate bf16 511294592 1572864 768 2048
|
| 101 |
+
L8.up bf16 514440320 1572864 768 2048
|
| 102 |
+
L8.down bf16 517586048 1572864 2048 768
|
| 103 |
+
L9.pre_a f32 520731776 768 768
|
| 104 |
+
L9.post_a f32 520734848 768 768
|
| 105 |
+
L9.pre_f f32 520737920 768 768
|
| 106 |
+
L9.post_f f32 520740992 768 768
|
| 107 |
+
L9.q bf16 520744064 589824 768 768
|
| 108 |
+
L9.k bf16 521923712 589824 768 768
|
| 109 |
+
L9.v bf16 523103360 589824 768 768
|
| 110 |
+
L9.o bf16 524283008 589824 768 768
|
| 111 |
+
L9.gate bf16 525462656 1572864 768 2048
|
| 112 |
+
L9.up bf16 528608384 1572864 768 2048
|
| 113 |
+
L9.down bf16 531754112 1572864 2048 768
|
| 114 |
+
L10.pre_a f32 534899840 768 768
|
| 115 |
+
L10.post_a f32 534902912 768 768
|
| 116 |
+
L10.pre_f f32 534905984 768 768
|
| 117 |
+
L10.post_f f32 534909056 768 768
|
| 118 |
+
L10.q bf16 534912128 589824 768 768
|
| 119 |
+
L10.k bf16 536091776 589824 768 768
|
| 120 |
+
L10.v bf16 537271424 589824 768 768
|
| 121 |
+
L10.o bf16 538451072 589824 768 768
|
| 122 |
+
L10.gate bf16 539630720 1572864 768 2048
|
| 123 |
+
L10.up bf16 542776448 1572864 768 2048
|
| 124 |
+
L10.down bf16 545922176 1572864 2048 768
|
| 125 |
+
L11.pre_a f32 549067904 768 768
|
| 126 |
+
L11.post_a f32 549070976 768 768
|
| 127 |
+
L11.pre_f f32 549074048 768 768
|
| 128 |
+
L11.post_f f32 549077120 768 768
|
| 129 |
+
L11.q bf16 549080192 589824 768 768
|
| 130 |
+
L11.k bf16 550259840 589824 768 768
|
| 131 |
+
L11.v bf16 551439488 589824 768 768
|
| 132 |
+
L11.o bf16 552619136 589824 768 768
|
| 133 |
+
L11.gate bf16 553798784 1572864 768 2048
|
| 134 |
+
L11.up bf16 556944512 1572864 768 2048
|
| 135 |
+
L11.down bf16 560090240 1572864 2048 768
|