Hugging Face's logo

Clocksmith
/
rdrr

Model card Files
xet
Community
Clocksmith commited on
Commit
ff91b97
·
verified ·
1 Parent(s): e28cbc4

Add EmbeddingGemma 300M RDRR shards

Browse files
Files changed (4) hide show
  1. .gitattributes +1 -0
  2. models/google-embeddinggemma-300m-wq4k-ef16/manifest.json +4320 -0
  3. models/google-embeddinggemma-300m-wq4k-ef16/tokenizer.json +3 -0
  4. models/google-embeddinggemma-300m-wq4k-ef16/tokenizer.model +3 -0
.gitattributes CHANGED
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  models/gemma-3-270m-it-wq4k-ef16/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  models/gemma-3-270m-it-wq4k-ef16/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ models/google-embeddinggemma-300m-wq4k-ef16/tokenizer.json filter=lfs diff=lfs merge=lfs -text
models/google-embeddinggemma-300m-wq4k-ef16/manifest.json ADDED
@@ -0,0 +1,4320 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": 1,
3
+ "modelId": "google-embeddinggemma-300m-wq4k-ef16",
4
+ "modelType": "embedding",
5
+ "quantization": "Q4_K_M",
6
+ "quantizationInfo": {
7
+ "weights": "q4k",
8
+ "embeddings": "f16",
9
+ "compute": "f32",
10
+ "layout": "row",
11
+ "variantTag": "wq4k-ef16"
12
+ },
13
+ "architecture": {
14
+ "numLayers": 24,
15
+ "hiddenSize": 768,
16
+ "intermediateSize": 1152,
17
+ "numAttentionHeads": 3,
18
+ "numKeyValueHeads": 1,
19
+ "headDim": 256,
20
+ "vocabSize": 262144,
21
+ "maxSeqLen": 2048,
22
+ "ropeTheta": 1000000
23
+ },
24
+ "moeConfig": null,
25
+ "inference": {
26
+ "schema": "doppler.execution/v0",
27
+ "presetId": "embeddinggemma",
28
+ "attention": {
29
+ "queryPreAttnScalar": 256,
30
+ "attnLogitSoftcapping": null,
31
+ "slidingWindow": 512,
32
+ "queryKeyNorm": true,
33
+ "causal": false,
34
+ "attentionBias": false
35
+ },
36
+ "normalization": {
37
+ "rmsNormEps": 0.000001,
38
+ "rmsNormWeightOffset": true,
39
+ "postAttentionNorm": true,
40
+ "preFeedforwardNorm": true,
41
+ "postFeedforwardNorm": true
42
+ },
43
+ "ffn": {
44
+ "activation": "gelu",
45
+ "gatedActivation": true,
46
+ "swigluLimit": null
47
+ },
48
+ "rope": {
49
+ "ropeTheta": 1000000,
50
+ "ropeLocalTheta": 10000,
51
+ "ropeScalingType": null,
52
+ "ropeScalingFactor": 1,
53
+ "yarnBetaFast": null,
54
+ "yarnBetaSlow": null,
55
+ "yarnOriginalMaxPos": null,
56
+ "ropeLocalScalingType": null,
57
+ "ropeLocalScalingFactor": 1,
58
+ "ropeLocalYarnBetaFast": null,
59
+ "ropeLocalYarnBetaSlow": null,
60
+ "ropeLocalYarnOriginalMaxPos": null
61
+ },
62
+ "output": {
63
+ "finalLogitSoftcapping": null,
64
+ "tieWordEmbeddings": true,
65
+ "scaleEmbeddings": true,
66
+ "embeddingTranspose": false,
67
+ "embeddingVocabSize": 262144
68
+ },
69
+ "layerPattern": {
70
+ "type": "every_n",
71
+ "globalPattern": null,
72
+ "period": 6,
73
+ "offset": 5
74
+ },
75
+ "chatTemplate": {
76
+ "type": null,
77
+ "enabled": false
78
+ },
79
+ "pipeline": null,
80
+ "sessionDefaults": {
81
+ "compute": {
82
+ "defaults": {
83
+ "activationDtype": "f32",
84
+ "mathDtype": "f32",
85
+ "accumDtype": "f32",
86
+ "outputDtype": "f32"
87
+ },
88
+ "kernelProfiles": [
89
+ {
90
+ "kernelRef": {
91
+ "id": "gather.main",
92
+ "version": "1.0.0",
93
+ "digest": "sha256:777991fb6e4b3b506e4493b47ee998afe541924ddd7c04e1eadf4cb7fd719ef8"
94
+ }
95
+ },
96
+ {
97
+ "kernelRef": {
98
+ "id": "rmsnorm.main",
99
+ "version": "1.0.0",
100
+ "digest": "sha256:c529986befb29a04b94d89744585923a7cef82baf4b2b0a243aa2431618622cc"
101
+ }
102
+ },
103
+ {
104
+ "kernelRef": {
105
+ "id": "matmul.f16w.f32a.main",
106
+ "version": "1.0.0",
107
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
108
+ }
109
+ },
110
+ {
111
+ "kernelRef": {
112
+ "id": "rope.main",
113
+ "version": "1.0.0",
114
+ "digest": "sha256:b639fe8a54508115c82c13c923bfea89f59c6e15a5bef66bfc34e12f0ab4e32f"
115
+ }
116
+ },
117
+ {
118
+ "kernelRef": {
119
+ "id": "attention.small.main",
120
+ "version": "1.0.0",
121
+ "digest": "sha256:6752ddd7ab53e6235c9b5b1a9515141c0d111df7fac9f4c0d7a38f9943490ed4"
122
+ }
123
+ },
124
+ {
125
+ "kernelRef": {
126
+ "id": "residual.main",
127
+ "version": "1.0.0",
128
+ "digest": "sha256:1fc456b14e2fb2bc9627107b4e51e7a2098f723b5ba6ab5542cd9455af99f423"
129
+ }
130
+ },
131
+ {
132
+ "kernelRef": {
133
+ "id": "gelu.main",
134
+ "version": "1.0.0",
135
+ "digest": "sha256:a9007ea08aaff98f9be08f1e0490a6bcf252883eac5513de876ab9ce918865e6"
136
+ }
137
+ },
138
+ {
139
+ "kernelRef": {
140
+ "id": "sample.sample.single.pass",
141
+ "version": "1.0.0",
142
+ "digest": "sha256:4412357e84113ee2f1bc0dc8bf89e314c2ab482c89c14ca016ea9949d16a9d0c"
143
+ }
144
+ }
145
+ ]
146
+ },
147
+ "kvcache": {
148
+ "kvDtype": "f32"
149
+ },
150
+ "decodeLoop": null
151
+ },
152
+ "execution": {
153
+ "steps": [
154
+ {
155
+ "id": "preLayer_both_0_embed",
156
+ "phase": "both",
157
+ "section": "preLayer",
158
+ "op": "embed",
159
+ "kernel": "gather.wgsl",
160
+ "entry": "main",
161
+ "weights": "embed_tokens",
162
+ "layers": "all",
163
+ "src": "state",
164
+ "dst": "state",
165
+ "kernelRef": {
166
+ "id": "gather.main",
167
+ "version": "1.0.0",
168
+ "digest": "sha256:777991fb6e4b3b506e4493b47ee998afe541924ddd7c04e1eadf4cb7fd719ef8"
169
+ }
170
+ },
171
+ {
172
+ "id": "layer_decode_1_input_norm",
173
+ "phase": "decode",
174
+ "section": "layer",
175
+ "op": "input_norm",
176
+ "kernel": "rmsnorm.wgsl",
177
+ "entry": "main",
178
+ "layers": "all",
179
+ "src": "state",
180
+ "dst": "state",
181
+ "kernelRef": {
182
+ "id": "rmsnorm.main",
183
+ "version": "1.0.0",
184
+ "digest": "sha256:c529986befb29a04b94d89744585923a7cef82baf4b2b0a243aa2431618622cc"
185
+ }
186
+ },
187
+ {
188
+ "id": "layer_decode_2_q_proj",
189
+ "phase": "decode",
190
+ "section": "layer",
191
+ "op": "q_proj",
192
+ "kernel": "matmul_f16w_f32a.wgsl",
193
+ "entry": "main",
194
+ "weights": "layer.{L}.self_attn.q_proj",
195
+ "layers": "all",
196
+ "src": "state",
197
+ "dst": "state",
198
+ "kernelRef": {
199
+ "id": "matmul.f16w.f32a.main",
200
+ "version": "1.0.0",
201
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
202
+ }
203
+ },
204
+ {
205
+ "id": "layer_decode_3_k_proj",
206
+ "phase": "decode",
207
+ "section": "layer",
208
+ "op": "k_proj",
209
+ "kernel": "matmul_f16w_f32a.wgsl",
210
+ "entry": "main",
211
+ "weights": "layer.{L}.self_attn.k_proj",
212
+ "layers": "all",
213
+ "src": "state",
214
+ "dst": "state",
215
+ "kernelRef": {
216
+ "id": "matmul.f16w.f32a.main",
217
+ "version": "1.0.0",
218
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
219
+ }
220
+ },
221
+ {
222
+ "id": "layer_decode_4_v_proj",
223
+ "phase": "decode",
224
+ "section": "layer",
225
+ "op": "v_proj",
226
+ "kernel": "matmul_f16w_f32a.wgsl",
227
+ "entry": "main",
228
+ "weights": "layer.{L}.self_attn.v_proj",
229
+ "layers": "all",
230
+ "src": "state",
231
+ "dst": "state",
232
+ "kernelRef": {
233
+ "id": "matmul.f16w.f32a.main",
234
+ "version": "1.0.0",
235
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
236
+ }
237
+ },
238
+ {
239
+ "id": "layer_decode_5_rope_q",
240
+ "phase": "decode",
241
+ "section": "layer",
242
+ "op": "rope_q",
243
+ "kernel": "rope.wgsl",
244
+ "entry": "main",
245
+ "layers": "all",
246
+ "src": "state",
247
+ "dst": "state",
248
+ "kernelRef": {
249
+ "id": "rope.main",
250
+ "version": "1.0.0",
251
+ "digest": "sha256:b639fe8a54508115c82c13c923bfea89f59c6e15a5bef66bfc34e12f0ab4e32f"
252
+ }
253
+ },
254
+ {
255
+ "id": "layer_decode_6_rope_k",
256
+ "phase": "decode",
257
+ "section": "layer",
258
+ "op": "rope_k",
259
+ "kernel": "rope.wgsl",
260
+ "entry": "main",
261
+ "layers": "all",
262
+ "src": "state",
263
+ "dst": "state",
264
+ "kernelRef": {
265
+ "id": "rope.main",
266
+ "version": "1.0.0",
267
+ "digest": "sha256:b639fe8a54508115c82c13c923bfea89f59c6e15a5bef66bfc34e12f0ab4e32f"
268
+ }
269
+ },
270
+ {
271
+ "id": "layer_decode_7_attention",
272
+ "phase": "decode",
273
+ "section": "layer",
274
+ "op": "attention",
275
+ "kernel": "attention_small.wgsl",
276
+ "entry": "main",
277
+ "layers": "all",
278
+ "src": "state",
279
+ "dst": "state",
280
+ "kernelRef": {
281
+ "id": "attention.small.main",
282
+ "version": "1.0.0",
283
+ "digest": "sha256:6752ddd7ab53e6235c9b5b1a9515141c0d111df7fac9f4c0d7a38f9943490ed4"
284
+ }
285
+ },
286
+ {
287
+ "id": "layer_decode_8_o_proj",
288
+ "phase": "decode",
289
+ "section": "layer",
290
+ "op": "o_proj",
291
+ "kernel": "matmul_f16w_f32a.wgsl",
292
+ "entry": "main",
293
+ "weights": "layer.{L}.self_attn.o_proj",
294
+ "layers": "all",
295
+ "src": "state",
296
+ "dst": "state",
297
+ "kernelRef": {
298
+ "id": "matmul.f16w.f32a.main",
299
+ "version": "1.0.0",
300
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
301
+ }
302
+ },
303
+ {
304
+ "id": "layer_decode_9_attn_residual",
305
+ "phase": "decode",
306
+ "section": "layer",
307
+ "op": "attn_residual",
308
+ "kernel": "residual.wgsl",
309
+ "entry": "main",
310
+ "layers": "all",
311
+ "src": "state",
312
+ "dst": "state",
313
+ "kernelRef": {
314
+ "id": "residual.main",
315
+ "version": "1.0.0",
316
+ "digest": "sha256:1fc456b14e2fb2bc9627107b4e51e7a2098f723b5ba6ab5542cd9455af99f423"
317
+ }
318
+ },
319
+ {
320
+ "id": "layer_decode_10_post_attn_norm",
321
+ "phase": "decode",
322
+ "section": "layer",
323
+ "op": "post_attn_norm",
324
+ "kernel": "rmsnorm.wgsl",
325
+ "entry": "main",
326
+ "layers": "all",
327
+ "src": "state",
328
+ "dst": "state",
329
+ "kernelRef": {
330
+ "id": "rmsnorm.main",
331
+ "version": "1.0.0",
332
+ "digest": "sha256:c529986befb29a04b94d89744585923a7cef82baf4b2b0a243aa2431618622cc"
333
+ }
334
+ },
335
+ {
336
+ "id": "layer_decode_11_gate_proj",
337
+ "phase": "decode",
338
+ "section": "layer",
339
+ "op": "gate_proj",
340
+ "kernel": "matmul_f16w_f32a.wgsl",
341
+ "entry": "main",
342
+ "weights": "layer.{L}.mlp.gate_proj",
343
+ "layers": "all",
344
+ "src": "state",
345
+ "dst": "state",
346
+ "kernelRef": {
347
+ "id": "matmul.f16w.f32a.main",
348
+ "version": "1.0.0",
349
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
350
+ }
351
+ },
352
+ {
353
+ "id": "layer_decode_12_up_proj",
354
+ "phase": "decode",
355
+ "section": "layer",
356
+ "op": "up_proj",
357
+ "kernel": "matmul_f16w_f32a.wgsl",
358
+ "entry": "main",
359
+ "weights": "layer.{L}.mlp.up_proj",
360
+ "layers": "all",
361
+ "src": "state",
362
+ "dst": "state",
363
+ "kernelRef": {
364
+ "id": "matmul.f16w.f32a.main",
365
+ "version": "1.0.0",
366
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
367
+ }
368
+ },
369
+ {
370
+ "id": "layer_decode_13_activation",
371
+ "phase": "decode",
372
+ "section": "layer",
373
+ "op": "activation",
374
+ "kernel": "gelu.wgsl",
375
+ "entry": "main",
376
+ "constants": {
377
+ "HAS_GATE": true
378
+ },
379
+ "layers": "all",
380
+ "src": "state",
381
+ "dst": "state",
382
+ "kernelRef": {
383
+ "id": "gelu.main",
384
+ "version": "1.0.0",
385
+ "digest": "sha256:a9007ea08aaff98f9be08f1e0490a6bcf252883eac5513de876ab9ce918865e6"
386
+ }
387
+ },
388
+ {
389
+ "id": "layer_decode_14_down_proj",
390
+ "phase": "decode",
391
+ "section": "layer",
392
+ "op": "down_proj",
393
+ "kernel": "matmul_f16w_f32a.wgsl",
394
+ "entry": "main",
395
+ "weights": "layer.{L}.mlp.down_proj",
396
+ "layers": "all",
397
+ "src": "state",
398
+ "dst": "state",
399
+ "kernelRef": {
400
+ "id": "matmul.f16w.f32a.main",
401
+ "version": "1.0.0",
402
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
403
+ }
404
+ },
405
+ {
406
+ "id": "layer_decode_15_ffn_residual",
407
+ "phase": "decode",
408
+ "section": "layer",
409
+ "op": "ffn_residual",
410
+ "kernel": "residual.wgsl",
411
+ "entry": "main",
412
+ "layers": "all",
413
+ "src": "state",
414
+ "dst": "state",
415
+ "kernelRef": {
416
+ "id": "residual.main",
417
+ "version": "1.0.0",
418
+ "digest": "sha256:1fc456b14e2fb2bc9627107b4e51e7a2098f723b5ba6ab5542cd9455af99f423"
419
+ }
420
+ },
421
+ {
422
+ "id": "layer_prefill_16_input_norm",
423
+ "phase": "prefill",
424
+ "section": "layer",
425
+ "op": "input_norm",
426
+ "kernel": "rmsnorm.wgsl",
427
+ "entry": "main",
428
+ "layers": "all",
429
+ "src": "state",
430
+ "dst": "state",
431
+ "kernelRef": {
432
+ "id": "rmsnorm.main",
433
+ "version": "1.0.0",
434
+ "digest": "sha256:c529986befb29a04b94d89744585923a7cef82baf4b2b0a243aa2431618622cc"
435
+ }
436
+ },
437
+ {
438
+ "id": "layer_prefill_17_q_proj",
439
+ "phase": "prefill",
440
+ "section": "layer",
441
+ "op": "q_proj",
442
+ "kernel": "matmul_f16w_f32a.wgsl",
443
+ "entry": "main",
444
+ "weights": "layer.{L}.self_attn.q_proj",
445
+ "layers": "all",
446
+ "src": "state",
447
+ "dst": "state",
448
+ "kernelRef": {
449
+ "id": "matmul.f16w.f32a.main",
450
+ "version": "1.0.0",
451
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
452
+ }
453
+ },
454
+ {
455
+ "id": "layer_prefill_18_k_proj",
456
+ "phase": "prefill",
457
+ "section": "layer",
458
+ "op": "k_proj",
459
+ "kernel": "matmul_f16w_f32a.wgsl",
460
+ "entry": "main",
461
+ "weights": "layer.{L}.self_attn.k_proj",
462
+ "layers": "all",
463
+ "src": "state",
464
+ "dst": "state",
465
+ "kernelRef": {
466
+ "id": "matmul.f16w.f32a.main",
467
+ "version": "1.0.0",
468
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
469
+ }
470
+ },
471
+ {
472
+ "id": "layer_prefill_19_v_proj",
473
+ "phase": "prefill",
474
+ "section": "layer",
475
+ "op": "v_proj",
476
+ "kernel": "matmul_f16w_f32a.wgsl",
477
+ "entry": "main",
478
+ "weights": "layer.{L}.self_attn.v_proj",
479
+ "layers": "all",
480
+ "src": "state",
481
+ "dst": "state",
482
+ "kernelRef": {
483
+ "id": "matmul.f16w.f32a.main",
484
+ "version": "1.0.0",
485
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
486
+ }
487
+ },
488
+ {
489
+ "id": "layer_prefill_20_rope_q",
490
+ "phase": "prefill",
491
+ "section": "layer",
492
+ "op": "rope_q",
493
+ "kernel": "rope.wgsl",
494
+ "entry": "main",
495
+ "layers": "all",
496
+ "src": "state",
497
+ "dst": "state",
498
+ "kernelRef": {
499
+ "id": "rope.main",
500
+ "version": "1.0.0",
501
+ "digest": "sha256:b639fe8a54508115c82c13c923bfea89f59c6e15a5bef66bfc34e12f0ab4e32f"
502
+ }
503
+ },
504
+ {
505
+ "id": "layer_prefill_21_rope_k",
506
+ "phase": "prefill",
507
+ "section": "layer",
508
+ "op": "rope_k",
509
+ "kernel": "rope.wgsl",
510
+ "entry": "main",
511
+ "layers": "all",
512
+ "src": "state",
513
+ "dst": "state",
514
+ "kernelRef": {
515
+ "id": "rope.main",
516
+ "version": "1.0.0",
517
+ "digest": "sha256:b639fe8a54508115c82c13c923bfea89f59c6e15a5bef66bfc34e12f0ab4e32f"
518
+ }
519
+ },
520
+ {
521
+ "id": "layer_prefill_22_attention",
522
+ "phase": "prefill",
523
+ "section": "layer",
524
+ "op": "attention",
525
+ "kernel": "attention_small.wgsl",
526
+ "entry": "main",
527
+ "layers": "all",
528
+ "src": "state",
529
+ "dst": "state",
530
+ "kernelRef": {
531
+ "id": "attention.small.main",
532
+ "version": "1.0.0",
533
+ "digest": "sha256:6752ddd7ab53e6235c9b5b1a9515141c0d111df7fac9f4c0d7a38f9943490ed4"
534
+ }
535
+ },
536
+ {
537
+ "id": "layer_prefill_23_o_proj",
538
+ "phase": "prefill",
539
+ "section": "layer",
540
+ "op": "o_proj",
541
+ "kernel": "matmul_f16w_f32a.wgsl",
542
+ "entry": "main",
543
+ "weights": "layer.{L}.self_attn.o_proj",
544
+ "layers": "all",
545
+ "src": "state",
546
+ "dst": "state",
547
+ "kernelRef": {
548
+ "id": "matmul.f16w.f32a.main",
549
+ "version": "1.0.0",
550
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
551
+ }
552
+ },
553
+ {
554
+ "id": "layer_prefill_24_attn_residual",
555
+ "phase": "prefill",
556
+ "section": "layer",
557
+ "op": "attn_residual",
558
+ "kernel": "residual.wgsl",
559
+ "entry": "main",
560
+ "layers": "all",
561
+ "src": "state",
562
+ "dst": "state",
563
+ "kernelRef": {
564
+ "id": "residual.main",
565
+ "version": "1.0.0",
566
+ "digest": "sha256:1fc456b14e2fb2bc9627107b4e51e7a2098f723b5ba6ab5542cd9455af99f423"
567
+ }
568
+ },
569
+ {
570
+ "id": "layer_prefill_25_post_attn_norm",
571
+ "phase": "prefill",
572
+ "section": "layer",
573
+ "op": "post_attn_norm",
574
+ "kernel": "rmsnorm.wgsl",
575
+ "entry": "main",
576
+ "layers": "all",
577
+ "src": "state",
578
+ "dst": "state",
579
+ "kernelRef": {
580
+ "id": "rmsnorm.main",
581
+ "version": "1.0.0",
582
+ "digest": "sha256:c529986befb29a04b94d89744585923a7cef82baf4b2b0a243aa2431618622cc"
583
+ }
584
+ },
585
+ {
586
+ "id": "layer_prefill_26_gate_proj",
587
+ "phase": "prefill",
588
+ "section": "layer",
589
+ "op": "gate_proj",
590
+ "kernel": "matmul_f16w_f32a.wgsl",
591
+ "entry": "main",
592
+ "weights": "layer.{L}.mlp.gate_proj",
593
+ "layers": "all",
594
+ "src": "state",
595
+ "dst": "state",
596
+ "kernelRef": {
597
+ "id": "matmul.f16w.f32a.main",
598
+ "version": "1.0.0",
599
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
600
+ }
601
+ },
602
+ {
603
+ "id": "layer_prefill_27_up_proj",
604
+ "phase": "prefill",
605
+ "section": "layer",
606
+ "op": "up_proj",
607
+ "kernel": "matmul_f16w_f32a.wgsl",
608
+ "entry": "main",
609
+ "weights": "layer.{L}.mlp.up_proj",
610
+ "layers": "all",
611
+ "src": "state",
612
+ "dst": "state",
613
+ "kernelRef": {
614
+ "id": "matmul.f16w.f32a.main",
615
+ "version": "1.0.0",
616
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
617
+ }
618
+ },
619
+ {
620
+ "id": "layer_prefill_28_activation",
621
+ "phase": "prefill",
622
+ "section": "layer",
623
+ "op": "activation",
624
+ "kernel": "gelu.wgsl",
625
+ "entry": "main",
626
+ "constants": {
627
+ "HAS_GATE": true
628
+ },
629
+ "layers": "all",
630
+ "src": "state",
631
+ "dst": "state",
632
+ "kernelRef": {
633
+ "id": "gelu.main",
634
+ "version": "1.0.0",
635
+ "digest": "sha256:a9007ea08aaff98f9be08f1e0490a6bcf252883eac5513de876ab9ce918865e6"
636
+ }
637
+ },
638
+ {
639
+ "id": "layer_prefill_29_down_proj",
640
+ "phase": "prefill",
641
+ "section": "layer",
642
+ "op": "down_proj",
643
+ "kernel": "matmul_f16w_f32a.wgsl",
644
+ "entry": "main",
645
+ "weights": "layer.{L}.mlp.down_proj",
646
+ "layers": "all",
647
+ "src": "state",
648
+ "dst": "state",
649
+ "kernelRef": {
650
+ "id": "matmul.f16w.f32a.main",
651
+ "version": "1.0.0",
652
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
653
+ }
654
+ },
655
+ {
656
+ "id": "layer_prefill_30_ffn_residual",
657
+ "phase": "prefill",
658
+ "section": "layer",
659
+ "op": "ffn_residual",
660
+ "kernel": "residual.wgsl",
661
+ "entry": "main",
662
+ "layers": "all",
663
+ "src": "state",
664
+ "dst": "state",
665
+ "kernelRef": {
666
+ "id": "residual.main",
667
+ "version": "1.0.0",
668
+ "digest": "sha256:1fc456b14e2fb2bc9627107b4e51e7a2098f723b5ba6ab5542cd9455af99f423"
669
+ }
670
+ },
671
+ {
672
+ "id": "postLayer_both_31_final_norm",
673
+ "phase": "both",
674
+ "section": "postLayer",
675
+ "op": "final_norm",
676
+ "kernel": "rmsnorm.wgsl",
677
+ "entry": "main",
678
+ "layers": "all",
679
+ "src": "state",
680
+ "dst": "state",
681
+ "kernelRef": {
682
+ "id": "rmsnorm.main",
683
+ "version": "1.0.0",
684
+ "digest": "sha256:c529986befb29a04b94d89744585923a7cef82baf4b2b0a243aa2431618622cc"
685
+ }
686
+ },
687
+ {
688
+ "id": "postLayer_both_32_lm_head",
689
+ "phase": "both",
690
+ "section": "postLayer",
691
+ "op": "lm_head",
692
+ "kernel": "matmul_f16w_f32a.wgsl",
693
+ "entry": "main",
694
+ "weights": "lm_head",
695
+ "layers": "all",
696
+ "src": "state",
697
+ "dst": "state",
698
+ "kernelRef": {
699
+ "id": "matmul.f16w.f32a.main",
700
+ "version": "1.0.0",
701
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
702
+ }
703
+ },
704
+ {
705
+ "id": "postLayer_both_33_lm_head_prefill",
706
+ "phase": "both",
707
+ "section": "postLayer",
708
+ "op": "lm_head_prefill",
709
+ "kernel": "matmul_f16w_f32a.wgsl",
710
+ "entry": "main",
711
+ "weights": "lm_head",
712
+ "layers": "all",
713
+ "src": "state",
714
+ "dst": "state",
715
+ "kernelRef": {
716
+ "id": "matmul.f16w.f32a.main",
717
+ "version": "1.0.0",
718
+ "digest": "sha256:027a8f1cd9713cbe0b0ada160bd175e0542bb90896ad85441b023522d9a1befc"
719
+ }
720
+ },
721
+ {
722
+ "id": "sampling_decode_34_sample",
723
+ "phase": "decode",
724
+ "section": "sampling",
725
+ "op": "sample",
726
+ "kernel": "sample.wgsl",
727
+ "entry": "sample_single_pass",
728
+ "layers": "all",
729
+ "src": "state",
730
+ "dst": "state",
731
+ "kernelRef": {
732
+ "id": "sample.sample.single.pass",
733
+ "version": "1.0.0",
734
+ "digest": "sha256:4412357e84113ee2f1bc0dc8bf89e314c2ab482c89c14ca016ea9949d16a9d0c"
735
+ }
736
+ }
737
+ ],
738
+ "policies": {
739
+ "precisionPrecedence": "step_then_kernel_profile_then_session_default",
740
+ "unsupportedPrecision": "error",
741
+ "dtypeTransition": "require_cast_step",
742
+ "unresolvedKernel": "error"
743
+ }
744
+ },
745
+ "defaultKernelPath": "embeddinggemma-q4k-dequant-f32a"
746
+ },
747
+ "shards": [
748
+ {
749
+ "index": 0,
750
+ "filename": "shard_00000.bin",
751
+ "size": 67108864,
752
+ "hash": "8ef63284d30fdd9a5c4d8801cc55fa617f6816e005aa1a8bb9152fe7a0bdf8db",
753
+ "offset": 0
754
+ },
755
+ {
756
+ "index": 1,
757
+ "filename": "shard_00001.bin",
758
+ "size": 67108864,
759
+ "hash": "35bf4acc82785e6885825b13ba1f50634534377b984257f0bfb7898bf39bdf47",
760
+ "offset": 67108864
761
+ },
762
+ {
763
+ "index": 2,
764
+ "filename": "shard_00002.bin",
765
+ "size": 67108864,
766
+ "hash": "f14be0098fe1ca7a69bee2d7470b87179cfec43339250e3bc988df76bbe1c742",
767
+ "offset": 134217728
768
+ },
769
+ {
770
+ "index": 3,
771
+ "filename": "shard_00003.bin",
772
+ "size": 67108864,
773
+ "hash": "c7624d5a1c060db21471d89731dc826c0ee3e11269e1dcd189c5fa25800f95fc",
774
+ "offset": 201326592
775
+ },
776
+ {
777
+ "index": 4,
778
+ "filename": "shard_00004.bin",
779
+ "size": 67108864,
780
+ "hash": "7d6652c3cb37a944d21ca68d275fa4e0c30eac800544bb5dec5bcf9231e75ac6",
781
+ "offset": 268435456
782
+ },
783
+ {
784
+ "index": 5,
785
+ "filename": "shard_00005.bin",
786
+ "size": 67108864,
787
+ "hash": "66926b5073c008f26cf0985ada86d5bde45af963e2b9a72be06c52d551dd9a6f",
788
+ "offset": 335544320
789
+ },
790
+ {
791
+ "index": 6,
792
+ "filename": "shard_00006.bin",
793
+ "size": 58739712,
794
+ "hash": "89c9576fb839a43cba6dded765f2bc46aabc9670babb5f9fce0bd0e8e73b1a40",
795
+ "offset": 402653184
796
+ }
797
+ ],
798
+ "tensors": {
799
+ "embed_tokens.weight": {
800
+ "spans": [
801
+ {
802
+ "shardIndex": 0,
803
+ "offset": 0,
804
+ "size": 67108864
805
+ },
806
+ {
807
+ "shardIndex": 1,
808
+ "offset": 0,
809
+ "size": 67108864
810
+ },
811
+ {
812
+ "shardIndex": 2,
813
+ "offset": 0,
814
+ "size": 67108864
815
+ },
816
+ {
817
+ "shardIndex": 3,
818
+ "offset": 0,
819
+ "size": 67108864
820
+ },
821
+ {
822
+ "shardIndex": 4,
823
+ "offset": 0,
824
+ "size": 67108864
825
+ },
826
+ {
827
+ "shardIndex": 5,
828
+ "offset": 0,
829
+ "size": 67108864
830
+ }
831
+ ],
832
+ "size": 402653184,
833
+ "shape": [
834
+ 262144,
835
+ 768
836
+ ],
837
+ "dtype": "F16",
838
+ "role": "embedding"
839
+ },
840
+ "layers.0.input_layernorm.weight": {
841
+ "shard": 6,
842
+ "offset": 0,
843
+ "size": 3072,
844
+ "shape": [
845
+ 768
846
+ ],
847
+ "dtype": "F32",
848
+ "role": "norm"
849
+ },
850
+ "layers.0.mlp.down_proj.weight": {
851
+ "shard": 6,
852
+ "offset": 3072,
853
+ "size": 552960,
854
+ "shape": [
855
+ 768,
856
+ 1152
857
+ ],
858
+ "dtype": "Q4_K_M",
859
+ "role": "matmul",
860
+ "layout": "row"
861
+ },
862
+ "layers.0.mlp.gate_proj.weight": {
863
+ "shard": 6,
864
+ "offset": 556032,
865
+ "size": 497664,
866
+ "shape": [
867
+ 1152,
868
+ 768
869
+ ],
870
+ "dtype": "Q4_K_M",
871
+ "role": "matmul",
872
+ "layout": "row"
873
+ },
874
+ "layers.0.mlp.up_proj.weight": {
875
+ "shard": 6,
876
+ "offset": 1053696,
877
+ "size": 497664,
878
+ "shape": [
879
+ 1152,
880
+ 768
881
+ ],
882
+ "dtype": "Q4_K_M",
883
+ "role": "matmul",
884
+ "layout": "row"
885
+ },
886
+ "layers.0.post_attention_layernorm.weight": {
887
+ "shard": 6,
888
+ "offset": 1551360,
889
+ "size": 3072,
890
+ "shape": [
891
+ 768
892
+ ],
893
+ "dtype": "F32",
894
+ "role": "norm"
895
+ },
896
+ "layers.0.post_feedforward_layernorm.weight": {
897
+ "shard": 6,
898
+ "offset": 1554432,
899
+ "size": 3072,
900
+ "shape": [
901
+ 768
902
+ ],
903
+ "dtype": "F32",
904
+ "role": "norm"
905
+ },
906
+ "layers.0.pre_feedforward_layernorm.weight": {
907
+ "shard": 6,
908
+ "offset": 1557504,
909
+ "size": 3072,
910
+ "shape": [
911
+ 768
912
+ ],
913
+ "dtype": "F32",
914
+ "role": "norm"
915
+ },
916
+ "layers.0.self_attn.k_norm.weight": {
917
+ "shard": 6,
918
+ "offset": 1560576,
919
+ "size": 1024,
920
+ "shape": [
921
+ 256
922
+ ],
923
+ "dtype": "F32",
924
+ "role": "norm"
925
+ },
926
+ "layers.0.self_attn.k_proj.weight": {
927
+ "shard": 6,
928
+ "offset": 1561600,
929
+ "size": 110592,
930
+ "shape": [
931
+ 256,
932
+ 768
933
+ ],
934
+ "dtype": "Q4_K_M",
935
+ "role": "matmul",
936
+ "layout": "row"
937
+ },
938
+ "layers.0.self_attn.o_proj.weight": {
939
+ "shard": 6,
940
+ "offset": 1672192,
941
+ "size": 331776,
942
+ "shape": [
943
+ 768,
944
+ 768
945
+ ],
946
+ "dtype": "Q4_K_M",
947
+ "role": "matmul",
948
+ "layout": "row"
949
+ },
950
+ "layers.0.self_attn.q_norm.weight": {
951
+ "shard": 6,
952
+ "offset": 2003968,
953
+ "size": 1024,
954
+ "shape": [
955
+ 256
956
+ ],
957
+ "dtype": "F32",
958
+ "role": "norm"
959
+ },
960
+ "layers.0.self_attn.q_proj.weight": {
961
+ "shard": 6,
962
+ "offset": 2004992,
963
+ "size": 331776,
964
+ "shape": [
965
+ 768,
966
+ 768
967
+ ],
968
+ "dtype": "Q4_K_M",
969
+ "role": "matmul",
970
+ "layout": "row"
971
+ },
972
+ "layers.0.self_attn.v_proj.weight": {
973
+ "shard": 6,
974
+ "offset": 2336768,
975
+ "size": 110592,
976
+ "shape": [
977
+ 256,
978
+ 768
979
+ ],
980
+ "dtype": "Q4_K_M",
981
+ "role": "matmul",
982
+ "layout": "row"
983
+ },
984
+ "layers.1.input_layernorm.weight": {
985
+ "shard": 6,
986
+ "offset": 2447360,
987
+ "size": 3072,
988
+ "shape": [
989
+ 768
990
+ ],
991
+ "dtype": "F32",
992
+ "role": "norm"
993
+ },
994
+ "layers.1.mlp.down_proj.weight": {
995
+ "shard": 6,
996
+ "offset": 2450432,
997
+ "size": 552960,
998
+ "shape": [
999
+ 768,
1000
+ 1152
1001
+ ],
1002
+ "dtype": "Q4_K_M",
1003
+ "role": "matmul",
1004
+ "layout": "row"
1005
+ },
1006
+ "layers.1.mlp.gate_proj.weight": {
1007
+ "shard": 6,
1008
+ "offset": 3003392,
1009
+ "size": 497664,
1010
+ "shape": [
1011
+ 1152,
1012
+ 768
1013
+ ],
1014
+ "dtype": "Q4_K_M",
1015
+ "role": "matmul",
1016
+ "layout": "row"
1017
+ },
1018
+ "layers.1.mlp.up_proj.weight": {
1019
+ "shard": 6,
1020
+ "offset": 3501056,
1021
+ "size": 497664,
1022
+ "shape": [
1023
+ 1152,
1024
+ 768
1025
+ ],
1026
+ "dtype": "Q4_K_M",
1027
+ "role": "matmul",
1028
+ "layout": "row"
1029
+ },
1030
+ "layers.1.post_attention_layernorm.weight": {
1031
+ "shard": 6,
1032
+ "offset": 3998720,
1033
+ "size": 3072,
1034
+ "shape": [
1035
+ 768
1036
+ ],
1037
+ "dtype": "F32",
1038
+ "role": "norm"
1039
+ },
1040
+ "layers.1.post_feedforward_layernorm.weight": {
1041
+ "shard": 6,
1042
+ "offset": 4001792,
1043
+ "size": 3072,
1044
+ "shape": [
1045
+ 768
1046
+ ],
1047
+ "dtype": "F32",
1048
+ "role": "norm"
1049
+ },
1050
+ "layers.1.pre_feedforward_layernorm.weight": {
1051
+ "shard": 6,
1052
+ "offset": 4004864,
1053
+ "size": 3072,
1054
+ "shape": [
1055
+ 768
1056
+ ],
1057
+ "dtype": "F32",
1058
+ "role": "norm"
1059
+ },
1060
+ "layers.1.self_attn.k_norm.weight": {
1061
+ "shard": 6,
1062
+ "offset": 4007936,
1063
+ "size": 1024,
1064
+ "shape": [
1065
+ 256
1066
+ ],
1067
+ "dtype": "F32",
1068
+ "role": "norm"
1069
+ },
1070
+ "layers.1.self_attn.k_proj.weight": {
1071
+ "shard": 6,
1072
+ "offset": 4008960,
1073
+ "size": 110592,
1074
+ "shape": [
1075
+ 256,
1076
+ 768
1077
+ ],
1078
+ "dtype": "Q4_K_M",
1079
+ "role": "matmul",
1080
+ "layout": "row"
1081
+ },
1082
+ "layers.1.self_attn.o_proj.weight": {
1083
+ "shard": 6,
1084
+ "offset": 4119552,
1085
+ "size": 331776,
1086
+ "shape": [
1087
+ 768,
1088
+ 768
1089
+ ],
1090
+ "dtype": "Q4_K_M",
1091
+ "role": "matmul",
1092
+ "layout": "row"
1093
+ },
1094
+ "layers.1.self_attn.q_norm.weight": {
1095
+ "shard": 6,
1096
+ "offset": 4451328,
1097
+ "size": 1024,
1098
+ "shape": [
1099
+ 256
1100
+ ],
1101
+ "dtype": "F32",
1102
+ "role": "norm"
1103
+ },
1104
+ "layers.1.self_attn.q_proj.weight": {
1105
+ "shard": 6,
1106
+ "offset": 4452352,
1107
+ "size": 331776,
1108
+ "shape": [
1109
+ 768,
1110
+ 768
1111
+ ],
1112
+ "dtype": "Q4_K_M",
1113
+ "role": "matmul",
1114
+ "layout": "row"
1115
+ },
1116
+ "layers.1.self_attn.v_proj.weight": {
1117
+ "shard": 6,
1118
+ "offset": 4784128,
1119
+ "size": 110592,
1120
+ "shape": [
1121
+ 256,
1122
+ 768
1123
+ ],
1124
+ "dtype": "Q4_K_M",
1125
+ "role": "matmul",
1126
+ "layout": "row"
1127
+ },
1128
+ "layers.10.input_layernorm.weight": {
1129
+ "shard": 6,
1130
+ "offset": 4894720,
1131
+ "size": 3072,
1132
+ "shape": [
1133
+ 768
1134
+ ],
1135
+ "dtype": "F32",
1136
+ "role": "norm"
1137
+ },
1138
+ "layers.10.mlp.down_proj.weight": {
1139
+ "shard": 6,
1140
+ "offset": 4897792,
1141
+ "size": 552960,
1142
+ "shape": [
1143
+ 768,
1144
+ 1152
1145
+ ],
1146
+ "dtype": "Q4_K_M",
1147
+ "role": "matmul",
1148
+ "layout": "row"
1149
+ },
1150
+ "layers.10.mlp.gate_proj.weight": {
1151
+ "shard": 6,
1152
+ "offset": 5450752,
1153
+ "size": 497664,
1154
+ "shape": [
1155
+ 1152,
1156
+ 768
1157
+ ],
1158
+ "dtype": "Q4_K_M",
1159
+ "role": "matmul",
1160
+ "layout": "row"
1161
+ },
1162
+ "layers.10.mlp.up_proj.weight": {
1163
+ "shard": 6,
1164
+ "offset": 5948416,
1165
+ "size": 497664,
1166
+ "shape": [
1167
+ 1152,
1168
+ 768
1169
+ ],
1170
+ "dtype": "Q4_K_M",
1171
+ "role": "matmul",
1172
+ "layout": "row"
1173
+ },
1174
+ "layers.10.post_attention_layernorm.weight": {
1175
+ "shard": 6,
1176
+ "offset": 6446080,
1177
+ "size": 3072,
1178
+ "shape": [
1179
+ 768
1180
+ ],
1181
+ "dtype": "F32",
1182
+ "role": "norm"
1183
+ },
1184
+ "layers.10.post_feedforward_layernorm.weight": {
1185
+ "shard": 6,
1186
+ "offset": 6449152,
1187
+ "size": 3072,
1188
+ "shape": [
1189
+ 768
1190
+ ],
1191
+ "dtype": "F32",
1192
+ "role": "norm"
1193
+ },
1194
+ "layers.10.pre_feedforward_layernorm.weight": {
1195
+ "shard": 6,
1196
+ "offset": 6452224,
1197
+ "size": 3072,
1198
+ "shape": [
1199
+ 768
1200
+ ],
1201
+ "dtype": "F32",
1202
+ "role": "norm"
1203
+ },
1204
+ "layers.10.self_attn.k_norm.weight": {
1205
+ "shard": 6,
1206
+ "offset": 6455296,
1207
+ "size": 1024,
1208
+ "shape": [
1209
+ 256
1210
+ ],
1211
+ "dtype": "F32",
1212
+ "role": "norm"
1213
+ },
1214
+ "layers.10.self_attn.k_proj.weight": {
1215
+ "shard": 6,
1216
+ "offset": 6456320,
1217
+ "size": 110592,
1218
+ "shape": [
1219
+ 256,
1220
+ 768
1221
+ ],
1222
+ "dtype": "Q4_K_M",
1223
+ "role": "matmul",
1224
+ "layout": "row"
1225
+ },
1226
+ "layers.10.self_attn.o_proj.weight": {
1227
+ "shard": 6,
1228
+ "offset": 6566912,
1229
+ "size": 331776,
1230
+ "shape": [
1231
+ 768,
1232
+ 768
1233
+ ],
1234
+ "dtype": "Q4_K_M",
1235
+ "role": "matmul",
1236
+ "layout": "row"
1237
+ },
1238
+ "layers.10.self_attn.q_norm.weight": {
1239
+ "shard": 6,
1240
+ "offset": 6898688,
1241
+ "size": 1024,
1242
+ "shape": [
1243
+ 256
1244
+ ],
1245
+ "dtype": "F32",
1246
+ "role": "norm"
1247
+ },
1248
+ "layers.10.self_attn.q_proj.weight": {
1249
+ "shard": 6,
1250
+ "offset": 6899712,
1251
+ "size": 331776,
1252
+ "shape": [
1253
+ 768,
1254
+ 768
1255
+ ],
1256
+ "dtype": "Q4_K_M",
1257
+ "role": "matmul",
1258
+ "layout": "row"
1259
+ },
1260
+ "layers.10.self_attn.v_proj.weight": {
1261
+ "shard": 6,
1262
+ "offset": 7231488,
1263
+ "size": 110592,
1264
+ "shape": [
1265
+ 256,
1266
+ 768
1267
+ ],
1268
+ "dtype": "Q4_K_M",
1269
+ "role": "matmul",
1270
+ "layout": "row"
1271
+ },
1272
+ "layers.11.input_layernorm.weight": {
1273
+ "shard": 6,
1274
+ "offset": 7342080,
1275
+ "size": 3072,
1276
+ "shape": [
1277
+ 768
1278
+ ],
1279
+ "dtype": "F32",
1280
+ "role": "norm"
1281
+ },
1282
+ "layers.11.mlp.down_proj.weight": {
1283
+ "shard": 6,
1284
+ "offset": 7345152,
1285
+ "size": 552960,
1286
+ "shape": [
1287
+ 768,
1288
+ 1152
1289
+ ],
1290
+ "dtype": "Q4_K_M",
1291
+ "role": "matmul",
1292
+ "layout": "row"
1293
+ },
1294
+ "layers.11.mlp.gate_proj.weight": {
1295
+ "shard": 6,
1296
+ "offset": 7898112,
1297
+ "size": 497664,
1298
+ "shape": [
1299
+ 1152,
1300
+ 768
1301
+ ],
1302
+ "dtype": "Q4_K_M",
1303
+ "role": "matmul",
1304
+ "layout": "row"
1305
+ },
1306
+ "layers.11.mlp.up_proj.weight": {
1307
+ "shard": 6,
1308
+ "offset": 8395776,
1309
+ "size": 497664,
1310
+ "shape": [
1311
+ 1152,
1312
+ 768
1313
+ ],
1314
+ "dtype": "Q4_K_M",
1315
+ "role": "matmul",
1316
+ "layout": "row"
1317
+ },
1318
+ "layers.11.post_attention_layernorm.weight": {
1319
+ "shard": 6,
1320
+ "offset": 8893440,
1321
+ "size": 3072,
1322
+ "shape": [
1323
+ 768
1324
+ ],
1325
+ "dtype": "F32",
1326
+ "role": "norm"
1327
+ },
1328
+ "layers.11.post_feedforward_layernorm.weight": {
1329
+ "shard": 6,
1330
+ "offset": 8896512,
1331
+ "size": 3072,
1332
+ "shape": [
1333
+ 768
1334
+ ],
1335
+ "dtype": "F32",
1336
+ "role": "norm"
1337
+ },
1338
+ "layers.11.pre_feedforward_layernorm.weight": {
1339
+ "shard": 6,
1340
+ "offset": 8899584,
1341
+ "size": 3072,
1342
+ "shape": [
1343
+ 768
1344
+ ],
1345
+ "dtype": "F32",
1346
+ "role": "norm"
1347
+ },
1348
+ "layers.11.self_attn.k_norm.weight": {
1349
+ "shard": 6,
1350
+ "offset": 8902656,
1351
+ "size": 1024,
1352
+ "shape": [
1353
+ 256
1354
+ ],
1355
+ "dtype": "F32",
1356
+ "role": "norm"
1357
+ },
1358
+ "layers.11.self_attn.k_proj.weight": {
1359
+ "shard": 6,
1360
+ "offset": 8903680,
1361
+ "size": 110592,
1362
+ "shape": [
1363
+ 256,
1364
+ 768
1365
+ ],
1366
+ "dtype": "Q4_K_M",
1367
+ "role": "matmul",
1368
+ "layout": "row"
1369
+ },
1370
+ "layers.11.self_attn.o_proj.weight": {
1371
+ "shard": 6,
1372
+ "offset": 9014272,
1373
+ "size": 331776,
1374
+ "shape": [
1375
+ 768,
1376
+ 768
1377
+ ],
1378
+ "dtype": "Q4_K_M",
1379
+ "role": "matmul",
1380
+ "layout": "row"
1381
+ },
1382
+ "layers.11.self_attn.q_norm.weight": {
1383
+ "shard": 6,
1384
+ "offset": 9346048,
1385
+ "size": 1024,
1386
+ "shape": [
1387
+ 256
1388
+ ],
1389
+ "dtype": "F32",
1390
+ "role": "norm"
1391
+ },
1392
+ "layers.11.self_attn.q_proj.weight": {
1393
+ "shard": 6,
1394
+ "offset": 9347072,
1395
+ "size": 331776,
1396
+ "shape": [
1397
+ 768,
1398
+ 768
1399
+ ],
1400
+ "dtype": "Q4_K_M",
1401
+ "role": "matmul",
1402
+ "layout": "row"
1403
+ },
1404
+ "layers.11.self_attn.v_proj.weight": {
1405
+ "shard": 6,
1406
+ "offset": 9678848,
1407
+ "size": 110592,
1408
+ "shape": [
1409
+ 256,
1410
+ 768
1411
+ ],
1412
+ "dtype": "Q4_K_M",
1413
+ "role": "matmul",
1414
+ "layout": "row"
1415
+ },
1416
+ "layers.12.input_layernorm.weight": {
1417
+ "shard": 6,
1418
+ "offset": 9789440,
1419
+ "size": 3072,
1420
+ "shape": [
1421
+ 768
1422
+ ],
1423
+ "dtype": "F32",
1424
+ "role": "norm"
1425
+ },
1426
+ "layers.12.mlp.down_proj.weight": {
1427
+ "shard": 6,
1428
+ "offset": 9792512,
1429
+ "size": 552960,
1430
+ "shape": [
1431
+ 768,
1432
+ 1152
1433
+ ],
1434
+ "dtype": "Q4_K_M",
1435
+ "role": "matmul",
1436
+ "layout": "row"
1437
+ },
1438
+ "layers.12.mlp.gate_proj.weight": {
1439
+ "shard": 6,
1440
+ "offset": 10345472,
1441
+ "size": 497664,
1442
+ "shape": [
1443
+ 1152,
1444
+ 768
1445
+ ],
1446
+ "dtype": "Q4_K_M",
1447
+ "role": "matmul",
1448
+ "layout": "row"
1449
+ },
1450
+ "layers.12.mlp.up_proj.weight": {
1451
+ "shard": 6,
1452
+ "offset": 10843136,
1453
+ "size": 497664,
1454
+ "shape": [
1455
+ 1152,
1456
+ 768
1457
+ ],
1458
+ "dtype": "Q4_K_M",
1459
+ "role": "matmul",
1460
+ "layout": "row"
1461
+ },
1462
+ "layers.12.post_attention_layernorm.weight": {
1463
+ "shard": 6,
1464
+ "offset": 11340800,
1465
+ "size": 3072,
1466
+ "shape": [
1467
+ 768
1468
+ ],
1469
+ "dtype": "F32",
1470
+ "role": "norm"
1471
+ },
1472
+ "layers.12.post_feedforward_layernorm.weight": {
1473
+ "shard": 6,
1474
+ "offset": 11343872,
1475
+ "size": 3072,
1476
+ "shape": [
1477
+ 768
1478
+ ],
1479
+ "dtype": "F32",
1480
+ "role": "norm"
1481
+ },
1482
+ "layers.12.pre_feedforward_layernorm.weight": {
1483
+ "shard": 6,
1484
+ "offset": 11346944,
1485
+ "size": 3072,
1486
+ "shape": [
1487
+ 768
1488
+ ],
1489
+ "dtype": "F32",
1490
+ "role": "norm"
1491
+ },
1492
+ "layers.12.self_attn.k_norm.weight": {
1493
+ "shard": 6,
1494
+ "offset": 11350016,
1495
+ "size": 1024,
1496
+ "shape": [
1497
+ 256
1498
+ ],
1499
+ "dtype": "F32",
1500
+ "role": "norm"
1501
+ },
1502
+ "layers.12.self_attn.k_proj.weight": {
1503
+ "shard": 6,
1504
+ "offset": 11351040,
1505
+ "size": 110592,
1506
+ "shape": [
1507
+ 256,
1508
+ 768
1509
+ ],
1510
+ "dtype": "Q4_K_M",
1511
+ "role": "matmul",
1512
+ "layout": "row"
1513
+ },
1514
+ "layers.12.self_attn.o_proj.weight": {
1515
+ "shard": 6,
1516
+ "offset": 11461632,
1517
+ "size": 331776,
1518
+ "shape": [
1519
+ 768,
1520
+ 768
1521
+ ],
1522
+ "dtype": "Q4_K_M",
1523
+ "role": "matmul",
1524
+ "layout": "row"
1525
+ },
1526
+ "layers.12.self_attn.q_norm.weight": {
1527
+ "shard": 6,
1528
+ "offset": 11793408,
1529
+ "size": 1024,
1530
+ "shape": [
1531
+ 256
1532
+ ],
1533
+ "dtype": "F32",
1534
+ "role": "norm"
1535
+ },
1536
+ "layers.12.self_attn.q_proj.weight": {
1537
+ "shard": 6,
1538
+ "offset": 11794432,
1539
+ "size": 331776,
1540
+ "shape": [
1541
+ 768,
1542
+ 768
1543
+ ],
1544
+ "dtype": "Q4_K_M",
1545
+ "role": "matmul",
1546
+ "layout": "row"
1547
+ },
1548
+ "layers.12.self_attn.v_proj.weight": {
1549
+ "shard": 6,
1550
+ "offset": 12126208,
1551
+ "size": 110592,
1552
+ "shape": [
1553
+ 256,
1554
+ 768
1555
+ ],
1556
+ "dtype": "Q4_K_M",
1557
+ "role": "matmul",
1558
+ "layout": "row"
1559
+ },
1560
+ "layers.13.input_layernorm.weight": {
1561
+ "shard": 6,
1562
+ "offset": 12236800,
1563
+ "size": 3072,
1564
+ "shape": [
1565
+ 768
1566
+ ],
1567
+ "dtype": "F32",
1568
+ "role": "norm"
1569
+ },
1570
+ "layers.13.mlp.down_proj.weight": {
1571
+ "shard": 6,
1572
+ "offset": 12239872,
1573
+ "size": 552960,
1574
+ "shape": [
1575
+ 768,
1576
+ 1152
1577
+ ],
1578
+ "dtype": "Q4_K_M",
1579
+ "role": "matmul",
1580
+ "layout": "row"
1581
+ },
1582
+ "layers.13.mlp.gate_proj.weight": {
1583
+ "shard": 6,
1584
+ "offset": 12792832,
1585
+ "size": 497664,
1586
+ "shape": [
1587
+ 1152,
1588
+ 768
1589
+ ],
1590
+ "dtype": "Q4_K_M",
1591
+ "role": "matmul",
1592
+ "layout": "row"
1593
+ },
1594
+ "layers.13.mlp.up_proj.weight": {
1595
+ "shard": 6,
1596
+ "offset": 13290496,
1597
+ "size": 497664,
1598
+ "shape": [
1599
+ 1152,
1600
+ 768
1601
+ ],
1602
+ "dtype": "Q4_K_M",
1603
+ "role": "matmul",
1604
+ "layout": "row"
1605
+ },
1606
+ "layers.13.post_attention_layernorm.weight": {
1607
+ "shard": 6,
1608
+ "offset": 13788160,
1609
+ "size": 3072,
1610
+ "shape": [
1611
+ 768
1612
+ ],
1613
+ "dtype": "F32",
1614
+ "role": "norm"
1615
+ },
1616
+ "layers.13.post_feedforward_layernorm.weight": {
1617
+ "shard": 6,
1618
+ "offset": 13791232,
1619
+ "size": 3072,
1620
+ "shape": [
1621
+ 768
1622
+ ],
1623
+ "dtype": "F32",
1624
+ "role": "norm"
1625
+ },
1626
+ "layers.13.pre_feedforward_layernorm.weight": {
1627
+ "shard": 6,
1628
+ "offset": 13794304,
1629
+ "size": 3072,
1630
+ "shape": [
1631
+ 768
1632
+ ],
1633
+ "dtype": "F32",
1634
+ "role": "norm"
1635
+ },
1636
+ "layers.13.self_attn.k_norm.weight": {
1637
+ "shard": 6,
1638
+ "offset": 13797376,
1639
+ "size": 1024,
1640
+ "shape": [
1641
+ 256
1642
+ ],
1643
+ "dtype": "F32",
1644
+ "role": "norm"
1645
+ },
1646
+ "layers.13.self_attn.k_proj.weight": {
1647
+ "shard": 6,
1648
+ "offset": 13798400,
1649
+ "size": 110592,
1650
+ "shape": [
1651
+ 256,
1652
+ 768
1653
+ ],
1654
+ "dtype": "Q4_K_M",
1655
+ "role": "matmul",
1656
+ "layout": "row"
1657
+ },
1658
+ "layers.13.self_attn.o_proj.weight": {
1659
+ "shard": 6,
1660
+ "offset": 13908992,
1661
+ "size": 331776,
1662
+ "shape": [
1663
+ 768,
1664
+ 768
1665
+ ],
1666
+ "dtype": "Q4_K_M",
1667
+ "role": "matmul",
1668
+ "layout": "row"
1669
+ },
1670
+ "layers.13.self_attn.q_norm.weight": {
1671
+ "shard": 6,
1672
+ "offset": 14240768,
1673
+ "size": 1024,
1674
+ "shape": [
1675
+ 256
1676
+ ],
1677
+ "dtype": "F32",
1678
+ "role": "norm"
1679
+ },
1680
+ "layers.13.self_attn.q_proj.weight": {
1681
+ "shard": 6,
1682
+ "offset": 14241792,
1683
+ "size": 331776,
1684
+ "shape": [
1685
+ 768,
1686
+ 768
1687
+ ],
1688
+ "dtype": "Q4_K_M",
1689
+ "role": "matmul",
1690
+ "layout": "row"
1691
+ },
1692
+ "layers.13.self_attn.v_proj.weight": {
1693
+ "shard": 6,
1694
+ "offset": 14573568,
1695
+ "size": 110592,
1696
+ "shape": [
1697
+ 256,
1698
+ 768
1699
+ ],
1700
+ "dtype": "Q4_K_M",
1701
+ "role": "matmul",
1702
+ "layout": "row"
1703
+ },
1704
+ "layers.14.input_layernorm.weight": {
1705
+ "shard": 6,
1706
+ "offset": 14684160,
1707
+ "size": 3072,
1708
+ "shape": [
1709
+ 768
1710
+ ],
1711
+ "dtype": "F32",
1712
+ "role": "norm"
1713
+ },
1714
+ "layers.14.mlp.down_proj.weight": {
1715
+ "shard": 6,
1716
+ "offset": 14687232,
1717
+ "size": 552960,
1718
+ "shape": [
1719
+ 768,
1720
+ 1152
1721
+ ],
1722
+ "dtype": "Q4_K_M",
1723
+ "role": "matmul",
1724
+ "layout": "row"
1725
+ },
1726
+ "layers.14.mlp.gate_proj.weight": {
1727
+ "shard": 6,
1728
+ "offset": 15240192,
1729
+ "size": 497664,
1730
+ "shape": [
1731
+ 1152,
1732
+ 768
1733
+ ],
1734
+ "dtype": "Q4_K_M",
1735
+ "role": "matmul",
1736
+ "layout": "row"
1737
+ },
1738
+ "layers.14.mlp.up_proj.weight": {
1739
+ "shard": 6,
1740
+ "offset": 15737856,
1741
+ "size": 497664,
1742
+ "shape": [
1743
+ 1152,
1744
+ 768
1745
+ ],
1746
+ "dtype": "Q4_K_M",
1747
+ "role": "matmul",
1748
+ "layout": "row"
1749
+ },
1750
+ "layers.14.post_attention_layernorm.weight": {
1751
+ "shard": 6,
1752
+ "offset": 16235520,
1753
+ "size": 3072,
1754
+ "shape": [
1755
+ 768
1756
+ ],
1757
+ "dtype": "F32",
1758
+ "role": "norm"
1759
+ },
1760
+ "layers.14.post_feedforward_layernorm.weight": {
1761
+ "shard": 6,
1762
+ "offset": 16238592,
1763
+ "size": 3072,
1764
+ "shape": [
1765
+ 768
1766
+ ],
1767
+ "dtype": "F32",
1768
+ "role": "norm"
1769
+ },
1770
+ "layers.14.pre_feedforward_layernorm.weight": {
1771
+ "shard": 6,
1772
+ "offset": 16241664,
1773
+ "size": 3072,
1774
+ "shape": [
1775
+ 768
1776
+ ],
1777
+ "dtype": "F32",
1778
+ "role": "norm"
1779
+ },
1780
+ "layers.14.self_attn.k_norm.weight": {
1781
+ "shard": 6,
1782
+ "offset": 16244736,
1783
+ "size": 1024,
1784
+ "shape": [
1785
+ 256
1786
+ ],
1787
+ "dtype": "F32",
1788
+ "role": "norm"
1789
+ },
1790
+ "layers.14.self_attn.k_proj.weight": {
1791
+ "shard": 6,
1792
+ "offset": 16245760,
1793
+ "size": 110592,
1794
+ "shape": [
1795
+ 256,
1796
+ 768
1797
+ ],
1798
+ "dtype": "Q4_K_M",
1799
+ "role": "matmul",
1800
+ "layout": "row"
1801
+ },
1802
+ "layers.14.self_attn.o_proj.weight": {
1803
+ "shard": 6,
1804
+ "offset": 16356352,
1805
+ "size": 331776,
1806
+ "shape": [
1807
+ 768,
1808
+ 768
1809
+ ],
1810
+ "dtype": "Q4_K_M",
1811
+ "role": "matmul",
1812
+ "layout": "row"
1813
+ },
1814
+ "layers.14.self_attn.q_norm.weight": {
1815
+ "shard": 6,
1816
+ "offset": 16688128,
1817
+ "size": 1024,
1818
+ "shape": [
1819
+ 256
1820
+ ],
1821
+ "dtype": "F32",
1822
+ "role": "norm"
1823
+ },
1824
+ "layers.14.self_attn.q_proj.weight": {
1825
+ "shard": 6,
1826
+ "offset": 16689152,
1827
+ "size": 331776,
1828
+ "shape": [
1829
+ 768,
1830
+ 768
1831
+ ],
1832
+ "dtype": "Q4_K_M",
1833
+ "role": "matmul",
1834
+ "layout": "row"
1835
+ },
1836
+ "layers.14.self_attn.v_proj.weight": {
1837
+ "shard": 6,
1838
+ "offset": 17020928,
1839
+ "size": 110592,
1840
+ "shape": [
1841
+ 256,
1842
+ 768
1843
+ ],
1844
+ "dtype": "Q4_K_M",
1845
+ "role": "matmul",
1846
+ "layout": "row"
1847
+ },
1848
+ "layers.15.input_layernorm.weight": {
1849
+ "shard": 6,
1850
+ "offset": 17131520,
1851
+ "size": 3072,
1852
+ "shape": [
1853
+ 768
1854
+ ],
1855
+ "dtype": "F32",
1856
+ "role": "norm"
1857
+ },
1858
+ "layers.15.mlp.down_proj.weight": {
1859
+ "shard": 6,
1860
+ "offset": 17134592,
1861
+ "size": 552960,
1862
+ "shape": [
1863
+ 768,
1864
+ 1152
1865
+ ],
1866
+ "dtype": "Q4_K_M",
1867
+ "role": "matmul",
1868
+ "layout": "row"
1869
+ },
1870
+ "layers.15.mlp.gate_proj.weight": {
1871
+ "shard": 6,
1872
+ "offset": 17687552,
1873
+ "size": 497664,
1874
+ "shape": [
1875
+ 1152,
1876
+ 768
1877
+ ],
1878
+ "dtype": "Q4_K_M",
1879
+ "role": "matmul",
1880
+ "layout": "row"
1881
+ },
1882
+ "layers.15.mlp.up_proj.weight": {
1883
+ "shard": 6,
1884
+ "offset": 18185216,
1885
+ "size": 497664,
1886
+ "shape": [
1887
+ 1152,
1888
+ 768
1889
+ ],
1890
+ "dtype": "Q4_K_M",
1891
+ "role": "matmul",
1892
+ "layout": "row"
1893
+ },
1894
+ "layers.15.post_attention_layernorm.weight": {
1895
+ "shard": 6,
1896
+ "offset": 18682880,
1897
+ "size": 3072,
1898
+ "shape": [
1899
+ 768
1900
+ ],
1901
+ "dtype": "F32",
1902
+ "role": "norm"
1903
+ },
1904
+ "layers.15.post_feedforward_layernorm.weight": {
1905
+ "shard": 6,
1906
+ "offset": 18685952,
1907
+ "size": 3072,
1908
+ "shape": [
1909
+ 768
1910
+ ],
1911
+ "dtype": "F32",
1912
+ "role": "norm"
1913
+ },
1914
+ "layers.15.pre_feedforward_layernorm.weight": {
1915
+ "shard": 6,
1916
+ "offset": 18689024,
1917
+ "size": 3072,
1918
+ "shape": [
1919
+ 768
1920
+ ],
1921
+ "dtype": "F32",
1922
+ "role": "norm"
1923
+ },
1924
+ "layers.15.self_attn.k_norm.weight": {
1925
+ "shard": 6,
1926
+ "offset": 18692096,
1927
+ "size": 1024,
1928
+ "shape": [
1929
+ 256
1930
+ ],
1931
+ "dtype": "F32",
1932
+ "role": "norm"
1933
+ },
1934
+ "layers.15.self_attn.k_proj.weight": {
1935
+ "shard": 6,
1936
+ "offset": 18693120,
1937
+ "size": 110592,
1938
+ "shape": [
1939
+ 256,
1940
+ 768
1941
+ ],
1942
+ "dtype": "Q4_K_M",
1943
+ "role": "matmul",
1944
+ "layout": "row"
1945
+ },
1946
+ "layers.15.self_attn.o_proj.weight": {
1947
+ "shard": 6,
1948
+ "offset": 18803712,
1949
+ "size": 331776,
1950
+ "shape": [
1951
+ 768,
1952
+ 768
1953
+ ],
1954
+ "dtype": "Q4_K_M",
1955
+ "role": "matmul",
1956
+ "layout": "row"
1957
+ },
1958
+ "layers.15.self_attn.q_norm.weight": {
1959
+ "shard": 6,
1960
+ "offset": 19135488,
1961
+ "size": 1024,
1962
+ "shape": [
1963
+ 256
1964
+ ],
1965
+ "dtype": "F32",
1966
+ "role": "norm"
1967
+ },
1968
+ "layers.15.self_attn.q_proj.weight": {
1969
+ "shard": 6,
1970
+ "offset": 19136512,
1971
+ "size": 331776,
1972
+ "shape": [
1973
+ 768,
1974
+ 768
1975
+ ],
1976
+ "dtype": "Q4_K_M",
1977
+ "role": "matmul",
1978
+ "layout": "row"
1979
+ },
1980
+ "layers.15.self_attn.v_proj.weight": {
1981
+ "shard": 6,
1982
+ "offset": 19468288,
1983
+ "size": 110592,
1984
+ "shape": [
1985
+ 256,
1986
+ 768
1987
+ ],
1988
+ "dtype": "Q4_K_M",
1989
+ "role": "matmul",
1990
+ "layout": "row"
1991
+ },
1992
+ "layers.16.input_layernorm.weight": {
1993
+ "shard": 6,
1994
+ "offset": 19578880,
1995
+ "size": 3072,
1996
+ "shape": [
1997
+ 768
1998
+ ],
1999
+ "dtype": "F32",
2000
+ "role": "norm"
2001
+ },
2002
+ "layers.16.mlp.down_proj.weight": {
2003
+ "shard": 6,
2004
+ "offset": 19581952,
2005
+ "size": 552960,
2006
+ "shape": [
2007
+ 768,
2008
+ 1152
2009
+ ],
2010
+ "dtype": "Q4_K_M",
2011
+ "role": "matmul",
2012
+ "layout": "row"
2013
+ },
2014
+ "layers.16.mlp.gate_proj.weight": {
2015
+ "shard": 6,
2016
+ "offset": 20134912,
2017
+ "size": 497664,
2018
+ "shape": [
2019
+ 1152,
2020
+ 768
2021
+ ],
2022
+ "dtype": "Q4_K_M",
2023
+ "role": "matmul",
2024
+ "layout": "row"
2025
+ },
2026
+ "layers.16.mlp.up_proj.weight": {
2027
+ "shard": 6,
2028
+ "offset": 20632576,
2029
+ "size": 497664,
2030
+ "shape": [
2031
+ 1152,
2032
+ 768
2033
+ ],
2034
+ "dtype": "Q4_K_M",
2035
+ "role": "matmul",
2036
+ "layout": "row"
2037
+ },
2038
+ "layers.16.post_attention_layernorm.weight": {
2039
+ "shard": 6,
2040
+ "offset": 21130240,
2041
+ "size": 3072,
2042
+ "shape": [
2043
+ 768
2044
+ ],
2045
+ "dtype": "F32",
2046
+ "role": "norm"
2047
+ },
2048
+ "layers.16.post_feedforward_layernorm.weight": {
2049
+ "shard": 6,
2050
+ "offset": 21133312,
2051
+ "size": 3072,
2052
+ "shape": [
2053
+ 768
2054
+ ],
2055
+ "dtype": "F32",
2056
+ "role": "norm"
2057
+ },
2058
+ "layers.16.pre_feedforward_layernorm.weight": {
2059
+ "shard": 6,
2060
+ "offset": 21136384,
2061
+ "size": 3072,
2062
+ "shape": [
2063
+ 768
2064
+ ],
2065
+ "dtype": "F32",
2066
+ "role": "norm"
2067
+ },
2068
+ "layers.16.self_attn.k_norm.weight": {
2069
+ "shard": 6,
2070
+ "offset": 21139456,
2071
+ "size": 1024,
2072
+ "shape": [
2073
+ 256
2074
+ ],
2075
+ "dtype": "F32",
2076
+ "role": "norm"
2077
+ },
2078
+ "layers.16.self_attn.k_proj.weight": {
2079
+ "shard": 6,
2080
+ "offset": 21140480,
2081
+ "size": 110592,
2082
+ "shape": [
2083
+ 256,
2084
+ 768
2085
+ ],
2086
+ "dtype": "Q4_K_M",
2087
+ "role": "matmul",
2088
+ "layout": "row"
2089
+ },
2090
+ "layers.16.self_attn.o_proj.weight": {
2091
+ "shard": 6,
2092
+ "offset": 21251072,
2093
+ "size": 331776,
2094
+ "shape": [
2095
+ 768,
2096
+ 768
2097
+ ],
2098
+ "dtype": "Q4_K_M",
2099
+ "role": "matmul",
2100
+ "layout": "row"
2101
+ },
2102
+ "layers.16.self_attn.q_norm.weight": {
2103
+ "shard": 6,
2104
+ "offset": 21582848,
2105
+ "size": 1024,
2106
+ "shape": [
2107
+ 256
2108
+ ],
2109
+ "dtype": "F32",
2110
+ "role": "norm"
2111
+ },
2112
+ "layers.16.self_attn.q_proj.weight": {
2113
+ "shard": 6,
2114
+ "offset": 21583872,
2115
+ "size": 331776,
2116
+ "shape": [
2117
+ 768,
2118
+ 768
2119
+ ],
2120
+ "dtype": "Q4_K_M",
2121
+ "role": "matmul",
2122
+ "layout": "row"
2123
+ },
2124
+ "layers.16.self_attn.v_proj.weight": {
2125
+ "shard": 6,
2126
+ "offset": 21915648,
2127
+ "size": 110592,
2128
+ "shape": [
2129
+ 256,
2130
+ 768
2131
+ ],
2132
+ "dtype": "Q4_K_M",
2133
+ "role": "matmul",
2134
+ "layout": "row"
2135
+ },
2136
+ "layers.17.input_layernorm.weight": {
2137
+ "shard": 6,
2138
+ "offset": 22026240,
2139
+ "size": 3072,
2140
+ "shape": [
2141
+ 768
2142
+ ],
2143
+ "dtype": "F32",
2144
+ "role": "norm"
2145
+ },
2146
+ "layers.17.mlp.down_proj.weight": {
2147
+ "shard": 6,
2148
+ "offset": 22029312,
2149
+ "size": 552960,
2150
+ "shape": [
2151
+ 768,
2152
+ 1152
2153
+ ],
2154
+ "dtype": "Q4_K_M",
2155
+ "role": "matmul",
2156
+ "layout": "row"
2157
+ },
2158
+ "layers.17.mlp.gate_proj.weight": {
2159
+ "shard": 6,
2160
+ "offset": 22582272,
2161
+ "size": 497664,
2162
+ "shape": [
2163
+ 1152,
2164
+ 768
2165
+ ],
2166
+ "dtype": "Q4_K_M",
2167
+ "role": "matmul",
2168
+ "layout": "row"
2169
+ },
2170
+ "layers.17.mlp.up_proj.weight": {
2171
+ "shard": 6,
2172
+ "offset": 23079936,
2173
+ "size": 497664,
2174
+ "shape": [
2175
+ 1152,
2176
+ 768
2177
+ ],
2178
+ "dtype": "Q4_K_M",
2179
+ "role": "matmul",
2180
+ "layout": "row"
2181
+ },
2182
+ "layers.17.post_attention_layernorm.weight": {
2183
+ "shard": 6,
2184
+ "offset": 23577600,
2185
+ "size": 3072,
2186
+ "shape": [
2187
+ 768
2188
+ ],
2189
+ "dtype": "F32",
2190
+ "role": "norm"
2191
+ },
2192
+ "layers.17.post_feedforward_layernorm.weight": {
2193
+ "shard": 6,
2194
+ "offset": 23580672,
2195
+ "size": 3072,
2196
+ "shape": [
2197
+ 768
2198
+ ],
2199
+ "dtype": "F32",
2200
+ "role": "norm"
2201
+ },
2202
+ "layers.17.pre_feedforward_layernorm.weight": {
2203
+ "shard": 6,
2204
+ "offset": 23583744,
2205
+ "size": 3072,
2206
+ "shape": [
2207
+ 768
2208
+ ],
2209
+ "dtype": "F32",
2210
+ "role": "norm"
2211
+ },
2212
+ "layers.17.self_attn.k_norm.weight": {
2213
+ "shard": 6,
2214
+ "offset": 23586816,
2215
+ "size": 1024,
2216
+ "shape": [
2217
+ 256
2218
+ ],
2219
+ "dtype": "F32",
2220
+ "role": "norm"
2221
+ },
2222
+ "layers.17.self_attn.k_proj.weight": {
2223
+ "shard": 6,
2224
+ "offset": 23587840,
2225
+ "size": 110592,
2226
+ "shape": [
2227
+ 256,
2228
+ 768
2229
+ ],
2230
+ "dtype": "Q4_K_M",
2231
+ "role": "matmul",
2232
+ "layout": "row"
2233
+ },
2234
+ "layers.17.self_attn.o_proj.weight": {
2235
+ "shard": 6,
2236
+ "offset": 23698432,
2237
+ "size": 331776,
2238
+ "shape": [
2239
+ 768,
2240
+ 768
2241
+ ],
2242
+ "dtype": "Q4_K_M",
2243
+ "role": "matmul",
2244
+ "layout": "row"
2245
+ },
2246
+ "layers.17.self_attn.q_norm.weight": {
2247
+ "shard": 6,
2248
+ "offset": 24030208,
2249
+ "size": 1024,
2250
+ "shape": [
2251
+ 256
2252
+ ],
2253
+ "dtype": "F32",
2254
+ "role": "norm"
2255
+ },
2256
+ "layers.17.self_attn.q_proj.weight": {
2257
+ "shard": 6,
2258
+ "offset": 24031232,
2259
+ "size": 331776,
2260
+ "shape": [
2261
+ 768,
2262
+ 768
2263
+ ],
2264
+ "dtype": "Q4_K_M",
2265
+ "role": "matmul",
2266
+ "layout": "row"
2267
+ },
2268
+ "layers.17.self_attn.v_proj.weight": {
2269
+ "shard": 6,
2270
+ "offset": 24363008,
2271
+ "size": 110592,
2272
+ "shape": [
2273
+ 256,
2274
+ 768
2275
+ ],
2276
+ "dtype": "Q4_K_M",
2277
+ "role": "matmul",
2278
+ "layout": "row"
2279
+ },
2280
+ "layers.18.input_layernorm.weight": {
2281
+ "shard": 6,
2282
+ "offset": 24473600,
2283
+ "size": 3072,
2284
+ "shape": [
2285
+ 768
2286
+ ],
2287
+ "dtype": "F32",
2288
+ "role": "norm"
2289
+ },
2290
+ "layers.18.mlp.down_proj.weight": {
2291
+ "shard": 6,
2292
+ "offset": 24476672,
2293
+ "size": 552960,
2294
+ "shape": [
2295
+ 768,
2296
+ 1152
2297
+ ],
2298
+ "dtype": "Q4_K_M",
2299
+ "role": "matmul",
2300
+ "layout": "row"
2301
+ },
2302
+ "layers.18.mlp.gate_proj.weight": {
2303
+ "shard": 6,
2304
+ "offset": 25029632,
2305
+ "size": 497664,
2306
+ "shape": [
2307
+ 1152,
2308
+ 768
2309
+ ],
2310
+ "dtype": "Q4_K_M",
2311
+ "role": "matmul",
2312
+ "layout": "row"
2313
+ },
2314
+ "layers.18.mlp.up_proj.weight": {
2315
+ "shard": 6,
2316
+ "offset": 25527296,
2317
+ "size": 497664,
2318
+ "shape": [
2319
+ 1152,
2320
+ 768
2321
+ ],
2322
+ "dtype": "Q4_K_M",
2323
+ "role": "matmul",
2324
+ "layout": "row"
2325
+ },
2326
+ "layers.18.post_attention_layernorm.weight": {
2327
+ "shard": 6,
2328
+ "offset": 26024960,
2329
+ "size": 3072,
2330
+ "shape": [
2331
+ 768
2332
+ ],
2333
+ "dtype": "F32",
2334
+ "role": "norm"
2335
+ },
2336
+ "layers.18.post_feedforward_layernorm.weight": {
2337
+ "shard": 6,
2338
+ "offset": 26028032,
2339
+ "size": 3072,
2340
+ "shape": [
2341
+ 768
2342
+ ],
2343
+ "dtype": "F32",
2344
+ "role": "norm"
2345
+ },
2346
+ "layers.18.pre_feedforward_layernorm.weight": {
2347
+ "shard": 6,
2348
+ "offset": 26031104,
2349
+ "size": 3072,
2350
+ "shape": [
2351
+ 768
2352
+ ],
2353
+ "dtype": "F32",
2354
+ "role": "norm"
2355
+ },
2356
+ "layers.18.self_attn.k_norm.weight": {
2357
+ "shard": 6,
2358
+ "offset": 26034176,
2359
+ "size": 1024,
2360
+ "shape": [
2361
+ 256
2362
+ ],
2363
+ "dtype": "F32",
2364
+ "role": "norm"
2365
+ },
2366
+ "layers.18.self_attn.k_proj.weight": {
2367
+ "shard": 6,
2368
+ "offset": 26035200,
2369
+ "size": 110592,
2370
+ "shape": [
2371
+ 256,
2372
+ 768
2373
+ ],
2374
+ "dtype": "Q4_K_M",
2375
+ "role": "matmul",
2376
+ "layout": "row"
2377
+ },
2378
+ "layers.18.self_attn.o_proj.weight": {
2379
+ "shard": 6,
2380
+ "offset": 26145792,
2381
+ "size": 331776,
2382
+ "shape": [
2383
+ 768,
2384
+ 768
2385
+ ],
2386
+ "dtype": "Q4_K_M",
2387
+ "role": "matmul",
2388
+ "layout": "row"
2389
+ },
2390
+ "layers.18.self_attn.q_norm.weight": {
2391
+ "shard": 6,
2392
+ "offset": 26477568,
2393
+ "size": 1024,
2394
+ "shape": [
2395
+ 256
2396
+ ],
2397
+ "dtype": "F32",
2398
+ "role": "norm"
2399
+ },
2400
+ "layers.18.self_attn.q_proj.weight": {
2401
+ "shard": 6,
2402
+ "offset": 26478592,
2403
+ "size": 331776,
2404
+ "shape": [
2405
+ 768,
2406
+ 768
2407
+ ],
2408
+ "dtype": "Q4_K_M",
2409
+ "role": "matmul",
2410
+ "layout": "row"
2411
+ },
2412
+ "layers.18.self_attn.v_proj.weight": {
2413
+ "shard": 6,
2414
+ "offset": 26810368,
2415
+ "size": 110592,
2416
+ "shape": [
2417
+ 256,
2418
+ 768
2419
+ ],
2420
+ "dtype": "Q4_K_M",
2421
+ "role": "matmul",
2422
+ "layout": "row"
2423
+ },
2424
+ "layers.19.input_layernorm.weight": {
2425
+ "shard": 6,
2426
+ "offset": 26920960,
2427
+ "size": 3072,
2428
+ "shape": [
2429
+ 768
2430
+ ],
2431
+ "dtype": "F32",
2432
+ "role": "norm"
2433
+ },
2434
+ "layers.19.mlp.down_proj.weight": {
2435
+ "shard": 6,
2436
+ "offset": 26924032,
2437
+ "size": 552960,
2438
+ "shape": [
2439
+ 768,
2440
+ 1152
2441
+ ],
2442
+ "dtype": "Q4_K_M",
2443
+ "role": "matmul",
2444
+ "layout": "row"
2445
+ },
2446
+ "layers.19.mlp.gate_proj.weight": {
2447
+ "shard": 6,
2448
+ "offset": 27476992,
2449
+ "size": 497664,
2450
+ "shape": [
2451
+ 1152,
2452
+ 768
2453
+ ],
2454
+ "dtype": "Q4_K_M",
2455
+ "role": "matmul",
2456
+ "layout": "row"
2457
+ },
2458
+ "layers.19.mlp.up_proj.weight": {
2459
+ "shard": 6,
2460
+ "offset": 27974656,
2461
+ "size": 497664,
2462
+ "shape": [
2463
+ 1152,
2464
+ 768
2465
+ ],
2466
+ "dtype": "Q4_K_M",
2467
+ "role": "matmul",
2468
+ "layout": "row"
2469
+ },
2470
+ "layers.19.post_attention_layernorm.weight": {
2471
+ "shard": 6,
2472
+ "offset": 28472320,
2473
+ "size": 3072,
2474
+ "shape": [
2475
+ 768
2476
+ ],
2477
+ "dtype": "F32",
2478
+ "role": "norm"
2479
+ },
2480
+ "layers.19.post_feedforward_layernorm.weight": {
2481
+ "shard": 6,
2482
+ "offset": 28475392,
2483
+ "size": 3072,
2484
+ "shape": [
2485
+ 768
2486
+ ],
2487
+ "dtype": "F32",
2488
+ "role": "norm"
2489
+ },
2490
+ "layers.19.pre_feedforward_layernorm.weight": {
2491
+ "shard": 6,
2492
+ "offset": 28478464,
2493
+ "size": 3072,
2494
+ "shape": [
2495
+ 768
2496
+ ],
2497
+ "dtype": "F32",
2498
+ "role": "norm"
2499
+ },
2500
+ "layers.19.self_attn.k_norm.weight": {
2501
+ "shard": 6,
2502
+ "offset": 28481536,
2503
+ "size": 1024,
2504
+ "shape": [
2505
+ 256
2506
+ ],
2507
+ "dtype": "F32",
2508
+ "role": "norm"
2509
+ },
2510
+ "layers.19.self_attn.k_proj.weight": {
2511
+ "shard": 6,
2512
+ "offset": 28482560,
2513
+ "size": 110592,
2514
+ "shape": [
2515
+ 256,
2516
+ 768
2517
+ ],
2518
+ "dtype": "Q4_K_M",
2519
+ "role": "matmul",
2520
+ "layout": "row"
2521
+ },
2522
+ "layers.19.self_attn.o_proj.weight": {
2523
+ "shard": 6,
2524
+ "offset": 28593152,
2525
+ "size": 331776,
2526
+ "shape": [
2527
+ 768,
2528
+ 768
2529
+ ],
2530
+ "dtype": "Q4_K_M",
2531
+ "role": "matmul",
2532
+ "layout": "row"
2533
+ },
2534
+ "layers.19.self_attn.q_norm.weight": {
2535
+ "shard": 6,
2536
+ "offset": 28924928,
2537
+ "size": 1024,
2538
+ "shape": [
2539
+ 256
2540
+ ],
2541
+ "dtype": "F32",
2542
+ "role": "norm"
2543
+ },
2544
+ "layers.19.self_attn.q_proj.weight": {
2545
+ "shard": 6,
2546
+ "offset": 28925952,
2547
+ "size": 331776,
2548
+ "shape": [
2549
+ 768,
2550
+ 768
2551
+ ],
2552
+ "dtype": "Q4_K_M",
2553
+ "role": "matmul",
2554
+ "layout": "row"
2555
+ },
2556
+ "layers.19.self_attn.v_proj.weight": {
2557
+ "shard": 6,
2558
+ "offset": 29257728,
2559
+ "size": 110592,
2560
+ "shape": [
2561
+ 256,
2562
+ 768
2563
+ ],
2564
+ "dtype": "Q4_K_M",
2565
+ "role": "matmul",
2566
+ "layout": "row"
2567
+ },
2568
+ "layers.2.input_layernorm.weight": {
2569
+ "shard": 6,
2570
+ "offset": 29368320,
2571
+ "size": 3072,
2572
+ "shape": [
2573
+ 768
2574
+ ],
2575
+ "dtype": "F32",
2576
+ "role": "norm"
2577
+ },
2578
+ "layers.2.mlp.down_proj.weight": {
2579
+ "shard": 6,
2580
+ "offset": 29371392,
2581
+ "size": 552960,
2582
+ "shape": [
2583
+ 768,
2584
+ 1152
2585
+ ],
2586
+ "dtype": "Q4_K_M",
2587
+ "role": "matmul",
2588
+ "layout": "row"
2589
+ },
2590
+ "layers.2.mlp.gate_proj.weight": {
2591
+ "shard": 6,
2592
+ "offset": 29924352,
2593
+ "size": 497664,
2594
+ "shape": [
2595
+ 1152,
2596
+ 768
2597
+ ],
2598
+ "dtype": "Q4_K_M",
2599
+ "role": "matmul",
2600
+ "layout": "row"
2601
+ },
2602
+ "layers.2.mlp.up_proj.weight": {
2603
+ "shard": 6,
2604
+ "offset": 30422016,
2605
+ "size": 497664,
2606
+ "shape": [
2607
+ 1152,
2608
+ 768
2609
+ ],
2610
+ "dtype": "Q4_K_M",
2611
+ "role": "matmul",
2612
+ "layout": "row"
2613
+ },
2614
+ "layers.2.post_attention_layernorm.weight": {
2615
+ "shard": 6,
2616
+ "offset": 30919680,
2617
+ "size": 3072,
2618
+ "shape": [
2619
+ 768
2620
+ ],
2621
+ "dtype": "F32",
2622
+ "role": "norm"
2623
+ },
2624
+ "layers.2.post_feedforward_layernorm.weight": {
2625
+ "shard": 6,
2626
+ "offset": 30922752,
2627
+ "size": 3072,
2628
+ "shape": [
2629
+ 768
2630
+ ],
2631
+ "dtype": "F32",
2632
+ "role": "norm"
2633
+ },
2634
+ "layers.2.pre_feedforward_layernorm.weight": {
2635
+ "shard": 6,
2636
+ "offset": 30925824,
2637
+ "size": 3072,
2638
+ "shape": [
2639
+ 768
2640
+ ],
2641
+ "dtype": "F32",
2642
+ "role": "norm"
2643
+ },
2644
+ "layers.2.self_attn.k_norm.weight": {
2645
+ "shard": 6,
2646
+ "offset": 30928896,
2647
+ "size": 1024,
2648
+ "shape": [
2649
+ 256
2650
+ ],
2651
+ "dtype": "F32",
2652
+ "role": "norm"
2653
+ },
2654
+ "layers.2.self_attn.k_proj.weight": {
2655
+ "shard": 6,
2656
+ "offset": 30929920,
2657
+ "size": 110592,
2658
+ "shape": [
2659
+ 256,
2660
+ 768
2661
+ ],
2662
+ "dtype": "Q4_K_M",
2663
+ "role": "matmul",
2664
+ "layout": "row"
2665
+ },
2666
+ "layers.2.self_attn.o_proj.weight": {
2667
+ "shard": 6,
2668
+ "offset": 31040512,
2669
+ "size": 331776,
2670
+ "shape": [
2671
+ 768,
2672
+ 768
2673
+ ],
2674
+ "dtype": "Q4_K_M",
2675
+ "role": "matmul",
2676
+ "layout": "row"
2677
+ },
2678
+ "layers.2.self_attn.q_norm.weight": {
2679
+ "shard": 6,
2680
+ "offset": 31372288,
2681
+ "size": 1024,
2682
+ "shape": [
2683
+ 256
2684
+ ],
2685
+ "dtype": "F32",
2686
+ "role": "norm"
2687
+ },
2688
+ "layers.2.self_attn.q_proj.weight": {
2689
+ "shard": 6,
2690
+ "offset": 31373312,
2691
+ "size": 331776,
2692
+ "shape": [
2693
+ 768,
2694
+ 768
2695
+ ],
2696
+ "dtype": "Q4_K_M",
2697
+ "role": "matmul",
2698
+ "layout": "row"
2699
+ },
2700
+ "layers.2.self_attn.v_proj.weight": {
2701
+ "shard": 6,
2702
+ "offset": 31705088,
2703
+ "size": 110592,
2704
+ "shape": [
2705
+ 256,
2706
+ 768
2707
+ ],
2708
+ "dtype": "Q4_K_M",
2709
+ "role": "matmul",
2710
+ "layout": "row"
2711
+ },
2712
+ "layers.20.input_layernorm.weight": {
2713
+ "shard": 6,
2714
+ "offset": 31815680,
2715
+ "size": 3072,
2716
+ "shape": [
2717
+ 768
2718
+ ],
2719
+ "dtype": "F32",
2720
+ "role": "norm"
2721
+ },
2722
+ "layers.20.mlp.down_proj.weight": {
2723
+ "shard": 6,
2724
+ "offset": 31818752,
2725
+ "size": 552960,
2726
+ "shape": [
2727
+ 768,
2728
+ 1152
2729
+ ],
2730
+ "dtype": "Q4_K_M",
2731
+ "role": "matmul",
2732
+ "layout": "row"
2733
+ },
2734
+ "layers.20.mlp.gate_proj.weight": {
2735
+ "shard": 6,
2736
+ "offset": 32371712,
2737
+ "size": 497664,
2738
+ "shape": [
2739
+ 1152,
2740
+ 768
2741
+ ],
2742
+ "dtype": "Q4_K_M",
2743
+ "role": "matmul",
2744
+ "layout": "row"
2745
+ },
2746
+ "layers.20.mlp.up_proj.weight": {
2747
+ "shard": 6,
2748
+ "offset": 32869376,
2749
+ "size": 497664,
2750
+ "shape": [
2751
+ 1152,
2752
+ 768
2753
+ ],
2754
+ "dtype": "Q4_K_M",
2755
+ "role": "matmul",
2756
+ "layout": "row"
2757
+ },
2758
+ "layers.20.post_attention_layernorm.weight": {
2759
+ "shard": 6,
2760
+ "offset": 33367040,
2761
+ "size": 3072,
2762
+ "shape": [
2763
+ 768
2764
+ ],
2765
+ "dtype": "F32",
2766
+ "role": "norm"
2767
+ },
2768
+ "layers.20.post_feedforward_layernorm.weight": {
2769
+ "shard": 6,
2770
+ "offset": 33370112,
2771
+ "size": 3072,
2772
+ "shape": [
2773
+ 768
2774
+ ],
2775
+ "dtype": "F32",
2776
+ "role": "norm"
2777
+ },
2778
+ "layers.20.pre_feedforward_layernorm.weight": {
2779
+ "shard": 6,
2780
+ "offset": 33373184,
2781
+ "size": 3072,
2782
+ "shape": [
2783
+ 768
2784
+ ],
2785
+ "dtype": "F32",
2786
+ "role": "norm"
2787
+ },
2788
+ "layers.20.self_attn.k_norm.weight": {
2789
+ "shard": 6,
2790
+ "offset": 33376256,
2791
+ "size": 1024,
2792
+ "shape": [
2793
+ 256
2794
+ ],
2795
+ "dtype": "F32",
2796
+ "role": "norm"
2797
+ },
2798
+ "layers.20.self_attn.k_proj.weight": {
2799
+ "shard": 6,
2800
+ "offset": 33377280,
2801
+ "size": 110592,
2802
+ "shape": [
2803
+ 256,
2804
+ 768
2805
+ ],
2806
+ "dtype": "Q4_K_M",
2807
+ "role": "matmul",
2808
+ "layout": "row"
2809
+ },
2810
+ "layers.20.self_attn.o_proj.weight": {
2811
+ "shard": 6,
2812
+ "offset": 33487872,
2813
+ "size": 331776,
2814
+ "shape": [
2815
+ 768,
2816
+ 768
2817
+ ],
2818
+ "dtype": "Q4_K_M",
2819
+ "role": "matmul",
2820
+ "layout": "row"
2821
+ },
2822
+ "layers.20.self_attn.q_norm.weight": {
2823
+ "shard": 6,
2824
+ "offset": 33819648,
2825
+ "size": 1024,
2826
+ "shape": [
2827
+ 256
2828
+ ],
2829
+ "dtype": "F32",
2830
+ "role": "norm"
2831
+ },
2832
+ "layers.20.self_attn.q_proj.weight": {
2833
+ "shard": 6,
2834
+ "offset": 33820672,
2835
+ "size": 331776,
2836
+ "shape": [
2837
+ 768,
2838
+ 768
2839
+ ],
2840
+ "dtype": "Q4_K_M",
2841
+ "role": "matmul",
2842
+ "layout": "row"
2843
+ },
2844
+ "layers.20.self_attn.v_proj.weight": {
2845
+ "shard": 6,
2846
+ "offset": 34152448,
2847
+ "size": 110592,
2848
+ "shape": [
2849
+ 256,
2850
+ 768
2851
+ ],
2852
+ "dtype": "Q4_K_M",
2853
+ "role": "matmul",
2854
+ "layout": "row"
2855
+ },
2856
+ "layers.21.input_layernorm.weight": {
2857
+ "shard": 6,
2858
+ "offset": 34263040,
2859
+ "size": 3072,
2860
+ "shape": [
2861
+ 768
2862
+ ],
2863
+ "dtype": "F32",
2864
+ "role": "norm"
2865
+ },
2866
+ "layers.21.mlp.down_proj.weight": {
2867
+ "shard": 6,
2868
+ "offset": 34266112,
2869
+ "size": 552960,
2870
+ "shape": [
2871
+ 768,
2872
+ 1152
2873
+ ],
2874
+ "dtype": "Q4_K_M",
2875
+ "role": "matmul",
2876
+ "layout": "row"
2877
+ },
2878
+ "layers.21.mlp.gate_proj.weight": {
2879
+ "shard": 6,
2880
+ "offset": 34819072,
2881
+ "size": 497664,
2882
+ "shape": [
2883
+ 1152,
2884
+ 768
2885
+ ],
2886
+ "dtype": "Q4_K_M",
2887
+ "role": "matmul",
2888
+ "layout": "row"
2889
+ },
2890
+ "layers.21.mlp.up_proj.weight": {
2891
+ "shard": 6,
2892
+ "offset": 35316736,
2893
+ "size": 497664,
2894
+ "shape": [
2895
+ 1152,
2896
+ 768
2897
+ ],
2898
+ "dtype": "Q4_K_M",
2899
+ "role": "matmul",
2900
+ "layout": "row"
2901
+ },
2902
+ "layers.21.post_attention_layernorm.weight": {
2903
+ "shard": 6,
2904
+ "offset": 35814400,
2905
+ "size": 3072,
2906
+ "shape": [
2907
+ 768
2908
+ ],
2909
+ "dtype": "F32",
2910
+ "role": "norm"
2911
+ },
2912
+ "layers.21.post_feedforward_layernorm.weight": {
2913
+ "shard": 6,
2914
+ "offset": 35817472,
2915
+ "size": 3072,
2916
+ "shape": [
2917
+ 768
2918
+ ],
2919
+ "dtype": "F32",
2920
+ "role": "norm"
2921
+ },
2922
+ "layers.21.pre_feedforward_layernorm.weight": {
2923
+ "shard": 6,
2924
+ "offset": 35820544,
2925
+ "size": 3072,
2926
+ "shape": [
2927
+ 768
2928
+ ],
2929
+ "dtype": "F32",
2930
+ "role": "norm"
2931
+ },
2932
+ "layers.21.self_attn.k_norm.weight": {
2933
+ "shard": 6,
2934
+ "offset": 35823616,
2935
+ "size": 1024,
2936
+ "shape": [
2937
+ 256
2938
+ ],
2939
+ "dtype": "F32",
2940
+ "role": "norm"
2941
+ },
2942
+ "layers.21.self_attn.k_proj.weight": {
2943
+ "shard": 6,
2944
+ "offset": 35824640,
2945
+ "size": 110592,
2946
+ "shape": [
2947
+ 256,
2948
+ 768
2949
+ ],
2950
+ "dtype": "Q4_K_M",
2951
+ "role": "matmul",
2952
+ "layout": "row"
2953
+ },
2954
+ "layers.21.self_attn.o_proj.weight": {
2955
+ "shard": 6,
2956
+ "offset": 35935232,
2957
+ "size": 331776,
2958
+ "shape": [
2959
+ 768,
2960
+ 768
2961
+ ],
2962
+ "dtype": "Q4_K_M",
2963
+ "role": "matmul",
2964
+ "layout": "row"
2965
+ },
2966
+ "layers.21.self_attn.q_norm.weight": {
2967
+ "shard": 6,
2968
+ "offset": 36267008,
2969
+ "size": 1024,
2970
+ "shape": [
2971
+ 256
2972
+ ],
2973
+ "dtype": "F32",
2974
+ "role": "norm"
2975
+ },
2976
+ "layers.21.self_attn.q_proj.weight": {
2977
+ "shard": 6,
2978
+ "offset": 36268032,
2979
+ "size": 331776,
2980
+ "shape": [
2981
+ 768,
2982
+ 768
2983
+ ],
2984
+ "dtype": "Q4_K_M",
2985
+ "role": "matmul",
2986
+ "layout": "row"
2987
+ },
2988
+ "layers.21.self_attn.v_proj.weight": {
2989
+ "shard": 6,
2990
+ "offset": 36599808,
2991
+ "size": 110592,
2992
+ "shape": [
2993
+ 256,
2994
+ 768
2995
+ ],
2996
+ "dtype": "Q4_K_M",
2997
+ "role": "matmul",
2998
+ "layout": "row"
2999
+ },
3000
+ "layers.22.input_layernorm.weight": {
3001
+ "shard": 6,
3002
+ "offset": 36710400,
3003
+ "size": 3072,
3004
+ "shape": [
3005
+ 768
3006
+ ],
3007
+ "dtype": "F32",
3008
+ "role": "norm"
3009
+ },
3010
+ "layers.22.mlp.down_proj.weight": {
3011
+ "shard": 6,
3012
+ "offset": 36713472,
3013
+ "size": 552960,
3014
+ "shape": [
3015
+ 768,
3016
+ 1152
3017
+ ],
3018
+ "dtype": "Q4_K_M",
3019
+ "role": "matmul",
3020
+ "layout": "row"
3021
+ },
3022
+ "layers.22.mlp.gate_proj.weight": {
3023
+ "shard": 6,
3024
+ "offset": 37266432,
3025
+ "size": 497664,
3026
+ "shape": [
3027
+ 1152,
3028
+ 768
3029
+ ],
3030
+ "dtype": "Q4_K_M",
3031
+ "role": "matmul",
3032
+ "layout": "row"
3033
+ },
3034
+ "layers.22.mlp.up_proj.weight": {
3035
+ "shard": 6,
3036
+ "offset": 37764096,
3037
+ "size": 497664,
3038
+ "shape": [
3039
+ 1152,
3040
+ 768
3041
+ ],
3042
+ "dtype": "Q4_K_M",
3043
+ "role": "matmul",
3044
+ "layout": "row"
3045
+ },
3046
+ "layers.22.post_attention_layernorm.weight": {
3047
+ "shard": 6,
3048
+ "offset": 38261760,
3049
+ "size": 3072,
3050
+ "shape": [
3051
+ 768
3052
+ ],
3053
+ "dtype": "F32",
3054
+ "role": "norm"
3055
+ },
3056
+ "layers.22.post_feedforward_layernorm.weight": {
3057
+ "shard": 6,
3058
+ "offset": 38264832,
3059
+ "size": 3072,
3060
+ "shape": [
3061
+ 768
3062
+ ],
3063
+ "dtype": "F32",
3064
+ "role": "norm"
3065
+ },
3066
+ "layers.22.pre_feedforward_layernorm.weight": {
3067
+ "shard": 6,
3068
+ "offset": 38267904,
3069
+ "size": 3072,
3070
+ "shape": [
3071
+ 768
3072
+ ],
3073
+ "dtype": "F32",
3074
+ "role": "norm"
3075
+ },
3076
+ "layers.22.self_attn.k_norm.weight": {
3077
+ "shard": 6,
3078
+ "offset": 38270976,
3079
+ "size": 1024,
3080
+ "shape": [
3081
+ 256
3082
+ ],
3083
+ "dtype": "F32",
3084
+ "role": "norm"
3085
+ },
3086
+ "layers.22.self_attn.k_proj.weight": {
3087
+ "shard": 6,
3088
+ "offset": 38272000,
3089
+ "size": 110592,
3090
+ "shape": [
3091
+ 256,
3092
+ 768
3093
+ ],
3094
+ "dtype": "Q4_K_M",
3095
+ "role": "matmul",
3096
+ "layout": "row"
3097
+ },
3098
+ "layers.22.self_attn.o_proj.weight": {
3099
+ "shard": 6,
3100
+ "offset": 38382592,
3101
+ "size": 331776,
3102
+ "shape": [
3103
+ 768,
3104
+ 768
3105
+ ],
3106
+ "dtype": "Q4_K_M",
3107
+ "role": "matmul",
3108
+ "layout": "row"
3109
+ },
3110
+ "layers.22.self_attn.q_norm.weight": {
3111
+ "shard": 6,
3112
+ "offset": 38714368,
3113
+ "size": 1024,
3114
+ "shape": [
3115
+ 256
3116
+ ],
3117
+ "dtype": "F32",
3118
+ "role": "norm"
3119
+ },
3120
+ "layers.22.self_attn.q_proj.weight": {
3121
+ "shard": 6,
3122
+ "offset": 38715392,
3123
+ "size": 331776,
3124
+ "shape": [
3125
+ 768,
3126
+ 768
3127
+ ],
3128
+ "dtype": "Q4_K_M",
3129
+ "role": "matmul",
3130
+ "layout": "row"
3131
+ },
3132
+ "layers.22.self_attn.v_proj.weight": {
3133
+ "shard": 6,
3134
+ "offset": 39047168,
3135
+ "size": 110592,
3136
+ "shape": [
3137
+ 256,
3138
+ 768
3139
+ ],
3140
+ "dtype": "Q4_K_M",
3141
+ "role": "matmul",
3142
+ "layout": "row"
3143
+ },
3144
+ "layers.23.input_layernorm.weight": {
3145
+ "shard": 6,
3146
+ "offset": 39157760,
3147
+ "size": 3072,
3148
+ "shape": [
3149
+ 768
3150
+ ],
3151
+ "dtype": "F32",
3152
+ "role": "norm"
3153
+ },
3154
+ "layers.23.mlp.down_proj.weight": {
3155
+ "shard": 6,
3156
+ "offset": 39160832,
3157
+ "size": 552960,
3158
+ "shape": [
3159
+ 768,
3160
+ 1152
3161
+ ],
3162
+ "dtype": "Q4_K_M",
3163
+ "role": "matmul",
3164
+ "layout": "row"
3165
+ },
3166
+ "layers.23.mlp.gate_proj.weight": {
3167
+ "shard": 6,
3168
+ "offset": 39713792,
3169
+ "size": 497664,
3170
+ "shape": [
3171
+ 1152,
3172
+ 768
3173
+ ],
3174
+ "dtype": "Q4_K_M",
3175
+ "role": "matmul",
3176
+ "layout": "row"
3177
+ },
3178
+ "layers.23.mlp.up_proj.weight": {
3179
+ "shard": 6,
3180
+ "offset": 40211456,
3181
+ "size": 497664,
3182
+ "shape": [
3183
+ 1152,
3184
+ 768
3185
+ ],
3186
+ "dtype": "Q4_K_M",
3187
+ "role": "matmul",
3188
+ "layout": "row"
3189
+ },
3190
+ "layers.23.post_attention_layernorm.weight": {
3191
+ "shard": 6,
3192
+ "offset": 40709120,
3193
+ "size": 3072,
3194
+ "shape": [
3195
+ 768
3196
+ ],
3197
+ "dtype": "F32",
3198
+ "role": "norm"
3199
+ },
3200
+ "layers.23.post_feedforward_layernorm.weight": {
3201
+ "shard": 6,
3202
+ "offset": 40712192,
3203
+ "size": 3072,
3204
+ "shape": [
3205
+ 768
3206
+ ],
3207
+ "dtype": "F32",
3208
+ "role": "norm"
3209
+ },
3210
+ "layers.23.pre_feedforward_layernorm.weight": {
3211
+ "shard": 6,
3212
+ "offset": 40715264,
3213
+ "size": 3072,
3214
+ "shape": [
3215
+ 768
3216
+ ],
3217
+ "dtype": "F32",
3218
+ "role": "norm"
3219
+ },
3220
+ "layers.23.self_attn.k_norm.weight": {
3221
+ "shard": 6,
3222
+ "offset": 40718336,
3223
+ "size": 1024,
3224
+ "shape": [
3225
+ 256
3226
+ ],
3227
+ "dtype": "F32",
3228
+ "role": "norm"
3229
+ },
3230
+ "layers.23.self_attn.k_proj.weight": {
3231
+ "shard": 6,
3232
+ "offset": 40719360,
3233
+ "size": 110592,
3234
+ "shape": [
3235
+ 256,
3236
+ 768
3237
+ ],
3238
+ "dtype": "Q4_K_M",
3239
+ "role": "matmul",
3240
+ "layout": "row"
3241
+ },
3242
+ "layers.23.self_attn.o_proj.weight": {
3243
+ "shard": 6,
3244
+ "offset": 40829952,
3245
+ "size": 331776,
3246
+ "shape": [
3247
+ 768,
3248
+ 768
3249
+ ],
3250
+ "dtype": "Q4_K_M",
3251
+ "role": "matmul",
3252
+ "layout": "row"
3253
+ },
3254
+ "layers.23.self_attn.q_norm.weight": {
3255
+ "shard": 6,
3256
+ "offset": 41161728,
3257
+ "size": 1024,
3258
+ "shape": [
3259
+ 256
3260
+ ],
3261
+ "dtype": "F32",
3262
+ "role": "norm"
3263
+ },
3264
+ "layers.23.self_attn.q_proj.weight": {
3265
+ "shard": 6,
3266
+ "offset": 41162752,
3267
+ "size": 331776,
3268
+ "shape": [
3269
+ 768,
3270
+ 768
3271
+ ],
3272
+ "dtype": "Q4_K_M",
3273
+ "role": "matmul",
3274
+ "layout": "row"
3275
+ },
3276
+ "layers.23.self_attn.v_proj.weight": {
3277
+ "shard": 6,
3278
+ "offset": 41494528,
3279
+ "size": 110592,
3280
+ "shape": [
3281
+ 256,
3282
+ 768
3283
+ ],
3284
+ "dtype": "Q4_K_M",
3285
+ "role": "matmul",
3286
+ "layout": "row"
3287
+ },
3288
+ "layers.3.input_layernorm.weight": {
3289
+ "shard": 6,
3290
+ "offset": 41605120,
3291
+ "size": 3072,
3292
+ "shape": [
3293
+ 768
3294
+ ],
3295
+ "dtype": "F32",
3296
+ "role": "norm"
3297
+ },
3298
+ "layers.3.mlp.down_proj.weight": {
3299
+ "shard": 6,
3300
+ "offset": 41608192,
3301
+ "size": 552960,
3302
+ "shape": [
3303
+ 768,
3304
+ 1152
3305
+ ],
3306
+ "dtype": "Q4_K_M",
3307
+ "role": "matmul",
3308
+ "layout": "row"
3309
+ },
3310
+ "layers.3.mlp.gate_proj.weight": {
3311
+ "shard": 6,
3312
+ "offset": 42161152,
3313
+ "size": 497664,
3314
+ "shape": [
3315
+ 1152,
3316
+ 768
3317
+ ],
3318
+ "dtype": "Q4_K_M",
3319
+ "role": "matmul",
3320
+ "layout": "row"
3321
+ },
3322
+ "layers.3.mlp.up_proj.weight": {
3323
+ "shard": 6,
3324
+ "offset": 42658816,
3325
+ "size": 497664,
3326
+ "shape": [
3327
+ 1152,
3328
+ 768
3329
+ ],
3330
+ "dtype": "Q4_K_M",
3331
+ "role": "matmul",
3332
+ "layout": "row"
3333
+ },
3334
+ "layers.3.post_attention_layernorm.weight": {
3335
+ "shard": 6,
3336
+ "offset": 43156480,
3337
+ "size": 3072,
3338
+ "shape": [
3339
+ 768
3340
+ ],
3341
+ "dtype": "F32",
3342
+ "role": "norm"
3343
+ },
3344
+ "layers.3.post_feedforward_layernorm.weight": {
3345
+ "shard": 6,
3346
+ "offset": 43159552,
3347
+ "size": 3072,
3348
+ "shape": [
3349
+ 768
3350
+ ],
3351
+ "dtype": "F32",
3352
+ "role": "norm"
3353
+ },
3354
+ "layers.3.pre_feedforward_layernorm.weight": {
3355
+ "shard": 6,
3356
+ "offset": 43162624,
3357
+ "size": 3072,
3358
+ "shape": [
3359
+ 768
3360
+ ],
3361
+ "dtype": "F32",
3362
+ "role": "norm"
3363
+ },
3364
+ "layers.3.self_attn.k_norm.weight": {
3365
+ "shard": 6,
3366
+ "offset": 43165696,
3367
+ "size": 1024,
3368
+ "shape": [
3369
+ 256
3370
+ ],
3371
+ "dtype": "F32",
3372
+ "role": "norm"
3373
+ },
3374
+ "layers.3.self_attn.k_proj.weight": {
3375
+ "shard": 6,
3376
+ "offset": 43166720,
3377
+ "size": 110592,
3378
+ "shape": [
3379
+ 256,
3380
+ 768
3381
+ ],
3382
+ "dtype": "Q4_K_M",
3383
+ "role": "matmul",
3384
+ "layout": "row"
3385
+ },
3386
+ "layers.3.self_attn.o_proj.weight": {
3387
+ "shard": 6,
3388
+ "offset": 43277312,
3389
+ "size": 331776,
3390
+ "shape": [
3391
+ 768,
3392
+ 768
3393
+ ],
3394
+ "dtype": "Q4_K_M",
3395
+ "role": "matmul",
3396
+ "layout": "row"
3397
+ },
3398
+ "layers.3.self_attn.q_norm.weight": {
3399
+ "shard": 6,
3400
+ "offset": 43609088,
3401
+ "size": 1024,
3402
+ "shape": [
3403
+ 256
3404
+ ],
3405
+ "dtype": "F32",
3406
+ "role": "norm"
3407
+ },
3408
+ "layers.3.self_attn.q_proj.weight": {
3409
+ "shard": 6,
3410
+ "offset": 43610112,
3411
+ "size": 331776,
3412
+ "shape": [
3413
+ 768,
3414
+ 768
3415
+ ],
3416
+ "dtype": "Q4_K_M",
3417
+ "role": "matmul",
3418
+ "layout": "row"
3419
+ },
3420
+ "layers.3.self_attn.v_proj.weight": {
3421
+ "shard": 6,
3422
+ "offset": 43941888,
3423
+ "size": 110592,
3424
+ "shape": [
3425
+ 256,
3426
+ 768
3427
+ ],
3428
+ "dtype": "Q4_K_M",
3429
+ "role": "matmul",
3430
+ "layout": "row"
3431
+ },
3432
+ "layers.4.input_layernorm.weight": {
3433
+ "shard": 6,
3434
+ "offset": 44052480,
3435
+ "size": 3072,
3436
+ "shape": [
3437
+ 768
3438
+ ],
3439
+ "dtype": "F32",
3440
+ "role": "norm"
3441
+ },
3442
+ "layers.4.mlp.down_proj.weight": {
3443
+ "shard": 6,
3444
+ "offset": 44055552,
3445
+ "size": 552960,
3446
+ "shape": [
3447
+ 768,
3448
+ 1152
3449
+ ],
3450
+ "dtype": "Q4_K_M",
3451
+ "role": "matmul",
3452
+ "layout": "row"
3453
+ },
3454
+ "layers.4.mlp.gate_proj.weight": {
3455
+ "shard": 6,
3456
+ "offset": 44608512,
3457
+ "size": 497664,
3458
+ "shape": [
3459
+ 1152,
3460
+ 768
3461
+ ],
3462
+ "dtype": "Q4_K_M",
3463
+ "role": "matmul",
3464
+ "layout": "row"
3465
+ },
3466
+ "layers.4.mlp.up_proj.weight": {
3467
+ "shard": 6,
3468
+ "offset": 45106176,
3469
+ "size": 497664,
3470
+ "shape": [
3471
+ 1152,
3472
+ 768
3473
+ ],
3474
+ "dtype": "Q4_K_M",
3475
+ "role": "matmul",
3476
+ "layout": "row"
3477
+ },
3478
+ "layers.4.post_attention_layernorm.weight": {
3479
+ "shard": 6,
3480
+ "offset": 45603840,
3481
+ "size": 3072,
3482
+ "shape": [
3483
+ 768
3484
+ ],
3485
+ "dtype": "F32",
3486
+ "role": "norm"
3487
+ },
3488
+ "layers.4.post_feedforward_layernorm.weight": {
3489
+ "shard": 6,
3490
+ "offset": 45606912,
3491
+ "size": 3072,
3492
+ "shape": [
3493
+ 768
3494
+ ],
3495
+ "dtype": "F32",
3496
+ "role": "norm"
3497
+ },
3498
+ "layers.4.pre_feedforward_layernorm.weight": {
3499
+ "shard": 6,
3500
+ "offset": 45609984,
3501
+ "size": 3072,
3502
+ "shape": [
3503
+ 768
3504
+ ],
3505
+ "dtype": "F32",
3506
+ "role": "norm"
3507
+ },
3508
+ "layers.4.self_attn.k_norm.weight": {
3509
+ "shard": 6,
3510
+ "offset": 45613056,
3511
+ "size": 1024,
3512
+ "shape": [
3513
+ 256
3514
+ ],
3515
+ "dtype": "F32",
3516
+ "role": "norm"
3517
+ },
3518
+ "layers.4.self_attn.k_proj.weight": {
3519
+ "shard": 6,
3520
+ "offset": 45614080,
3521
+ "size": 110592,
3522
+ "shape": [
3523
+ 256,
3524
+ 768
3525
+ ],
3526
+ "dtype": "Q4_K_M",
3527
+ "role": "matmul",
3528
+ "layout": "row"
3529
+ },
3530
+ "layers.4.self_attn.o_proj.weight": {
3531
+ "shard": 6,
3532
+ "offset": 45724672,
3533
+ "size": 331776,
3534
+ "shape": [
3535
+ 768,
3536
+ 768
3537
+ ],
3538
+ "dtype": "Q4_K_M",
3539
+ "role": "matmul",
3540
+ "layout": "row"
3541
+ },
3542
+ "layers.4.self_attn.q_norm.weight": {
3543
+ "shard": 6,
3544
+ "offset": 46056448,
3545
+ "size": 1024,
3546
+ "shape": [
3547
+ 256
3548
+ ],
3549
+ "dtype": "F32",
3550
+ "role": "norm"
3551
+ },
3552
+ "layers.4.self_attn.q_proj.weight": {
3553
+ "shard": 6,
3554
+ "offset": 46057472,
3555
+ "size": 331776,
3556
+ "shape": [
3557
+ 768,
3558
+ 768
3559
+ ],
3560
+ "dtype": "Q4_K_M",
3561
+ "role": "matmul",
3562
+ "layout": "row"
3563
+ },
3564
+ "layers.4.self_attn.v_proj.weight": {
3565
+ "shard": 6,
3566
+ "offset": 46389248,
3567
+ "size": 110592,
3568
+ "shape": [
3569
+ 256,
3570
+ 768
3571
+ ],
3572
+ "dtype": "Q4_K_M",
3573
+ "role": "matmul",
3574
+ "layout": "row"
3575
+ },
3576
+ "layers.5.input_layernorm.weight": {
3577
+ "shard": 6,
3578
+ "offset": 46499840,
3579
+ "size": 3072,
3580
+ "shape": [
3581
+ 768
3582
+ ],
3583
+ "dtype": "F32",
3584
+ "role": "norm"
3585
+ },
3586
+ "layers.5.mlp.down_proj.weight": {
3587
+ "shard": 6,
3588
+ "offset": 46502912,
3589
+ "size": 552960,
3590
+ "shape": [
3591
+ 768,
3592
+ 1152
3593
+ ],
3594
+ "dtype": "Q4_K_M",
3595
+ "role": "matmul",
3596
+ "layout": "row"
3597
+ },
3598
+ "layers.5.mlp.gate_proj.weight": {
3599
+ "shard": 6,
3600
+ "offset": 47055872,
3601
+ "size": 497664,
3602
+ "shape": [
3603
+ 1152,
3604
+ 768
3605
+ ],
3606
+ "dtype": "Q4_K_M",
3607
+ "role": "matmul",
3608
+ "layout": "row"
3609
+ },
3610
+ "layers.5.mlp.up_proj.weight": {
3611
+ "shard": 6,
3612
+ "offset": 47553536,
3613
+ "size": 497664,
3614
+ "shape": [
3615
+ 1152,
3616
+ 768
3617
+ ],
3618
+ "dtype": "Q4_K_M",
3619
+ "role": "matmul",
3620
+ "layout": "row"
3621
+ },
3622
+ "layers.5.post_attention_layernorm.weight": {
3623
+ "shard": 6,
3624
+ "offset": 48051200,
3625
+ "size": 3072,
3626
+ "shape": [
3627
+ 768
3628
+ ],
3629
+ "dtype": "F32",
3630
+ "role": "norm"
3631
+ },
3632
+ "layers.5.post_feedforward_layernorm.weight": {
3633
+ "shard": 6,
3634
+ "offset": 48054272,
3635
+ "size": 3072,
3636
+ "shape": [
3637
+ 768
3638
+ ],
3639
+ "dtype": "F32",
3640
+ "role": "norm"
3641
+ },
3642
+ "layers.5.pre_feedforward_layernorm.weight": {
3643
+ "shard": 6,
3644
+ "offset": 48057344,
3645
+ "size": 3072,
3646
+ "shape": [
3647
+ 768
3648
+ ],
3649
+ "dtype": "F32",
3650
+ "role": "norm"
3651
+ },
3652
+ "layers.5.self_attn.k_norm.weight": {
3653
+ "shard": 6,
3654
+ "offset": 48060416,
3655
+ "size": 1024,
3656
+ "shape": [
3657
+ 256
3658
+ ],
3659
+ "dtype": "F32",
3660
+ "role": "norm"
3661
+ },
3662
+ "layers.5.self_attn.k_proj.weight": {
3663
+ "shard": 6,
3664
+ "offset": 48061440,
3665
+ "size": 110592,
3666
+ "shape": [
3667
+ 256,
3668
+ 768
3669
+ ],
3670
+ "dtype": "Q4_K_M",
3671
+ "role": "matmul",
3672
+ "layout": "row"
3673
+ },
3674
+ "layers.5.self_attn.o_proj.weight": {
3675
+ "shard": 6,
3676
+ "offset": 48172032,
3677
+ "size": 331776,
3678
+ "shape": [
3679
+ 768,
3680
+ 768
3681
+ ],
3682
+ "dtype": "Q4_K_M",
3683
+ "role": "matmul",
3684
+ "layout": "row"
3685
+ },
3686
+ "layers.5.self_attn.q_norm.weight": {
3687
+ "shard": 6,
3688
+ "offset": 48503808,
3689
+ "size": 1024,
3690
+ "shape": [
3691
+ 256
3692
+ ],
3693
+ "dtype": "F32",
3694
+ "role": "norm"
3695
+ },
3696
+ "layers.5.self_attn.q_proj.weight": {
3697
+ "shard": 6,
3698
+ "offset": 48504832,
3699
+ "size": 331776,
3700
+ "shape": [
3701
+ 768,
3702
+ 768
3703
+ ],
3704
+ "dtype": "Q4_K_M",
3705
+ "role": "matmul",
3706
+ "layout": "row"
3707
+ },
3708
+ "layers.5.self_attn.v_proj.weight": {
3709
+ "shard": 6,
3710
+ "offset": 48836608,
3711
+ "size": 110592,
3712
+ "shape": [
3713
+ 256,
3714
+ 768
3715
+ ],
3716
+ "dtype": "Q4_K_M",
3717
+ "role": "matmul",
3718
+ "layout": "row"
3719
+ },
3720
+ "layers.6.input_layernorm.weight": {
3721
+ "shard": 6,
3722
+ "offset": 48947200,
3723
+ "size": 3072,
3724
+ "shape": [
3725
+ 768
3726
+ ],
3727
+ "dtype": "F32",
3728
+ "role": "norm"
3729
+ },
3730
+ "layers.6.mlp.down_proj.weight": {
3731
+ "shard": 6,
3732
+ "offset": 48950272,
3733
+ "size": 552960,
3734
+ "shape": [
3735
+ 768,
3736
+ 1152
3737
+ ],
3738
+ "dtype": "Q4_K_M",
3739
+ "role": "matmul",
3740
+ "layout": "row"
3741
+ },
3742
+ "layers.6.mlp.gate_proj.weight": {
3743
+ "shard": 6,
3744
+ "offset": 49503232,
3745
+ "size": 497664,
3746
+ "shape": [
3747
+ 1152,
3748
+ 768
3749
+ ],
3750
+ "dtype": "Q4_K_M",
3751
+ "role": "matmul",
3752
+ "layout": "row"
3753
+ },
3754
+ "layers.6.mlp.up_proj.weight": {
3755
+ "shard": 6,
3756
+ "offset": 50000896,
3757
+ "size": 497664,
3758
+ "shape": [
3759
+ 1152,
3760
+ 768
3761
+ ],
3762
+ "dtype": "Q4_K_M",
3763
+ "role": "matmul",
3764
+ "layout": "row"
3765
+ },
3766
+ "layers.6.post_attention_layernorm.weight": {
3767
+ "shard": 6,
3768
+ "offset": 50498560,
3769
+ "size": 3072,
3770
+ "shape": [
3771
+ 768
3772
+ ],
3773
+ "dtype": "F32",
3774
+ "role": "norm"
3775
+ },
3776
+ "layers.6.post_feedforward_layernorm.weight": {
3777
+ "shard": 6,
3778
+ "offset": 50501632,
3779
+ "size": 3072,
3780
+ "shape": [
3781
+ 768
3782
+ ],
3783
+ "dtype": "F32",
3784
+ "role": "norm"
3785
+ },
3786
+ "layers.6.pre_feedforward_layernorm.weight": {
3787
+ "shard": 6,
3788
+ "offset": 50504704,
3789
+ "size": 3072,
3790
+ "shape": [
3791
+ 768
3792
+ ],
3793
+ "dtype": "F32",
3794
+ "role": "norm"
3795
+ },
3796
+ "layers.6.self_attn.k_norm.weight": {
3797
+ "shard": 6,
3798
+ "offset": 50507776,
3799
+ "size": 1024,
3800
+ "shape": [
3801
+ 256
3802
+ ],
3803
+ "dtype": "F32",
3804
+ "role": "norm"
3805
+ },
3806
+ "layers.6.self_attn.k_proj.weight": {
3807
+ "shard": 6,
3808
+ "offset": 50508800,
3809
+ "size": 110592,
3810
+ "shape": [
3811
+ 256,
3812
+ 768
3813
+ ],
3814
+ "dtype": "Q4_K_M",
3815
+ "role": "matmul",
3816
+ "layout": "row"
3817
+ },
3818
+ "layers.6.self_attn.o_proj.weight": {
3819
+ "shard": 6,
3820
+ "offset": 50619392,
3821
+ "size": 331776,
3822
+ "shape": [
3823
+ 768,
3824
+ 768
3825
+ ],
3826
+ "dtype": "Q4_K_M",
3827
+ "role": "matmul",
3828
+ "layout": "row"
3829
+ },
3830
+ "layers.6.self_attn.q_norm.weight": {
3831
+ "shard": 6,
3832
+ "offset": 50951168,
3833
+ "size": 1024,
3834
+ "shape": [
3835
+ 256
3836
+ ],
3837
+ "dtype": "F32",
3838
+ "role": "norm"
3839
+ },
3840
+ "layers.6.self_attn.q_proj.weight": {
3841
+ "shard": 6,
3842
+ "offset": 50952192,
3843
+ "size": 331776,
3844
+ "shape": [
3845
+ 768,
3846
+ 768
3847
+ ],
3848
+ "dtype": "Q4_K_M",
3849
+ "role": "matmul",
3850
+ "layout": "row"
3851
+ },
3852
+ "layers.6.self_attn.v_proj.weight": {
3853
+ "shard": 6,
3854
+ "offset": 51283968,
3855
+ "size": 110592,
3856
+ "shape": [
3857
+ 256,
3858
+ 768
3859
+ ],
3860
+ "dtype": "Q4_K_M",
3861
+ "role": "matmul",
3862
+ "layout": "row"
3863
+ },
3864
+ "layers.7.input_layernorm.weight": {
3865
+ "shard": 6,
3866
+ "offset": 51394560,
3867
+ "size": 3072,
3868
+ "shape": [
3869
+ 768
3870
+ ],
3871
+ "dtype": "F32",
3872
+ "role": "norm"
3873
+ },
3874
+ "layers.7.mlp.down_proj.weight": {
3875
+ "shard": 6,
3876
+ "offset": 51397632,
3877
+ "size": 552960,
3878
+ "shape": [
3879
+ 768,
3880
+ 1152
3881
+ ],
3882
+ "dtype": "Q4_K_M",
3883
+ "role": "matmul",
3884
+ "layout": "row"
3885
+ },
3886
+ "layers.7.mlp.gate_proj.weight": {
3887
+ "shard": 6,
3888
+ "offset": 51950592,
3889
+ "size": 497664,
3890
+ "shape": [
3891
+ 1152,
3892
+ 768
3893
+ ],
3894
+ "dtype": "Q4_K_M",
3895
+ "role": "matmul",
3896
+ "layout": "row"
3897
+ },
3898
+ "layers.7.mlp.up_proj.weight": {
3899
+ "shard": 6,
3900
+ "offset": 52448256,
3901
+ "size": 497664,
3902
+ "shape": [
3903
+ 1152,
3904
+ 768
3905
+ ],
3906
+ "dtype": "Q4_K_M",
3907
+ "role": "matmul",
3908
+ "layout": "row"
3909
+ },
3910
+ "layers.7.post_attention_layernorm.weight": {
3911
+ "shard": 6,
3912
+ "offset": 52945920,
3913
+ "size": 3072,
3914
+ "shape": [
3915
+ 768
3916
+ ],
3917
+ "dtype": "F32",
3918
+ "role": "norm"
3919
+ },
3920
+ "layers.7.post_feedforward_layernorm.weight": {
3921
+ "shard": 6,
3922
+ "offset": 52948992,
3923
+ "size": 3072,
3924
+ "shape": [
3925
+ 768
3926
+ ],
3927
+ "dtype": "F32",
3928
+ "role": "norm"
3929
+ },
3930
+ "layers.7.pre_feedforward_layernorm.weight": {
3931
+ "shard": 6,
3932
+ "offset": 52952064,
3933
+ "size": 3072,
3934
+ "shape": [
3935
+ 768
3936
+ ],
3937
+ "dtype": "F32",
3938
+ "role": "norm"
3939
+ },
3940
+ "layers.7.self_attn.k_norm.weight": {
3941
+ "shard": 6,
3942
+ "offset": 52955136,
3943
+ "size": 1024,
3944
+ "shape": [
3945
+ 256
3946
+ ],
3947
+ "dtype": "F32",
3948
+ "role": "norm"
3949
+ },
3950
+ "layers.7.self_attn.k_proj.weight": {
3951
+ "shard": 6,
3952
+ "offset": 52956160,
3953
+ "size": 110592,
3954
+ "shape": [
3955
+ 256,
3956
+ 768
3957
+ ],
3958
+ "dtype": "Q4_K_M",
3959
+ "role": "matmul",
3960
+ "layout": "row"
3961
+ },
3962
+ "layers.7.self_attn.o_proj.weight": {
3963
+ "shard": 6,
3964
+ "offset": 53066752,
3965
+ "size": 331776,
3966
+ "shape": [
3967
+ 768,
3968
+ 768
3969
+ ],
3970
+ "dtype": "Q4_K_M",
3971
+ "role": "matmul",
3972
+ "layout": "row"
3973
+ },
3974
+ "layers.7.self_attn.q_norm.weight": {
3975
+ "shard": 6,
3976
+ "offset": 53398528,
3977
+ "size": 1024,
3978
+ "shape": [
3979
+ 256
3980
+ ],
3981
+ "dtype": "F32",
3982
+ "role": "norm"
3983
+ },
3984
+ "layers.7.self_attn.q_proj.weight": {
3985
+ "shard": 6,
3986
+ "offset": 53399552,
3987
+ "size": 331776,
3988
+ "shape": [
3989
+ 768,
3990
+ 768
3991
+ ],
3992
+ "dtype": "Q4_K_M",
3993
+ "role": "matmul",
3994
+ "layout": "row"
3995
+ },
3996
+ "layers.7.self_attn.v_proj.weight": {
3997
+ "shard": 6,
3998
+ "offset": 53731328,
3999
+ "size": 110592,
4000
+ "shape": [
4001
+ 256,
4002
+ 768
4003
+ ],
4004
+ "dtype": "Q4_K_M",
4005
+ "role": "matmul",
4006
+ "layout": "row"
4007
+ },
4008
+ "layers.8.input_layernorm.weight": {
4009
+ "shard": 6,
4010
+ "offset": 53841920,
4011
+ "size": 3072,
4012
+ "shape": [
4013
+ 768
4014
+ ],
4015
+ "dtype": "F32",
4016
+ "role": "norm"
4017
+ },
4018
+ "layers.8.mlp.down_proj.weight": {
4019
+ "shard": 6,
4020
+ "offset": 53844992,
4021
+ "size": 552960,
4022
+ "shape": [
4023
+ 768,
4024
+ 1152
4025
+ ],
4026
+ "dtype": "Q4_K_M",
4027
+ "role": "matmul",
4028
+ "layout": "row"
4029
+ },
4030
+ "layers.8.mlp.gate_proj.weight": {
4031
+ "shard": 6,
4032
+ "offset": 54397952,
4033
+ "size": 497664,
4034
+ "shape": [
4035
+ 1152,
4036
+ 768
4037
+ ],
4038
+ "dtype": "Q4_K_M",
4039
+ "role": "matmul",
4040
+ "layout": "row"
4041
+ },
4042
+ "layers.8.mlp.up_proj.weight": {
4043
+ "shard": 6,
4044
+ "offset": 54895616,
4045
+ "size": 497664,
4046
+ "shape": [
4047
+ 1152,
4048
+ 768
4049
+ ],
4050
+ "dtype": "Q4_K_M",
4051
+ "role": "matmul",
4052
+ "layout": "row"
4053
+ },
4054
+ "layers.8.post_attention_layernorm.weight": {
4055
+ "shard": 6,
4056
+ "offset": 55393280,
4057
+ "size": 3072,
4058
+ "shape": [
4059
+ 768
4060
+ ],
4061
+ "dtype": "F32",
4062
+ "role": "norm"
4063
+ },
4064
+ "layers.8.post_feedforward_layernorm.weight": {
4065
+ "shard": 6,
4066
+ "offset": 55396352,
4067
+ "size": 3072,
4068
+ "shape": [
4069
+ 768
4070
+ ],
4071
+ "dtype": "F32",
4072
+ "role": "norm"
4073
+ },
4074
+ "layers.8.pre_feedforward_layernorm.weight": {
4075
+ "shard": 6,
4076
+ "offset": 55399424,
4077
+ "size": 3072,
4078
+ "shape": [
4079
+ 768
4080
+ ],
4081
+ "dtype": "F32",
4082
+ "role": "norm"
4083
+ },
4084
+ "layers.8.self_attn.k_norm.weight": {
4085
+ "shard": 6,
4086
+ "offset": 55402496,
4087
+ "size": 1024,
4088
+ "shape": [
4089
+ 256
4090
+ ],
4091
+ "dtype": "F32",
4092
+ "role": "norm"
4093
+ },
4094
+ "layers.8.self_attn.k_proj.weight": {
4095
+ "shard": 6,
4096
+ "offset": 55403520,
4097
+ "size": 110592,
4098
+ "shape": [
4099
+ 256,
4100
+ 768
4101
+ ],
4102
+ "dtype": "Q4_K_M",
4103
+ "role": "matmul",
4104
+ "layout": "row"
4105
+ },
4106
+ "layers.8.self_attn.o_proj.weight": {
4107
+ "shard": 6,
4108
+ "offset": 55514112,
4109
+ "size": 331776,
4110
+ "shape": [
4111
+ 768,
4112
+ 768
4113
+ ],
4114
+ "dtype": "Q4_K_M",
4115
+ "role": "matmul",
4116
+ "layout": "row"
4117
+ },
4118
+ "layers.8.self_attn.q_norm.weight": {
4119
+ "shard": 6,
4120
+ "offset": 55845888,
4121
+ "size": 1024,
4122
+ "shape": [
4123
+ 256
4124
+ ],
4125
+ "dtype": "F32",
4126
+ "role": "norm"
4127
+ },
4128
+ "layers.8.self_attn.q_proj.weight": {
4129
+ "shard": 6,
4130
+ "offset": 55846912,
4131
+ "size": 331776,
4132
+ "shape": [
4133
+ 768,
4134
+ 768
4135
+ ],
4136
+ "dtype": "Q4_K_M",
4137
+ "role": "matmul",
4138
+ "layout": "row"
4139
+ },
4140
+ "layers.8.self_attn.v_proj.weight": {
4141
+ "shard": 6,
4142
+ "offset": 56178688,
4143
+ "size": 110592,
4144
+ "shape": [
4145
+ 256,
4146
+ 768
4147
+ ],
4148
+ "dtype": "Q4_K_M",
4149
+ "role": "matmul",
4150
+ "layout": "row"
4151
+ },
4152
+ "layers.9.input_layernorm.weight": {
4153
+ "shard": 6,
4154
+ "offset": 56289280,
4155
+ "size": 3072,
4156
+ "shape": [
4157
+ 768
4158
+ ],
4159
+ "dtype": "F32",
4160
+ "role": "norm"
4161
+ },
4162
+ "layers.9.mlp.down_proj.weight": {
4163
+ "shard": 6,
4164
+ "offset": 56292352,
4165
+ "size": 552960,
4166
+ "shape": [
4167
+ 768,
4168
+ 1152
4169
+ ],
4170
+ "dtype": "Q4_K_M",
4171
+ "role": "matmul",
4172
+ "layout": "row"
4173
+ },
4174
+ "layers.9.mlp.gate_proj.weight": {
4175
+ "shard": 6,
4176
+ "offset": 56845312,
4177
+ "size": 497664,
4178
+ "shape": [
4179
+ 1152,
4180
+ 768
4181
+ ],
4182
+ "dtype": "Q4_K_M",
4183
+ "role": "matmul",
4184
+ "layout": "row"
4185
+ },
4186
+ "layers.9.mlp.up_proj.weight": {
4187
+ "shard": 6,
4188
+ "offset": 57342976,
4189
+ "size": 497664,
4190
+ "shape": [
4191
+ 1152,
4192
+ 768
4193
+ ],
4194
+ "dtype": "Q4_K_M",
4195
+ "role": "matmul",
4196
+ "layout": "row"
4197
+ },
4198
+ "layers.9.post_attention_layernorm.weight": {
4199
+ "shard": 6,
4200
+ "offset": 57840640,
4201
+ "size": 3072,
4202
+ "shape": [
4203
+ 768
4204
+ ],
4205
+ "dtype": "F32",
4206
+ "role": "norm"
4207
+ },
4208
+ "layers.9.post_feedforward_layernorm.weight": {
4209
+ "shard": 6,
4210
+ "offset": 57843712,
4211
+ "size": 3072,
4212
+ "shape": [
4213
+ 768
4214
+ ],
4215
+ "dtype": "F32",
4216
+ "role": "norm"
4217
+ },
4218
+ "layers.9.pre_feedforward_layernorm.weight": {
4219
+ "shard": 6,
4220
+ "offset": 57846784,
4221
+ "size": 3072,
4222
+ "shape": [
4223
+ 768
4224
+ ],
4225
+ "dtype": "F32",
4226
+ "role": "norm"
4227
+ },
4228
+ "layers.9.self_attn.k_norm.weight": {
4229
+ "shard": 6,
4230
+ "offset": 57849856,
4231
+ "size": 1024,
4232
+ "shape": [
4233
+ 256
4234
+ ],
4235
+ "dtype": "F32",
4236
+ "role": "norm"
4237
+ },
4238
+ "layers.9.self_attn.k_proj.weight": {
4239
+ "shard": 6,
4240
+ "offset": 57850880,
4241
+ "size": 110592,
4242
+ "shape": [
4243
+ 256,
4244
+ 768
4245
+ ],
4246
+ "dtype": "Q4_K_M",
4247
+ "role": "matmul",
4248
+ "layout": "row"
4249
+ },
4250
+ "layers.9.self_attn.o_proj.weight": {
4251
+ "shard": 6,
4252
+ "offset": 57961472,
4253
+ "size": 331776,
4254
+ "shape": [
4255
+ 768,
4256
+ 768
4257
+ ],
4258
+ "dtype": "Q4_K_M",
4259
+ "role": "matmul",
4260
+ "layout": "row"
4261
+ },
4262
+ "layers.9.self_attn.q_norm.weight": {
4263
+ "shard": 6,
4264
+ "offset": 58293248,
4265
+ "size": 1024,
4266
+ "shape": [
4267
+ 256
4268
+ ],
4269
+ "dtype": "F32",
4270
+ "role": "norm"
4271
+ },
4272
+ "layers.9.self_attn.q_proj.weight": {
4273
+ "shard": 6,
4274
+ "offset": 58294272,
4275
+ "size": 331776,
4276
+ "shape": [
4277
+ 768,
4278
+ 768
4279
+ ],
4280
+ "dtype": "Q4_K_M",
4281
+ "role": "matmul",
4282
+ "layout": "row"
4283
+ },
4284
+ "layers.9.self_attn.v_proj.weight": {
4285
+ "shard": 6,
4286
+ "offset": 58626048,
4287
+ "size": 110592,
4288
+ "shape": [
4289
+ 256,
4290
+ 768
4291
+ ],
4292
+ "dtype": "Q4_K_M",
4293
+ "role": "matmul",
4294
+ "layout": "row"
4295
+ },
4296
+ "norm.weight": {
4297
+ "shard": 6,
4298
+ "offset": 58736640,
4299
+ "size": 3072,
4300
+ "shape": [
4301
+ 768
4302
+ ],
4303
+ "dtype": "F32",
4304
+ "role": "norm"
4305
+ }
4306
+ },
4307
+ "totalSize": 461392896,
4308
+ "hashAlgorithm": "blake3",
4309
+ "eos_token_id": 1,
4310
+ "metadata": {
4311
+ "source": "convert-core",
4312
+ "convertedAt": "2026-02-26T22:42:49.563Z",
4313
+ "hasTokenizer": true
4314
+ },
4315
+ "tokenizer": {
4316
+ "type": "bundled",
4317
+ "vocabSize": 3119,
4318
+ "file": "tokenizer.json"
4319
+ }
4320
+ }
models/google-embeddinggemma-300m-wq4k-ef16/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c0e995aeadfa58597548cd285a3c955775baa020b982fa3b5930b6d733d8d9b
3
+ size 14386429
models/google-embeddinggemma-300m-wq4k-ef16/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
3
+ size 4689074