| [sdq] MoE 層 23 / 共 24 層（dense lead 1）：expert 權重合計 3.90 GiB |
| [sdq] 槽位大小：小的 1327104 B（12 層）／大的 1531904 B（11 層） |
| [sdq] 使用 O_DIRECT 讀取模型檔（1 個 shard，不佔用 kernel page cache） |
| [sdq] I/O 段平行：12 條讀取執行緒 × 640 KB 對齊緩衝（gate/up/down 同時讀，每段再切 1 塊） |
| [sdq] RAM 預算 1.00 GiB：已用(RSS) 0.02 GiB、預留 0.68 GiB → arena 0.30 GiB |
| [sdq] page-map: n_layer=24 n_expert=128 n_expert_used=8 n_embd=1536 n_ff_exp=512 | L1E0 gate=456826720/442368 up=514684768/442368 down=373606240/645120 (off/size) ne0=1536 ne1=512 d_ne0=512 d_ne1=1536 |
| [sdq] expert pager ready: /root/app/ling/tiny/models/Ling-3.0-tiny-Q4_K_M.gguf |
| [sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） |
| [sdq] slot=1.46 MiB (1531904 B) slots=225 arena=0.29 GiB (RAM 實測 RSS=0.31 GiB) |
| [sdq] 槽位分兩種：小 1.27 MiB × 169 + 大 1.46 MiB × 56（平均 1.31 MiB） |
| [sdq] prefetch=off fadvise_dontneed=1 |
| llama_model_loader: loaded meta data with 53 key-value pairs and 526 tensors from /root/app/ling/tiny/models/Ling-3.0-tiny-Q4_K_M.gguf (version GGUF V3 (latest)) |
| llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. |
| llama_model_loader: - kv 0: general.architecture str = bailingmoe3 |
| llama_model_loader: - kv 1: general.type str = model |
| llama_model_loader: - kv 2: general.sampling.top_k i32 = 20 |
| llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000 |
| llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000 |
| llama_model_loader: - kv 5: general.name str = Ling-3.0-tiny |
| llama_model_loader: - kv 6: general.size_label str = 128x1.0B |
| llama_model_loader: - kv 7: general.license str = mit |
| llama_model_loader: - kv 8: general.tags arr[str,1] = ["text-generation"] |
| llama_model_loader: - kv 9: bailingmoe3.block_count u32 = 24 |
| llama_model_loader: - kv 10: bailingmoe3.context_length u32 = 131072 |
| llama_model_loader: - kv 11: bailingmoe3.embedding_length u32 = 1536 |
| llama_model_loader: - kv 12: bailingmoe3.feed_forward_length u32 = 4608 |
| llama_model_loader: - kv 13: bailingmoe3.attention.head_count u32 = 16 |
| llama_model_loader: - kv 14: bailingmoe3.attention.head_count_kv arr[i32,24] = [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, ... |
| llama_model_loader: - kv 15: bailingmoe3.rope.freq_base f32 = 6000000.000000 |
| llama_model_loader: - kv 16: bailingmoe3.attention.layer_norm_rms_epsilon f32 = 0.000001 |
| llama_model_loader: - kv 17: bailingmoe3.expert_count u32 = 128 |
| llama_model_loader: - kv 18: bailingmoe3.expert_used_count u32 = 8 |
| llama_model_loader: - kv 19: bailingmoe3.expert_group_count u32 = 8 |
| llama_model_loader: - kv 20: bailingmoe3.expert_group_used_count u32 = 4 |
| llama_model_loader: - kv 21: bailingmoe3.expert_gating_func u32 = 2 |
| llama_model_loader: - kv 22: bailingmoe3.attention.key_length u32 = 576 |
| llama_model_loader: - kv 23: bailingmoe3.attention.value_length u32 = 128 |
| llama_model_loader: - kv 24: bailingmoe3.vocab_size u32 = 157184 |
| llama_model_loader: - kv 25: bailingmoe3.ssm.conv_kernel u32 = 4 |
| llama_model_loader: - kv 26: bailingmoe3.kda.head_dim u32 = 128 |
| llama_model_loader: - kv 27: bailingmoe3.kda.safe_gate bool = true |
| llama_model_loader: - kv 28: bailingmoe3.kda.gate_lower_bound f32 = -5.000000 |
| llama_model_loader: - kv 29: bailingmoe3.attention.q_lora_rank u32 = 256 |
| llama_model_loader: - kv 30: bailingmoe3.attention.kv_lora_rank u32 = 512 |
| llama_model_loader: - kv 31: bailingmoe3.rope.dimension_count u32 = 64 |
| llama_model_loader: - kv 32: bailingmoe3.attention.key_length_mla u32 = 192 |
| llama_model_loader: - kv 33: bailingmoe3.attention.value_length_mla u32 = 128 |
| llama_model_loader: - kv 34: bailingmoe3.expert_feed_forward_length u32 = 512 |
| llama_model_loader: - kv 35: bailingmoe3.expert_shared_feed_forward_length u32 = 512 |
| llama_model_loader: - kv 36: bailingmoe3.expert_shared_count u32 = 1 |
| llama_model_loader: - kv 37: bailingmoe3.leading_dense_block_count u32 = 1 |
| llama_model_loader: - kv 38: bailingmoe3.expert_weights_scale f32 = 2.500000 |
| llama_model_loader: - kv 39: bailingmoe3.expert_weights_norm bool = true |
| llama_model_loader: - kv 40: tokenizer.ggml.model str = gpt2 |
| llama_model_loader: - kv 41: tokenizer.ggml.pre str = bailingmoe2 |
| llama_model_loader: - kv 42: tokenizer.ggml.tokens arr[str,157184] = ["!", "\"", "#", "$", "%", "&", "'", ... |
| llama_model_loader: - kv 43: tokenizer.ggml.token_type arr[i32,157184] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... |
| llama_model_loader: - kv 44: tokenizer.ggml.merges arr[str,156635] = ["Ġ Ġ", "Ġ t", "i n", "Ġ a", "h e... |
| llama_model_loader: - kv 45: tokenizer.ggml.bos_token_id u32 = 156891 |
| llama_model_loader: - kv 46: tokenizer.ggml.eos_token_id u32 = 156895 |
| llama_model_loader: - kv 47: tokenizer.ggml.padding_token_id u32 = 156892 |
| llama_model_loader: - kv 48: tokenizer.ggml.add_bos_token bool = false |
| llama_model_loader: - kv 49: tokenizer.ggml.add_eos_token bool = false |
| llama_model_loader: - kv 50: tokenizer.chat_template str = {#- Bailing V3 chat template -#}\n{#- ... |
| llama_model_loader: - kv 51: general.quantization_version u32 = 2 |
| llama_model_loader: - kv 52: general.file_type u32 = 15 |
| llama_model_loader: - type f32: 215 tensors |
| llama_model_loader: - type q5_0: 6 tensors |
| llama_model_loader: - type q4_K: 272 tensors |
| llama_model_loader: - type q6_K: 33 tensors |
| print_info: file format = GGUF V3 (latest) |
| print_info: file type = Q4_K - Medium |
| print_info: file size = 4.49 GiB (4.88 BPW) |
| init_tokenizer: initializing tokenizer for type 2 |
| load: 0 unused tokens |
| load: control token: 157151 '<role>' is not marked as EOG |
| load: control token: 157145 '<|reserved_token_250|>' is not marked as EOG |
| load: control token: 157128 '<|reserved_token_233|>' is not marked as EOG |
| load: control token: 157126 '<|reserved_token_231|>' is not marked as EOG |
| load: control token: 157125 '<|reserved_token_230|>' is not marked as EOG |
| load: control token: 157124 '<|reserved_token_229|>' is not marked as EOG |
| load: control token: 157122 '<|reserved_token_227|>' is not marked as EOG |
| load: control token: 157120 '<|reserved_token_225|>' is not marked as EOG |
| load: control token: 157117 '<|reserved_token_222|>' is not marked as EOG |
| load: control token: 157116 '<|reserved_token_221|>' is not marked as EOG |
| load: control token: 157107 '<|reserved_token_212|>' is not marked as EOG |
| load: control token: 157106 '<|reserved_token_211|>' is not marked as EOG |
| load: control token: 157105 '<|reserved_token_210|>' is not marked as EOG |
| load: control token: 157104 '<|reserved_token_209|>' is not marked as EOG |
| load: control token: 157101 '<|reserved_token_206|>' is not marked as EOG |
| load: control token: 157100 '<|reserved_token_205|>' is not marked as EOG |
| load: control token: 157096 '<|reserved_token_201|>' is not marked as EOG |
| load: control token: 157095 '<|reserved_token_200|>' is not marked as EOG |
| load: control token: 157094 '<|reserved_token_199|>' is not marked as EOG |
| load: control token: 157093 '<|reserved_token_198|>' is not marked as EOG |
| load: control token: 157092 '<|reserved_token_197|>' is not marked as EOG |
| load: control token: 157090 '<|reserved_token_195|>' is not marked as EOG |
| load: control token: 157082 '<|reserved_token_187|>' is not marked as EOG |
| load: control token: 157081 '<|reserved_token_186|>' is not marked as EOG |
| load: control token: 157080 '<|reserved_token_185|>' is not marked as EOG |
| load: control token: 157079 '<|reserved_token_184|>' is not marked as EOG |
| load: control token: 157076 '<|reserved_token_181|>' is not marked as EOG |
| load: control token: 157072 '<|reserved_token_177|>' is not marked as EOG |
| load: control token: 157070 '<|reserved_token_175|>' is not marked as EOG |
| load: control token: 157068 '<|reserved_token_173|>' is not marked as EOG |
| load: control token: 157067 '<|reserved_token_172|>' is not marked as EOG |
| load: control token: 157066 '<|reserved_token_171|>' is not marked as EOG |
| load: control token: 157064 '<|reserved_token_169|>' is not marked as EOG |
| load: control token: 157062 '<|reserved_token_167|>' is not marked as EOG |
| load: control token: 157058 '<|reserved_token_163|>' is not marked as EOG |
| load: control token: 157053 '<|reserved_token_158|>' is not marked as EOG |
| load: control token: 157052 '<|reserved_token_157|>' is not marked as EOG |
| load: control token: 157050 '<|reserved_token_155|>' is not marked as EOG |
| load: control token: 157048 '<|reserved_token_153|>' is not marked as EOG |
| load: control token: 157047 '<|reserved_token_152|>' is not marked as EOG |
| load: control token: 157044 '<|reserved_token_149|>' is not marked as EOG |
| load: control token: 157043 '<|reserved_token_148|>' is not marked as EOG |
| load: control token: 157040 '<|reserved_token_145|>' is not marked as EOG |
| load: control token: 157034 '<|reserved_token_139|>' is not marked as EOG |
| load: control token: 157031 '<|reserved_token_136|>' is not marked as EOG |
| load: control token: 157028 '<|reserved_token_133|>' is not marked as EOG |
| load: control token: 157022 '<|reserved_token_127|>' is not marked as EOG |
| load: control token: 157020 '<|reserved_token_125|>' is not marked as EOG |
| load: control token: 157016 '<|reserved_token_121|>' is not marked as EOG |
| load: control token: 157011 '<|reserved_token_116|>' is not marked as EOG |
| load: control token: 157009 '<|reserved_token_114|>' is not marked as EOG |
| load: control token: 157005 '<|reserved_token_110|>' is not marked as EOG |
| load: control token: 157001 '<|reserved_token_106|>' is not marked as EOG |
| load: control token: 156999 '<|reserved_token_104|>' is not marked as EOG |
| load: control token: 156998 '<|reserved_token_103|>' is not marked as EOG |
| load: control token: 156997 '<|reserved_token_102|>' is not marked as EOG |
| load: control token: 156994 '<|reserved_token_99|>' is not marked as EOG |
| load: control token: 156990 '<|reserved_token_95|>' is not marked as EOG |
| load: control token: 156986 '<|reserved_token_91|>' is not marked as EOG |
| load: control token: 156985 '<|reserved_token_90|>' is not marked as EOG |
| load: control token: 156984 '<|reserved_token_89|>' is not marked as EOG |
| load: control token: 156981 '<|reserved_token_86|>' is not marked as EOG |
| load: control token: 156977 '<|reserved_token_82|>' is not marked as EOG |
| load: control token: 156973 '<|reserved_token_78|>' is not marked as EOG |
| load: control token: 156971 '<|reserved_token_76|>' is not marked as EOG |
| load: control token: 156969 '<|reserved_token_74|>' is not marked as EOG |
| load: control token: 156968 '<|reserved_token_73|>' is not marked as EOG |
| load: control token: 156967 '<|reserved_token_72|>' is not marked as EOG |
| load: control token: 156966 '<|reserved_token_71|>' is not marked as EOG |
| load: control token: 156965 '<|reserved_token_70|>' is not marked as EOG |
| load: control token: 156963 '<|reserved_token_68|>' is not marked as EOG |
| load: control token: 156962 '<|reserved_token_67|>' is not marked as EOG |
| load: control token: 156961 '<|reserved_token_66|>' is not marked as EOG |
| load: control token: 156959 '<|reserved_token_64|>' is not marked as EOG |
| load: control token: 156954 '<|reserved_token_59|>' is not marked as EOG |
| load: control token: 156951 '<|reserved_token_56|>' is not marked as EOG |
| load: control token: 156950 '<|reserved_token_55|>' is not marked as EOG |
| load: control token: 156948 '<|reserved_token_53|>' is not marked as EOG |
| load: control token: 156944 '<|reserved_token_49|>' is not marked as EOG |
| load: control token: 156943 '<|reserved_token_48|>' is not marked as EOG |
| load: control token: 156937 '<|reserved_token_42|>' is not marked as EOG |
| load: control token: 156936 '<|reserved_token_41|>' is not marked as EOG |
| load: control token: 156935 '<|reserved_token_40|>' is not marked as EOG |
| load: control token: 156931 '<|reserved_token_36|>' is not marked as EOG |
| load: control token: 156929 '<|reserved_token_34|>' is not marked as EOG |
| load: control token: 156926 '<|reserved_token_31|>' is not marked as EOG |
| load: control token: 156921 '<|reserved_token_26|>' is not marked as EOG |
| load: control token: 156919 '<|reserved_token_24|>' is not marked as EOG |
| load: control token: 156917 '<|reserved_token_22|>' is not marked as EOG |
| load: control token: 156916 '<|reserved_token_21|>' is not marked as EOG |
| load: control token: 156912 '<|reserved_token_17|>' is not marked as EOG |
| load: control token: 156902 '<|fim_end|>' is not marked as EOG |
| load: control token: 156900 '<|fim_start|>' is not marked as EOG |
| load: control token: 156894 '[gMASK]' is not marked as EOG |
| load: control token: 156891 '<|startoftext|>' is not marked as EOG |
| load: control token: 157111 '<|reserved_token_216|>' is not marked as EOG |
| load: control token: 157110 '<|reserved_token_215|>' is not marked as EOG |
| load: control token: 156987 '<|reserved_token_92|>' is not marked as EOG |
| load: control token: 157036 '<|reserved_token_141|>' is not marked as EOG |
| load: control token: 157138 '<|reserved_token_243|>' is not marked as EOG |
| load: control token: 157023 '<|reserved_token_128|>' is not marked as EOG |
| load: control token: 157014 '<|reserved_token_119|>' is not marked as EOG |
| load: control token: 156976 '<|reserved_token_81|>' is not marked as EOG |
| load: control token: 157141 '<|reserved_token_246|>' is not marked as EOG |
| load: control token: 157051 '<|reserved_token_156|>' is not marked as EOG |
| load: control token: 157098 '<|reserved_token_203|>' is not marked as EOG |
| load: control token: 157049 '<|reserved_token_154|>' is not marked as EOG |
| load: control token: 157008 '<|reserved_token_113|>' is not marked as EOG |
| load: control token: 157039 '<|reserved_token_144|>' is not marked as EOG |
| load: control token: 157129 '<|reserved_token_234|>' is not marked as EOG |
| load: control token: 156924 '<|reserved_token_29|>' is not marked as EOG |
| load: control token: 157102 '<|reserved_token_207|>' is not marked as EOG |
| load: control token: 157142 '<|reserved_token_247|>' is not marked as EOG |
| load: control token: 157038 '<|reserved_token_143|>' is not marked as EOG |
| load: control token: 157042 '<|reserved_token_147|>' is not marked as EOG |
| load: control token: 157109 '<|reserved_token_214|>' is not marked as EOG |
| load: control token: 157010 '<|reserved_token_115|>' is not marked as EOG |
| load: control token: 156988 '<|reserved_token_93|>' is not marked as EOG |
| load: control token: 157085 '<|reserved_token_190|>' is not marked as EOG |
| load: control token: 156970 '<|reserved_token_75|>' is not marked as EOG |
| load: control token: 156922 '<|reserved_token_27|>' is not marked as EOG |
| load: control token: 157150 '<|reserved_token_255|>' is not marked as EOG |
| load: control token: 157033 '<|reserved_token_138|>' is not marked as EOG |
| load: control token: 157071 '<|reserved_token_176|>' is not marked as EOG |
| load: control token: 157148 '<|reserved_token_253|>' is not marked as EOG |
| load: control token: 157056 '<|reserved_token_161|>' is not marked as EOG |
| load: control token: 156980 '<|reserved_token_85|>' is not marked as EOG |
| load: control token: 156942 '<|reserved_token_47|>' is not marked as EOG |
| load: control token: 157063 '<|reserved_token_168|>' is not marked as EOG |
| load: control token: 156955 '<|reserved_token_60|>' is not marked as EOG |
| load: control token: 157041 '<|reserved_token_146|>' is not marked as EOG |
| load: control token: 157003 '<|reserved_token_108|>' is not marked as EOG |
| load: control token: 157114 '<|reserved_token_219|>' is not marked as EOG |
| load: control token: 157091 '<|reserved_token_196|>' is not marked as EOG |
| load: control token: 157132 '<|reserved_token_237|>' is not marked as EOG |
| load: control token: 156895 '<|role_end|>' is not marked as EOG |
| load: control token: 157002 '<|reserved_token_107|>' is not marked as EOG |
| load: control token: 156941 '<|reserved_token_46|>' is not marked as EOG |
| load: control token: 156910 '<|reserved_token_15|>' is not marked as EOG |
| load: control token: 156918 '<|reserved_token_23|>' is not marked as EOG |
| load: control token: 157113 '<|reserved_token_218|>' is not marked as EOG |
| load: control token: 157103 '<|reserved_token_208|>' is not marked as EOG |
| load: control token: 156974 '<|reserved_token_79|>' is not marked as EOG |
| load: control token: 156932 '<|reserved_token_37|>' is not marked as EOG |
| load: control token: 157146 '<|reserved_token_251|>' is not marked as EOG |
| load: control token: 157133 '<|reserved_token_238|>' is not marked as EOG |
| load: control token: 157026 '<|reserved_token_131|>' is not marked as EOG |
| load: control token: 157027 '<|reserved_token_132|>' is not marked as EOG |
| load: control token: 157019 '<|reserved_token_124|>' is not marked as EOG |
| load: control token: 157055 '<|reserved_token_160|>' is not marked as EOG |
| load: control token: 157024 '<|reserved_token_129|>' is not marked as EOG |
| load: control token: 157127 '<|reserved_token_232|>' is not marked as EOG |
| load: control token: 156893 '[CLS]' is not marked as EOG |
| load: control token: 157035 '<|reserved_token_140|>' is not marked as EOG |
| load: control token: 156972 '<|reserved_token_77|>' is not marked as EOG |
| load: control token: 157006 '<|reserved_token_111|>' is not marked as EOG |
| load: control token: 157012 '<|reserved_token_117|>' is not marked as EOG |
| load: control token: 157139 '<|reserved_token_244|>' is not marked as EOG |
| load: control token: 157013 '<|reserved_token_118|>' is not marked as EOG |
| load: control token: 157046 '<|reserved_token_151|>' is not marked as EOG |
| load: control token: 157140 '<|reserved_token_245|>' is not marked as EOG |
| load: control token: 156914 '<|reserved_token_19|>' is not marked as EOG |
| load: control token: 157077 '<|reserved_token_182|>' is not marked as EOG |
| load: control token: 156939 '<|reserved_token_44|>' is not marked as EOG |
| load: control token: 156940 '<|reserved_token_45|>' is not marked as EOG |
| load: control token: 156956 '<|reserved_token_61|>' is not marked as EOG |
| load: control token: 156957 '<|reserved_token_62|>' is not marked as EOG |
| load: control token: 156993 '<|reserved_token_98|>' is not marked as EOG |
| load: control token: 156925 '<|reserved_token_30|>' is not marked as EOG |
| load: control token: 157135 '<|reserved_token_240|>' is not marked as EOG |
| load: control token: 156992 '<|reserved_token_97|>' is not marked as EOG |
| load: control token: 157015 '<|reserved_token_120|>' is not marked as EOG |
| load: control token: 157087 '<|reserved_token_192|>' is not marked as EOG |
| load: control token: 157083 '<|reserved_token_188|>' is not marked as EOG |
| load: control token: 157144 '<|reserved_token_249|>' is not marked as EOG |
| load: control token: 156934 '<|reserved_token_39|>' is not marked as EOG |
| load: control token: 157143 '<|reserved_token_248|>' is not marked as EOG |
| load: control token: 156930 '<|reserved_token_35|>' is not marked as EOG |
| load: control token: 157060 '<|reserved_token_165|>' is not marked as EOG |
| load: control token: 157131 '<|reserved_token_236|>' is not marked as EOG |
| load: control token: 156952 '<|reserved_token_57|>' is not marked as EOG |
| load: control token: 157065 '<|reserved_token_170|>' is not marked as EOG |
| load: control token: 156982 '<|reserved_token_87|>' is not marked as EOG |
| load: control token: 157097 '<|reserved_token_202|>' is not marked as EOG |
| load: control token: 157118 '<|reserved_token_223|>' is not marked as EOG |
| load: control token: 156945 '<|reserved_token_50|>' is not marked as EOG |
| load: control token: 156991 '<|reserved_token_96|>' is not marked as EOG |
| load: control token: 157086 '<|reserved_token_191|>' is not marked as EOG |
| load: control token: 157074 '<|reserved_token_179|>' is not marked as EOG |
| load: control token: 156975 '<|reserved_token_80|>' is not marked as EOG |
| load: control token: 156938 '<|reserved_token_43|>' is not marked as EOG |
| load: control token: 156995 '<|reserved_token_100|>' is not marked as EOG |
| load: control token: 156979 '<|reserved_token_84|>' is not marked as EOG |
| load: control token: 157004 '<|reserved_token_109|>' is not marked as EOG |
| load: control token: 157108 '<|reserved_token_213|>' is not marked as EOG |
| load: control token: 156953 '<|reserved_token_58|>' is not marked as EOG |
| load: control token: 157088 '<|reserved_token_193|>' is not marked as EOG |
| load: control token: 156960 '<|reserved_token_65|>' is not marked as EOG |
| load: control token: 156949 '<|reserved_token_54|>' is not marked as EOG |
| load: control token: 156927 '<|reserved_token_32|>' is not marked as EOG |
| load: control token: 157059 '<|reserved_token_164|>' is not marked as EOG |
| load: control token: 157099 '<|reserved_token_204|>' is not marked as EOG |
| load: control token: 157030 '<|reserved_token_135|>' is not marked as EOG |
| load: control token: 157119 '<|reserved_token_224|>' is not marked as EOG |
| load: control token: 157061 '<|reserved_token_166|>' is not marked as EOG |
| load: control token: 156996 '<|reserved_token_101|>' is not marked as EOG |
| load: control token: 156947 '<|reserved_token_52|>' is not marked as EOG |
| load: control token: 157000 '<|reserved_token_105|>' is not marked as EOG |
| load: control token: 157134 '<|reserved_token_239|>' is not marked as EOG |
| load: control token: 156915 '<|reserved_token_20|>' is not marked as EOG |
| load: control token: 157057 '<|reserved_token_162|>' is not marked as EOG |
| load: control token: 157069 '<|reserved_token_174|>' is not marked as EOG |
| load: control token: 156920 '<|reserved_token_25|>' is not marked as EOG |
| load: control token: 157112 '<|reserved_token_217|>' is not marked as EOG |
| load: control token: 156928 '<|reserved_token_33|>' is not marked as EOG |
| load: control token: 157075 '<|reserved_token_180|>' is not marked as EOG |
| load: control token: 156923 '<|reserved_token_28|>' is not marked as EOG |
| load: control token: 156983 '<|reserved_token_88|>' is not marked as EOG |
| load: control token: 157152 '</role>' is not marked as EOG |
| load: control token: 156946 '<|reserved_token_51|>' is not marked as EOG |
| load: control token: 157025 '<|reserved_token_130|>' is not marked as EOG |
| load: control token: 156989 '<|reserved_token_94|>' is not marked as EOG |
| load: control token: 157073 '<|reserved_token_178|>' is not marked as EOG |
| load: control token: 157149 '<|reserved_token_254|>' is not marked as EOG |
| load: control token: 157007 '<|reserved_token_112|>' is not marked as EOG |
| load: control token: 157017 '<|reserved_token_122|>' is not marked as EOG |
| load: control token: 157123 '<|reserved_token_228|>' is not marked as EOG |
| load: control token: 156964 '<|reserved_token_69|>' is not marked as EOG |
| load: control token: 156978 '<|reserved_token_83|>' is not marked as EOG |
| load: control token: 156958 '<|reserved_token_63|>' is not marked as EOG |
| load: control token: 156909 '<|reserved_token_14|>' is not marked as EOG |
| load: control token: 157089 '<|reserved_token_194|>' is not marked as EOG |
| load: control token: 157045 '<|reserved_token_150|>' is not marked as EOG |
| load: control token: 157037 '<|reserved_token_142|>' is not marked as EOG |
| load: control token: 156913 '<|reserved_token_18|>' is not marked as EOG |
| load: control token: 157136 '<|reserved_token_241|>' is not marked as EOG |
| load: control token: 156933 '<|reserved_token_38|>' is not marked as EOG |
| load: control token: 157130 '<|reserved_token_235|>' is not marked as EOG |
| load: control token: 157084 '<|reserved_token_189|>' is not marked as EOG |
| load: control token: 157018 '<|reserved_token_123|>' is not marked as EOG |
| load: control token: 157054 '<|reserved_token_159|>' is not marked as EOG |
| load: control token: 157121 '<|reserved_token_226|>' is not marked as EOG |
| load: control token: 157021 '<|reserved_token_126|>' is not marked as EOG |
| load: control token: 157115 '<|reserved_token_220|>' is not marked as EOG |
| load: control token: 156901 '<|fim_hole|>' is not marked as EOG |
| load: control token: 157137 '<|reserved_token_242|>' is not marked as EOG |
| load: control token: 157147 '<|reserved_token_252|>' is not marked as EOG |
| load: control token: 157029 '<|reserved_token_134|>' is not marked as EOG |
| load: control token: 157032 '<|reserved_token_137|>' is not marked as EOG |
| load: control token: 156911 '<|reserved_token_16|>' is not marked as EOG |
| load: control token: 157078 '<|reserved_token_183|>' is not marked as EOG |
| load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect |
| load: printing all EOG tokens: |
| load: - 156892 ('<|endoftext|>') |
| load: - 156895 ('<|role_end|>') |
| load: special tokens cache size = 262 |
| load: token to piece cache size = 1.0009 MB |
| print_info: arch = bailingmoe3 |
| print_info: vocab_only = 0 |
| print_info: no_alloc = 0 |
| print_info: n_ctx_train = 131072 |
| print_info: n_embd_inp = 1536 |
| print_info: n_embd = 1536 |
| print_info: n_embd_out = 1536 |
| print_info: n_layer = 24 |
| print_info: n_layer_all = 24 |
| print_info: n_head = 16 |
| print_info: n_head_kv = [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1] |
| print_info: n_rot = 64 |
| print_info: n_swa = 0 |
| print_info: is_swa_any = 0 |
| print_info: non_causal_type = 0 |
| print_info: n_embd_head_k = 576 |
| print_info: n_embd_head_v = 128 |
| print_info: n_gqa = [0, 0, 0, 16, 0, 0, 0, 16, 0, 0, 0, 16, 0, 0, 0, 16, 0, 0, 0, 16, 0, 0, 0, 16] |
| print_info: n_embd_k_gqa = [0, 0, 0, 576, 0, 0, 0, 576, 0, 0, 0, 576, 0, 0, 0, 576, 0, 0, 0, 576, 0, 0, 0, 576] |
| print_info: n_embd_v_gqa = [0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128] |
| print_info: f_norm_eps = 0.0e+00 |
| print_info: f_norm_rms_eps = 1.0e-06 |
| print_info: f_clamp_kqv = 0.0e+00 |
| print_info: f_max_alibi_bias = 0.0e+00 |
| print_info: f_logit_scale = 0.0e+00 |
| print_info: f_attn_scale = 0.0e+00 |
| print_info: f_attn_value_scale = 0.0000 |
| print_info: n_ff = 4608 |
| print_info: n_expert = 128 |
| print_info: n_expert_used = 8 |
| print_info: n_expert_groups = 8 |
| print_info: n_group_used = 4 |
| print_info: causal attn = 1 |
| print_info: pooling type = -1 |
| print_info: rope type = 0 |
| print_info: rope scaling = linear |
| print_info: freq_base_train = 6000000.0 |
| print_info: freq_scale_train = 1 |
| print_info: n_ctx_orig_yarn = 131072 |
| print_info: rope_yarn_log_mul = 0.0000 |
| print_info: rope_finetuned = unknown |
| print_info: model type = 7.9B.A1.3B |
| print_info: model params = 7.89 B |
| print_info: general.name = Ling-3.0-tiny |
| print_info: n_layer_dense_lead = 1 |
| print_info: n_ff_exp = 512 |
| print_info: n_ff_shexp = 512 |
| print_info: n_expert_shared = 1 |
| print_info: expert_weights_scale = 2.5 |
| print_info: expert_weights_norm = 1 |
| print_info: expert_gating_func = sigmoid |
| print_info: n_layer_nextn = 0 |
| print_info: vocab type = BPE |
| print_info: n_vocab = 157184 |
| print_info: n_merges = 156635 |
| print_info: BOS token = 156891 '<|startoftext|>' |
| print_info: EOS token = 156895 '<|role_end|>' |
| print_info: EOT token = 156892 '<|endoftext|>' |
| print_info: PAD token = 156892 '<|endoftext|>' |
| print_info: LF token = 198 'Ċ' |
| print_info: EOG token = 156892 '<|endoftext|>' |
| print_info: EOG token = 156895 '<|role_end|>' |
| print_info: max token length = 154 |
| load_tensors: loading model tensors, this can take a while... (load_mode = mmap) |
| load_tensors: layer 0 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 1 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 2 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 3 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 4 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 5 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 6 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 7 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 8 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 9 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 10 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 11 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 12 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 13 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 14 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 15 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 16 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 17 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 18 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 19 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 20 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 21 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 22 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 23 assigned to device CPU, is_swa = 0 |
| load_tensors: layer 24 assigned to device CPU, is_swa = 0 |
| create_tensor: loading tensor token_embd.weight |
| create_tensor: loading tensor output_norm.weight |
| create_tensor: loading tensor output.weight |
| create_tensor: loading tensor blk.0.attn_norm.weight |
| create_tensor: loading tensor blk.0.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.0.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.0.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.0.attn_qkv.weight |
| create_tensor: loading tensor blk.0.attn_q.weight |
| create_tensor: loading tensor blk.0.attn_k.weight |
| create_tensor: loading tensor blk.0.attn_v.weight |
| create_tensor: loading tensor blk.0.attn_q.bias |
| create_tensor: loading tensor blk.0.attn_k.bias |
| create_tensor: loading tensor blk.0.attn_v.bias |
| create_tensor: loading tensor blk.0.ssm_f_a.weight |
| create_tensor: loading tensor blk.0.ssm_beta.weight |
| create_tensor: loading tensor blk.0.ssm_a |
| create_tensor: loading tensor blk.0.ssm_dt.bias |
| create_tensor: loading tensor blk.0.ssm_g_a.weight |
| create_tensor: loading tensor blk.0.ssm_norm.weight |
| create_tensor: loading tensor blk.0.attn_output.weight |
| create_tensor: loading tensor blk.0.ffn_norm.weight |
| create_tensor: loading tensor blk.0.ffn_gate.weight |
| create_tensor: loading tensor blk.0.ffn_up.weight |
| create_tensor: loading tensor blk.0.ffn_down.weight |
| create_tensor: loading tensor blk.1.attn_norm.weight |
| create_tensor: loading tensor blk.1.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.1.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.1.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.1.attn_qkv.weight |
| create_tensor: loading tensor blk.1.attn_q.weight |
| create_tensor: loading tensor blk.1.attn_k.weight |
| create_tensor: loading tensor blk.1.attn_v.weight |
| create_tensor: loading tensor blk.1.attn_q.bias |
| create_tensor: loading tensor blk.1.attn_k.bias |
| create_tensor: loading tensor blk.1.attn_v.bias |
| create_tensor: loading tensor blk.1.ssm_f_a.weight |
| create_tensor: loading tensor blk.1.ssm_beta.weight |
| create_tensor: loading tensor blk.1.ssm_a |
| create_tensor: loading tensor blk.1.ssm_dt.bias |
| create_tensor: loading tensor blk.1.ssm_g_a.weight |
| create_tensor: loading tensor blk.1.ssm_norm.weight |
| create_tensor: loading tensor blk.1.attn_output.weight |
| create_tensor: loading tensor blk.1.ffn_norm.weight |
| create_tensor: loading tensor blk.1.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.1.exp_probs_b.bias |
| create_tensor: loading tensor blk.1.ffn_gate_exps.weight |
| tensor blk.1.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.1.ffn_up_exps.weight |
| tensor blk.1.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.1.ffn_down_exps.weight |
| tensor blk.1.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.1.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.1.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.1.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.2.attn_norm.weight |
| create_tensor: loading tensor blk.2.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.2.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.2.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.2.attn_qkv.weight |
| create_tensor: loading tensor blk.2.attn_q.weight |
| create_tensor: loading tensor blk.2.attn_k.weight |
| create_tensor: loading tensor blk.2.attn_v.weight |
| create_tensor: loading tensor blk.2.attn_q.bias |
| create_tensor: loading tensor blk.2.attn_k.bias |
| create_tensor: loading tensor blk.2.attn_v.bias |
| create_tensor: loading tensor blk.2.ssm_f_a.weight |
| create_tensor: loading tensor blk.2.ssm_beta.weight |
| create_tensor: loading tensor blk.2.ssm_a |
| create_tensor: loading tensor blk.2.ssm_dt.bias |
| create_tensor: loading tensor blk.2.ssm_g_a.weight |
| create_tensor: loading tensor blk.2.ssm_norm.weight |
| create_tensor: loading tensor blk.2.attn_output.weight |
| create_tensor: loading tensor blk.2.ffn_norm.weight |
| create_tensor: loading tensor blk.2.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.2.exp_probs_b.bias |
| create_tensor: loading tensor blk.2.ffn_gate_exps.weight |
| tensor blk.2.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.2.ffn_up_exps.weight |
| tensor blk.2.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.2.ffn_down_exps.weight |
| tensor blk.2.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.2.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.2.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.2.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.3.attn_norm.weight |
| create_tensor: loading tensor blk.3.attn_q_a.weight |
| create_tensor: loading tensor blk.3.attn_q_a_norm.weight |
| create_tensor: loading tensor blk.3.attn_q_b.weight |
| create_tensor: loading tensor blk.3.attn_kv_a_mqa.weight |
| create_tensor: loading tensor blk.3.attn_kv_a_norm.weight |
| create_tensor: loading tensor blk.3.attn_k_b.weight |
| create_tensor: loading tensor blk.3.attn_v_b.weight |
| create_tensor: loading tensor blk.3.attn_gate.weight |
| create_tensor: loading tensor blk.3.attn_output.weight |
| create_tensor: loading tensor blk.3.ffn_norm.weight |
| create_tensor: loading tensor blk.3.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.3.exp_probs_b.bias |
| create_tensor: loading tensor blk.3.ffn_gate_exps.weight |
| tensor blk.3.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.3.ffn_up_exps.weight |
| tensor blk.3.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.3.ffn_down_exps.weight |
| tensor blk.3.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.3.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.3.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.3.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.4.attn_norm.weight |
| create_tensor: loading tensor blk.4.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.4.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.4.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.4.attn_qkv.weight |
| create_tensor: loading tensor blk.4.attn_q.weight |
| create_tensor: loading tensor blk.4.attn_k.weight |
| create_tensor: loading tensor blk.4.attn_v.weight |
| create_tensor: loading tensor blk.4.attn_q.bias |
| create_tensor: loading tensor blk.4.attn_k.bias |
| create_tensor: loading tensor blk.4.attn_v.bias |
| create_tensor: loading tensor blk.4.ssm_f_a.weight |
| create_tensor: loading tensor blk.4.ssm_beta.weight |
| create_tensor: loading tensor blk.4.ssm_a |
| create_tensor: loading tensor blk.4.ssm_dt.bias |
| create_tensor: loading tensor blk.4.ssm_g_a.weight |
| create_tensor: loading tensor blk.4.ssm_norm.weight |
| create_tensor: loading tensor blk.4.attn_output.weight |
| create_tensor: loading tensor blk.4.ffn_norm.weight |
| create_tensor: loading tensor blk.4.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.4.exp_probs_b.bias |
| create_tensor: loading tensor blk.4.ffn_gate_exps.weight |
| tensor blk.4.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.4.ffn_up_exps.weight |
| tensor blk.4.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.4.ffn_down_exps.weight |
| tensor blk.4.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.4.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.4.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.4.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.5.attn_norm.weight |
| create_tensor: loading tensor blk.5.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.5.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.5.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.5.attn_qkv.weight |
| create_tensor: loading tensor blk.5.attn_q.weight |
| create_tensor: loading tensor blk.5.attn_k.weight |
| create_tensor: loading tensor blk.5.attn_v.weight |
| create_tensor: loading tensor blk.5.attn_q.bias |
| create_tensor: loading tensor blk.5.attn_k.bias |
| create_tensor: loading tensor blk.5.attn_v.bias |
| create_tensor: loading tensor blk.5.ssm_f_a.weight |
| create_tensor: loading tensor blk.5.ssm_beta.weight |
| create_tensor: loading tensor blk.5.ssm_a |
| create_tensor: loading tensor blk.5.ssm_dt.bias |
| create_tensor: loading tensor blk.5.ssm_g_a.weight |
| create_tensor: loading tensor blk.5.ssm_norm.weight |
| create_tensor: loading tensor blk.5.attn_output.weight |
| create_tensor: loading tensor blk.5.ffn_norm.weight |
| create_tensor: loading tensor blk.5.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.5.exp_probs_b.bias |
| create_tensor: loading tensor blk.5.ffn_gate_exps.weight |
| tensor blk.5.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.5.ffn_up_exps.weight |
| tensor blk.5.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.5.ffn_down_exps.weight |
| tensor blk.5.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.5.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.5.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.5.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.6.attn_norm.weight |
| create_tensor: loading tensor blk.6.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.6.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.6.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.6.attn_qkv.weight |
| create_tensor: loading tensor blk.6.attn_q.weight |
| create_tensor: loading tensor blk.6.attn_k.weight |
| create_tensor: loading tensor blk.6.attn_v.weight |
| create_tensor: loading tensor blk.6.attn_q.bias |
| create_tensor: loading tensor blk.6.attn_k.bias |
| create_tensor: loading tensor blk.6.attn_v.bias |
| create_tensor: loading tensor blk.6.ssm_f_a.weight |
| create_tensor: loading tensor blk.6.ssm_beta.weight |
| create_tensor: loading tensor blk.6.ssm_a |
| create_tensor: loading tensor blk.6.ssm_dt.bias |
| create_tensor: loading tensor blk.6.ssm_g_a.weight |
| create_tensor: loading tensor blk.6.ssm_norm.weight |
| create_tensor: loading tensor blk.6.attn_output.weight |
| create_tensor: loading tensor blk.6.ffn_norm.weight |
| create_tensor: loading tensor blk.6.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.6.exp_probs_b.bias |
| create_tensor: loading tensor blk.6.ffn_gate_exps.weight |
| tensor blk.6.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.6.ffn_up_exps.weight |
| tensor blk.6.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.6.ffn_down_exps.weight |
| tensor blk.6.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.6.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.6.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.6.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.7.attn_norm.weight |
| create_tensor: loading tensor blk.7.attn_q_a.weight |
| create_tensor: loading tensor blk.7.attn_q_a_norm.weight |
| create_tensor: loading tensor blk.7.attn_q_b.weight |
| create_tensor: loading tensor blk.7.attn_kv_a_mqa.weight |
| create_tensor: loading tensor blk.7.attn_kv_a_norm.weight |
| create_tensor: loading tensor blk.7.attn_k_b.weight |
| create_tensor: loading tensor blk.7.attn_v_b.weight |
| create_tensor: loading tensor blk.7.attn_gate.weight |
| create_tensor: loading tensor blk.7.attn_output.weight |
| create_tensor: loading tensor blk.7.ffn_norm.weight |
| create_tensor: loading tensor blk.7.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.7.exp_probs_b.bias |
| create_tensor: loading tensor blk.7.ffn_gate_exps.weight |
| tensor blk.7.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.7.ffn_up_exps.weight |
| tensor blk.7.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.7.ffn_down_exps.weight |
| tensor blk.7.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.7.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.7.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.7.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.8.attn_norm.weight |
| create_tensor: loading tensor blk.8.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.8.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.8.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.8.attn_qkv.weight |
| create_tensor: loading tensor blk.8.attn_q.weight |
| create_tensor: loading tensor blk.8.attn_k.weight |
| create_tensor: loading tensor blk.8.attn_v.weight |
| create_tensor: loading tensor blk.8.attn_q.bias |
| create_tensor: loading tensor blk.8.attn_k.bias |
| create_tensor: loading tensor blk.8.attn_v.bias |
| create_tensor: loading tensor blk.8.ssm_f_a.weight |
| create_tensor: loading tensor blk.8.ssm_beta.weight |
| create_tensor: loading tensor blk.8.ssm_a |
| create_tensor: loading tensor blk.8.ssm_dt.bias |
| create_tensor: loading tensor blk.8.ssm_g_a.weight |
| create_tensor: loading tensor blk.8.ssm_norm.weight |
| create_tensor: loading tensor blk.8.attn_output.weight |
| create_tensor: loading tensor blk.8.ffn_norm.weight |
| create_tensor: loading tensor blk.8.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.8.exp_probs_b.bias |
| create_tensor: loading tensor blk.8.ffn_gate_exps.weight |
| tensor blk.8.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.8.ffn_up_exps.weight |
| tensor blk.8.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.8.ffn_down_exps.weight |
| tensor blk.8.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.8.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.8.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.8.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.9.attn_norm.weight |
| create_tensor: loading tensor blk.9.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.9.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.9.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.9.attn_qkv.weight |
| create_tensor: loading tensor blk.9.attn_q.weight |
| create_tensor: loading tensor blk.9.attn_k.weight |
| create_tensor: loading tensor blk.9.attn_v.weight |
| create_tensor: loading tensor blk.9.attn_q.bias |
| create_tensor: loading tensor blk.9.attn_k.bias |
| create_tensor: loading tensor blk.9.attn_v.bias |
| create_tensor: loading tensor blk.9.ssm_f_a.weight |
| create_tensor: loading tensor blk.9.ssm_beta.weight |
| create_tensor: loading tensor blk.9.ssm_a |
| create_tensor: loading tensor blk.9.ssm_dt.bias |
| create_tensor: loading tensor blk.9.ssm_g_a.weight |
| create_tensor: loading tensor blk.9.ssm_norm.weight |
| create_tensor: loading tensor blk.9.attn_output.weight |
| create_tensor: loading tensor blk.9.ffn_norm.weight |
| create_tensor: loading tensor blk.9.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.9.exp_probs_b.bias |
| create_tensor: loading tensor blk.9.ffn_gate_exps.weight |
| tensor blk.9.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.9.ffn_up_exps.weight |
| tensor blk.9.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.9.ffn_down_exps.weight |
| tensor blk.9.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.9.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.9.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.9.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.10.attn_norm.weight |
| create_tensor: loading tensor blk.10.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.10.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.10.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.10.attn_qkv.weight |
| create_tensor: loading tensor blk.10.attn_q.weight |
| create_tensor: loading tensor blk.10.attn_k.weight |
| create_tensor: loading tensor blk.10.attn_v.weight |
| create_tensor: loading tensor blk.10.attn_q.bias |
| create_tensor: loading tensor blk.10.attn_k.bias |
| create_tensor: loading tensor blk.10.attn_v.bias |
| create_tensor: loading tensor blk.10.ssm_f_a.weight |
| create_tensor: loading tensor blk.10.ssm_beta.weight |
| create_tensor: loading tensor blk.10.ssm_a |
| create_tensor: loading tensor blk.10.ssm_dt.bias |
| create_tensor: loading tensor blk.10.ssm_g_a.weight |
| create_tensor: loading tensor blk.10.ssm_norm.weight |
| create_tensor: loading tensor blk.10.attn_output.weight |
| create_tensor: loading tensor blk.10.ffn_norm.weight |
| create_tensor: loading tensor blk.10.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.10.exp_probs_b.bias |
| create_tensor: loading tensor blk.10.ffn_gate_exps.weight |
| tensor blk.10.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.10.ffn_up_exps.weight |
| tensor blk.10.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.10.ffn_down_exps.weight |
| tensor blk.10.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.10.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.10.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.10.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.11.attn_norm.weight |
| create_tensor: loading tensor blk.11.attn_q_a.weight |
| create_tensor: loading tensor blk.11.attn_q_a_norm.weight |
| create_tensor: loading tensor blk.11.attn_q_b.weight |
| create_tensor: loading tensor blk.11.attn_kv_a_mqa.weight |
| create_tensor: loading tensor blk.11.attn_kv_a_norm.weight |
| create_tensor: loading tensor blk.11.attn_k_b.weight |
| create_tensor: loading tensor blk.11.attn_v_b.weight |
| create_tensor: loading tensor blk.11.attn_gate.weight |
| create_tensor: loading tensor blk.11.attn_output.weight |
| create_tensor: loading tensor blk.11.ffn_norm.weight |
| create_tensor: loading tensor blk.11.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.11.exp_probs_b.bias |
| create_tensor: loading tensor blk.11.ffn_gate_exps.weight |
| tensor blk.11.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.11.ffn_up_exps.weight |
| tensor blk.11.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.11.ffn_down_exps.weight |
| tensor blk.11.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.11.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.11.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.11.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.12.attn_norm.weight |
| create_tensor: loading tensor blk.12.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.12.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.12.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.12.attn_qkv.weight |
| create_tensor: loading tensor blk.12.attn_q.weight |
| create_tensor: loading tensor blk.12.attn_k.weight |
| create_tensor: loading tensor blk.12.attn_v.weight |
| create_tensor: loading tensor blk.12.attn_q.bias |
| create_tensor: loading tensor blk.12.attn_k.bias |
| create_tensor: loading tensor blk.12.attn_v.bias |
| create_tensor: loading tensor blk.12.ssm_f_a.weight |
| create_tensor: loading tensor blk.12.ssm_beta.weight |
| create_tensor: loading tensor blk.12.ssm_a |
| create_tensor: loading tensor blk.12.ssm_dt.bias |
| create_tensor: loading tensor blk.12.ssm_g_a.weight |
| create_tensor: loading tensor blk.12.ssm_norm.weight |
| create_tensor: loading tensor blk.12.attn_output.weight |
| create_tensor: loading tensor blk.12.ffn_norm.weight |
| create_tensor: loading tensor blk.12.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.12.exp_probs_b.bias |
| create_tensor: loading tensor blk.12.ffn_gate_exps.weight |
| tensor blk.12.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.12.ffn_up_exps.weight |
| tensor blk.12.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.12.ffn_down_exps.weight |
| tensor blk.12.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.12.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.12.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.12.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.13.attn_norm.weight |
| create_tensor: loading tensor blk.13.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.13.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.13.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.13.attn_qkv.weight |
| create_tensor: loading tensor blk.13.attn_q.weight |
| create_tensor: loading tensor blk.13.attn_k.weight |
| create_tensor: loading tensor blk.13.attn_v.weight |
| create_tensor: loading tensor blk.13.attn_q.bias |
| create_tensor: loading tensor blk.13.attn_k.bias |
| create_tensor: loading tensor blk.13.attn_v.bias |
| create_tensor: loading tensor blk.13.ssm_f_a.weight |
| create_tensor: loading tensor blk.13.ssm_beta.weight |
| create_tensor: loading tensor blk.13.ssm_a |
| create_tensor: loading tensor blk.13.ssm_dt.bias |
| create_tensor: loading tensor blk.13.ssm_g_a.weight |
| create_tensor: loading tensor blk.13.ssm_norm.weight |
| create_tensor: loading tensor blk.13.attn_output.weight |
| create_tensor: loading tensor blk.13.ffn_norm.weight |
| create_tensor: loading tensor blk.13.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.13.exp_probs_b.bias |
| create_tensor: loading tensor blk.13.ffn_gate_exps.weight |
| tensor blk.13.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.13.ffn_up_exps.weight |
| tensor blk.13.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.13.ffn_down_exps.weight |
| tensor blk.13.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.13.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.13.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.13.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.14.attn_norm.weight |
| create_tensor: loading tensor blk.14.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.14.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.14.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.14.attn_qkv.weight |
| create_tensor: loading tensor blk.14.attn_q.weight |
| create_tensor: loading tensor blk.14.attn_k.weight |
| create_tensor: loading tensor blk.14.attn_v.weight |
| create_tensor: loading tensor blk.14.attn_q.bias |
| create_tensor: loading tensor blk.14.attn_k.bias |
| create_tensor: loading tensor blk.14.attn_v.bias |
| create_tensor: loading tensor blk.14.ssm_f_a.weight |
| create_tensor: loading tensor blk.14.ssm_beta.weight |
| create_tensor: loading tensor blk.14.ssm_a |
| create_tensor: loading tensor blk.14.ssm_dt.bias |
| create_tensor: loading tensor blk.14.ssm_g_a.weight |
| create_tensor: loading tensor blk.14.ssm_norm.weight |
| create_tensor: loading tensor blk.14.attn_output.weight |
| create_tensor: loading tensor blk.14.ffn_norm.weight |
| create_tensor: loading tensor blk.14.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.14.exp_probs_b.bias |
| create_tensor: loading tensor blk.14.ffn_gate_exps.weight |
| tensor blk.14.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.14.ffn_up_exps.weight |
| tensor blk.14.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.14.ffn_down_exps.weight |
| tensor blk.14.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.14.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.14.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.14.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.15.attn_norm.weight |
| create_tensor: loading tensor blk.15.attn_q_a.weight |
| create_tensor: loading tensor blk.15.attn_q_a_norm.weight |
| create_tensor: loading tensor blk.15.attn_q_b.weight |
| create_tensor: loading tensor blk.15.attn_kv_a_mqa.weight |
| create_tensor: loading tensor blk.15.attn_kv_a_norm.weight |
| create_tensor: loading tensor blk.15.attn_k_b.weight |
| create_tensor: loading tensor blk.15.attn_v_b.weight |
| create_tensor: loading tensor blk.15.attn_gate.weight |
| create_tensor: loading tensor blk.15.attn_output.weight |
| create_tensor: loading tensor blk.15.ffn_norm.weight |
| create_tensor: loading tensor blk.15.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.15.exp_probs_b.bias |
| create_tensor: loading tensor blk.15.ffn_gate_exps.weight |
| tensor blk.15.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.15.ffn_up_exps.weight |
| tensor blk.15.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.15.ffn_down_exps.weight |
| tensor blk.15.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.15.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.15.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.15.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.16.attn_norm.weight |
| create_tensor: loading tensor blk.16.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.16.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.16.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.16.attn_qkv.weight |
| create_tensor: loading tensor blk.16.attn_q.weight |
| create_tensor: loading tensor blk.16.attn_k.weight |
| create_tensor: loading tensor blk.16.attn_v.weight |
| create_tensor: loading tensor blk.16.attn_q.bias |
| create_tensor: loading tensor blk.16.attn_k.bias |
| create_tensor: loading tensor blk.16.attn_v.bias |
| create_tensor: loading tensor blk.16.ssm_f_a.weight |
| create_tensor: loading tensor blk.16.ssm_beta.weight |
| create_tensor: loading tensor blk.16.ssm_a |
| create_tensor: loading tensor blk.16.ssm_dt.bias |
| create_tensor: loading tensor blk.16.ssm_g_a.weight |
| create_tensor: loading tensor blk.16.ssm_norm.weight |
| create_tensor: loading tensor blk.16.attn_output.weight |
| create_tensor: loading tensor blk.16.ffn_norm.weight |
| create_tensor: loading tensor blk.16.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.16.exp_probs_b.bias |
| create_tensor: loading tensor blk.16.ffn_gate_exps.weight |
| tensor blk.16.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.16.ffn_up_exps.weight |
| tensor blk.16.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.16.ffn_down_exps.weight |
| tensor blk.16.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.16.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.16.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.16.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.17.attn_norm.weight |
| create_tensor: loading tensor blk.17.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.17.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.17.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.17.attn_qkv.weight |
| create_tensor: loading tensor blk.17.attn_q.weight |
| create_tensor: loading tensor blk.17.attn_k.weight |
| create_tensor: loading tensor blk.17.attn_v.weight |
| create_tensor: loading tensor blk.17.attn_q.bias |
| create_tensor: loading tensor blk.17.attn_k.bias |
| create_tensor: loading tensor blk.17.attn_v.bias |
| create_tensor: loading tensor blk.17.ssm_f_a.weight |
| create_tensor: loading tensor blk.17.ssm_beta.weight |
| create_tensor: loading tensor blk.17.ssm_a |
| create_tensor: loading tensor blk.17.ssm_dt.bias |
| create_tensor: loading tensor blk.17.ssm_g_a.weight |
| create_tensor: loading tensor blk.17.ssm_norm.weight |
| create_tensor: loading tensor blk.17.attn_output.weight |
| create_tensor: loading tensor blk.17.ffn_norm.weight |
| create_tensor: loading tensor blk.17.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.17.exp_probs_b.bias |
| create_tensor: loading tensor blk.17.ffn_gate_exps.weight |
| tensor blk.17.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.17.ffn_up_exps.weight |
| tensor blk.17.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.17.ffn_down_exps.weight |
| tensor blk.17.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.17.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.17.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.17.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.18.attn_norm.weight |
| create_tensor: loading tensor blk.18.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.18.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.18.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.18.attn_qkv.weight |
| create_tensor: loading tensor blk.18.attn_q.weight |
| create_tensor: loading tensor blk.18.attn_k.weight |
| create_tensor: loading tensor blk.18.attn_v.weight |
| create_tensor: loading tensor blk.18.attn_q.bias |
| create_tensor: loading tensor blk.18.attn_k.bias |
| create_tensor: loading tensor blk.18.attn_v.bias |
| create_tensor: loading tensor blk.18.ssm_f_a.weight |
| create_tensor: loading tensor blk.18.ssm_beta.weight |
| create_tensor: loading tensor blk.18.ssm_a |
| create_tensor: loading tensor blk.18.ssm_dt.bias |
| create_tensor: loading tensor blk.18.ssm_g_a.weight |
| create_tensor: loading tensor blk.18.ssm_norm.weight |
| create_tensor: loading tensor blk.18.attn_output.weight |
| create_tensor: loading tensor blk.18.ffn_norm.weight |
| create_tensor: loading tensor blk.18.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.18.exp_probs_b.bias |
| create_tensor: loading tensor blk.18.ffn_gate_exps.weight |
| tensor blk.18.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.18.ffn_up_exps.weight |
| tensor blk.18.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.18.ffn_down_exps.weight |
| tensor blk.18.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.18.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.18.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.18.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.19.attn_norm.weight |
| create_tensor: loading tensor blk.19.attn_q_a.weight |
| create_tensor: loading tensor blk.19.attn_q_a_norm.weight |
| create_tensor: loading tensor blk.19.attn_q_b.weight |
| create_tensor: loading tensor blk.19.attn_kv_a_mqa.weight |
| create_tensor: loading tensor blk.19.attn_kv_a_norm.weight |
| create_tensor: loading tensor blk.19.attn_k_b.weight |
| create_tensor: loading tensor blk.19.attn_v_b.weight |
| create_tensor: loading tensor blk.19.attn_gate.weight |
| create_tensor: loading tensor blk.19.attn_output.weight |
| create_tensor: loading tensor blk.19.ffn_norm.weight |
| create_tensor: loading tensor blk.19.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.19.exp_probs_b.bias |
| create_tensor: loading tensor blk.19.ffn_gate_exps.weight |
| tensor blk.19.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.19.ffn_up_exps.weight |
| tensor blk.19.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.19.ffn_down_exps.weight |
| tensor blk.19.ffn_down_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.19.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.19.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.19.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.20.attn_norm.weight |
| create_tensor: loading tensor blk.20.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.20.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.20.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.20.attn_qkv.weight |
| create_tensor: loading tensor blk.20.attn_q.weight |
| create_tensor: loading tensor blk.20.attn_k.weight |
| create_tensor: loading tensor blk.20.attn_v.weight |
| create_tensor: loading tensor blk.20.attn_q.bias |
| create_tensor: loading tensor blk.20.attn_k.bias |
| create_tensor: loading tensor blk.20.attn_v.bias |
| create_tensor: loading tensor blk.20.ssm_f_a.weight |
| create_tensor: loading tensor blk.20.ssm_beta.weight |
| create_tensor: loading tensor blk.20.ssm_a |
| create_tensor: loading tensor blk.20.ssm_dt.bias |
| create_tensor: loading tensor blk.20.ssm_g_a.weight |
| create_tensor: loading tensor blk.20.ssm_norm.weight |
| create_tensor: loading tensor blk.20.attn_output.weight |
| create_tensor: loading tensor blk.20.ffn_norm.weight |
| create_tensor: loading tensor blk.20.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.20.exp_probs_b.bias |
| create_tensor: loading tensor blk.20.ffn_gate_exps.weight |
| tensor blk.20.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.20.ffn_up_exps.weight |
| tensor blk.20.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.20.ffn_down_exps.weight |
| tensor blk.20.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.20.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.20.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.20.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.21.attn_norm.weight |
| create_tensor: loading tensor blk.21.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.21.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.21.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.21.attn_qkv.weight |
| create_tensor: loading tensor blk.21.attn_q.weight |
| create_tensor: loading tensor blk.21.attn_k.weight |
| create_tensor: loading tensor blk.21.attn_v.weight |
| create_tensor: loading tensor blk.21.attn_q.bias |
| create_tensor: loading tensor blk.21.attn_k.bias |
| create_tensor: loading tensor blk.21.attn_v.bias |
| create_tensor: loading tensor blk.21.ssm_f_a.weight |
| create_tensor: loading tensor blk.21.ssm_beta.weight |
| create_tensor: loading tensor blk.21.ssm_a |
| create_tensor: loading tensor blk.21.ssm_dt.bias |
| create_tensor: loading tensor blk.21.ssm_g_a.weight |
| create_tensor: loading tensor blk.21.ssm_norm.weight |
| create_tensor: loading tensor blk.21.attn_output.weight |
| create_tensor: loading tensor blk.21.ffn_norm.weight |
| create_tensor: loading tensor blk.21.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.21.exp_probs_b.bias |
| create_tensor: loading tensor blk.21.ffn_gate_exps.weight |
| tensor blk.21.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.21.ffn_up_exps.weight |
| tensor blk.21.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.21.ffn_down_exps.weight |
| tensor blk.21.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.21.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.21.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.21.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.22.attn_norm.weight |
| create_tensor: loading tensor blk.22.ssm_conv1d_q.weight |
| create_tensor: loading tensor blk.22.ssm_conv1d_k.weight |
| create_tensor: loading tensor blk.22.ssm_conv1d_v.weight |
| create_tensor: loading tensor blk.22.attn_qkv.weight |
| create_tensor: loading tensor blk.22.attn_q.weight |
| create_tensor: loading tensor blk.22.attn_k.weight |
| create_tensor: loading tensor blk.22.attn_v.weight |
| create_tensor: loading tensor blk.22.attn_q.bias |
| create_tensor: loading tensor blk.22.attn_k.bias |
| create_tensor: loading tensor blk.22.attn_v.bias |
| create_tensor: loading tensor blk.22.ssm_f_a.weight |
| create_tensor: loading tensor blk.22.ssm_beta.weight |
| create_tensor: loading tensor blk.22.ssm_a |
| create_tensor: loading tensor blk.22.ssm_dt.bias |
| create_tensor: loading tensor blk.22.ssm_g_a.weight |
| create_tensor: loading tensor blk.22.ssm_norm.weight |
| create_tensor: loading tensor blk.22.attn_output.weight |
| create_tensor: loading tensor blk.22.ffn_norm.weight |
| create_tensor: loading tensor blk.22.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.22.exp_probs_b.bias |
| create_tensor: loading tensor blk.22.ffn_gate_exps.weight |
| tensor blk.22.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.22.ffn_up_exps.weight |
| tensor blk.22.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.22.ffn_down_exps.weight |
| tensor blk.22.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.22.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.22.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.22.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.23.attn_norm.weight |
| create_tensor: loading tensor blk.23.attn_q_a.weight |
| create_tensor: loading tensor blk.23.attn_q_a_norm.weight |
| create_tensor: loading tensor blk.23.attn_q_b.weight |
| create_tensor: loading tensor blk.23.attn_kv_a_mqa.weight |
| create_tensor: loading tensor blk.23.attn_kv_a_norm.weight |
| create_tensor: loading tensor blk.23.attn_k_b.weight |
| create_tensor: loading tensor blk.23.attn_v_b.weight |
| create_tensor: loading tensor blk.23.attn_gate.weight |
| create_tensor: loading tensor blk.23.attn_output.weight |
| create_tensor: loading tensor blk.23.ffn_norm.weight |
| create_tensor: loading tensor blk.23.ffn_gate_inp.weight |
| create_tensor: loading tensor blk.23.exp_probs_b.bias |
| create_tensor: loading tensor blk.23.ffn_gate_exps.weight |
| tensor blk.23.ffn_gate_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.23.ffn_up_exps.weight |
| tensor blk.23.ffn_up_exps.weight (54 MiB q4_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.23.ffn_down_exps.weight |
| tensor blk.23.ffn_down_exps.weight (78 MiB q6_K) buffer type overridden to SDQ_ExpertSSD |
| create_tensor: loading tensor blk.23.ffn_gate_shexp.weight |
| create_tensor: loading tensor blk.23.ffn_up_shexp.weight |
| create_tensor: loading tensor blk.23.ffn_down_shexp.weight |
| create_tensor: loading tensor blk.0.attn_q.scale |
| create_tensor: loading tensor blk.0.attn_k.scale |
| create_tensor: loading tensor blk.0.attn_v.scale |
| create_tensor: loading tensor blk.0.attn_output.scale |
| create_tensor: loading tensor blk.0.ffn_gate.scale |
| create_tensor: loading tensor blk.0.ffn_down.scale |
| create_tensor: loading tensor blk.0.ffn_up.scale |
| create_tensor: loading tensor blk.0.ssm_beta.scale |
| create_tensor: loading tensor blk.0.attn_q.input_scale |
| create_tensor: loading tensor blk.0.attn_k.input_scale |
| create_tensor: loading tensor blk.0.attn_v.input_scale |
| create_tensor: loading tensor blk.0.attn_output.input_scale |
| create_tensor: loading tensor blk.0.ffn_gate.input_scale |
| create_tensor: loading tensor blk.0.ffn_down.input_scale |
| create_tensor: loading tensor blk.0.ffn_up.input_scale |
| create_tensor: loading tensor blk.0.ssm_beta.input_scale |
| create_tensor: loading tensor blk.1.attn_q.scale |
| create_tensor: loading tensor blk.1.attn_k.scale |
| create_tensor: loading tensor blk.1.attn_v.scale |
| create_tensor: loading tensor blk.1.attn_output.scale |
| create_tensor: loading tensor blk.1.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.1.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.1.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.1.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.1.ffn_down_exps.scale |
| create_tensor: loading tensor blk.1.ffn_up_exps.scale |
| create_tensor: loading tensor blk.1.ssm_beta.scale |
| create_tensor: loading tensor blk.1.attn_q.input_scale |
| create_tensor: loading tensor blk.1.attn_k.input_scale |
| create_tensor: loading tensor blk.1.attn_v.input_scale |
| create_tensor: loading tensor blk.1.attn_output.input_scale |
| create_tensor: loading tensor blk.1.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.1.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.1.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.1.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.1.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.1.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.1.ssm_beta.input_scale |
| create_tensor: loading tensor blk.2.attn_q.scale |
| create_tensor: loading tensor blk.2.attn_k.scale |
| create_tensor: loading tensor blk.2.attn_v.scale |
| create_tensor: loading tensor blk.2.attn_output.scale |
| create_tensor: loading tensor blk.2.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.2.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.2.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.2.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.2.ffn_down_exps.scale |
| create_tensor: loading tensor blk.2.ffn_up_exps.scale |
| create_tensor: loading tensor blk.2.ssm_beta.scale |
| create_tensor: loading tensor blk.2.attn_q.input_scale |
| create_tensor: loading tensor blk.2.attn_k.input_scale |
| create_tensor: loading tensor blk.2.attn_v.input_scale |
| create_tensor: loading tensor blk.2.attn_output.input_scale |
| create_tensor: loading tensor blk.2.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.2.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.2.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.2.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.2.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.2.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.2.ssm_beta.input_scale |
| create_tensor: loading tensor blk.3.attn_output.scale |
| create_tensor: loading tensor blk.3.attn_gate.scale |
| create_tensor: loading tensor blk.3.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.3.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.3.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.3.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.3.ffn_down_exps.scale |
| create_tensor: loading tensor blk.3.ffn_up_exps.scale |
| create_tensor: loading tensor blk.3.attn_output.input_scale |
| create_tensor: loading tensor blk.3.attn_gate.input_scale |
| create_tensor: loading tensor blk.3.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.3.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.3.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.3.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.3.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.3.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.4.attn_q.scale |
| create_tensor: loading tensor blk.4.attn_k.scale |
| create_tensor: loading tensor blk.4.attn_v.scale |
| create_tensor: loading tensor blk.4.attn_output.scale |
| create_tensor: loading tensor blk.4.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.4.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.4.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.4.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.4.ffn_down_exps.scale |
| create_tensor: loading tensor blk.4.ffn_up_exps.scale |
| create_tensor: loading tensor blk.4.ssm_beta.scale |
| create_tensor: loading tensor blk.4.attn_q.input_scale |
| create_tensor: loading tensor blk.4.attn_k.input_scale |
| create_tensor: loading tensor blk.4.attn_v.input_scale |
| create_tensor: loading tensor blk.4.attn_output.input_scale |
| create_tensor: loading tensor blk.4.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.4.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.4.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.4.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.4.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.4.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.4.ssm_beta.input_scale |
| create_tensor: loading tensor blk.5.attn_q.scale |
| create_tensor: loading tensor blk.5.attn_k.scale |
| create_tensor: loading tensor blk.5.attn_v.scale |
| create_tensor: loading tensor blk.5.attn_output.scale |
| create_tensor: loading tensor blk.5.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.5.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.5.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.5.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.5.ffn_down_exps.scale |
| create_tensor: loading tensor blk.5.ffn_up_exps.scale |
| create_tensor: loading tensor blk.5.ssm_beta.scale |
| create_tensor: loading tensor blk.5.attn_q.input_scale |
| create_tensor: loading tensor blk.5.attn_k.input_scale |
| create_tensor: loading tensor blk.5.attn_v.input_scale |
| create_tensor: loading tensor blk.5.attn_output.input_scale |
| create_tensor: loading tensor blk.5.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.5.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.5.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.5.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.5.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.5.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.5.ssm_beta.input_scale |
| create_tensor: loading tensor blk.6.attn_q.scale |
| create_tensor: loading tensor blk.6.attn_k.scale |
| create_tensor: loading tensor blk.6.attn_v.scale |
| create_tensor: loading tensor blk.6.attn_output.scale |
| create_tensor: loading tensor blk.6.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.6.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.6.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.6.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.6.ffn_down_exps.scale |
| create_tensor: loading tensor blk.6.ffn_up_exps.scale |
| create_tensor: loading tensor blk.6.ssm_beta.scale |
| create_tensor: loading tensor blk.6.attn_q.input_scale |
| create_tensor: loading tensor blk.6.attn_k.input_scale |
| create_tensor: loading tensor blk.6.attn_v.input_scale |
| create_tensor: loading tensor blk.6.attn_output.input_scale |
| create_tensor: loading tensor blk.6.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.6.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.6.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.6.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.6.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.6.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.6.ssm_beta.input_scale |
| create_tensor: loading tensor blk.7.attn_output.scale |
| create_tensor: loading tensor blk.7.attn_gate.scale |
| create_tensor: loading tensor blk.7.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.7.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.7.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.7.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.7.ffn_down_exps.scale |
| create_tensor: loading tensor blk.7.ffn_up_exps.scale |
| create_tensor: loading tensor blk.7.attn_output.input_scale |
| create_tensor: loading tensor blk.7.attn_gate.input_scale |
| create_tensor: loading tensor blk.7.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.7.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.7.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.7.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.7.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.7.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.8.attn_q.scale |
| create_tensor: loading tensor blk.8.attn_k.scale |
| create_tensor: loading tensor blk.8.attn_v.scale |
| create_tensor: loading tensor blk.8.attn_output.scale |
| create_tensor: loading tensor blk.8.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.8.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.8.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.8.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.8.ffn_down_exps.scale |
| create_tensor: loading tensor blk.8.ffn_up_exps.scale |
| create_tensor: loading tensor blk.8.ssm_beta.scale |
| create_tensor: loading tensor blk.8.attn_q.input_scale |
| create_tensor: loading tensor blk.8.attn_k.input_scale |
| create_tensor: loading tensor blk.8.attn_v.input_scale |
| create_tensor: loading tensor blk.8.attn_output.input_scale |
| create_tensor: loading tensor blk.8.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.8.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.8.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.8.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.8.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.8.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.8.ssm_beta.input_scale |
| create_tensor: loading tensor blk.9.attn_q.scale |
| create_tensor: loading tensor blk.9.attn_k.scale |
| create_tensor: loading tensor blk.9.attn_v.scale |
| create_tensor: loading tensor blk.9.attn_output.scale |
| create_tensor: loading tensor blk.9.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.9.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.9.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.9.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.9.ffn_down_exps.scale |
| create_tensor: loading tensor blk.9.ffn_up_exps.scale |
| create_tensor: loading tensor blk.9.ssm_beta.scale |
| create_tensor: loading tensor blk.9.attn_q.input_scale |
| create_tensor: loading tensor blk.9.attn_k.input_scale |
| create_tensor: loading tensor blk.9.attn_v.input_scale |
| create_tensor: loading tensor blk.9.attn_output.input_scale |
| create_tensor: loading tensor blk.9.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.9.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.9.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.9.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.9.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.9.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.9.ssm_beta.input_scale |
| create_tensor: loading tensor blk.10.attn_q.scale |
| create_tensor: loading tensor blk.10.attn_k.scale |
| create_tensor: loading tensor blk.10.attn_v.scale |
| create_tensor: loading tensor blk.10.attn_output.scale |
| create_tensor: loading tensor blk.10.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.10.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.10.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.10.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.10.ffn_down_exps.scale |
| create_tensor: loading tensor blk.10.ffn_up_exps.scale |
| create_tensor: loading tensor blk.10.ssm_beta.scale |
| create_tensor: loading tensor blk.10.attn_q.input_scale |
| create_tensor: loading tensor blk.10.attn_k.input_scale |
| create_tensor: loading tensor blk.10.attn_v.input_scale |
| create_tensor: loading tensor blk.10.attn_output.input_scale |
| create_tensor: loading tensor blk.10.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.10.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.10.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.10.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.10.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.10.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.10.ssm_beta.input_scale |
| create_tensor: loading tensor blk.11.attn_output.scale |
| create_tensor: loading tensor blk.11.attn_gate.scale |
| create_tensor: loading tensor blk.11.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.11.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.11.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.11.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.11.ffn_down_exps.scale |
| create_tensor: loading tensor blk.11.ffn_up_exps.scale |
| create_tensor: loading tensor blk.11.attn_output.input_scale |
| create_tensor: loading tensor blk.11.attn_gate.input_scale |
| create_tensor: loading tensor blk.11.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.11.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.11.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.11.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.11.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.11.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.12.attn_q.scale |
| create_tensor: loading tensor blk.12.attn_k.scale |
| create_tensor: loading tensor blk.12.attn_v.scale |
| create_tensor: loading tensor blk.12.attn_output.scale |
| create_tensor: loading tensor blk.12.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.12.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.12.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.12.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.12.ffn_down_exps.scale |
| create_tensor: loading tensor blk.12.ffn_up_exps.scale |
| create_tensor: loading tensor blk.12.ssm_beta.scale |
| create_tensor: loading tensor blk.12.attn_q.input_scale |
| create_tensor: loading tensor blk.12.attn_k.input_scale |
| create_tensor: loading tensor blk.12.attn_v.input_scale |
| create_tensor: loading tensor blk.12.attn_output.input_scale |
| create_tensor: loading tensor blk.12.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.12.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.12.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.12.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.12.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.12.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.12.ssm_beta.input_scale |
| create_tensor: loading tensor blk.13.attn_q.scale |
| create_tensor: loading tensor blk.13.attn_k.scale |
| create_tensor: loading tensor blk.13.attn_v.scale |
| create_tensor: loading tensor blk.13.attn_output.scale |
| create_tensor: loading tensor blk.13.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.13.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.13.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.13.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.13.ffn_down_exps.scale |
| create_tensor: loading tensor blk.13.ffn_up_exps.scale |
| create_tensor: loading tensor blk.13.ssm_beta.scale |
| create_tensor: loading tensor blk.13.attn_q.input_scale |
| create_tensor: loading tensor blk.13.attn_k.input_scale |
| create_tensor: loading tensor blk.13.attn_v.input_scale |
| create_tensor: loading tensor blk.13.attn_output.input_scale |
| create_tensor: loading tensor blk.13.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.13.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.13.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.13.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.13.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.13.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.13.ssm_beta.input_scale |
| create_tensor: loading tensor blk.14.attn_q.scale |
| create_tensor: loading tensor blk.14.attn_k.scale |
| create_tensor: loading tensor blk.14.attn_v.scale |
| create_tensor: loading tensor blk.14.attn_output.scale |
| create_tensor: loading tensor blk.14.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.14.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.14.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.14.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.14.ffn_down_exps.scale |
| create_tensor: loading tensor blk.14.ffn_up_exps.scale |
| create_tensor: loading tensor blk.14.ssm_beta.scale |
| create_tensor: loading tensor blk.14.attn_q.input_scale |
| create_tensor: loading tensor blk.14.attn_k.input_scale |
| create_tensor: loading tensor blk.14.attn_v.input_scale |
| create_tensor: loading tensor blk.14.attn_output.input_scale |
| create_tensor: loading tensor blk.14.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.14.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.14.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.14.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.14.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.14.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.14.ssm_beta.input_scale |
| create_tensor: loading tensor blk.15.attn_output.scale |
| create_tensor: loading tensor blk.15.attn_gate.scale |
| create_tensor: loading tensor blk.15.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.15.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.15.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.15.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.15.ffn_down_exps.scale |
| create_tensor: loading tensor blk.15.ffn_up_exps.scale |
| create_tensor: loading tensor blk.15.attn_output.input_scale |
| create_tensor: loading tensor blk.15.attn_gate.input_scale |
| create_tensor: loading tensor blk.15.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.15.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.15.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.15.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.15.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.15.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.16.attn_q.scale |
| create_tensor: loading tensor blk.16.attn_k.scale |
| create_tensor: loading tensor blk.16.attn_v.scale |
| create_tensor: loading tensor blk.16.attn_output.scale |
| create_tensor: loading tensor blk.16.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.16.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.16.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.16.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.16.ffn_down_exps.scale |
| create_tensor: loading tensor blk.16.ffn_up_exps.scale |
| create_tensor: loading tensor blk.16.ssm_beta.scale |
| create_tensor: loading tensor blk.16.attn_q.input_scale |
| create_tensor: loading tensor blk.16.attn_k.input_scale |
| create_tensor: loading tensor blk.16.attn_v.input_scale |
| create_tensor: loading tensor blk.16.attn_output.input_scale |
| create_tensor: loading tensor blk.16.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.16.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.16.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.16.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.16.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.16.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.16.ssm_beta.input_scale |
| create_tensor: loading tensor blk.17.attn_q.scale |
| create_tensor: loading tensor blk.17.attn_k.scale |
| create_tensor: loading tensor blk.17.attn_v.scale |
| create_tensor: loading tensor blk.17.attn_output.scale |
| create_tensor: loading tensor blk.17.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.17.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.17.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.17.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.17.ffn_down_exps.scale |
| create_tensor: loading tensor blk.17.ffn_up_exps.scale |
| create_tensor: loading tensor blk.17.ssm_beta.scale |
| create_tensor: loading tensor blk.17.attn_q.input_scale |
| create_tensor: loading tensor blk.17.attn_k.input_scale |
| create_tensor: loading tensor blk.17.attn_v.input_scale |
| create_tensor: loading tensor blk.17.attn_output.input_scale |
| create_tensor: loading tensor blk.17.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.17.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.17.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.17.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.17.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.17.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.17.ssm_beta.input_scale |
| create_tensor: loading tensor blk.18.attn_q.scale |
| create_tensor: loading tensor blk.18.attn_k.scale |
| create_tensor: loading tensor blk.18.attn_v.scale |
| create_tensor: loading tensor blk.18.attn_output.scale |
| create_tensor: loading tensor blk.18.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.18.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.18.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.18.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.18.ffn_down_exps.scale |
| create_tensor: loading tensor blk.18.ffn_up_exps.scale |
| create_tensor: loading tensor blk.18.ssm_beta.scale |
| create_tensor: loading tensor blk.18.attn_q.input_scale |
| create_tensor: loading tensor blk.18.attn_k.input_scale |
| create_tensor: loading tensor blk.18.attn_v.input_scale |
| create_tensor: loading tensor blk.18.attn_output.input_scale |
| create_tensor: loading tensor blk.18.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.18.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.18.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.18.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.18.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.18.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.18.ssm_beta.input_scale |
| create_tensor: loading tensor blk.19.attn_output.scale |
| create_tensor: loading tensor blk.19.attn_gate.scale |
| create_tensor: loading tensor blk.19.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.19.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.19.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.19.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.19.ffn_down_exps.scale |
| create_tensor: loading tensor blk.19.ffn_up_exps.scale |
| create_tensor: loading tensor blk.19.attn_output.input_scale |
| create_tensor: loading tensor blk.19.attn_gate.input_scale |
| create_tensor: loading tensor blk.19.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.19.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.19.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.19.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.19.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.19.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.20.attn_q.scale |
| create_tensor: loading tensor blk.20.attn_k.scale |
| create_tensor: loading tensor blk.20.attn_v.scale |
| create_tensor: loading tensor blk.20.attn_output.scale |
| create_tensor: loading tensor blk.20.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.20.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.20.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.20.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.20.ffn_down_exps.scale |
| create_tensor: loading tensor blk.20.ffn_up_exps.scale |
| create_tensor: loading tensor blk.20.ssm_beta.scale |
| create_tensor: loading tensor blk.20.attn_q.input_scale |
| create_tensor: loading tensor blk.20.attn_k.input_scale |
| create_tensor: loading tensor blk.20.attn_v.input_scale |
| create_tensor: loading tensor blk.20.attn_output.input_scale |
| create_tensor: loading tensor blk.20.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.20.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.20.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.20.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.20.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.20.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.20.ssm_beta.input_scale |
| create_tensor: loading tensor blk.21.attn_q.scale |
| create_tensor: loading tensor blk.21.attn_k.scale |
| create_tensor: loading tensor blk.21.attn_v.scale |
| create_tensor: loading tensor blk.21.attn_output.scale |
| create_tensor: loading tensor blk.21.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.21.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.21.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.21.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.21.ffn_down_exps.scale |
| create_tensor: loading tensor blk.21.ffn_up_exps.scale |
| create_tensor: loading tensor blk.21.ssm_beta.scale |
| create_tensor: loading tensor blk.21.attn_q.input_scale |
| create_tensor: loading tensor blk.21.attn_k.input_scale |
| create_tensor: loading tensor blk.21.attn_v.input_scale |
| create_tensor: loading tensor blk.21.attn_output.input_scale |
| create_tensor: loading tensor blk.21.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.21.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.21.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.21.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.21.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.21.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.21.ssm_beta.input_scale |
| create_tensor: loading tensor blk.22.attn_q.scale |
| create_tensor: loading tensor blk.22.attn_k.scale |
| create_tensor: loading tensor blk.22.attn_v.scale |
| create_tensor: loading tensor blk.22.attn_output.scale |
| create_tensor: loading tensor blk.22.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.22.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.22.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.22.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.22.ffn_down_exps.scale |
| create_tensor: loading tensor blk.22.ffn_up_exps.scale |
| create_tensor: loading tensor blk.22.ssm_beta.scale |
| create_tensor: loading tensor blk.22.attn_q.input_scale |
| create_tensor: loading tensor blk.22.attn_k.input_scale |
| create_tensor: loading tensor blk.22.attn_v.input_scale |
| create_tensor: loading tensor blk.22.attn_output.input_scale |
| create_tensor: loading tensor blk.22.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.22.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.22.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.22.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.22.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.22.ffn_up_shexp.input_scale |
| create_tensor: loading tensor blk.22.ssm_beta.input_scale |
| create_tensor: loading tensor blk.23.attn_output.scale |
| create_tensor: loading tensor blk.23.attn_gate.scale |
| create_tensor: loading tensor blk.23.ffn_gate_shexp.scale |
| create_tensor: loading tensor blk.23.ffn_down_shexp.scale |
| create_tensor: loading tensor blk.23.ffn_up_shexp.scale |
| create_tensor: loading tensor blk.23.ffn_gate_exps.scale |
| create_tensor: loading tensor blk.23.ffn_down_exps.scale |
| create_tensor: loading tensor blk.23.ffn_up_exps.scale |
| create_tensor: loading tensor blk.23.attn_output.input_scale |
| create_tensor: loading tensor blk.23.attn_gate.input_scale |
| create_tensor: loading tensor blk.23.ffn_gate_exps.input_scale |
| create_tensor: loading tensor blk.23.ffn_down_exps.input_scale |
| create_tensor: loading tensor blk.23.ffn_up_exps.input_scale |
| create_tensor: loading tensor blk.23.ffn_gate_shexp.input_scale |
| create_tensor: loading tensor blk.23.ffn_down_shexp.input_scale |
| create_tensor: loading tensor blk.23.ffn_up_shexp.input_scale |
| done_getting_tensors: tensor 'blk.1.ffn_gate_exps.weight' (q4_K) (and 68 others) cannot be used with preferred buffer type CPU, using SDQ_ExpertSSD instead |
| load_tensors: CPU_Mapped model buffer size = 4594.21 MiB |
| load_tensors: SDQ_ExpertSSD model buffer size = 0.27 MiB |
| ............................................................................. |
| llama_context: constructing llama_context |
| llama_context: n_seq_max = 1 |
| llama_context: n_ctx = 256 |
| llama_context: n_ctx_seq = 256 |
| llama_context: n_batch = 256 |
| llama_context: n_ubatch = 32 |
| llama_context: causal_attn = 1 |
| llama_context: flash_attn = auto |
| llama_context: kv_unified = false |
| llama_context: freq_base = 6000000.0 |
| llama_context: freq_scale = 1 |
| llama_context: n_rs_seq = 0 |
| llama_context: n_outputs_max = 256 |
| llama_context: n_outputs_max_per_seq = 1 |
| llama_context: n_ctx_seq (256) < n_ctx_train (131072) -- the full capacity of the model will not be utilized |
| set_abort_callback: call |
| llama_context: CPU output buffer size = 0.60 MiB |
| llama_kv_cache: layer 0: filtered |
| llama_kv_cache: layer 1: filtered |
| llama_kv_cache: layer 2: filtered |
| llama_kv_cache: layer 3: dev = CPU |
| llama_kv_cache: layer 4: filtered |
| llama_kv_cache: layer 5: filtered |
| llama_kv_cache: layer 6: filtered |
| llama_kv_cache: layer 7: dev = CPU |
| llama_kv_cache: layer 8: filtered |
| llama_kv_cache: layer 9: filtered |
| llama_kv_cache: layer 10: filtered |
| llama_kv_cache: layer 11: dev = CPU |
| llama_kv_cache: layer 12: filtered |
| llama_kv_cache: layer 13: filtered |
| llama_kv_cache: layer 14: filtered |
| llama_kv_cache: layer 15: dev = CPU |
| llama_kv_cache: layer 16: filtered |
| llama_kv_cache: layer 17: filtered |
| llama_kv_cache: layer 18: filtered |
| llama_kv_cache: layer 19: dev = CPU |
| llama_kv_cache: layer 20: filtered |
| llama_kv_cache: layer 21: filtered |
| llama_kv_cache: layer 22: filtered |
| llama_kv_cache: layer 23: dev = CPU |
| llama_kv_cache: CPU KV buffer size = 1.69 MiB |
| llama_kv_cache: size = 1.69 MiB ( 256 cells, 6 layers, 1/1 seqs), K (f16): 1.69 MiB, V (f16): 0.00 MiB |
| llama_kv_cache: n_rot_k = 0, n_embd_head_k_all = 576 |
| llama_kv_cache: n_rot_v = 0, n_embd_head_k_all = 0 |
| llama_memory_recurrent, layer 0: dev = CPU |
| llama_memory_recurrent, layer 1: dev = CPU |
| llama_memory_recurrent, layer 2: dev = CPU |
| llama_memory_recurrent: layer 3: skipped |
| llama_memory_recurrent, layer 4: dev = CPU |
| llama_memory_recurrent, layer 5: dev = CPU |
| llama_memory_recurrent, layer 6: dev = CPU |
| llama_memory_recurrent: layer 7: skipped |
| llama_memory_recurrent, layer 8: dev = CPU |
| llama_memory_recurrent, layer 9: dev = CPU |
| llama_memory_recurrent, layer 10: dev = CPU |
| llama_memory_recurrent: layer 11: skipped |
| llama_memory_recurrent, layer 12: dev = CPU |
| llama_memory_recurrent, layer 13: dev = CPU |
| llama_memory_recurrent, layer 14: dev = CPU |
| llama_memory_recurrent: layer 15: skipped |
| llama_memory_recurrent, layer 16: dev = CPU |
| llama_memory_recurrent, layer 17: dev = CPU |
| llama_memory_recurrent, layer 18: dev = CPU |
| llama_memory_recurrent: layer 19: skipped |
| llama_memory_recurrent, layer 20: dev = CPU |
| llama_memory_recurrent, layer 21: dev = CPU |
| llama_memory_recurrent, layer 22: dev = CPU |
| llama_memory_recurrent: layer 23: skipped |
| llama_memory_recurrent: CPU RS buffer size = 19.27 MiB |
| llama_memory_recurrent: size = 19.27 MiB ( 1 cells, 24 layers, 1 seqs 0 rs_seq), R (f32): 1.27 MiB, S (f32): 18.00 MiB, P (f32): 0.00 MiB |
| llama_context: enumerating backends |
| llama_context: backend_ptrs.size() = 1 |
| sched_reserve: reserving ... |
| sched_reserve: max_nodes = 16832 |
| sched_reserve: reserving full memory module |
| sched_reserve: worst-case: n_tokens = 32, n_seqs = 1, n_outputs = 1 |
| graph_reserve: reserving a graph for ubatch with n_tokens = 1, n_seqs = 1, n_outputs = 1 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2662' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2663' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_361' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_369' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_815' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_823' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1269' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1277' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1723' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1731' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2177' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2185' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2631' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2639' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_222' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_262' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_426' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_466' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_551' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_591' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_676' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_716' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_880' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_920' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1005' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1045' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1130' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1170' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1334' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1374' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1459' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1499' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1584' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1624' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1788' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1828' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1913' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1953' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2038' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2078' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2242' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2282' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2367' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2407' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2492' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2532' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 1, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 1, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_379' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_833' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1287' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1741' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2195' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2649' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| resolve_fused_ops: Flash Attention enabled |
| resolve_fused_ops: resolving fused DeepSeek V4 HC support: |
| graph_reserve: reserving a graph for ubatch with n_tokens = 1, n_seqs = 1, n_outputs = 1 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2662' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2663' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_361' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_369' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_815' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_823' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1269' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1277' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1723' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1731' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2177' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2185' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2631' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2639' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_222' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_262' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_426' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_466' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_551' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_591' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_676' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_716' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_880' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_920' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1005' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1045' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1130' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1170' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1334' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1374' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1459' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1499' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1584' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1624' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1788' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1828' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1913' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1953' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2038' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2078' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2242' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2282' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2367' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2407' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2492' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2532' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 1, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 1, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_379' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_833' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1287' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1741' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2195' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2649' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| resolve_fused_ops: fused DeepSeek V4 HC pre enabled |
| graph_reserve: reserving a graph for ubatch with n_tokens = 1, n_seqs = 1, n_outputs = 1 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2662' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2663' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_361' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_369' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_815' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_823' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1269' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1277' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1723' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1731' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2177' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2185' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2631' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2639' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_222' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_262' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_426' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_466' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_551' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_591' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_676' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_716' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_880' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_920' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1005' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1045' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1130' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1170' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1334' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1374' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1459' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1499' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1584' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1624' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1788' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1828' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1913' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1953' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2038' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2078' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2242' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2282' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2367' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2407' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2492' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2532' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 1, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 1, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_379' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_833' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1287' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1741' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2195' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2649' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| resolve_fused_ops: fused DeepSeek V4 HC comb enabled |
| graph_reserve: reserving a graph for ubatch with n_tokens = 1, n_seqs = 1, n_outputs = 1 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2662' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2663' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_361' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_369' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_815' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_823' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1269' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1277' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1723' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1731' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2177' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2185' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2631' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2639' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_222' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_262' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_426' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_466' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_551' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_591' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_676' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_716' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_880' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_920' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1005' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1045' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1130' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1170' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1334' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1374' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1459' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1499' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1584' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1624' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1788' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1828' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1913' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1953' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2038' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2078' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2242' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2282' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2367' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2407' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2492' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2532' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 1, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 1, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_379' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_833' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1287' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1741' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2195' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2649' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| resolve_fused_ops: fused DeepSeek V4 HC post enabled |
| graph_reserve: reserving a graph for ubatch with n_tokens = 32, n_seqs = 1, n_outputs = 32 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2690' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2691' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_363' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_371' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_822' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_830' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1281' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1289' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1740' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1748' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2199' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2207' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2658' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2666' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_223' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_263' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_430' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_470' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_556' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_596' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_682' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_722' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_889' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_929' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1015' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1055' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1141' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1181' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1348' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1388' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1474' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1514' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1600' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1640' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1807' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1847' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1933' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1973' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2059' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2099' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2266' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2306' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2392' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2432' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2518' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2558' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 32, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 32, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_381' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_840' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_1299' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_1758' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_2217' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_2676' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 32, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| graph_reserve: reserving a graph for ubatch with n_tokens = 1, n_seqs = 1, n_outputs = 1 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2662' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2663' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 1, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_361' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_369' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_815' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_823' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1269' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1277' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1723' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1731' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2177' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2185' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2631' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2639' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_222' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_262' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_426' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_466' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_551' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_591' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_676' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_716' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_880' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_920' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1005' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1045' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1130' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1170' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1334' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1374' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1459' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1499' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1584' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1624' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1788' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1828' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1913' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1953' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2038' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2078' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2242' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2282' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2367' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2407' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2492' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2532' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 1, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 1, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_379' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_833' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1287' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_1741' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2195' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 1, 1, 1 }] is used by node 'node_2649' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 1, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| graph_reserve: reserving a graph for ubatch with n_tokens = 32, n_seqs = 1, n_outputs = 32 |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2690' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' out_ids' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2691' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l3 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l7 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l11 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l15 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l19 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_k_idxs' [i64, ne = { 32, 1, 1, 1 }] is used by node 'cache_k_l23 (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_363' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_371' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_822' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_830' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1281' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1289' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1740' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1748' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2199' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2207' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2658' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' inp_pos' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_2666' (ROPE) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_7' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_41' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_84' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_137' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_223' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_263' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_430' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_470' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_556' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_596' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_682' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_722' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_889' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_929' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1015' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1055' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1141' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1181' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1348' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1388' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1474' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1514' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1600' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1640' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1807' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1847' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1933' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_1973' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2059' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2099' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2266' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2306' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2392' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2432' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2518' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' rs_s_copy' [i32, ne = { 1, 1, 1, 1 }] is used by node 'node_2558' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_slots' [i64, ne = { 32, 1, 1, 1 }] is used by node ' (view)' (SET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_embd' [f32, ne = { 1536, 32, 1, 1 }] is used by node 'node_2' (DUP) |
| llama_graph_n_input_tensors: input tensor ' inp_mixed_tokens' [i32, ne = { 32, 1, 1, 1 }] is used by node 'node_1' (GET_ROWS) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_381' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_840' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_1299' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_1758' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_2217' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' attn_inp_kq_mask' [f16, ne = { 256, 32, 1, 1 }] is used by node 'node_2676' (FLASH_ATTN_EXT) |
| llama_graph_n_input_tensors: input tensor ' inp_tokens' [i32, ne = { 32, 1, 1, 1 }] is used by node 'model.input_embed' (GET_ROWS) |
| sched_reserve: CPU compute buffer size = 20.33 MiB |
| sched_reserve: graph (pp bs=32, tg bs=1): nodes = 2733 / 2704, splits = 1 / 1, input objects = 4 / 4, input tensors = 9 / 9 |
| sched_reserve: reserve took 10.18 ms, sched copies = 1 |
| [sdq] 記憶體（載入 + pager 後）：匿名 RSS 382 MB / 檔案對映 RSS 7 MB / 總 RSS 390 MB / 峰值 390 MB / swap 0 MB |
| [sdq-selftest] L20 E17 n_tok=4 計算完成（gate q4_K / up q4_K / down q6_K） |
| [sdq] page-map: n_layer=24 n_expert=128 n_expert_used=8 n_embd=1536 n_ff_exp=512 | L1E0 gate=456826720/442368 up=514684768/442368 down=373606240/645120 (off/size) ne0=1536 ne1=512 d_ne0=512 d_ne1=1536 |
| [sdq] expert pager ready: /root/app/ling/tiny/models/Ling-3.0-tiny-Q4_K_M.gguf |
| [sdq] 熱門 expert 釘選：0 個（無剖面，用動態 LRU/LFU） |
| [sdq] slot=1.46 MiB (1531904 B) slots=225 arena=0.29 GiB (RAM 實測 RSS=0.38 GiB) |
| [sdq] 槽位分兩種：小 1.27 MiB × 169 + 大 1.46 MiB × 56（平均 1.31 MiB） |
| [sdq] prefetch=off fadvise_dontneed=1 |
| ~llama_context: CPU compute buffer size is 20.3291 MiB, matches expectation of 20.3291 MiB |
| [sdq] 載入模型：/root/app/ling/tiny/models/Ling-3.0-tiny-Q4_K_M.gguf |
| [sdq] ctx=256 threads=4 ubatch=32 RAM(RSS)=390 MB 模型載入 0.5s |
| [sdq] selftest 結果已寫到 results/selftest_l20e17.bin（layer=20 expert=17 seed=1234 n_embd=1536 n_ff=512） |
|
|
