Phase-3 coding-agentic QLoRA checkpoint (merged) + adapter + eval artifacts

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,86 @@
+---
+base_model: WarlordHermes/Magidonia-24B-v4.3-creative-ORPO
+license: apache-2.0
+tags:
+  - qlora
+  - axolotl
+  - mistral
+  - coding
+  - agentic
+  - phase3
+  - reasoning
+language:
+  - en
+---
+
+# Magidonia-24B Coding-Agentic (phase 3 checkpoint)
+
+**This is the phase-3 checkpoint of a multi-phase coding-agentic reasoning fine-tune. It is
+*not* the final release.** A further phase-4 continuation (real agentic tool-use traces) is
+planned/pending as of this upload — check the
+[repo commit history / discussions](https://huggingface.co/groxaxo/Magidonia-24B-coding-agentic-v1)
+for whether a phase-4 revision has since been added under a separate tag.
+
+A QLoRA reasoning adapter trained on
+[`WarlordHermes/Magidonia-24B-v4.3-creative-ORPO`](https://huggingface.co/WarlordHermes/Magidonia-24B-v4.3-creative-ORPO)
+(itself a fine-tune of `TheDrummer/Magidonia-24B-v4.3`, Mistral-Small-24B lineage), merged into
+the base weights. Trained via a chained 2k→4k→8k sequence-length curriculum with a
+coding-agentic-focused dataset mix.
+
+## Training recipe
+
+- Base: `WarlordHermes/Magidonia-24B-v4.3-creative-ORPO` (Mistral-Small-24B, 32q/8kv heads, 40
+  layers, ByteLevel "Tekken" tokenizer)
+- Method: QLoRA (4-bit NF4 double-quant), r=64, alpha=128, rslora, target modules
+  `[q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj]`
+- FSDP2, 3× RTX 3090, `axolotl`
+- 3-phase chained length curriculum: seq_len 2048 → 4096 → 8192 (each phase resumes the
+  previous phase's adapter), 1 epoch, lr 8e-5 cosine, `sample_packing: true`
+- **Dataset mix** (~13.7k rows): `angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k` (~33%),
+  `Roman1111111/claude-sonnet-4.6-120000x` grade≥9.0 filtered (~33%, includes hard
+  systems-programming), `Roman1111111/claude-opus-4.6-10000x` simple-math (~18%),
+  `nohurry/Opus-4.6-Reasoning-3000x-filtered` (~17%),
+  `AlicanKiraz0/Agentic-Chain-of-Thought-Coding-SFT-Dataset-v1.1` (~3%, meta-leaked rows filtered)
+- Reasoning format: literal `<thinking>...</thinking>` tags wrapping the model's reasoning
+  before its final answer, response-only (assistant-only) loss masking
+
+## ⚠️ Important: reasoning-tag convention
+
+This model was trained to emit **literal `<thinking>...</thinking>` tags** as plain text in its
+output. The base Mistral/Magidonia chat template has its own *separate* native convention: it
+injects a `[THINK]...[/THINK]` instruction when the system prompt contains the literal string
+`/think`. **Do not put `/think` in the system prompt when using this model** — doing so mixes
+the template's native instruction with this model's learned literal-tag behavior and will
+produce inconsistent/duplicated reasoning framing. Use plain system prompts.
+
+## Evaluation
+
+**MMLU: 69.9%** (unweighted mean across all 57 standard MMLU subjects, 0-shot, `lm-evaluation-harness`).
+
+Methodology note: multiple-choice loglikelihood tasks like MMLU move very little for a
+reasoning/creative-focused LoRA and are a rough sanity check, not a precise measurement of what
+the fine-tune changed. This number should not be directly compared to Mistral's official
+5-shot benchmark for the base model — different shot count, different quantization (this eval
+ran in 4-bit on a single GPU; naive multi-GPU pipeline-sharded inference was found to corrupt
+results for this model and was avoided), and this is a derivative, not the base.
+
+Qualitative check: clean-process reload + generation on fixed probes shows coherent step-by-step
+`<thinking>` reasoning, correct tokenization/spacing, and no prompt echo.
+
+## License and provenance
+
+- Base model license: **Apache-2.0** (per `WarlordHermes/Magidonia-24B-v4.3-creative-ORPO`'s
+  card metadata).
+- This derivative is released under **Apache-2.0** to match the base model's license metadata
+  where permitted.
+- Training data was drawn from public Hugging Face datasets (listed above). **This is not legal
+  advice** — the upstream terms of each training dataset (several are synthetic/distilled from
+  other models) have not been independently audited by the uploader. Users are responsible for
+  checking dataset/source terms for their own use case before relying on this model commercially.
+
+## Known limitations
+
+- Phase-3 checkpoint only — has not yet seen the planned agentic tool-use continuation data.
+- QLoRA merge; quality profile reflects a 4-bit-trained adapter merged into bf16 weights.
+- MMLU (a general-knowledge benchmark) is not a strong signal for this model's actual target
+  behavior (reasoning-formatted, increasingly coding/agentic-focused generation).

adapter/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "WarlordHermes/Magidonia-24B-v4.3-creative-ORPO",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj"
+  ],
+  "target_parameters": [],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa1480596d0ec5637d9017c73772afddb9fdb11e38754cb86e552f79b5810fa1
+size 1478568312

chat_template.jinja

@@ -0,0 +1,112 @@
+{%- set default_system_message = 'First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.\n\nYour thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.[/THINK]Here, provide a self-contained response.' %}
+ 
+{{- bos_token }}
+ 
+{#- Extract system message if present -#}
+{%- if messages[0]['role'] == 'system' %}
+    {%- if messages[0]['content'] is string %}
+        {%- set raw_system_message = messages[0]['content'] %}
+    {%- else %}
+        {%- set raw_system_message = messages[0]['content'][0]['text'] %}
+    {%- endif %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set raw_system_message = "" %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+ 
+{#- Detect THINK flag by searching for exact phrase "/think" -#}
+{%- if "/think" in raw_system_message %}
+    {%- set THINK = True %}
+{%- else %}
+    {%- set THINK = False %}
+{%- endif %}
+ 
+{#- Apply logic depending on THINK flag -#}
+{%- if THINK %}
+    {%- if raw_system_message|length > 0 %}
+        {%- set system_message = default_system_message + "\n\n" + raw_system_message %}
+    {%- else %}
+        {%- set system_message = default_system_message %}
+    {%- endif %}
+    {{- '[SYSTEM_PROMPT]' + system_message + '[/SYSTEM_PROMPT]' }}
+{%- else %}
+    {%- if raw_system_message|length > 0 %}
+        {{- '[SYSTEM_PROMPT]' + raw_system_message + '[/SYSTEM_PROMPT]' }}
+    {%- endif %}
+{%- endif %}
+ 
+ 
+{#- Tool description appended ONLY to last user message. Edits made by Unsloth #}
+{%- set tools_description = "" %}
+{%- set has_tools = false %}
+ 
+{%- if tools is defined and tools is not none and tools|length > 0 %}
+    {%- set has_tools = true %}
+    {%- set tools_description = "[AVAILABLE_TOOLS]" + (tools | tojson) + "[/AVAILABLE_TOOLS]" %}
+    {{- tools_description }}
+{%- endif %}
+ 
+{%- for message in loop_messages %}
+    {%- if message['role'] == 'user' %}
+ 
+        {%- if message['content'] is string %}
+            {{- '[INST]' + message['content'] + '[/INST]' }}
+        {%- else %}
+            {{- '[INST]' }}
+            {%- for block in message['content'] %}
+                {%- if block['type'] == 'text' %}
+                    {%- if block['text'] is defined %}
+                        {{- block['text'] }}
+                    {%- else %}
+                        {{- block['content'] }}
+                    {%- endif %}
+                {%- elif block['type'] in ['image', 'image_url'] %}
+                    {{- '[IMG]' }}
+                {%- else %}
+                    {{- raise_exception('Only text and image blocks are supported in message content!') }}
+                {%- endif %}
+            {%- endfor %}
+            {{- '[/INST]' }}
+        {%- endif %}
+ 
+    {%- elif message['role'] == 'system' %}
+        {%- if message['content'] is string %}
+            {{- '[SYSTEM_PROMPT]' + message['content'] + '[/SYSTEM_PROMPT]' }}
+        {%- else %}
+            {{- '[SYSTEM_PROMPT]' + message['content'][0]['text'] + '[/SYSTEM_PROMPT]' }}
+        {%- endif %}
+ 
+    {%- elif message['role'] == 'assistant' %}
+        {%- if message['content'] is string %}
+            {{- message['content'] }}
+        {%- elif message['content'] is iterable  %}
+            {{- message['content'][0]['text'] }}
+        {%- endif %}
+ 
+        {%- if message['tool_calls'] is defined and message['tool_calls'] is not none %}
+            {%- for tool in message['tool_calls'] %}
+                {%- set arguments = tool['function']['arguments'] %}
+                {%- if arguments is not string %}
+                    {%- set arguments = arguments|tojson %}
+                {%- endif %}
+                {{- "[TOOL_CALLS]" + tool['function']['name'] + "[ARGS]" + arguments }}
+            {%- endfor %}
+        {%- endif %}
+ 
+        {{- eos_token }}
+ 
+    {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
+        {%- if message.content is defined and message.content.content is defined %}
+            {%- set content = message.content.content %}
+        {%- else %}
+            {%- set content = message.content %}
+        {%- endif %}
+        {{- "[TOOL_RESULTS]" + content|string + "[/TOOL_RESULTS]" }}
+ 
+    {%- else %}
+        {{- raise_exception('Only user, system, assistant and tool roles are supported!') }}
+    {%- endif %}
+{%- endfor %}
+ 
+{#- Licensed under the Apache License, Version 2.0 (the "License") #}

config.json

@@ -0,0 +1,31 @@
+{
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 32768,
+  "max_position_embeddings": 131072,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 8,
+  "pad_token_id": 11,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 1000000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.10.2",
+  "unsloth_version": "2025.12.5",
+  "use_cache": false,
+  "vocab_size": 131072
+}

eval/fixed_probe_outputs.md

@@ -0,0 +1,58 @@
+# Fixed generation probes (merged model, clean process, single-GPU 4-bit)
+
+Tokenizer spacing check (plain `AutoTokenizer.from_pretrained(model_dir)`, no special-casing):
+- `'The quick brown fox'` -> `'The quick brown fox'` OK
+- `'a b c'` -> `'a b c'` OK
+- `'17 * 23 = 391, correct?'` -> `'17 * 23 = 391, correct?'` OK
+
+## Prompt: What is 17 * 23? Show your reasoning.
+```
+<thinking>
+This is a simple multiplication problem. Let me work through it step by step.
+
+17 × 23
+
+I can think of this as:
+- 17 × 20 = 340
+- 17 × 3 = 51
+- 340 + 51 = 391
+
+Alternatively, I can use the distributive property:
+- 17 × 23 = 17 × (20 + 3) = 17×20 + 17×3 = 340 + 51 = 391
+
+Let me verify: 391 ÷ 17 = 23 ✓
+...
+</thinking>
+# 17 × 23 = 391
+```
+`<thinking></thinking>` present: yes. Prompt echo: no.
+
+## Prompt: If a train travels 60 km in 45 minutes, what is its average speed in km/h?
+```
+<thinking>
+Okay, so I'm tackling a basic speed problem: a train travels 60 km in 45 minutes, and I need to
+find the average speed in km/h. ... speed equals distance divided by time ... need to convert
+the time to hours (h) before doing the division ...
+```
+Cut off at max_new_tokens (this is a generation-length artifact, not a formatting bug). Prompt echo: no.
+
+## Prompt: Write a vivid two-sentence story about an old lighthouse keeper.
+```
+<thinking>
+The user wants a vivid two-sentence story about an old lighthouse keeper. The key is to make it
+feel like a complete story with a beginning, middle, and end compressed into two sentences ...
+```
+Cut off at max_new_tokens. Prompt echo: no.
+
+## Prompt: Explain, briefly, why the sky is blue.
+```
+<thinking>
+The user wants a brief explanation of why the sky is blue ... Rayleigh scattering is proportional
+to 1/lambda^4 (blue light is scattered more than red) ...
+</thinking>
+
+Sunlight is white lig[ht...]
+```
+`<thinking></thinking>` present: yes. Prompt echo: no.
+
+All 4 probes: correctly spaced, coherent, no prompt echo, on-topic reasoning.

eval/mmlu_0shot_summary.json

@@ -0,0 +1,65 @@
+{
+  "overall_mmlu_unweighted_mean": 0.6989,
+  "n_subjects": 57,
+  "shots": 0,
+  "methodology": "single-GPU 4-bit inference per subject-group (data-parallel across 3 GPUs); naive multi-GPU pipeline sharding (parallelize=True) was found to corrupt inference for this model and was avoided",
+  "per_subject_acc": {
+    "mmlu_moral_scenarios": 0.34301675977653634,
+    "mmlu_abstract_algebra": 0.35,
+    "mmlu_college_mathematics": 0.39,
+    "mmlu_formal_logic": 0.40476190476190477,
+    "mmlu_high_school_mathematics": 0.4444444444444444,
+    "mmlu_machine_learning": 0.48214285714285715,
+    "mmlu_virology": 0.4939759036144578,
+    "mmlu_professional_law": 0.49608865710560623,
+    "mmlu_econometrics": 0.5087719298245614,
+    "mmlu_high_school_physics": 0.5099337748344371,
+    "mmlu_global_facts": 0.51,
+    "mmlu_college_chemistry": 0.52,
+    "mmlu_college_physics": 0.5294117647058824,
+    "mmlu_professional_accounting": 0.5531914893617021,
+    "mmlu_elementary_mathematics": 0.5873015873015873,
+    "mmlu_college_computer_science": 0.6,
+    "mmlu_high_school_statistics": 0.6157407407407407,
+    "mmlu_high_school_chemistry": 0.6551724137931034,
+    "mmlu_conceptual_physics": 0.6723404255319149,
+    "mmlu_business_ethics": 0.68,
+    "mmlu_college_medicine": 0.6820809248554913,
+    "mmlu_electrical_engineering": 0.696551724137931,
+    "mmlu_human_aging": 0.7040358744394619,
+    "mmlu_computer_security": 0.71,
+    "mmlu_high_school_computer_science": 0.71,
+    "mmlu_professional_psychology": 0.7107843137254902,
+    "mmlu_high_school_macroeconomics": 0.7128205128205128,
+    "mmlu_philosophy": 0.7266881028938906,
+    "mmlu_anatomy": 0.7333333333333333,
+    "mmlu_professional_medicine": 0.7463235294117647,
+    "mmlu_public_relations": 0.7545454545454545,
+    "mmlu_high_school_microeconomics": 0.7605042016806722,
+    "mmlu_clinical_knowledge": 0.769811320754717,
+    "mmlu_security_studies": 0.7714285714285715,
+    "mmlu_moral_disputes": 0.7745664739884393,
+    "mmlu_human_sexuality": 0.7862595419847328,
+    "mmlu_medical_genetics": 0.79,
+    "mmlu_prehistory": 0.7962962962962963,
+    "mmlu_jurisprudence": 0.8148148148148148,
+    "mmlu_logical_fallacies": 0.8159509202453987,
+    "mmlu_high_school_european_history": 0.8181818181818182,
+    "mmlu_nutrition": 0.8300653594771242,
+    "mmlu_international_law": 0.8347107438016529,
+    "mmlu_world_religions": 0.8362573099415205,
+    "mmlu_miscellaneous": 0.8365261813537676,
+    "mmlu_high_school_us_history": 0.8382352941176471,
+    "mmlu_high_school_geography": 0.8383838383838383,
+    "mmlu_college_biology": 0.8402777777777778,
+    "mmlu_astronomy": 0.8421052631578947,
+    "mmlu_high_school_world_history": 0.8481012658227848,
+    "mmlu_high_school_biology": 0.8548387096774194,
+    "mmlu_high_school_psychology": 0.8623853211009175,
+    "mmlu_sociology": 0.8706467661691543,
+    "mmlu_high_school_government_and_politics": 0.8808290155440415,
+    "mmlu_management": 0.883495145631068,
+    "mmlu_marketing": 0.8888888888888888,
+    "mmlu_us_foreign_policy": 0.92
+  }
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 11,
+  "transformers_version": "5.10.2",
+  "use_cache": false
+}

model-00001-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c26214518357eebfbf9745eff4cf006f29c56f0b236541b15e26c8e0d9b72e1
+size 4907389312

model-00002-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:177b3a979f250fe2876506afebc1d4eeed4cbbd1de033cd56ed0f257a21718b4
+size 4781592792

model-00003-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3db9a420483fabbf69a51b590c5cd7c2f5524ed8d5ccc4f3748b8a3a83a30c1f
+size 4781592784

model-00004-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bc56046a013e536bfd679ff8bf20a1779f62585a2e6997de12b1f3d7a31df7d
+size 4886471600

model-00005-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3bf0d7ad1d094ccc4f34aeae52a54e17c5c3517a7369e8e42c5d8b59f28f4b2
+size 4781592832

model-00006-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:297e858406441357926847a8038317478e59f36efb95c0dd0fafd96d041fbcf1
+size 4781592816

model-00007-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfb4cd31032dca7d85817c131bb8b67b19df917a4960fe05e823d6db690ec5f9
+size 4886471600

model-00008-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79f1f3c05e40d9a62cf8705634a3d3f98c53d5c7f193a2f26d887f220bb7709d
+size 4781592832

model-00009-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25c36f2e0088597754ffa7415bc4f981655b0f5b737aa057ce3d203cb80eac2b
+size 4781592816

model-00010-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a78d2f0d4873229444b962d6ef2f332a83b0aed8af8bf8f1974fef7e86e496f3
+size 3774959488

model.safetensors.index.json

@@ -0,0 +1,371 @@
+{
+  "metadata": {
+    "total_parameters": 23572403200,
+    "total_size": 47144806400
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00001-of-00010.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00010.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00010.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "model.layers.36.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.36.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "model.layers.36.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.36.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.36.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.36.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.37.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.38.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.39.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "model.norm.weight": "model-00010-of-00010.safetensors"
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:158915c7c928301e0d32f1da363e45c294d907bd4c64a8e855f0bd1a47b9e870
+size 17078001

tokenizer_config.json

@@ -0,0 +1,18 @@
+{
+  "add_prefix_space": null,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": true,
+  "legacy": true,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "processor_class": "PixtralProcessor",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}