diff --git a/DeepSeek_V4.pdf b/DeepSeek_V4.pdf index d230d3fcaa804eb261f7776c2ff9b6c46f9fbca2..3c5b511f3a753dfb2c48154a61edb7d183109f70 100644 --- a/DeepSeek_V4.pdf +++ b/DeepSeek_V4.pdf @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:f4cbe4fcbd2888b25b2890a98cc6ef4ce0489df7c93e140b6f853c451d3f5c52 -size 4479907 +oid sha256:8a03dadae71894de1515da33e296f0df1dbeed3e7f4bf0badd087f9af77f29e9 +size 4480407 diff --git a/README.md b/README.md index 44c09cde9f73a04dd591f8de55cbd67f4ef31a78..87521b09505a221a44d8bfa824c848b00d9cfcf0 100644 --- a/README.md +++ b/README.md @@ -1,57 +1,240 @@ --- license: mit library_name: transformers -base_model: -- deepseek-ai/DeepSeek-V4-Flash --- - - -This model is an int4 model with group_size 128 of [deepseek-ai/DeepSeek-V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) generated by [intel/auto-round](https://github.com/intel/auto-round) with RTN mode. Please follow the license of the original model. - - +# DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence + + + + + +
+ DeepSeek-V4 +
+
+
+ + Homepage + + + Chat + +
+
+ + Hugging Face + + + Twitter Follow + +
+
+ + License + +
+ +

+ Technical Report👁️ +

+ +## Introduction + +We present a preview version of **DeepSeek-V4** series, including two strong Mixture-of-Experts (MoE) language models — **DeepSeek-V4-Pro** with 1.6T parameters (49B activated) and **DeepSeek-V4-Flash** with 284B parameters (13B activated) — both supporting a context length of **one million tokens**. + +DeepSeek-V4 series incorporate several key upgrades in architecture and optimization: + +1. **Hybrid Attention Architecture:** We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only **27% of single-token inference FLOPs** and **10% of KV cache** compared with DeepSeek-V3.2. +2. **Manifold-Constrained Hyper-Connections (mHC):** We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity. +3. **Muon Optimizer:** We employ the Muon optimizer for faster convergence and greater training stability. + +We pre-train both models on more than **32T** diverse and high-quality tokens, followed by a comprehensive post-training pipeline. The post-training features a two-stage paradigm: independent cultivation of domain-specific experts (through SFT and RL with GRPO), followed by unified model consolidation via on-policy distillation, integrating distinct proficiencies across diverse domains into a single model. + +**DeepSeek-V4-Pro-Max**, the maximum reasoning effort mode of DeepSeek-V4-Pro, significantly advances the knowledge capabilities of open-source models, firmly establishing itself as the best open-source model available today. It achieves top-tier performance in coding benchmarks and significantly bridges the gap with leading closed-source models on reasoning and agentic tasks. Meanwhile, **DeepSeek-V4-Flash-Max** achieves comparable reasoning performance to the Pro version when given a larger thinking budget, though its smaller parameter scale naturally places it slightly behind on pure knowledge tasks and the most complex agentic workflows. + +
+ +
+ +## Model Downloads + +
+ +| **Model** | **#Total Params** | **#Activated Params** | **Context Length** | **Precision** | **Download** | +| :---: | :---: | :---: | :---: | :---: | :---: | +| DeepSeek-V4-Flash-Base | 284B | 13B | 1M | FP8 Mixed | [HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Base) \| [ModelScope](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Flash-Base) | +| DeepSeek-V4-Flash | 284B | 13B | 1M | FP4 + FP8 Mixed* | [HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) \| [ModelScope](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Flash) | +| DeepSeek-V4-Pro-Base | 1.6T | 49B | 1M | FP8 Mixed | [HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-Base) \| [ModelScope](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Pro-Base) | +| DeepSeek-V4-Pro | 1.6T | 49B | 1M | FP4 + FP8 Mixed* | [HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) \| [ModelScope](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Pro) | + +
+ +*\*FP4 + FP8 Mixed: MoE expert parameters use FP4 precision; most other parameters use FP8.* + +## Evaluation Results + +### Base Model + +
+ +| Benchmark (Metric) | # Shots | DeepSeek-V3.2-Base | DeepSeek-V4-Flash-Base | DeepSeek-V4-Pro-Base | +| :--- | :---: | :---: | :---: | :---: | +| Architecture | - | MoE | MoE | MoE | +| # Activated Params | - | 37B | 13B | 49B | +| # Total Params | - | 671B | 284B | 1.6T | +| **World Knowledge** | | | | | +| AGIEval (EM) | 0-shot | 80.1 | 82.6 | **83.1** | +| MMLU (EM) | 5-shot | 87.8 | 88.7 | **90.1** | +| MMLU-Redux (EM) | 5-shot | 87.5 | 89.4 | **90.8** | +| MMLU-Pro (EM) | 5-shot | 65.5 | 68.3 | **73.5** | +| MMMLU (EM) | 5-shot | 87.9 | 88.8 | **90.3** | +| C-Eval (EM) | 5-shot | 90.4 | 92.1 | **93.1** | +| CMMLU (EM) | 5-shot | 88.9 | 90.4 | **90.8** | +| MultiLoKo (EM) | 5-shot | 38.7 | 42.2 | **51.1** | +| Simple-QA verified (EM) | 25-shot | 28.3 | 30.1 | **55.2** | +| SuperGPQA (EM) | 5-shot | 45.0 | 46.5 | **53.9** | +| FACTS Parametric (EM) | 25-shot | 27.1 | 33.9 | **62.6** | +| TriviaQA (EM) | 5-shot | 83.3 | 82.8 | **85.6** | +| **Language & Reasoning** | | | | | +| BBH (EM) | 3-shot | **87.6** | 86.9 | 87.5 | +| DROP (F1) | 1-shot | 88.2 | 88.6 | **88.7** | +| HellaSwag (EM) | 0-shot | 86.4 | 85.7 | **88.0** | +| WinoGrande (EM) | 0-shot | 78.9 | 79.5 | **81.5** | +| CLUEWSC (EM) | 5-shot | 83.5 | 82.2 | **85.2** | +| **Code & Math** | | | | | +| BigCodeBench (Pass@1) | 3-shot | **63.9** | 56.8 | 59.2 | +| HumanEval (Pass@1) | 0-shot | 62.8 | 69.5 | **76.8** | +| GSM8K (EM) | 8-shot | 91.1 | 90.8 | **92.6** | +| MATH (EM) | 4-shot | 60.5 | 57.4 | **64.5** | +| MGSM (EM) | 8-shot | 81.3 | **85.7** | 84.4 | +| CMath (EM) | 3-shot | 92.6 | **93.6** | 90.9 | +| **Long Context** | | | | | +| LongBench-V2 (EM) | 1-shot | 40.2 | 44.7 | **51.5** | + +
+ +### Instruct Model + +DeepSeek-V4-Pro and DeepSeek-V4-Flash both support three reasoning effort modes: + +| Reasoning Mode | Characteristics | Typical Use Cases | Response Format | +| :--- | :--- | :--- | :--- | +| Non-think | Fast, intuitive responses | Routine daily tasks, low-risk decisions | `` summary | +| Think High | Conscious logical analysis, slower but more accurate | Complex problem-solving, planning | `` thinking `` summary | +| Think Max | Push reasoning to its fullest extent | Exploring the boundary of model reasoning capability | Special system prompt + `` thinking `` summary | + +#### DeepSeek-V4-Pro-Max vs Frontier Models + +
+ +| Benchmark (Metric) | Opus-4.6 Max | GPT-5.4 xHigh | Gemini-3.1-Pro High | K2.6 Thinking | GLM-5.1 Thinking | DS-V4-Pro Max | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | +| **Knowledge & Reasoning** | | | | | | | +| MMLU-Pro (EM) | 89.1 | 87.5 | **91.0** | 87.1 | 86.0 | 87.5 | +| SimpleQA-Verified (Pass@1) | 46.2 | 45.3 | **75.6** | 36.9 | 38.1 | 57.9 | +| Chinese-SimpleQA (Pass@1) | 76.4 | 76.8 | **85.9** | 75.9 | 75.0 | 84.4 | +| GPQA Diamond (Pass@1) | 91.3 | 93.0 | **94.3** | 90.5 | 86.2 | 90.1 | +| HLE (Pass@1) | 40.0 | 39.8 | **44.4** | 36.4 | 34.7 | 37.7 | +| LiveCodeBench (Pass@1) | 88.8 | - | 91.7 | 89.6 | - | **93.5** | +| Codeforces (Rating) | - | 3168 | 3052 | - | - | **3206** | +| HMMT 2026 Feb (Pass@1) | 96.2 | **97.7** | 94.7 | 92.7 | 89.4 | 95.2 | +| IMOAnswerBench (Pass@1) | 75.3 | **91.4** | 81.0 | 86.0 | 83.8 | 89.8 | +| Apex (Pass@1) | 34.5 | 54.1 | **60.9** | 24.0 | 11.5 | 38.3 | +| Apex Shortlist (Pass@1) | 85.9 | 78.1 | 89.1 | 75.5 | 72.4 | **90.2** | +| **Long Context** | | | | | | | +| MRCR 1M (MMR) | **92.9** | - | 76.3 | - | - | 83.5 | +| CorpusQA 1M (ACC) | **71.7** | - | 53.8 | - | - | 62.0 | +| **Agentic** | | | | | | | +| Terminal Bench 2.0 (Acc) | 65.4 | **75.1** | 68.5 | 66.7 | 63.5 | 67.9 | +| SWE Verified (Resolved) | **80.8** | - | 80.6 | 80.2 | - | 80.6 | +| SWE Pro (Resolved) | 57.3 | 57.7 | 54.2 | **58.6** | 58.4 | 55.4 | +| SWE Multilingual (Resolved) | **77.5** | - | - | 76.7 | 73.3 | 76.2 | +| BrowseComp (Pass@1) | 83.7 | 82.7 | **85.9** | 83.2 | 79.3 | 83.4 | +| HLE w/ tools (Pass@1) | 53.1 | 52.0 | 51.6 | **54.0** | 50.4 | 48.2 | +| GDPval-AA (Elo) | 1619 | **1674** | 1314 | 1482 | 1535 | 1554 | +| MCPAtlas Public (Pass@1) | **73.8** | 67.2 | 69.2 | 66.6 | 71.8 | 73.6 | +| Toolathlon (Pass@1) | 47.2 | **54.6** | 48.8 | 50.0 | 40.7 | 51.8 | + +
+ +#### Comparison across Modes + +
+ +| Benchmark (Metric) | V4-Flash Non-Think | V4-Flash High | V4-Flash Max | V4-Pro Non-Think | V4-Pro High | V4-Pro Max | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | +| **Knowledge & Reasoning** | | | | | | | +| MMLU-Pro (EM) | 83.0 | 86.4 | 86.2 | 82.9 | 87.1 | **87.5** | +| SimpleQA-Verified (Pass@1) | 23.1 | 28.9 | 34.1 | 45.0 | 46.2 | **57.9** | +| Chinese-SimpleQA (Pass@1) | 71.5 | 73.2 | 78.9 | 75.8 | 77.7 | **84.4** | +| GPQA Diamond (Pass@1) | 71.2 | 87.4 | 88.1 | 72.9 | 89.1 | **90.1** | +| HLE (Pass@1) | 8.1 | 29.4 | 34.8 | 7.7 | 34.5 | **37.7** | +| LiveCodeBench (Pass@1) | 55.2 | 88.4 | 91.6 | 56.8 | 89.8 | **93.5** | +| Codeforces (Rating) | - | 2816 | 3052 | - | 2919 | **3206** | +| HMMT 2026 Feb (Pass@1) | 40.8 | 91.9 | 94.8 | 31.7 | 94.0 | **95.2** | +| IMOAnswerBench (Pass@1) | 41.9 | 85.1 | 88.4 | 35.3 | 88.0 | **89.8** | +| Apex (Pass@1) | 1.0 | 19.1 | 33.0 | 0.4 | 27.4 | **38.3** | +| Apex Shortlist (Pass@1) | 9.3 | 72.1 | 85.7 | 9.2 | 85.5 | **90.2** | +| **Long Context** | | | | | | | +| MRCR 1M (MMR) | 37.5 | 76.9 | 78.7 | 44.7 | 83.3 | **83.5** | +| CorpusQA 1M (ACC) | 15.5 | 59.3 | 60.5 | 35.6 | 56.5 | **62.0** | +| **Agentic** | | | | | | | +| Terminal Bench 2.0 (Acc) | 49.1 | 56.6 | 56.9 | 59.1 | 63.3 | **67.9** | +| SWE Verified (Resolved) | 73.7 | 78.6 | 79.0 | 73.6 | 79.4 | **80.6** | +| SWE Pro (Resolved) | 49.1 | 52.3 | 52.6 | 52.1 | 54.4 | **55.4** | +| SWE Multilingual (Resolved) | 69.7 | 70.2 | 73.3 | 69.8 | 74.1 | **76.2** | +| BrowseComp (Pass@1) | - | 53.5 | 73.2 | - | 80.4 | **83.4** | +| HLE w/ tools (Pass@1) | - | 40.3 | 45.1 | - | 44.7 | **48.2** | +| MCPAtlas (Pass@1) | 64.0 | 67.4 | 69.0 | 69.4 | **74.2** | 73.6 | +| GDPval-AA (Elo) | - | - | 1395 | - | - | **1554** | +| Toolathlon (Pass@1) | 40.7 | 43.5 | 47.8 | 46.3 | 49.0 | **51.8** | + +
+ +## Chat Template + +This release does not include a Jinja-format chat template. Instead, we provide a dedicated `encoding` folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the [`encoding`](encoding/README.md) folder for full documentation. + +A brief example: + +```python +from encoding_dsv4 import encode_messages, parse_message_from_completion_text + +messages = [ + {"role": "user", "content": "hello"}, + {"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."}, + {"role": "user", "content": "1+1=?"} +] + +# messages -> string +prompt = encode_messages(messages, thinking_mode="thinking") + +# string -> tokens +import transformers +tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Pro") +tokens = tokenizer.encode(prompt) +``` ## How to Run Locally -**vLLM and Sglang is not supported currently: https://huggingface.co/Intel/DeepSeek-V4-Flash-W4A16-AutoRound/discussions/1** - Please refer to the [inference](inference/README.md) folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos. For local deployment, we recommend setting the sampling parameters to `temperature = 1.0, top_p = 1.0`. For the Think Max reasoning mode, we recommend setting the context window to at least **384K** tokens. +## License -## Generate the Model - -~~~bash -auto-round deepseek-ai/DeepSeek-V4-Flash \ - --model_free \ - --scheme W4A16 \ - --ignore_layers compressor,indexer.weights_proj \ - --layer_config "{'wo_a':{bits:16}}" \ - --output_dir "./DeepSeek-V4-Flash-W4A16" -~~~ - - - -## Ethical Considerations and Limitations - -The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs. - -Therefore, before deploying any applications of the model, developers should perform safety testing. - -## Caveats and Recommendations - -Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. - -Here are a couple of useful links to learn more about Intel's AI software: - -- [Intel Neural Compressor](https://github.com/intel/neural-compressor) - -## Disclaimer +This repository and the model weights are licensed under the [MIT License](LICENSE). -The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes. +## Citation -## Cite +``` +@misc{deepseekai2026deepseekv4, + title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence}, + author={DeepSeek-AI}, + year={2026}, +} +``` -@article{cheng2023optimize, title={Optimize weight rounding via signed gradient descent for the quantization of llms}, author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi}, journal={arXiv preprint arXiv:2309.05516}, year={2023} } +## Contact -[arxiv](https://arxiv.org/abs/2309.05516) [github](https://github.com/intel/auto-round) \ No newline at end of file +If you have any questions, please raise an issue or contact us at [service@deepseek.com](service@deepseek.com). diff --git a/config.json b/config.json index c5b688a3908b2da7ac5b237dcae3bf224aabb0f4..f7bfef418c31e46c7c35b6e418d99ea62ee5c969 100644 --- a/config.json +++ b/config.json @@ -6,6 +6,7 @@ "attention_dropout": 0.0, "bos_token_id": 0, "eos_token_id": 1, + "expert_dtype": "fp4", "hc_eps": 1e-06, "hc_mult": 4, "hc_sinkhorn_iters": 20, @@ -41,15 +42,950 @@ "data_type": "int", "iters": 0, "model_free": true, - "autoround_version": "0.13.0", + "autoround_version": "0.14.0", "extra_config": { + "wo_a": { + "bits": 16 + }, "embed": { "bits": 16, "data_type": "float" }, + "layers.0.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.0.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.1.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.1.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.2.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.3.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.3.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.3.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.3.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.4.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.5.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.5.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.5.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.5.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.6.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.7.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.7.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.7.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.7.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.8.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.9.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.9.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.9.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.9.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.10.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.11.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.11.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.11.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.11.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.12.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.13.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.13.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.13.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.13.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.14.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.15.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.15.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.15.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.15.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.16.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.17.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.17.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.17.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.17.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.18.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.19.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.19.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.19.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.19.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.20.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.21.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.21.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.21.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.21.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.22.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.23.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.23.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.23.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.23.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.24.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.25.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.25.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.25.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.25.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.26.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.27.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.27.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.27.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.27.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.28.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.29.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.29.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.29.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.29.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.30.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.31.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.31.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.31.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.31.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.32.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.33.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.33.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.33.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.33.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.34.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.35.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.35.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.35.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.35.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.36.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.37.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.37.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.37.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.37.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.38.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.39.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.39.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.39.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.39.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.40.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.41.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.41.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.41.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.41.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.42.ffn.gate": { + "bits": 16, + "data_type": "float" + }, "head": { "bits": 16, "data_type": "float" + }, + "mtp.0.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "mtp.0.ffn.gate": { + "bits": 16, + "data_type": "float" } } }, diff --git a/inference/README.md b/inference/README.md index f5bb078d74ed517ddc8d5ea7d9bf7202ea18d954..e11577229e5e5b6f82f687f7ed755096b3edd8d6 100644 --- a/inference/README.md +++ b/inference/README.md @@ -3,9 +3,9 @@ First convert huggingface model weight files to the format of this project. ```bash export EXPERTS=256 -export MP=8 -export CONFIG=config_w4a16.json -python convert_w4a16.py --hf-ckpt-path ${HF_CKPT_PATH} --save-path ${SAVE_PATH} --n-experts ${EXPERTS} --model-parallel ${MP} +export MP=4 +export CONFIG=config.json +python convert.py --hf-ckpt-path ${HF_CKPT_PATH} --save-path ${SAVE_PATH} --n-experts ${EXPERTS} --model-parallel ${MP} ``` Then chat with DeepSeek model at will! @@ -23,3 +23,4 @@ Or multi nodes inference. torchrun --nnodes ${NODES} --nproc-per-node $((MP / NODES)) --node-rank $RANK --master-addr $ADDR generate.py --ckpt-path ${SAVE_PATH} --config ${CONFIG} --input-file ${FILE} ``` +If you want to use fp8, just remove `"expert_dtype": "fp4"` in `config.json` and specify `--expert-dtype fp8` in `convert.py`. diff --git a/inference/generate.py b/inference/generate.py index 8d141392e0b131e89e5461f87db98f8270c59441..c35c80305e5173f6b26e394a64134a75a3f17e28 100644 --- a/inference/generate.py +++ b/inference/generate.py @@ -1,6 +1,4 @@ import os -# Prevent gptqmodel from setting CUDA_DEVICE_ORDER=PCI_BUS_ID (breaks multi-GPU on some systems) -os.environ.setdefault("CUDA_DEVICE_ORDER", "FASTEST_FIRST") import json import sys from argparse import ArgumentParser @@ -102,8 +100,6 @@ def main( tokenizer = AutoTokenizer.from_pretrained(ckpt_path) print("load model") load_model(model, os.path.join(ckpt_path, f"model{rank}-mp{world_size}.safetensors"), strict=False) - if args.dtype == "w4a16": - model.init_woq_layers() torch.set_default_device("cuda") print("I'm DeepSeek 👋") diff --git a/inference/model.py b/inference/model.py index 52eba6065064b65183819aa054964a496bb97010..167ade8fdb35c0fbaabf4735b6696ef7a32f1588 100644 --- a/inference/model.py +++ b/inference/model.py @@ -16,35 +16,9 @@ world_size = 1 rank = 0 block_size = 128 fp4_block_size = 32 -w4a16_group_size = 128 default_dtype = torch.bfloat16 scale_fmt = None scale_dtype = torch.float32 -w4a16_mode = False # set in Transformer.__init__ when args.dtype == "w4a16" - - -def dequantize_w4a16(qweight: torch.Tensor, qzeros: torch.Tensor, scales: torch.Tensor, - group_size: int = 128) -> torch.Tensor: - """Auto-round / auto_gptq W4A16 packing -> BF16 weight [out, in]. - - qweight: int32 [in/8, out], LSB-first 4-bit packed along input dim - qzeros : int32 [in/g, out/8], LSB-first 4-bit packed along output dim - scales : bf16 [in/g, out] - """ - in_packed, out_features = qweight.shape - in_features = in_packed * 8 - n_groups = scales.shape[0] - device = qweight.device - shifts = torch.arange(0, 32, 4, device=device, dtype=torch.int32) - w = (qweight.unsqueeze(1) >> shifts.view(1, 8, 1)) & 0xF # [in/8, 8, out] - w = w.reshape(in_features, out_features).to(torch.float32) - z = (qzeros.unsqueeze(2) >> shifts.view(1, 1, 8)) & 0xF # [in/g, out/8, 8] - z = z.reshape(n_groups, out_features).to(torch.float32) + 1.0 # GPTQ stores zero - 1 - s = scales.to(torch.float32) - w = w.view(n_groups, group_size, out_features) - deq = (w - z.unsqueeze(1)) * s.unsqueeze(1) - deq = deq.view(in_features, out_features) - return deq.t().contiguous().to(torch.bfloat16) @contextmanager @@ -62,7 +36,7 @@ class ModelArgs: """Model hyperparameters. Field names match the config JSON keys.""" max_batch_size: int = 4 max_seq_len: int = 4096 - dtype: Literal["bf16", "fp8", "w4a16"] = "fp8" + dtype: Literal["bf16", "fp8"] = "fp8" scale_fmt: Literal[None, "ue8m0"] = "ue8m0" expert_dtype: Literal[None, "fp4"] = None scale_dtype: Literal["fp32", "fp8"] = "fp8" @@ -153,34 +127,8 @@ class Linear(nn.Module): super().__init__() self.in_features = in_features self.out_features = out_features - # In a W4A16 build every Linear becomes W4A16 regardless of the dtype the - # original FP8/FP4 model wanted. The non-quant special cases (RMSNorm, - # embed, attn_sink, etc.) are NOT instances of `Linear`, so they are - # untouched. - if w4a16_mode: - dtype = "w4a16" - else: - dtype = dtype or default_dtype - self.is_w4a16 = (dtype == "w4a16") - if self.is_w4a16: - assert in_features % 8 == 0 and in_features % w4a16_group_size == 0 - assert out_features % 8 == 0 - self.group_size = w4a16_group_size - self.qweight = nn.Parameter( - torch.empty(in_features // 8, out_features, dtype=torch.int32), - requires_grad=False, - ) - self.qzeros = nn.Parameter( - torch.empty(in_features // self.group_size, out_features // 8, dtype=torch.int32), - requires_grad=False, - ) - self.scales = nn.Parameter( - torch.empty(in_features // self.group_size, out_features, dtype=torch.bfloat16), - requires_grad=False, - ) - self.register_parameter("weight", None) - self.register_parameter("scale", None) - elif dtype == torch.float4_e2m1fn_x2: + dtype = dtype or default_dtype + if dtype == torch.float4_e2m1fn_x2: # FP4: weight is [out, in//2] in float4_e2m1fn_x2, logically [out, in] in fp4 # Scale is [out, in//32] in float8_e8m0fnu (1 scale per 32 fp4 elements along K) self.weight = nn.Parameter(torch.empty(out_features, in_features // 2, dtype=torch.float4_e2m1fn_x2)) @@ -200,51 +148,7 @@ class Linear(nn.Module): else: self.register_parameter("bias", None) - def init_woq(self, QuantLinear): - """Create a QuantLinear from loaded GPTQ parameters.""" - if not self.is_w4a16: - return - # Marlin requires out_features % 64 == 0; fall back to manual dequant - if self.out_features % 64 != 0: - self._woq = None - return - dev = self.qweight.device - layer = QuantLinear( - bits=4, group_size=self.group_size, - in_features=self.in_features, out_features=self.out_features, - bias=False, desc_act=False, sym=True, register_buffers=True, - ) - layer = layer.to(dev) - layer.qweight.copy_(self.qweight.data) - layer.qzeros.copy_(self.qzeros.data) - layer.scales.copy_(self.scales.to(layer.scales.dtype).data) - layer.g_idx.copy_(torch.arange(self.in_features, dtype=torch.int32, device=dev) // self.group_size) - layer.post_init() - self._woq = layer - # Free original parameters to save memory - self.qweight = None - self.qzeros = None - self.scales = None - - def get_weight(self) -> torch.Tensor: - """Return the dequantised BF16 weight [out, in]. For non-W4A16 modes - returns ``self.weight`` unchanged. Used only for wo_a einsum path.""" - if self.is_w4a16: - if self._woq is not None: - return dequantize_w4a16(self._woq.qweight, self._woq.qzeros, self._woq.scales, self.group_size) - return dequantize_w4a16(self.qweight, self.qzeros, self.scales, self.group_size) - return self.weight - def forward(self, x: torch.Tensor) -> torch.Tensor: - if self.is_w4a16: - if hasattr(self, '_woq') and self._woq is not None: - y = self._woq(x.to(torch.bfloat16)) - else: - w = dequantize_w4a16(self.qweight, self.qzeros, self.scales, self.group_size) - y = F.linear(x.to(w.dtype), w) - if self.bias is not None: - y = y + self.bias - return y.type_as(x) return linear(x, self.weight, self.bias) @@ -256,8 +160,6 @@ class ColumnParallelLinear(Linear): super().__init__(in_features, self.part_out_features, bias, dtype) def forward(self, x: torch.Tensor) -> torch.Tensor: - if self.is_w4a16: - return Linear.forward(self, x) return linear(x, self.weight, self.bias) @@ -269,14 +171,7 @@ class RowParallelLinear(Linear): super().__init__(self.part_in_features, out_features, bias, dtype) def forward(self, x: torch.Tensor) -> torch.Tensor: - if self.is_w4a16: - if hasattr(self, '_woq') and self._woq is not None: - y = self._woq(x.to(torch.bfloat16)) - else: - w = dequantize_w4a16(self.qweight, self.qzeros, self.scales, self.group_size) - y = F.linear(x.to(w.dtype), w) - else: - y = linear(x, self.weight, None) + y = linear(x, self.weight, None) if world_size > 1: y = y.float() dist.all_reduce(y) @@ -638,12 +533,12 @@ class Attention(nn.Module): o = sparse_attn(q, self.kv_cache[:bsz], self.attn_sink, topk_idxs, self.softmax_scale) apply_rotary_emb(o[..., -rd:], freqs_cis, True) - # o: apply wo_a per-group projection then wo_b - # Flatten groups into the feature dim, call wo_a as a normal linear, then reshape back. - # Equivalent to the per-group einsum when wo_a weight is block-diagonal across groups - # (always true here since n_local_groups = n_groups/world_size = 1 for 8-GPU deploy). + # o o = o.view(bsz, seqlen, self.n_local_groups, -1) - o = self.wo_a(o.flatten(2)).view(bsz, seqlen, self.n_local_groups, self.o_lora_rank) + wo_a = self.wo_a.weight.view(self.n_local_groups, self.o_lora_rank, -1) + # NOTE: wo_a is FP8 in checkpoint; could do FP8 einsum here for better perf, + # but using BF16 for simplicity. + o = torch.einsum("bsgd,grd->bsgr", o, wo_a) x = self.wo_b(o.flatten(2)) return x @@ -659,59 +554,15 @@ class Gate(nn.Module): self.score_func = args.score_func self.route_scale = args.route_scale self.hash = layer_id < args.n_hash_layers - self.is_w4a16 = w4a16_mode - if self.is_w4a16: - in_f, out_f = args.dim, args.n_routed_experts - assert in_f % w4a16_group_size == 0 and out_f % 8 == 0 - self.group_size = w4a16_group_size - self.qweight = nn.Parameter( - torch.empty(in_f // 8, out_f, dtype=torch.int32), requires_grad=False) - self.qzeros = nn.Parameter( - torch.empty(in_f // self.group_size, out_f // 8, dtype=torch.int32), requires_grad=False) - self.scales = nn.Parameter( - torch.empty(in_f // self.group_size, out_f, dtype=torch.bfloat16), requires_grad=False) - self.register_parameter("weight", None) - else: - self.weight = nn.Parameter(torch.empty(args.n_routed_experts, args.dim)) + self.weight = nn.Parameter(torch.empty(args.n_routed_experts, args.dim)) if self.hash: self.tid2eid = nn.Parameter(torch.empty(args.vocab_size, args.n_activated_experts, dtype=torch.int32), requires_grad=False) self.bias = None else: self.bias = nn.Parameter(torch.empty(args.n_routed_experts, dtype=torch.float32)) - def init_woq(self, QuantLinear): - if not self.is_w4a16: - return - dev = self.qweight.device - in_f, out_f = self.dim, self.qweight.shape[1] - if out_f % 64 != 0: - self._woq = None - return - layer = QuantLinear( - bits=4, group_size=self.group_size, - in_features=in_f, out_features=out_f, - bias=False, desc_act=False, sym=True, register_buffers=True, - ) - layer = layer.to(dev) - layer.qweight.copy_(self.qweight.data) - layer.qzeros.copy_(self.qzeros.data) - layer.scales.copy_(self.scales.to(layer.scales.dtype).data) - layer.g_idx.copy_(torch.arange(in_f, dtype=torch.int32, device=dev) // self.group_size) - layer.post_init() - self._woq = layer - self.qweight = None - self.qzeros = None - self.scales = None - def forward(self, x: torch.Tensor, input_ids: Optional[torch.Tensor] = None) -> Tuple[torch.Tensor, torch.Tensor]: - if self.is_w4a16: - if hasattr(self, '_woq') and self._woq is not None: - scores = self._woq(x.to(torch.bfloat16)).float() - else: - w = dequantize_w4a16(self.qweight, self.qzeros, self.scales, self.group_size) - scores = F.linear(x.to(w.dtype), w).float() - else: - scores = linear(x.float(), self.weight.float()) + scores = linear(x.float(), self.weight.float()) if self.score_func == "softmax": scores = scores.softmax(dim=-1) elif self.score_func == "sigmoid": @@ -773,7 +624,6 @@ class MoE(nn.Module): self.experts = nn.ModuleList([Expert(args.dim, args.moe_inter_dim, dtype=expert_dtype, swiglu_limit=args.swiglu_limit) if self.experts_start_idx <= i < self.experts_end_idx else None for i in range(self.n_routed_experts)]) assert args.n_shared_experts == 1 - # no swiglu_limit self.shared_experts = Expert(args.dim, args.moe_inter_dim, swiglu_limit=args.swiglu_limit) def forward(self, x: torch.Tensor, input_ids: torch.Tensor) -> torch.Tensor: @@ -859,7 +709,7 @@ class ParallelHead(nn.Module): self.norm_eps = norm_eps self.hc_eps = hc_eps self.part_vocab_size = (vocab_size // world_size) - # lm_head is always stored as bf16 (even in W4A16 checkpoints); use fp32 for logit precision + # lm_head in the checkpoint is stored in bf16, while the parameter here is stored in fp32 for easier computation of logits later. self.weight = nn.Parameter(torch.empty(self.part_vocab_size, self.dim, dtype=torch.float32)) def get_logits(self, x): @@ -920,18 +770,12 @@ class Transformer(nn.Module): """Full DeepSeek-V4 model: embed -> HC-expand -> N blocks -> HC-head -> logits. Sets global state (world_size, rank, default_dtype, scale_fmt, scale_dtype) in __init__.""" def __init__(self, args: ModelArgs): - global world_size, rank, default_dtype, scale_fmt, scale_dtype, w4a16_mode + global world_size, rank, default_dtype, scale_fmt, scale_dtype world_size = dist.get_world_size() if dist.is_initialized() else 1 rank = dist.get_rank() if dist.is_initialized() else 0 - w4a16_mode = (args.dtype == "w4a16") - if w4a16_mode: - default_dtype = torch.bfloat16 - scale_fmt = None - scale_dtype = torch.float32 - else: - default_dtype = torch.float8_e4m3fn if args.dtype == "fp8" else torch.bfloat16 - scale_fmt = "ue8m0" if args.scale_dtype == "fp8" else args.scale_fmt - scale_dtype = torch.float8_e8m0fnu if args.scale_dtype == "fp8" else torch.float32 + default_dtype = torch.float8_e4m3fn if args.dtype == "fp8" else torch.bfloat16 + scale_fmt = "ue8m0" if args.scale_dtype == "fp8" else args.scale_fmt + scale_dtype = torch.float8_e8m0fnu if args.scale_dtype == "fp8" else torch.float32 super().__init__() self.max_seq_len = args.max_seq_len self.norm_eps = args.norm_eps @@ -954,15 +798,6 @@ class Transformer(nn.Module): self.hc_head_base = nn.Parameter(torch.empty(hc_mult)) self.hc_head_scale = nn.Parameter(torch.empty(1)) - def init_woq_layers(self): - """After load_model(), convert all W4A16 parameters into QuantLinear layers.""" - # from gptqmodel.nn_modules.qlinear.tritonv2 import TritonV2QuantLinear as QuantLinear - from gptqmodel.nn_modules.qlinear.marlin import MarlinQuantLinear as QuantLinear - for module in self.modules(): - if hasattr(module, 'init_woq') and module is not self: - module.init_woq(QuantLinear) - torch.cuda.empty_cache() - @torch.inference_mode() def forward(self, input_ids: torch.Tensor, start_pos: int = 0): h = self.embed(input_ids) diff --git a/inference/requirements.txt b/inference/requirements.txt index d30adfcbd3f859502058257f7f8bafa247132fcd..7e1cc78c33123582c00538a830d792cd2ea2f516 100644 --- a/inference/requirements.txt +++ b/inference/requirements.txt @@ -2,5 +2,4 @@ torch>=2.10.0 transformers>=5.0.0 safetensors>=0.7.0 fast_hadamard_transform -tilelang==0.1.8 -gptqmodel==6.0.3 +tilelang==0.1.8 \ No newline at end of file diff --git a/model-00002-of-00046.safetensors b/model-00002-of-00046.safetensors index c7d0b0b4ecb223ced81e7e81e15cd7eccc944420..e2bc253422920294b5b7bf4e3d2a35ddbe31dc18 100644 --- a/model-00002-of-00046.safetensors +++ b/model-00002-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:e02f607a26f45708944d8b6490a826f11f83e13ebeabbdd7921be492589b1406 -size 3427419616 +oid sha256:6618a5361a98e3320b904118ec44b24344d80e5431dafd2b711555ccde44d366 +size 3477095680 diff --git a/model-00003-of-00046.safetensors b/model-00003-of-00046.safetensors index c7d3326a3631da40f959bf65427b9f4df51a177e..cf290e0915a7a5ba0f835ee2b46554bcd3588753 100644 --- a/model-00003-of-00046.safetensors +++ b/model-00003-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:f7b1bd86d868e69226fcb113a3b6f71dc5caada7861459cc65a802727a1cd8cc -size 3427419616 +oid sha256:9d0a73917baa25d1c01e3a42df95aba1dc2cb68c558e518bce9d0e1a196110fe +size 3477095680 diff --git a/model-00004-of-00046.safetensors b/model-00004-of-00046.safetensors index 36959dde8d6496a2656e87ed45e0f63581ab6630..53ae4ac0dcefc3f7c177a2e66116820fc6557e62 100644 --- a/model-00004-of-00046.safetensors +++ b/model-00004-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:a797c4f0b787c68ff149f2d81f9ce73debd5d964915119dd0ff3cd0c4241df64 -size 3437385848 +oid sha256:3589581d62d9281b8ad1da35b95bfedef84cfcb923bc8577a048952dd2e98a59 +size 3502972736 diff --git a/model-00005-of-00046.safetensors b/model-00005-of-00046.safetensors index aa313a8ed4d7a8a69fecb7b6182ca667396bcb3a..e064c12a4c6f2b3827ca793aa50ff90780b557b6 100644 --- a/model-00005-of-00046.safetensors +++ b/model-00005-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:f5e88868fc91f138ceaa83da68066b7b622fe1c51a231776d2559b37dcec842c -size 3423658256 +oid sha256:a4206f35baed1f28eaef83248318a6cb69af37dd6d26c57fb300d1f7eaed419f +size 3479543432 diff --git a/model-00006-of-00046.safetensors b/model-00006-of-00046.safetensors index 8fd0efb9a73d8a8750543ff23c88ec393e5985f5..8403cfe169e00519dcd0a4a02370418e1931a494 100644 --- a/model-00006-of-00046.safetensors +++ b/model-00006-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:1ddfe6f1d4bb4bebecf4aedd614bb126f07d004ddbdc8826f0a67db2c792d7a9 -size 3431181376 +oid sha256:d559bcb1d22582a78c746c7922e7e7e2be793106409b0036b289adfe10dbdb0a +size 3496768264 diff --git a/model-00007-of-00046.safetensors b/model-00007-of-00046.safetensors index 5afe6c49d73b072690cccc69b0e74a553ad90660..690b6e5586c0b6f6023746860f4f7908ba857fdf 100644 --- a/model-00007-of-00046.safetensors +++ b/model-00007-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:28a6375b1d220b3eb3a375634eff3ab84b16abc5f4c08cd4dc0c3199b3d7cbed -size 3423658256 +oid sha256:9c7cbd22f0499b820bb97dff5c0d5be431ca5b77e154651ec897ab12d01eb2eb +size 3479543432 diff --git a/model-00008-of-00046.safetensors b/model-00008-of-00046.safetensors index 8f330e502702e02e7c9a6d1567f32c56b3d1dba6..a0448560700653a132b3b9596e3a68ac9197a931 100644 --- a/model-00008-of-00046.safetensors +++ b/model-00008-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0d8d1ac243d5a509d394a425d6784a021a9b3574d83cb9eaeb940857993f4e70 -size 3431181376 +oid sha256:b32125218dffc7048a6e198da9782e09e6d810d3cec5759c38d08d7d5ed0684a +size 3496768264 diff --git a/model-00009-of-00046.safetensors b/model-00009-of-00046.safetensors index 1dcd9b325de0605853ed03ce6d9c9398b4f5c820..86fdbaf4047f6d2ad13a65a475234ecd8175b890 100644 --- a/model-00009-of-00046.safetensors +++ b/model-00009-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:9d09ee1d9b392a33f720fe22e413fa50b18bcc5e9e711d97c52af257efc00b89 -size 3423658256 +oid sha256:d86d105bb06228e43730db3b9e374da67ba4b54ba53e4531b7662d4d9f872f70 +size 3479543432 diff --git a/model-00010-of-00046.safetensors b/model-00010-of-00046.safetensors index 03dd61bc4762ba6d4c4bf1534a51bd5d932985b0..d51c74c4bd376888bb2c59383230047274d77bed 100644 --- a/model-00010-of-00046.safetensors +++ b/model-00010-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0a8d867d31d0fcd49b16824a9c28644b0bea07d4268ec727915ca389165d041f -size 3431181376 +oid sha256:e292a801f3fa6a94e27aa0b531e7bffa7a4d8881defaeb5de615903dc791dea6 +size 3496768264 diff --git a/model-00011-of-00046.safetensors b/model-00011-of-00046.safetensors index f78bc18b6a07422658fadcc147c7342c2d38d11a..8a72f939d14ceab8a46aabc5a9d84d17a63d0fdb 100644 --- a/model-00011-of-00046.safetensors +++ b/model-00011-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:277248da439d1c4ca1eab81e0a189521415b3102e27018626f32656efc4691e4 -size 3423658256 +oid sha256:24105aec63df32300dba0d2aae2e8b3a4bc8e1a36261b68b81e9898f001d0a85 +size 3479543432 diff --git a/model-00012-of-00046.safetensors b/model-00012-of-00046.safetensors index 64b54a9db872400d236e67592c0376654afd94d2..d31c722d247555d8068df052ef14011ea8f9e57b 100644 --- a/model-00012-of-00046.safetensors +++ b/model-00012-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:e290660f0daf12abd56fe037ec4fcecfea71d16b3c5f14d570e857e79812b0de -size 3431183736 +oid sha256:27cd663d032f20849d3d0fe4a033569429d49ca5be2255b81e6ddf4c5c62cc13 +size 3496770616 diff --git a/model-00013-of-00046.safetensors b/model-00013-of-00046.safetensors index d782b6fb9fc94fbbcd717f69f9d5bf6fa78fb2c2..7feb8768899b8115676f8bddc76e1e2276d1a3c3 100644 --- a/model-00013-of-00046.safetensors +++ b/model-00013-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:cac0880c2dd03be152e9be5434874709b30b7ffc994a046ce8d59f47b128a770 -size 3423660608 +oid sha256:cf26562d7ac818e48b09be54c940620f7d326ba2072d8ca55f3a5926db3570fd +size 3479545776 diff --git a/model-00014-of-00046.safetensors b/model-00014-of-00046.safetensors index fe929e9ebb64e9c0239f9f840acf5498cf19e261..19e8593da8135c74c6bb7631db11fb3ebec86a86 100644 --- a/model-00014-of-00046.safetensors +++ b/model-00014-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:3a938bd97f3b19b18d96646a56022d02cfcfa1a5812f6e1d16fae299355c60bd -size 3431183736 +oid sha256:a977b89d8b041bf73f3811971a3d834521dbf96f20ce5c37cbd8446a5ccf354a +size 3496770616 diff --git a/model-00015-of-00046.safetensors b/model-00015-of-00046.safetensors index bbcf3c9d909fd6bcccded50cf673ffa9e397daa0..b59b82b81d7fd57ad076c1ec446a484e2fc0320c 100644 --- a/model-00015-of-00046.safetensors +++ b/model-00015-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:fc8e4843ac478bf53faffd22f4087d18d7e1f5c5bdbf7b8ccf0ba2a997721ed7 -size 3423660608 +oid sha256:04ca5679a66fc7743360f3a8bc9a49f38132f00b3aacf17e68b0753948ef91c9 +size 3479545776 diff --git a/model-00016-of-00046.safetensors b/model-00016-of-00046.safetensors index d67fbf244bcfaf4f2f2b16e3481f5d9f946a0239..51f9768bfb19491253309cfaac0cf67f53cd7a8d 100644 --- a/model-00016-of-00046.safetensors +++ b/model-00016-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:d8ef297a833beb89de788024d24ba44de660e8fccb15ac0086356c54f0bf2a38 -size 3431183736 +oid sha256:7983e469cadec35df9a3353d21ef80d5ff81aee56ac340ea19b5aa8aab95726f +size 3496770616 diff --git a/model-00017-of-00046.safetensors b/model-00017-of-00046.safetensors index 669a35c3da8fb71cbdef36c853b585bc7ce92991..bf55ea27c518bde06ecac57d22900c0bb8e451bc 100644 --- a/model-00017-of-00046.safetensors +++ b/model-00017-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:837f53cdf4a22acca4aa23caec95bd1eed44ce9cdddb7be71bb4918170322e07 -size 3423660608 +oid sha256:a8b0fc6a864e2deb724b24b54860985d59bdaba67b3908854f962c87ee805f45 +size 3479545776 diff --git a/model-00018-of-00046.safetensors b/model-00018-of-00046.safetensors index 2fc9b05e5be5a4db434b425ebf9ac269e1c9c828..8e53821eb847a5fd288912f80c8085a7e0257a85 100644 --- a/model-00018-of-00046.safetensors +++ b/model-00018-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:d2611d2e5274408335d2ceb2caa73c3715682fc7cbb211d9a31e6679410cfab2 -size 3431183736 +oid sha256:bbb00df23b6e2a91706b44a16b3cd85b53c805c23840beabd80cf8318f580b1a +size 3496770616 diff --git a/model-00019-of-00046.safetensors b/model-00019-of-00046.safetensors index 744e308583e9a86237835f29bd22c4341905613b..e3f863be0ad885a8fb47c831935140a15a69d9f8 100644 --- a/model-00019-of-00046.safetensors +++ b/model-00019-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:e8fef4da43c21c9246aa0ee58417cf3c7915e64a49b93757cd8622aba60d0cd9 -size 3423660608 +oid sha256:2a2e0858ccb9d60bdb2ecc8547ee83437b8875168b7fe7259815083c805e43bd +size 3479545776 diff --git a/model-00020-of-00046.safetensors b/model-00020-of-00046.safetensors index c466b84763645dae25b2d1e74e980dc5e71aa42c..98a6fe6baf9e517dcb4b8355d63d69381bae1cc0 100644 --- a/model-00020-of-00046.safetensors +++ b/model-00020-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:2e99e9b30c6551d26b50b0c316bdf270ea6599c493ba69187f4db8f6a0d1674f -size 3431183736 +oid sha256:cf8bb4d99f392a2106f6602518feb48c2627bb076ebec657594dad071766e754 +size 3496770616 diff --git a/model-00021-of-00046.safetensors b/model-00021-of-00046.safetensors index 1a93acfc81ad1cf648db59c0dedd72eefdd475e4..8a4ae167eeadeb461b25434081f61df18ea7b6b8 100644 --- a/model-00021-of-00046.safetensors +++ b/model-00021-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:aa0ce177331636f80c30510b32625a2968a8cdb9817ab553a667ee484b55b0de -size 3423660608 +oid sha256:be8e098683557b6e44374cb7b8ca1b7d754be3034a9758b7e9d87288251f8283 +size 3479545776 diff --git a/model-00022-of-00046.safetensors b/model-00022-of-00046.safetensors index 319f75720128ee3cde51c6da98a561276f41a285..686bc1d993cc4c82d4acc5131812cb540d333f89 100644 --- a/model-00022-of-00046.safetensors +++ b/model-00022-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:1beaa5dad0904caf68bd6779e6ea9feb4ac7606b9304bbd46f7fce344a9fe1bf -size 3431183736 +oid sha256:63e6f9ff02eb0ca214408c353727ae6ddacd9a443df6893063a39fad5bc8da02 +size 3496770616 diff --git a/model-00023-of-00046.safetensors b/model-00023-of-00046.safetensors index 98f770a9810002e47e3389147bc9f2b6846dd06d..8326f9bc1128a1f99b3948429a7cffd99993901f 100644 --- a/model-00023-of-00046.safetensors +++ b/model-00023-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:46d6a3cd9e9621d56b679a80ac0ec16810fadce36ac460c2b8954204ed79c840 -size 3423660608 +oid sha256:4e2bc0b3988718211272fba9abdb5fe4f133f1b6ccc7492b1bdfada5e405da7c +size 3479545776 diff --git a/model-00024-of-00046.safetensors b/model-00024-of-00046.safetensors index ef176de7c9b65a65a75aba5f311257f40da2a5b6..e06fe4f33880c4bf69c24903c879a860c13a7dc7 100644 --- a/model-00024-of-00046.safetensors +++ b/model-00024-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:85c3a76c7572800bf9a0e18284d9dbfdf309ce74256d51a77406d19bd2889620 -size 3431183736 +oid sha256:eacf61cb808296c33e3bfc8d8876bb1465e113bf1a0146ed89d5debd3be9ec21 +size 3496770616 diff --git a/model-00025-of-00046.safetensors b/model-00025-of-00046.safetensors index afafa26a7534c80de66501540273251212ff3a17..b9557000b21b218c8b1ac53ae225a69246e66481 100644 --- a/model-00025-of-00046.safetensors +++ b/model-00025-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0ef227c55d8d63126dcf3971472f48d49a5761b533105153be72a8764fda858d -size 3423660608 +oid sha256:f2c45ce2ff4642ccc223dad6901471a0567269aef8bbf565860f3e08add4e413 +size 3479545776 diff --git a/model-00026-of-00046.safetensors b/model-00026-of-00046.safetensors index c4d6e92b464937ae72ffeac4fe728775fad159b4..028ba3351fac5576454d54312aa9aa90a58c0116 100644 --- a/model-00026-of-00046.safetensors +++ b/model-00026-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:f4ab1aff714b9a04da874ae3ca9f111c673a05a9ed98eb9afbf532900019a372 -size 3431183736 +oid sha256:ade2380ab274963f77ffb0d5f1a196e6b073af01492a819bcc5225a7cf73e27d +size 3496770616 diff --git a/model-00027-of-00046.safetensors b/model-00027-of-00046.safetensors index 4c3ac725271bcdb264789942d8a9cf09e5777237..5bffcdfc43e6ae28493c4e784261b10fb8ce09ba 100644 --- a/model-00027-of-00046.safetensors +++ b/model-00027-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:d100dc2fd6a40477e3cd2c1acf2563a62e19bbc8236229151de3cc8afb6c35e8 -size 3423660608 +oid sha256:0f95d38eee88befab5d33275b7f0c1d3bc2d8bbbc165e4a0218f03deaf05001b +size 3479545776 diff --git a/model-00028-of-00046.safetensors b/model-00028-of-00046.safetensors index def3ba50f6acd929e9620895984cf64980d5e02e..e9fea4501ed7df9fa825817b03d6f8348b07412f 100644 --- a/model-00028-of-00046.safetensors +++ b/model-00028-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:585f545aa1a0fd2ba90b0bf73c4d4e782d021b5409f8732aaf672626e8b880e0 -size 3431183736 +oid sha256:0644622a898b12accbd66ae648da69abbc55e707ac2ebf0c242f16e5022c165e +size 3496770616 diff --git a/model-00029-of-00046.safetensors b/model-00029-of-00046.safetensors index a6451cafe1724a59d891ad973f5a8547168ebdd0..1a896042d11f4d79017dccc639aee520aba71837 100644 --- a/model-00029-of-00046.safetensors +++ b/model-00029-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:e881fd2800c332d92de455b5e66870448bd89875fbe253349e051c2092bee67a -size 3423660608 +oid sha256:310c3af9c0b3f8d56167c8095c75ad6c69786571a0b8e050cb9187e5df4bede9 +size 3479545776 diff --git a/model-00030-of-00046.safetensors b/model-00030-of-00046.safetensors index eab5556f44ee62d622ebd14073f4a9241f89c3e3..21c1a8ddd970afd4dec122e078aee43d31d63148 100644 --- a/model-00030-of-00046.safetensors +++ b/model-00030-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:da88e3df8c32036b00adb3870df1c1563ec24fbc3587b2b409385bd98435be53 -size 3431183736 +oid sha256:182e5de93c63c5e9d50efae769bc1e7f64c130c4e6e802bc529ca541607d3a53 +size 3496770616 diff --git a/model-00031-of-00046.safetensors b/model-00031-of-00046.safetensors index dbb8e4e5fbb9a172b72524a05268bfa30827cd43..29674eb74d690b595ed456018002633210dee4d0 100644 --- a/model-00031-of-00046.safetensors +++ b/model-00031-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:e2b2f58bd586eeb14105211900148722d57bcd98b7dec6e411a2742b0c1fb774 -size 3423660608 +oid sha256:b621a2e8540c459e08b6285bb2c25df1c4d6cad486f8c3abb5abc312dc6aa306 +size 3479545776 diff --git a/model-00032-of-00046.safetensors b/model-00032-of-00046.safetensors index 15144e47532932efdf7026d41269ec7629a5c2b7..ba855948f440b0e40842f1f2d389d852c5964061 100644 --- a/model-00032-of-00046.safetensors +++ b/model-00032-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:66f14267c11cbe1fab8acf06a1bbe00fb9ffdd5647ee44e114b21f6799adb54d -size 3431183736 +oid sha256:f8cc5af0a73949f00442412e1a299b97637af208f80cb07cae1eb48cf243a42c +size 3496770616 diff --git a/model-00033-of-00046.safetensors b/model-00033-of-00046.safetensors index f3d662ad88656c1955b907af9ed912b33d1d6b36..1ad652536f7faca6c9183679e0475a0a29e3e67f 100644 --- a/model-00033-of-00046.safetensors +++ b/model-00033-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:b19498d784401b1d84ef1391a2b3ac96df34272ada471c96292e61258f326bc3 -size 3423660608 +oid sha256:afda71aac8f47c2c18615e0ddf244da3146d3550f2baee2edbbf5c5b614d6937 +size 3479545776 diff --git a/model-00034-of-00046.safetensors b/model-00034-of-00046.safetensors index 93ad1d8885ef545e4553ae862d82e8880451dbc4..237d08ae1eaddd450bd65dbcd9b8674bea753ad4 100644 --- a/model-00034-of-00046.safetensors +++ b/model-00034-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:1c543d6e20eaa4aace7200221ebe58580b44db25b70cb8891c57cbe822309008 -size 3431183736 +oid sha256:21ecf6ab3a1ff7c6962540a33b01d7e6bee7f42d24a3588ddb66602618da6dca +size 3496770616 diff --git a/model-00035-of-00046.safetensors b/model-00035-of-00046.safetensors index c6d79692f67cd5e2f7bba1a825a28ef92ff5cacd..3576612a4cecf0b3704428b73cd14994fca5ed0c 100644 --- a/model-00035-of-00046.safetensors +++ b/model-00035-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:96cdf8718ca753c080ccd425f4e73c90cf6af7500502f3b9197f36225c9d4949 -size 3423660608 +oid sha256:781ddbbf31f30a59cfd2b9a0d27ada5596b229779af7969921aea008f228d615 +size 3479545776 diff --git a/model-00036-of-00046.safetensors b/model-00036-of-00046.safetensors index 7bf8b58af14e7e7b2031bb51fd04f4dcf62bcb19..8bf8c8fddde91ee02ad347dc063192dd0cf23e77 100644 --- a/model-00036-of-00046.safetensors +++ b/model-00036-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:2cdae8b3fea57596b0b1e75d7e25e988c96163ce6b6877d989feec9611a16a76 -size 3431183736 +oid sha256:2b4cdbb8be17e42c8eb19f370df4b06e5262aae07a9ea348947effb78db25c58 +size 3496770616 diff --git a/model-00037-of-00046.safetensors b/model-00037-of-00046.safetensors index 048aa4b9bfc37d6ec63524061ec8fdecc9b2abb7..3a935097b3ddff09db3b7a09274e3166a0a95849 100644 --- a/model-00037-of-00046.safetensors +++ b/model-00037-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:a8532307c61e5b4baa57d3a49c50cb9cabc8a39b3477ecf4de0eb2523d1cb01f -size 3423660608 +oid sha256:4e2b208a2a4887aa434ea02143fcca905d2209cb0953c0258e8e930b50e2d6cf +size 3479545776 diff --git a/model-00038-of-00046.safetensors b/model-00038-of-00046.safetensors index dc632ca8e38ba1b4ecdc9af47dc6252daaa837c4..6b68fbfd7fdeb390c784ac9330a12480544f6e39 100644 --- a/model-00038-of-00046.safetensors +++ b/model-00038-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:6aaaa2e5372f786618e3265802bc03d23a66dafed7834d7f5a39ddff7dbaa09f -size 3431183736 +oid sha256:e91f54779818309a096a4abeadbbbd36617317706d65aa4e31362fa4b371aa51 +size 3496770616 diff --git a/model-00039-of-00046.safetensors b/model-00039-of-00046.safetensors index 15f484057f87cc6a1ef16c016cf729fe43d56839..50927c77adbb2609d82a87feff74bce6ffa928ea 100644 --- a/model-00039-of-00046.safetensors +++ b/model-00039-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:5d539be25733a9f174ce380ef2120a8392c5c469ea77d2faf980e6079dc18e1c -size 3423660608 +oid sha256:b5879afa52366dac9af42da20acb117222373067612f107042832c8ed96e4963 +size 3479545776 diff --git a/model-00040-of-00046.safetensors b/model-00040-of-00046.safetensors index b5c577bbf52bae643cb43146f438793bdc68ca45..8873d344c70abc5973df4dc5a819588b83c5d546 100644 --- a/model-00040-of-00046.safetensors +++ b/model-00040-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0e4939f98e60a79c1e8c0d6ef861191925a55c2ee128e974ca1bf740f80612ff -size 3431183736 +oid sha256:b2ca6d4421ebec09f14830273b95fa7102a97cbe60733046a5c2612c750dd1bd +size 3496770616 diff --git a/model-00041-of-00046.safetensors b/model-00041-of-00046.safetensors index f708587c90e5bfb26dad45f59fc6917156294802..75203587f13c8e5029da8ab92f9c80539c39ed01 100644 --- a/model-00041-of-00046.safetensors +++ b/model-00041-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:11668f1dc99abd3bbf274599d373dc839bd96df7e0113290e7e4b7590fad7466 -size 3423660608 +oid sha256:54680691e6eb715f82f31522af745045c913705f23ef5730a059332fad2ef77b +size 3479545776 diff --git a/model-00042-of-00046.safetensors b/model-00042-of-00046.safetensors index e1def8f6c9622554219d43886688be566e81002d..8050e34ecff76d3575b1f1648eeef3abeb254198 100644 --- a/model-00042-of-00046.safetensors +++ b/model-00042-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:6ddaae1f547ad5dfa7e595862c9513f19f88c9056de76ec74343fb34ffe7e4fc -size 3431183736 +oid sha256:ecd566bb6f100bcaf728a198c0cd0299995bdbf5e041650f47d48ec43264d2f8 +size 3496770616 diff --git a/model-00043-of-00046.safetensors b/model-00043-of-00046.safetensors index 9c7f5d763bac04559f18f9d5f5d2ef77a4f91ff2..7b54f4832d59d3d9c7aafa5035f6c11fba87bbf2 100644 --- a/model-00043-of-00046.safetensors +++ b/model-00043-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0ceffbe4e3437d0b7eb5e42ca3feed75816f704aa7b32637d2e3da9bda24a0b2 -size 3423660608 +oid sha256:db7a7d5a4e627f536b37afa7e679b520222d9dc7996271f2cb28782a48c0e704 +size 3479545776 diff --git a/model-00044-of-00046.safetensors b/model-00044-of-00046.safetensors index e1cdd1c0664bd8eeb2f7c1d380980c84f7bea19c..3c65cef7dd6319ba10c340f4ca471820a43c4266 100644 --- a/model-00044-of-00046.safetensors +++ b/model-00044-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:904f05584b895b34f70ba25f8a22b59fa5ee3a845253262016e91d3a7f5f468e -size 3431183736 +oid sha256:4b8128fd119dc6eaa64de3b1d18b0a46f80b21c87b06bbc373756cb3ed2094f1 +size 3496770616 diff --git a/model-00046-of-00046.safetensors b/model-00046-of-00046.safetensors index 7ecc8a06ac7388a322841197c3f3b1681a917180..b75d868ebba461e2582bd8e73ea7f8fc96330127 100644 --- a/model-00046-of-00046.safetensors +++ b/model-00046-of-00046.safetensors @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:13ab4077fbf65b726535aec1cafce96758dd44bea0513126e31a22ecfd770b16 -size 3438928548 +oid sha256:d45a083e13fdd59e9247e3aa4f4ad8da681a160ef36152e75d9739c93720d9d2 +size 3488604620 diff --git a/model.safetensors.index.json b/model.safetensors.index.json index 9a1a0103ed2ba81f318f88d6c15cf27dc2af1a01..fb8d9d4cb045364eb0a41a2d644b776445bc7f35 100644 --- a/model.safetensors.index.json +++ b/model.safetensors.index.json @@ -10,9 +10,7 @@ "layers.0.attn.wkv.qweight": "model-00002-of-00046.safetensors", "layers.0.attn.wkv.qzeros": "model-00002-of-00046.safetensors", "layers.0.attn.wkv.scales": "model-00002-of-00046.safetensors", - "layers.0.attn.wo_a.qweight": "model-00002-of-00046.safetensors", - "layers.0.attn.wo_a.qzeros": "model-00002-of-00046.safetensors", - "layers.0.attn.wo_a.scales": "model-00002-of-00046.safetensors", + "layers.0.attn.wo_a.weight": "model-00002-of-00046.safetensors", "layers.0.attn.wo_b.qweight": "model-00002-of-00046.safetensors", "layers.0.attn.wo_b.qzeros": "model-00002-of-00046.safetensors", "layers.0.attn.wo_b.scales": "model-00002-of-00046.safetensors", @@ -2327,8 +2325,6 @@ "layers.0.ffn.experts.99.w3.qweight": "model-00002-of-00046.safetensors", "layers.0.ffn.experts.99.w3.qzeros": "model-00002-of-00046.safetensors", "layers.0.ffn.experts.99.w3.scales": "model-00002-of-00046.safetensors", - "layers.0.ffn.gate.tid2eid": "model-00002-of-00046.safetensors", - "layers.0.ffn.gate.weight": "model-00002-of-00046.safetensors", "layers.0.ffn.shared_experts.w1.qweight": "model-00002-of-00046.safetensors", "layers.0.ffn.shared_experts.w1.qzeros": "model-00002-of-00046.safetensors", "layers.0.ffn.shared_experts.w1.scales": "model-00002-of-00046.safetensors", @@ -2345,15 +2341,15 @@ "layers.0.hc_ffn_base": "model-00002-of-00046.safetensors", "layers.0.hc_ffn_fn": "model-00002-of-00046.safetensors", "layers.0.hc_ffn_scale": "model-00002-of-00046.safetensors", + "layers.0.ffn.gate.tid2eid": "model-00002-of-00046.safetensors", + "layers.0.ffn.gate.weight": "model-00002-of-00046.safetensors", "layers.1.attn.attn_sink": "model-00003-of-00046.safetensors", "layers.1.attn.kv_norm.weight": "model-00003-of-00046.safetensors", "layers.1.attn.q_norm.weight": "model-00003-of-00046.safetensors", "layers.1.attn.wkv.qweight": "model-00003-of-00046.safetensors", "layers.1.attn.wkv.qzeros": "model-00003-of-00046.safetensors", "layers.1.attn.wkv.scales": "model-00003-of-00046.safetensors", - "layers.1.attn.wo_a.qweight": "model-00003-of-00046.safetensors", - "layers.1.attn.wo_a.qzeros": "model-00003-of-00046.safetensors", - "layers.1.attn.wo_a.scales": "model-00003-of-00046.safetensors", + "layers.1.attn.wo_a.weight": "model-00003-of-00046.safetensors", "layers.1.attn.wo_b.qweight": "model-00003-of-00046.safetensors", "layers.1.attn.wo_b.qzeros": "model-00003-of-00046.safetensors", "layers.1.attn.wo_b.scales": "model-00003-of-00046.safetensors", @@ -4668,8 +4664,6 @@ "layers.1.ffn.experts.99.w3.qweight": "model-00003-of-00046.safetensors", "layers.1.ffn.experts.99.w3.qzeros": "model-00003-of-00046.safetensors", "layers.1.ffn.experts.99.w3.scales": "model-00003-of-00046.safetensors", - "layers.1.ffn.gate.tid2eid": "model-00003-of-00046.safetensors", - "layers.1.ffn.gate.weight": "model-00003-of-00046.safetensors", "layers.1.ffn.shared_experts.w1.qweight": "model-00003-of-00046.safetensors", "layers.1.ffn.shared_experts.w1.qzeros": "model-00003-of-00046.safetensors", "layers.1.ffn.shared_experts.w1.scales": "model-00003-of-00046.safetensors", @@ -4686,26 +4680,11 @@ "layers.1.hc_ffn_base": "model-00003-of-00046.safetensors", "layers.1.hc_ffn_fn": "model-00003-of-00046.safetensors", "layers.1.hc_ffn_scale": "model-00003-of-00046.safetensors", + "layers.1.ffn.gate.tid2eid": "model-00003-of-00046.safetensors", + "layers.1.ffn.gate.weight": "model-00003-of-00046.safetensors", "layers.2.attn.attn_sink": "model-00004-of-00046.safetensors", "layers.2.attn.compressor.ape": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.norm.weight": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.wgate.qweight": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.wgate.qzeros": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.wgate.scales": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.wkv.qweight": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.wkv.qzeros": "model-00004-of-00046.safetensors", - "layers.2.attn.compressor.wkv.scales": "model-00004-of-00046.safetensors", "layers.2.attn.indexer.compressor.ape": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.norm.weight": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.wgate.qweight": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.wgate.qzeros": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.wgate.scales": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.wkv.qweight": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.wkv.qzeros": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.compressor.wkv.scales": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.weights_proj.qweight": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.weights_proj.qzeros": "model-00004-of-00046.safetensors", - "layers.2.attn.indexer.weights_proj.scales": "model-00004-of-00046.safetensors", "layers.2.attn.indexer.wq_b.qweight": "model-00004-of-00046.safetensors", "layers.2.attn.indexer.wq_b.qzeros": "model-00004-of-00046.safetensors", "layers.2.attn.indexer.wq_b.scales": "model-00004-of-00046.safetensors", @@ -4714,9 +4693,7 @@ "layers.2.attn.wkv.qweight": "model-00004-of-00046.safetensors", "layers.2.attn.wkv.qzeros": "model-00004-of-00046.safetensors", "layers.2.attn.wkv.scales": "model-00004-of-00046.safetensors", - "layers.2.attn.wo_a.qweight": "model-00004-of-00046.safetensors", - "layers.2.attn.wo_a.qzeros": "model-00004-of-00046.safetensors", - "layers.2.attn.wo_a.scales": "model-00004-of-00046.safetensors", + "layers.2.attn.wo_a.weight": "model-00004-of-00046.safetensors", "layers.2.attn.wo_b.qweight": "model-00004-of-00046.safetensors", "layers.2.attn.wo_b.qzeros": "model-00004-of-00046.safetensors", "layers.2.attn.wo_b.scales": "model-00004-of-00046.safetensors", @@ -7031,8 +7008,6 @@ "layers.2.ffn.experts.99.w3.qweight": "model-00004-of-00046.safetensors", "layers.2.ffn.experts.99.w3.qzeros": "model-00004-of-00046.safetensors", "layers.2.ffn.experts.99.w3.scales": "model-00004-of-00046.safetensors", - "layers.2.ffn.gate.tid2eid": "model-00004-of-00046.safetensors", - "layers.2.ffn.gate.weight": "model-00004-of-00046.safetensors", "layers.2.ffn.shared_experts.w1.qweight": "model-00004-of-00046.safetensors", "layers.2.ffn.shared_experts.w1.qzeros": "model-00004-of-00046.safetensors", "layers.2.ffn.shared_experts.w1.scales": "model-00004-of-00046.safetensors", @@ -7049,23 +7024,23 @@ "layers.2.hc_ffn_base": "model-00004-of-00046.safetensors", "layers.2.hc_ffn_fn": "model-00004-of-00046.safetensors", "layers.2.hc_ffn_scale": "model-00004-of-00046.safetensors", + "layers.2.attn.compressor.norm.weight": "model-00004-of-00046.safetensors", + "layers.2.attn.compressor.wgate.weight": "model-00004-of-00046.safetensors", + "layers.2.attn.compressor.wkv.weight": "model-00004-of-00046.safetensors", + "layers.2.attn.indexer.compressor.norm.weight": "model-00004-of-00046.safetensors", + "layers.2.attn.indexer.compressor.wgate.weight": "model-00004-of-00046.safetensors", + "layers.2.attn.indexer.compressor.wkv.weight": "model-00004-of-00046.safetensors", + "layers.2.attn.indexer.weights_proj.weight": "model-00004-of-00046.safetensors", + "layers.2.ffn.gate.tid2eid": "model-00004-of-00046.safetensors", + "layers.2.ffn.gate.weight": "model-00004-of-00046.safetensors", "layers.3.attn.attn_sink": "model-00005-of-00046.safetensors", "layers.3.attn.compressor.ape": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.norm.weight": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.wgate.qweight": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.wgate.qzeros": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.wgate.scales": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.wkv.qweight": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.wkv.qzeros": "model-00005-of-00046.safetensors", - "layers.3.attn.compressor.wkv.scales": "model-00005-of-00046.safetensors", "layers.3.attn.kv_norm.weight": "model-00005-of-00046.safetensors", "layers.3.attn.q_norm.weight": "model-00005-of-00046.safetensors", "layers.3.attn.wkv.qweight": "model-00005-of-00046.safetensors", "layers.3.attn.wkv.qzeros": "model-00005-of-00046.safetensors", "layers.3.attn.wkv.scales": "model-00005-of-00046.safetensors", - "layers.3.attn.wo_a.qweight": "model-00005-of-00046.safetensors", - "layers.3.attn.wo_a.qzeros": "model-00005-of-00046.safetensors", - "layers.3.attn.wo_a.scales": "model-00005-of-00046.safetensors", + "layers.3.attn.wo_a.weight": "model-00005-of-00046.safetensors", "layers.3.attn.wo_b.qweight": "model-00005-of-00046.safetensors", "layers.3.attn.wo_b.qzeros": "model-00005-of-00046.safetensors", "layers.3.attn.wo_b.scales": "model-00005-of-00046.safetensors", @@ -9380,8 +9355,6 @@ "layers.3.ffn.experts.99.w3.qweight": "model-00005-of-00046.safetensors", "layers.3.ffn.experts.99.w3.qzeros": "model-00005-of-00046.safetensors", "layers.3.ffn.experts.99.w3.scales": "model-00005-of-00046.safetensors", - "layers.3.ffn.gate.bias": "model-00005-of-00046.safetensors", - "layers.3.ffn.gate.weight": "model-00005-of-00046.safetensors", "layers.3.ffn.shared_experts.w1.qweight": "model-00005-of-00046.safetensors", "layers.3.ffn.shared_experts.w1.qzeros": "model-00005-of-00046.safetensors", "layers.3.ffn.shared_experts.w1.scales": "model-00005-of-00046.safetensors", @@ -9398,26 +9371,14 @@ "layers.3.hc_ffn_base": "model-00005-of-00046.safetensors", "layers.3.hc_ffn_fn": "model-00005-of-00046.safetensors", "layers.3.hc_ffn_scale": "model-00005-of-00046.safetensors", + "layers.3.attn.compressor.norm.weight": "model-00005-of-00046.safetensors", + "layers.3.attn.compressor.wgate.weight": "model-00005-of-00046.safetensors", + "layers.3.attn.compressor.wkv.weight": "model-00005-of-00046.safetensors", + "layers.3.ffn.gate.bias": "model-00005-of-00046.safetensors", + "layers.3.ffn.gate.weight": "model-00005-of-00046.safetensors", "layers.4.attn.attn_sink": "model-00006-of-00046.safetensors", "layers.4.attn.compressor.ape": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.norm.weight": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.wgate.qweight": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.wgate.qzeros": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.wgate.scales": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.wkv.qweight": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.wkv.qzeros": "model-00006-of-00046.safetensors", - "layers.4.attn.compressor.wkv.scales": "model-00006-of-00046.safetensors", "layers.4.attn.indexer.compressor.ape": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.norm.weight": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.wgate.qweight": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.wgate.qzeros": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.wgate.scales": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.wkv.qweight": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.wkv.qzeros": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.compressor.wkv.scales": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.weights_proj.qweight": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.weights_proj.qzeros": "model-00006-of-00046.safetensors", - "layers.4.attn.indexer.weights_proj.scales": "model-00006-of-00046.safetensors", "layers.4.attn.indexer.wq_b.qweight": "model-00006-of-00046.safetensors", "layers.4.attn.indexer.wq_b.qzeros": "model-00006-of-00046.safetensors", "layers.4.attn.indexer.wq_b.scales": "model-00006-of-00046.safetensors", @@ -9426,9 +9387,7 @@ "layers.4.attn.wkv.qweight": "model-00006-of-00046.safetensors", "layers.4.attn.wkv.qzeros": "model-00006-of-00046.safetensors", "layers.4.attn.wkv.scales": "model-00006-of-00046.safetensors", - "layers.4.attn.wo_a.qweight": "model-00006-of-00046.safetensors", - "layers.4.attn.wo_a.qzeros": "model-00006-of-00046.safetensors", - "layers.4.attn.wo_a.scales": "model-00006-of-00046.safetensors", + "layers.4.attn.wo_a.weight": "model-00006-of-00046.safetensors", "layers.4.attn.wo_b.qweight": "model-00006-of-00046.safetensors", "layers.4.attn.wo_b.qzeros": "model-00006-of-00046.safetensors", "layers.4.attn.wo_b.scales": "model-00006-of-00046.safetensors", @@ -11743,8 +11702,6 @@ "layers.4.ffn.experts.99.w3.qweight": "model-00006-of-00046.safetensors", "layers.4.ffn.experts.99.w3.qzeros": "model-00006-of-00046.safetensors", "layers.4.ffn.experts.99.w3.scales": "model-00006-of-00046.safetensors", - "layers.4.ffn.gate.bias": "model-00006-of-00046.safetensors", - "layers.4.ffn.gate.weight": "model-00006-of-00046.safetensors", "layers.4.ffn.shared_experts.w1.qweight": "model-00006-of-00046.safetensors", "layers.4.ffn.shared_experts.w1.qzeros": "model-00006-of-00046.safetensors", "layers.4.ffn.shared_experts.w1.scales": "model-00006-of-00046.safetensors", @@ -11761,23 +11718,23 @@ "layers.4.hc_ffn_base": "model-00006-of-00046.safetensors", "layers.4.hc_ffn_fn": "model-00006-of-00046.safetensors", "layers.4.hc_ffn_scale": "model-00006-of-00046.safetensors", + "layers.4.attn.compressor.norm.weight": "model-00006-of-00046.safetensors", + "layers.4.attn.compressor.wgate.weight": "model-00006-of-00046.safetensors", + "layers.4.attn.compressor.wkv.weight": "model-00006-of-00046.safetensors", + "layers.4.attn.indexer.compressor.norm.weight": "model-00006-of-00046.safetensors", + "layers.4.attn.indexer.compressor.wgate.weight": "model-00006-of-00046.safetensors", + "layers.4.attn.indexer.compressor.wkv.weight": "model-00006-of-00046.safetensors", + "layers.4.attn.indexer.weights_proj.weight": "model-00006-of-00046.safetensors", + "layers.4.ffn.gate.bias": "model-00006-of-00046.safetensors", + "layers.4.ffn.gate.weight": "model-00006-of-00046.safetensors", "layers.5.attn.attn_sink": "model-00007-of-00046.safetensors", "layers.5.attn.compressor.ape": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.norm.weight": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.wgate.qweight": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.wgate.qzeros": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.wgate.scales": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.wkv.qweight": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.wkv.qzeros": "model-00007-of-00046.safetensors", - "layers.5.attn.compressor.wkv.scales": "model-00007-of-00046.safetensors", "layers.5.attn.kv_norm.weight": "model-00007-of-00046.safetensors", "layers.5.attn.q_norm.weight": "model-00007-of-00046.safetensors", "layers.5.attn.wkv.qweight": "model-00007-of-00046.safetensors", "layers.5.attn.wkv.qzeros": "model-00007-of-00046.safetensors", "layers.5.attn.wkv.scales": "model-00007-of-00046.safetensors", - "layers.5.attn.wo_a.qweight": "model-00007-of-00046.safetensors", - "layers.5.attn.wo_a.qzeros": "model-00007-of-00046.safetensors", - "layers.5.attn.wo_a.scales": "model-00007-of-00046.safetensors", + "layers.5.attn.wo_a.weight": "model-00007-of-00046.safetensors", "layers.5.attn.wo_b.qweight": "model-00007-of-00046.safetensors", "layers.5.attn.wo_b.qzeros": "model-00007-of-00046.safetensors", "layers.5.attn.wo_b.scales": "model-00007-of-00046.safetensors", @@ -14092,8 +14049,6 @@ "layers.5.ffn.experts.99.w3.qweight": "model-00007-of-00046.safetensors", "layers.5.ffn.experts.99.w3.qzeros": "model-00007-of-00046.safetensors", "layers.5.ffn.experts.99.w3.scales": "model-00007-of-00046.safetensors", - "layers.5.ffn.gate.bias": "model-00007-of-00046.safetensors", - "layers.5.ffn.gate.weight": "model-00007-of-00046.safetensors", "layers.5.ffn.shared_experts.w1.qweight": "model-00007-of-00046.safetensors", "layers.5.ffn.shared_experts.w1.qzeros": "model-00007-of-00046.safetensors", "layers.5.ffn.shared_experts.w1.scales": "model-00007-of-00046.safetensors", @@ -14110,26 +14065,14 @@ "layers.5.hc_ffn_base": "model-00007-of-00046.safetensors", "layers.5.hc_ffn_fn": "model-00007-of-00046.safetensors", "layers.5.hc_ffn_scale": "model-00007-of-00046.safetensors", + "layers.5.attn.compressor.norm.weight": "model-00007-of-00046.safetensors", + "layers.5.attn.compressor.wgate.weight": "model-00007-of-00046.safetensors", + "layers.5.attn.compressor.wkv.weight": "model-00007-of-00046.safetensors", + "layers.5.ffn.gate.bias": "model-00007-of-00046.safetensors", + "layers.5.ffn.gate.weight": "model-00007-of-00046.safetensors", "layers.6.attn.attn_sink": "model-00008-of-00046.safetensors", "layers.6.attn.compressor.ape": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.norm.weight": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.wgate.qweight": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.wgate.qzeros": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.wgate.scales": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.wkv.qweight": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.wkv.qzeros": "model-00008-of-00046.safetensors", - "layers.6.attn.compressor.wkv.scales": "model-00008-of-00046.safetensors", "layers.6.attn.indexer.compressor.ape": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.norm.weight": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.wgate.qweight": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.wgate.qzeros": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.wgate.scales": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.wkv.qweight": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.wkv.qzeros": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.compressor.wkv.scales": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.weights_proj.qweight": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.weights_proj.qzeros": "model-00008-of-00046.safetensors", - "layers.6.attn.indexer.weights_proj.scales": "model-00008-of-00046.safetensors", "layers.6.attn.indexer.wq_b.qweight": "model-00008-of-00046.safetensors", "layers.6.attn.indexer.wq_b.qzeros": "model-00008-of-00046.safetensors", "layers.6.attn.indexer.wq_b.scales": "model-00008-of-00046.safetensors", @@ -14138,9 +14081,7 @@ "layers.6.attn.wkv.qweight": "model-00008-of-00046.safetensors", "layers.6.attn.wkv.qzeros": "model-00008-of-00046.safetensors", "layers.6.attn.wkv.scales": "model-00008-of-00046.safetensors", - "layers.6.attn.wo_a.qweight": "model-00008-of-00046.safetensors", - "layers.6.attn.wo_a.qzeros": "model-00008-of-00046.safetensors", - "layers.6.attn.wo_a.scales": "model-00008-of-00046.safetensors", + "layers.6.attn.wo_a.weight": "model-00008-of-00046.safetensors", "layers.6.attn.wo_b.qweight": "model-00008-of-00046.safetensors", "layers.6.attn.wo_b.qzeros": "model-00008-of-00046.safetensors", "layers.6.attn.wo_b.scales": "model-00008-of-00046.safetensors", @@ -16455,8 +16396,6 @@ "layers.6.ffn.experts.99.w3.qweight": "model-00008-of-00046.safetensors", "layers.6.ffn.experts.99.w3.qzeros": "model-00008-of-00046.safetensors", "layers.6.ffn.experts.99.w3.scales": "model-00008-of-00046.safetensors", - "layers.6.ffn.gate.bias": "model-00008-of-00046.safetensors", - "layers.6.ffn.gate.weight": "model-00008-of-00046.safetensors", "layers.6.ffn.shared_experts.w1.qweight": "model-00008-of-00046.safetensors", "layers.6.ffn.shared_experts.w1.qzeros": "model-00008-of-00046.safetensors", "layers.6.ffn.shared_experts.w1.scales": "model-00008-of-00046.safetensors", @@ -16473,23 +16412,23 @@ "layers.6.hc_ffn_base": "model-00008-of-00046.safetensors", "layers.6.hc_ffn_fn": "model-00008-of-00046.safetensors", "layers.6.hc_ffn_scale": "model-00008-of-00046.safetensors", + "layers.6.attn.compressor.norm.weight": "model-00008-of-00046.safetensors", + "layers.6.attn.compressor.wgate.weight": "model-00008-of-00046.safetensors", + "layers.6.attn.compressor.wkv.weight": "model-00008-of-00046.safetensors", + "layers.6.attn.indexer.compressor.norm.weight": "model-00008-of-00046.safetensors", + "layers.6.attn.indexer.compressor.wgate.weight": "model-00008-of-00046.safetensors", + "layers.6.attn.indexer.compressor.wkv.weight": "model-00008-of-00046.safetensors", + "layers.6.attn.indexer.weights_proj.weight": "model-00008-of-00046.safetensors", + "layers.6.ffn.gate.bias": "model-00008-of-00046.safetensors", + "layers.6.ffn.gate.weight": "model-00008-of-00046.safetensors", "layers.7.attn.attn_sink": "model-00009-of-00046.safetensors", "layers.7.attn.compressor.ape": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.norm.weight": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.wgate.qweight": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.wgate.qzeros": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.wgate.scales": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.wkv.qweight": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.wkv.qzeros": "model-00009-of-00046.safetensors", - "layers.7.attn.compressor.wkv.scales": "model-00009-of-00046.safetensors", "layers.7.attn.kv_norm.weight": "model-00009-of-00046.safetensors", "layers.7.attn.q_norm.weight": "model-00009-of-00046.safetensors", "layers.7.attn.wkv.qweight": "model-00009-of-00046.safetensors", "layers.7.attn.wkv.qzeros": "model-00009-of-00046.safetensors", "layers.7.attn.wkv.scales": "model-00009-of-00046.safetensors", - "layers.7.attn.wo_a.qweight": "model-00009-of-00046.safetensors", - "layers.7.attn.wo_a.qzeros": "model-00009-of-00046.safetensors", - "layers.7.attn.wo_a.scales": "model-00009-of-00046.safetensors", + "layers.7.attn.wo_a.weight": "model-00009-of-00046.safetensors", "layers.7.attn.wo_b.qweight": "model-00009-of-00046.safetensors", "layers.7.attn.wo_b.qzeros": "model-00009-of-00046.safetensors", "layers.7.attn.wo_b.scales": "model-00009-of-00046.safetensors", @@ -18804,8 +18743,6 @@ "layers.7.ffn.experts.99.w3.qweight": "model-00009-of-00046.safetensors", "layers.7.ffn.experts.99.w3.qzeros": "model-00009-of-00046.safetensors", "layers.7.ffn.experts.99.w3.scales": "model-00009-of-00046.safetensors", - "layers.7.ffn.gate.bias": "model-00009-of-00046.safetensors", - "layers.7.ffn.gate.weight": "model-00009-of-00046.safetensors", "layers.7.ffn.shared_experts.w1.qweight": "model-00009-of-00046.safetensors", "layers.7.ffn.shared_experts.w1.qzeros": "model-00009-of-00046.safetensors", "layers.7.ffn.shared_experts.w1.scales": "model-00009-of-00046.safetensors", @@ -18822,26 +18759,14 @@ "layers.7.hc_ffn_base": "model-00009-of-00046.safetensors", "layers.7.hc_ffn_fn": "model-00009-of-00046.safetensors", "layers.7.hc_ffn_scale": "model-00009-of-00046.safetensors", + "layers.7.attn.compressor.norm.weight": "model-00009-of-00046.safetensors", + "layers.7.attn.compressor.wgate.weight": "model-00009-of-00046.safetensors", + "layers.7.attn.compressor.wkv.weight": "model-00009-of-00046.safetensors", + "layers.7.ffn.gate.bias": "model-00009-of-00046.safetensors", + "layers.7.ffn.gate.weight": "model-00009-of-00046.safetensors", "layers.8.attn.attn_sink": "model-00010-of-00046.safetensors", "layers.8.attn.compressor.ape": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.norm.weight": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.wgate.qweight": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.wgate.qzeros": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.wgate.scales": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.wkv.qweight": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.wkv.qzeros": "model-00010-of-00046.safetensors", - "layers.8.attn.compressor.wkv.scales": "model-00010-of-00046.safetensors", "layers.8.attn.indexer.compressor.ape": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.norm.weight": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.wgate.qweight": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.wgate.qzeros": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.wgate.scales": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.wkv.qweight": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.wkv.qzeros": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.compressor.wkv.scales": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.weights_proj.qweight": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.weights_proj.qzeros": "model-00010-of-00046.safetensors", - "layers.8.attn.indexer.weights_proj.scales": "model-00010-of-00046.safetensors", "layers.8.attn.indexer.wq_b.qweight": "model-00010-of-00046.safetensors", "layers.8.attn.indexer.wq_b.qzeros": "model-00010-of-00046.safetensors", "layers.8.attn.indexer.wq_b.scales": "model-00010-of-00046.safetensors", @@ -18850,9 +18775,7 @@ "layers.8.attn.wkv.qweight": "model-00010-of-00046.safetensors", "layers.8.attn.wkv.qzeros": "model-00010-of-00046.safetensors", "layers.8.attn.wkv.scales": "model-00010-of-00046.safetensors", - "layers.8.attn.wo_a.qweight": "model-00010-of-00046.safetensors", - "layers.8.attn.wo_a.qzeros": "model-00010-of-00046.safetensors", - "layers.8.attn.wo_a.scales": "model-00010-of-00046.safetensors", + "layers.8.attn.wo_a.weight": "model-00010-of-00046.safetensors", "layers.8.attn.wo_b.qweight": "model-00010-of-00046.safetensors", "layers.8.attn.wo_b.qzeros": "model-00010-of-00046.safetensors", "layers.8.attn.wo_b.scales": "model-00010-of-00046.safetensors", @@ -21167,8 +21090,6 @@ "layers.8.ffn.experts.99.w3.qweight": "model-00010-of-00046.safetensors", "layers.8.ffn.experts.99.w3.qzeros": "model-00010-of-00046.safetensors", "layers.8.ffn.experts.99.w3.scales": "model-00010-of-00046.safetensors", - "layers.8.ffn.gate.bias": "model-00010-of-00046.safetensors", - "layers.8.ffn.gate.weight": "model-00010-of-00046.safetensors", "layers.8.ffn.shared_experts.w1.qweight": "model-00010-of-00046.safetensors", "layers.8.ffn.shared_experts.w1.qzeros": "model-00010-of-00046.safetensors", "layers.8.ffn.shared_experts.w1.scales": "model-00010-of-00046.safetensors", @@ -21185,23 +21106,23 @@ "layers.8.hc_ffn_base": "model-00010-of-00046.safetensors", "layers.8.hc_ffn_fn": "model-00010-of-00046.safetensors", "layers.8.hc_ffn_scale": "model-00010-of-00046.safetensors", + "layers.8.attn.compressor.norm.weight": "model-00010-of-00046.safetensors", + "layers.8.attn.compressor.wgate.weight": "model-00010-of-00046.safetensors", + "layers.8.attn.compressor.wkv.weight": "model-00010-of-00046.safetensors", + "layers.8.attn.indexer.compressor.norm.weight": "model-00010-of-00046.safetensors", + "layers.8.attn.indexer.compressor.wgate.weight": "model-00010-of-00046.safetensors", + "layers.8.attn.indexer.compressor.wkv.weight": "model-00010-of-00046.safetensors", + "layers.8.attn.indexer.weights_proj.weight": "model-00010-of-00046.safetensors", + "layers.8.ffn.gate.bias": "model-00010-of-00046.safetensors", + "layers.8.ffn.gate.weight": "model-00010-of-00046.safetensors", "layers.9.attn.attn_sink": "model-00011-of-00046.safetensors", "layers.9.attn.compressor.ape": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.norm.weight": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.wgate.qweight": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.wgate.qzeros": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.wgate.scales": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.wkv.qweight": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.wkv.qzeros": "model-00011-of-00046.safetensors", - "layers.9.attn.compressor.wkv.scales": "model-00011-of-00046.safetensors", "layers.9.attn.kv_norm.weight": "model-00011-of-00046.safetensors", "layers.9.attn.q_norm.weight": "model-00011-of-00046.safetensors", "layers.9.attn.wkv.qweight": "model-00011-of-00046.safetensors", "layers.9.attn.wkv.qzeros": "model-00011-of-00046.safetensors", "layers.9.attn.wkv.scales": "model-00011-of-00046.safetensors", - "layers.9.attn.wo_a.qweight": "model-00011-of-00046.safetensors", - "layers.9.attn.wo_a.qzeros": "model-00011-of-00046.safetensors", - "layers.9.attn.wo_a.scales": "model-00011-of-00046.safetensors", + "layers.9.attn.wo_a.weight": "model-00011-of-00046.safetensors", "layers.9.attn.wo_b.qweight": "model-00011-of-00046.safetensors", "layers.9.attn.wo_b.qzeros": "model-00011-of-00046.safetensors", "layers.9.attn.wo_b.scales": "model-00011-of-00046.safetensors", @@ -23516,8 +23437,6 @@ "layers.9.ffn.experts.99.w3.qweight": "model-00011-of-00046.safetensors", "layers.9.ffn.experts.99.w3.qzeros": "model-00011-of-00046.safetensors", "layers.9.ffn.experts.99.w3.scales": "model-00011-of-00046.safetensors", - "layers.9.ffn.gate.bias": "model-00011-of-00046.safetensors", - "layers.9.ffn.gate.weight": "model-00011-of-00046.safetensors", "layers.9.ffn.shared_experts.w1.qweight": "model-00011-of-00046.safetensors", "layers.9.ffn.shared_experts.w1.qzeros": "model-00011-of-00046.safetensors", "layers.9.ffn.shared_experts.w1.scales": "model-00011-of-00046.safetensors", @@ -23534,26 +23453,14 @@ "layers.9.hc_ffn_base": "model-00011-of-00046.safetensors", "layers.9.hc_ffn_fn": "model-00011-of-00046.safetensors", "layers.9.hc_ffn_scale": "model-00011-of-00046.safetensors", + "layers.9.attn.compressor.norm.weight": "model-00011-of-00046.safetensors", + "layers.9.attn.compressor.wgate.weight": "model-00011-of-00046.safetensors", + "layers.9.attn.compressor.wkv.weight": "model-00011-of-00046.safetensors", + "layers.9.ffn.gate.bias": "model-00011-of-00046.safetensors", + "layers.9.ffn.gate.weight": "model-00011-of-00046.safetensors", "layers.10.attn.attn_sink": "model-00012-of-00046.safetensors", "layers.10.attn.compressor.ape": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.norm.weight": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.wgate.qweight": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.wgate.qzeros": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.wgate.scales": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.wkv.qweight": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.wkv.qzeros": "model-00012-of-00046.safetensors", - "layers.10.attn.compressor.wkv.scales": "model-00012-of-00046.safetensors", "layers.10.attn.indexer.compressor.ape": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.norm.weight": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.wgate.qweight": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.wgate.qzeros": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.wgate.scales": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.wkv.qweight": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.wkv.qzeros": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.compressor.wkv.scales": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.weights_proj.qweight": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.weights_proj.qzeros": "model-00012-of-00046.safetensors", - "layers.10.attn.indexer.weights_proj.scales": "model-00012-of-00046.safetensors", "layers.10.attn.indexer.wq_b.qweight": "model-00012-of-00046.safetensors", "layers.10.attn.indexer.wq_b.qzeros": "model-00012-of-00046.safetensors", "layers.10.attn.indexer.wq_b.scales": "model-00012-of-00046.safetensors", @@ -23562,9 +23469,7 @@ "layers.10.attn.wkv.qweight": "model-00012-of-00046.safetensors", "layers.10.attn.wkv.qzeros": "model-00012-of-00046.safetensors", "layers.10.attn.wkv.scales": "model-00012-of-00046.safetensors", - "layers.10.attn.wo_a.qweight": "model-00012-of-00046.safetensors", - "layers.10.attn.wo_a.qzeros": "model-00012-of-00046.safetensors", - "layers.10.attn.wo_a.scales": "model-00012-of-00046.safetensors", + "layers.10.attn.wo_a.weight": "model-00012-of-00046.safetensors", "layers.10.attn.wo_b.qweight": "model-00012-of-00046.safetensors", "layers.10.attn.wo_b.qzeros": "model-00012-of-00046.safetensors", "layers.10.attn.wo_b.scales": "model-00012-of-00046.safetensors", @@ -25879,8 +25784,6 @@ "layers.10.ffn.experts.99.w3.qweight": "model-00012-of-00046.safetensors", "layers.10.ffn.experts.99.w3.qzeros": "model-00012-of-00046.safetensors", "layers.10.ffn.experts.99.w3.scales": "model-00012-of-00046.safetensors", - "layers.10.ffn.gate.bias": "model-00012-of-00046.safetensors", - "layers.10.ffn.gate.weight": "model-00012-of-00046.safetensors", "layers.10.ffn.shared_experts.w1.qweight": "model-00012-of-00046.safetensors", "layers.10.ffn.shared_experts.w1.qzeros": "model-00012-of-00046.safetensors", "layers.10.ffn.shared_experts.w1.scales": "model-00012-of-00046.safetensors", @@ -25897,23 +25800,23 @@ "layers.10.hc_ffn_base": "model-00012-of-00046.safetensors", "layers.10.hc_ffn_fn": "model-00012-of-00046.safetensors", "layers.10.hc_ffn_scale": "model-00012-of-00046.safetensors", + "layers.10.attn.compressor.norm.weight": "model-00012-of-00046.safetensors", + "layers.10.attn.compressor.wgate.weight": "model-00012-of-00046.safetensors", + "layers.10.attn.compressor.wkv.weight": "model-00012-of-00046.safetensors", + "layers.10.attn.indexer.compressor.norm.weight": "model-00012-of-00046.safetensors", + "layers.10.attn.indexer.compressor.wgate.weight": "model-00012-of-00046.safetensors", + "layers.10.attn.indexer.compressor.wkv.weight": "model-00012-of-00046.safetensors", + "layers.10.attn.indexer.weights_proj.weight": "model-00012-of-00046.safetensors", + "layers.10.ffn.gate.bias": "model-00012-of-00046.safetensors", + "layers.10.ffn.gate.weight": "model-00012-of-00046.safetensors", "layers.11.attn.attn_sink": "model-00013-of-00046.safetensors", "layers.11.attn.compressor.ape": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.norm.weight": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.wgate.qweight": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.wgate.qzeros": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.wgate.scales": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.wkv.qweight": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.wkv.qzeros": "model-00013-of-00046.safetensors", - "layers.11.attn.compressor.wkv.scales": "model-00013-of-00046.safetensors", "layers.11.attn.kv_norm.weight": "model-00013-of-00046.safetensors", "layers.11.attn.q_norm.weight": "model-00013-of-00046.safetensors", "layers.11.attn.wkv.qweight": "model-00013-of-00046.safetensors", "layers.11.attn.wkv.qzeros": "model-00013-of-00046.safetensors", "layers.11.attn.wkv.scales": "model-00013-of-00046.safetensors", - "layers.11.attn.wo_a.qweight": "model-00013-of-00046.safetensors", - "layers.11.attn.wo_a.qzeros": "model-00013-of-00046.safetensors", - "layers.11.attn.wo_a.scales": "model-00013-of-00046.safetensors", + "layers.11.attn.wo_a.weight": "model-00013-of-00046.safetensors", "layers.11.attn.wo_b.qweight": "model-00013-of-00046.safetensors", "layers.11.attn.wo_b.qzeros": "model-00013-of-00046.safetensors", "layers.11.attn.wo_b.scales": "model-00013-of-00046.safetensors", @@ -28228,8 +28131,6 @@ "layers.11.ffn.experts.99.w3.qweight": "model-00013-of-00046.safetensors", "layers.11.ffn.experts.99.w3.qzeros": "model-00013-of-00046.safetensors", "layers.11.ffn.experts.99.w3.scales": "model-00013-of-00046.safetensors", - "layers.11.ffn.gate.bias": "model-00013-of-00046.safetensors", - "layers.11.ffn.gate.weight": "model-00013-of-00046.safetensors", "layers.11.ffn.shared_experts.w1.qweight": "model-00013-of-00046.safetensors", "layers.11.ffn.shared_experts.w1.qzeros": "model-00013-of-00046.safetensors", "layers.11.ffn.shared_experts.w1.scales": "model-00013-of-00046.safetensors", @@ -28246,26 +28147,14 @@ "layers.11.hc_ffn_base": "model-00013-of-00046.safetensors", "layers.11.hc_ffn_fn": "model-00013-of-00046.safetensors", "layers.11.hc_ffn_scale": "model-00013-of-00046.safetensors", + "layers.11.attn.compressor.norm.weight": "model-00013-of-00046.safetensors", + "layers.11.attn.compressor.wgate.weight": "model-00013-of-00046.safetensors", + "layers.11.attn.compressor.wkv.weight": "model-00013-of-00046.safetensors", + "layers.11.ffn.gate.bias": "model-00013-of-00046.safetensors", + "layers.11.ffn.gate.weight": "model-00013-of-00046.safetensors", "layers.12.attn.attn_sink": "model-00014-of-00046.safetensors", "layers.12.attn.compressor.ape": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.norm.weight": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.wgate.qweight": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.wgate.qzeros": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.wgate.scales": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.wkv.qweight": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.wkv.qzeros": "model-00014-of-00046.safetensors", - "layers.12.attn.compressor.wkv.scales": "model-00014-of-00046.safetensors", "layers.12.attn.indexer.compressor.ape": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.norm.weight": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.wgate.qweight": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.wgate.qzeros": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.wgate.scales": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.wkv.qweight": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.wkv.qzeros": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.compressor.wkv.scales": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.weights_proj.qweight": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.weights_proj.qzeros": "model-00014-of-00046.safetensors", - "layers.12.attn.indexer.weights_proj.scales": "model-00014-of-00046.safetensors", "layers.12.attn.indexer.wq_b.qweight": "model-00014-of-00046.safetensors", "layers.12.attn.indexer.wq_b.qzeros": "model-00014-of-00046.safetensors", "layers.12.attn.indexer.wq_b.scales": "model-00014-of-00046.safetensors", @@ -28274,9 +28163,7 @@ "layers.12.attn.wkv.qweight": "model-00014-of-00046.safetensors", "layers.12.attn.wkv.qzeros": "model-00014-of-00046.safetensors", "layers.12.attn.wkv.scales": "model-00014-of-00046.safetensors", - "layers.12.attn.wo_a.qweight": "model-00014-of-00046.safetensors", - "layers.12.attn.wo_a.qzeros": "model-00014-of-00046.safetensors", - "layers.12.attn.wo_a.scales": "model-00014-of-00046.safetensors", + "layers.12.attn.wo_a.weight": "model-00014-of-00046.safetensors", "layers.12.attn.wo_b.qweight": "model-00014-of-00046.safetensors", "layers.12.attn.wo_b.qzeros": "model-00014-of-00046.safetensors", "layers.12.attn.wo_b.scales": "model-00014-of-00046.safetensors", @@ -30591,8 +30478,6 @@ "layers.12.ffn.experts.99.w3.qweight": "model-00014-of-00046.safetensors", "layers.12.ffn.experts.99.w3.qzeros": "model-00014-of-00046.safetensors", "layers.12.ffn.experts.99.w3.scales": "model-00014-of-00046.safetensors", - "layers.12.ffn.gate.bias": "model-00014-of-00046.safetensors", - "layers.12.ffn.gate.weight": "model-00014-of-00046.safetensors", "layers.12.ffn.shared_experts.w1.qweight": "model-00014-of-00046.safetensors", "layers.12.ffn.shared_experts.w1.qzeros": "model-00014-of-00046.safetensors", "layers.12.ffn.shared_experts.w1.scales": "model-00014-of-00046.safetensors", @@ -30609,23 +30494,23 @@ "layers.12.hc_ffn_base": "model-00014-of-00046.safetensors", "layers.12.hc_ffn_fn": "model-00014-of-00046.safetensors", "layers.12.hc_ffn_scale": "model-00014-of-00046.safetensors", + "layers.12.attn.compressor.norm.weight": "model-00014-of-00046.safetensors", + "layers.12.attn.compressor.wgate.weight": "model-00014-of-00046.safetensors", + "layers.12.attn.compressor.wkv.weight": "model-00014-of-00046.safetensors", + "layers.12.attn.indexer.compressor.norm.weight": "model-00014-of-00046.safetensors", + "layers.12.attn.indexer.compressor.wgate.weight": "model-00014-of-00046.safetensors", + "layers.12.attn.indexer.compressor.wkv.weight": "model-00014-of-00046.safetensors", + "layers.12.attn.indexer.weights_proj.weight": "model-00014-of-00046.safetensors", + "layers.12.ffn.gate.bias": "model-00014-of-00046.safetensors", + "layers.12.ffn.gate.weight": "model-00014-of-00046.safetensors", "layers.13.attn.attn_sink": "model-00015-of-00046.safetensors", "layers.13.attn.compressor.ape": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.norm.weight": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.wgate.qweight": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.wgate.qzeros": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.wgate.scales": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.wkv.qweight": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.wkv.qzeros": "model-00015-of-00046.safetensors", - "layers.13.attn.compressor.wkv.scales": "model-00015-of-00046.safetensors", "layers.13.attn.kv_norm.weight": "model-00015-of-00046.safetensors", "layers.13.attn.q_norm.weight": "model-00015-of-00046.safetensors", "layers.13.attn.wkv.qweight": "model-00015-of-00046.safetensors", "layers.13.attn.wkv.qzeros": "model-00015-of-00046.safetensors", "layers.13.attn.wkv.scales": "model-00015-of-00046.safetensors", - "layers.13.attn.wo_a.qweight": "model-00015-of-00046.safetensors", - "layers.13.attn.wo_a.qzeros": "model-00015-of-00046.safetensors", - "layers.13.attn.wo_a.scales": "model-00015-of-00046.safetensors", + "layers.13.attn.wo_a.weight": "model-00015-of-00046.safetensors", "layers.13.attn.wo_b.qweight": "model-00015-of-00046.safetensors", "layers.13.attn.wo_b.qzeros": "model-00015-of-00046.safetensors", "layers.13.attn.wo_b.scales": "model-00015-of-00046.safetensors", @@ -32940,8 +32825,6 @@ "layers.13.ffn.experts.99.w3.qweight": "model-00015-of-00046.safetensors", "layers.13.ffn.experts.99.w3.qzeros": "model-00015-of-00046.safetensors", "layers.13.ffn.experts.99.w3.scales": "model-00015-of-00046.safetensors", - "layers.13.ffn.gate.bias": "model-00015-of-00046.safetensors", - "layers.13.ffn.gate.weight": "model-00015-of-00046.safetensors", "layers.13.ffn.shared_experts.w1.qweight": "model-00015-of-00046.safetensors", "layers.13.ffn.shared_experts.w1.qzeros": "model-00015-of-00046.safetensors", "layers.13.ffn.shared_experts.w1.scales": "model-00015-of-00046.safetensors", @@ -32958,26 +32841,14 @@ "layers.13.hc_ffn_base": "model-00015-of-00046.safetensors", "layers.13.hc_ffn_fn": "model-00015-of-00046.safetensors", "layers.13.hc_ffn_scale": "model-00015-of-00046.safetensors", + "layers.13.attn.compressor.norm.weight": "model-00015-of-00046.safetensors", + "layers.13.attn.compressor.wgate.weight": "model-00015-of-00046.safetensors", + "layers.13.attn.compressor.wkv.weight": "model-00015-of-00046.safetensors", + "layers.13.ffn.gate.bias": "model-00015-of-00046.safetensors", + "layers.13.ffn.gate.weight": "model-00015-of-00046.safetensors", "layers.14.attn.attn_sink": "model-00016-of-00046.safetensors", "layers.14.attn.compressor.ape": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.norm.weight": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.wgate.qweight": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.wgate.qzeros": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.wgate.scales": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.wkv.qweight": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.wkv.qzeros": "model-00016-of-00046.safetensors", - "layers.14.attn.compressor.wkv.scales": "model-00016-of-00046.safetensors", "layers.14.attn.indexer.compressor.ape": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.norm.weight": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.wgate.qweight": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.wgate.qzeros": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.wgate.scales": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.wkv.qweight": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.wkv.qzeros": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.compressor.wkv.scales": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.weights_proj.qweight": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.weights_proj.qzeros": "model-00016-of-00046.safetensors", - "layers.14.attn.indexer.weights_proj.scales": "model-00016-of-00046.safetensors", "layers.14.attn.indexer.wq_b.qweight": "model-00016-of-00046.safetensors", "layers.14.attn.indexer.wq_b.qzeros": "model-00016-of-00046.safetensors", "layers.14.attn.indexer.wq_b.scales": "model-00016-of-00046.safetensors", @@ -32986,9 +32857,7 @@ "layers.14.attn.wkv.qweight": "model-00016-of-00046.safetensors", "layers.14.attn.wkv.qzeros": "model-00016-of-00046.safetensors", "layers.14.attn.wkv.scales": "model-00016-of-00046.safetensors", - "layers.14.attn.wo_a.qweight": "model-00016-of-00046.safetensors", - "layers.14.attn.wo_a.qzeros": "model-00016-of-00046.safetensors", - "layers.14.attn.wo_a.scales": "model-00016-of-00046.safetensors", + "layers.14.attn.wo_a.weight": "model-00016-of-00046.safetensors", "layers.14.attn.wo_b.qweight": "model-00016-of-00046.safetensors", "layers.14.attn.wo_b.qzeros": "model-00016-of-00046.safetensors", "layers.14.attn.wo_b.scales": "model-00016-of-00046.safetensors", @@ -35303,8 +35172,6 @@ "layers.14.ffn.experts.99.w3.qweight": "model-00016-of-00046.safetensors", "layers.14.ffn.experts.99.w3.qzeros": "model-00016-of-00046.safetensors", "layers.14.ffn.experts.99.w3.scales": "model-00016-of-00046.safetensors", - "layers.14.ffn.gate.bias": "model-00016-of-00046.safetensors", - "layers.14.ffn.gate.weight": "model-00016-of-00046.safetensors", "layers.14.ffn.shared_experts.w1.qweight": "model-00016-of-00046.safetensors", "layers.14.ffn.shared_experts.w1.qzeros": "model-00016-of-00046.safetensors", "layers.14.ffn.shared_experts.w1.scales": "model-00016-of-00046.safetensors", @@ -35321,23 +35188,23 @@ "layers.14.hc_ffn_base": "model-00016-of-00046.safetensors", "layers.14.hc_ffn_fn": "model-00016-of-00046.safetensors", "layers.14.hc_ffn_scale": "model-00016-of-00046.safetensors", + "layers.14.attn.compressor.norm.weight": "model-00016-of-00046.safetensors", + "layers.14.attn.compressor.wgate.weight": "model-00016-of-00046.safetensors", + "layers.14.attn.compressor.wkv.weight": "model-00016-of-00046.safetensors", + "layers.14.attn.indexer.compressor.norm.weight": "model-00016-of-00046.safetensors", + "layers.14.attn.indexer.compressor.wgate.weight": "model-00016-of-00046.safetensors", + "layers.14.attn.indexer.compressor.wkv.weight": "model-00016-of-00046.safetensors", + "layers.14.attn.indexer.weights_proj.weight": "model-00016-of-00046.safetensors", + "layers.14.ffn.gate.bias": "model-00016-of-00046.safetensors", + "layers.14.ffn.gate.weight": "model-00016-of-00046.safetensors", "layers.15.attn.attn_sink": "model-00017-of-00046.safetensors", "layers.15.attn.compressor.ape": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.norm.weight": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.wgate.qweight": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.wgate.qzeros": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.wgate.scales": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.wkv.qweight": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.wkv.qzeros": "model-00017-of-00046.safetensors", - "layers.15.attn.compressor.wkv.scales": "model-00017-of-00046.safetensors", "layers.15.attn.kv_norm.weight": "model-00017-of-00046.safetensors", "layers.15.attn.q_norm.weight": "model-00017-of-00046.safetensors", "layers.15.attn.wkv.qweight": "model-00017-of-00046.safetensors", "layers.15.attn.wkv.qzeros": "model-00017-of-00046.safetensors", "layers.15.attn.wkv.scales": "model-00017-of-00046.safetensors", - "layers.15.attn.wo_a.qweight": "model-00017-of-00046.safetensors", - "layers.15.attn.wo_a.qzeros": "model-00017-of-00046.safetensors", - "layers.15.attn.wo_a.scales": "model-00017-of-00046.safetensors", + "layers.15.attn.wo_a.weight": "model-00017-of-00046.safetensors", "layers.15.attn.wo_b.qweight": "model-00017-of-00046.safetensors", "layers.15.attn.wo_b.qzeros": "model-00017-of-00046.safetensors", "layers.15.attn.wo_b.scales": "model-00017-of-00046.safetensors", @@ -37652,8 +37519,6 @@ "layers.15.ffn.experts.99.w3.qweight": "model-00017-of-00046.safetensors", "layers.15.ffn.experts.99.w3.qzeros": "model-00017-of-00046.safetensors", "layers.15.ffn.experts.99.w3.scales": "model-00017-of-00046.safetensors", - "layers.15.ffn.gate.bias": "model-00017-of-00046.safetensors", - "layers.15.ffn.gate.weight": "model-00017-of-00046.safetensors", "layers.15.ffn.shared_experts.w1.qweight": "model-00017-of-00046.safetensors", "layers.15.ffn.shared_experts.w1.qzeros": "model-00017-of-00046.safetensors", "layers.15.ffn.shared_experts.w1.scales": "model-00017-of-00046.safetensors", @@ -37670,26 +37535,14 @@ "layers.15.hc_ffn_base": "model-00017-of-00046.safetensors", "layers.15.hc_ffn_fn": "model-00017-of-00046.safetensors", "layers.15.hc_ffn_scale": "model-00017-of-00046.safetensors", + "layers.15.attn.compressor.norm.weight": "model-00017-of-00046.safetensors", + "layers.15.attn.compressor.wgate.weight": "model-00017-of-00046.safetensors", + "layers.15.attn.compressor.wkv.weight": "model-00017-of-00046.safetensors", + "layers.15.ffn.gate.bias": "model-00017-of-00046.safetensors", + "layers.15.ffn.gate.weight": "model-00017-of-00046.safetensors", "layers.16.attn.attn_sink": "model-00018-of-00046.safetensors", "layers.16.attn.compressor.ape": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.norm.weight": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.wgate.qweight": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.wgate.qzeros": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.wgate.scales": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.wkv.qweight": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.wkv.qzeros": "model-00018-of-00046.safetensors", - "layers.16.attn.compressor.wkv.scales": "model-00018-of-00046.safetensors", "layers.16.attn.indexer.compressor.ape": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.norm.weight": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.wgate.qweight": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.wgate.qzeros": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.wgate.scales": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.wkv.qweight": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.wkv.qzeros": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.compressor.wkv.scales": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.weights_proj.qweight": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.weights_proj.qzeros": "model-00018-of-00046.safetensors", - "layers.16.attn.indexer.weights_proj.scales": "model-00018-of-00046.safetensors", "layers.16.attn.indexer.wq_b.qweight": "model-00018-of-00046.safetensors", "layers.16.attn.indexer.wq_b.qzeros": "model-00018-of-00046.safetensors", "layers.16.attn.indexer.wq_b.scales": "model-00018-of-00046.safetensors", @@ -37698,9 +37551,7 @@ "layers.16.attn.wkv.qweight": "model-00018-of-00046.safetensors", "layers.16.attn.wkv.qzeros": "model-00018-of-00046.safetensors", "layers.16.attn.wkv.scales": "model-00018-of-00046.safetensors", - "layers.16.attn.wo_a.qweight": "model-00018-of-00046.safetensors", - "layers.16.attn.wo_a.qzeros": "model-00018-of-00046.safetensors", - "layers.16.attn.wo_a.scales": "model-00018-of-00046.safetensors", + "layers.16.attn.wo_a.weight": "model-00018-of-00046.safetensors", "layers.16.attn.wo_b.qweight": "model-00018-of-00046.safetensors", "layers.16.attn.wo_b.qzeros": "model-00018-of-00046.safetensors", "layers.16.attn.wo_b.scales": "model-00018-of-00046.safetensors", @@ -40015,8 +39866,6 @@ "layers.16.ffn.experts.99.w3.qweight": "model-00018-of-00046.safetensors", "layers.16.ffn.experts.99.w3.qzeros": "model-00018-of-00046.safetensors", "layers.16.ffn.experts.99.w3.scales": "model-00018-of-00046.safetensors", - "layers.16.ffn.gate.bias": "model-00018-of-00046.safetensors", - "layers.16.ffn.gate.weight": "model-00018-of-00046.safetensors", "layers.16.ffn.shared_experts.w1.qweight": "model-00018-of-00046.safetensors", "layers.16.ffn.shared_experts.w1.qzeros": "model-00018-of-00046.safetensors", "layers.16.ffn.shared_experts.w1.scales": "model-00018-of-00046.safetensors", @@ -40033,23 +39882,23 @@ "layers.16.hc_ffn_base": "model-00018-of-00046.safetensors", "layers.16.hc_ffn_fn": "model-00018-of-00046.safetensors", "layers.16.hc_ffn_scale": "model-00018-of-00046.safetensors", + "layers.16.attn.compressor.norm.weight": "model-00018-of-00046.safetensors", + "layers.16.attn.compressor.wgate.weight": "model-00018-of-00046.safetensors", + "layers.16.attn.compressor.wkv.weight": "model-00018-of-00046.safetensors", + "layers.16.attn.indexer.compressor.norm.weight": "model-00018-of-00046.safetensors", + "layers.16.attn.indexer.compressor.wgate.weight": "model-00018-of-00046.safetensors", + "layers.16.attn.indexer.compressor.wkv.weight": "model-00018-of-00046.safetensors", + "layers.16.attn.indexer.weights_proj.weight": "model-00018-of-00046.safetensors", + "layers.16.ffn.gate.bias": "model-00018-of-00046.safetensors", + "layers.16.ffn.gate.weight": "model-00018-of-00046.safetensors", "layers.17.attn.attn_sink": "model-00019-of-00046.safetensors", "layers.17.attn.compressor.ape": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.norm.weight": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.wgate.qweight": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.wgate.qzeros": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.wgate.scales": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.wkv.qweight": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.wkv.qzeros": "model-00019-of-00046.safetensors", - "layers.17.attn.compressor.wkv.scales": "model-00019-of-00046.safetensors", "layers.17.attn.kv_norm.weight": "model-00019-of-00046.safetensors", "layers.17.attn.q_norm.weight": "model-00019-of-00046.safetensors", "layers.17.attn.wkv.qweight": "model-00019-of-00046.safetensors", "layers.17.attn.wkv.qzeros": "model-00019-of-00046.safetensors", "layers.17.attn.wkv.scales": "model-00019-of-00046.safetensors", - "layers.17.attn.wo_a.qweight": "model-00019-of-00046.safetensors", - "layers.17.attn.wo_a.qzeros": "model-00019-of-00046.safetensors", - "layers.17.attn.wo_a.scales": "model-00019-of-00046.safetensors", + "layers.17.attn.wo_a.weight": "model-00019-of-00046.safetensors", "layers.17.attn.wo_b.qweight": "model-00019-of-00046.safetensors", "layers.17.attn.wo_b.qzeros": "model-00019-of-00046.safetensors", "layers.17.attn.wo_b.scales": "model-00019-of-00046.safetensors", @@ -42364,8 +42213,6 @@ "layers.17.ffn.experts.99.w3.qweight": "model-00019-of-00046.safetensors", "layers.17.ffn.experts.99.w3.qzeros": "model-00019-of-00046.safetensors", "layers.17.ffn.experts.99.w3.scales": "model-00019-of-00046.safetensors", - "layers.17.ffn.gate.bias": "model-00019-of-00046.safetensors", - "layers.17.ffn.gate.weight": "model-00019-of-00046.safetensors", "layers.17.ffn.shared_experts.w1.qweight": "model-00019-of-00046.safetensors", "layers.17.ffn.shared_experts.w1.qzeros": "model-00019-of-00046.safetensors", "layers.17.ffn.shared_experts.w1.scales": "model-00019-of-00046.safetensors", @@ -42382,26 +42229,14 @@ "layers.17.hc_ffn_base": "model-00019-of-00046.safetensors", "layers.17.hc_ffn_fn": "model-00019-of-00046.safetensors", "layers.17.hc_ffn_scale": "model-00019-of-00046.safetensors", + "layers.17.attn.compressor.norm.weight": "model-00019-of-00046.safetensors", + "layers.17.attn.compressor.wgate.weight": "model-00019-of-00046.safetensors", + "layers.17.attn.compressor.wkv.weight": "model-00019-of-00046.safetensors", + "layers.17.ffn.gate.bias": "model-00019-of-00046.safetensors", + "layers.17.ffn.gate.weight": "model-00019-of-00046.safetensors", "layers.18.attn.attn_sink": "model-00020-of-00046.safetensors", "layers.18.attn.compressor.ape": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.norm.weight": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.wgate.qweight": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.wgate.qzeros": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.wgate.scales": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.wkv.qweight": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.wkv.qzeros": "model-00020-of-00046.safetensors", - "layers.18.attn.compressor.wkv.scales": "model-00020-of-00046.safetensors", "layers.18.attn.indexer.compressor.ape": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.norm.weight": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.wgate.qweight": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.wgate.qzeros": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.wgate.scales": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.wkv.qweight": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.wkv.qzeros": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.compressor.wkv.scales": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.weights_proj.qweight": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.weights_proj.qzeros": "model-00020-of-00046.safetensors", - "layers.18.attn.indexer.weights_proj.scales": "model-00020-of-00046.safetensors", "layers.18.attn.indexer.wq_b.qweight": "model-00020-of-00046.safetensors", "layers.18.attn.indexer.wq_b.qzeros": "model-00020-of-00046.safetensors", "layers.18.attn.indexer.wq_b.scales": "model-00020-of-00046.safetensors", @@ -42410,9 +42245,7 @@ "layers.18.attn.wkv.qweight": "model-00020-of-00046.safetensors", "layers.18.attn.wkv.qzeros": "model-00020-of-00046.safetensors", "layers.18.attn.wkv.scales": "model-00020-of-00046.safetensors", - "layers.18.attn.wo_a.qweight": "model-00020-of-00046.safetensors", - "layers.18.attn.wo_a.qzeros": "model-00020-of-00046.safetensors", - "layers.18.attn.wo_a.scales": "model-00020-of-00046.safetensors", + "layers.18.attn.wo_a.weight": "model-00020-of-00046.safetensors", "layers.18.attn.wo_b.qweight": "model-00020-of-00046.safetensors", "layers.18.attn.wo_b.qzeros": "model-00020-of-00046.safetensors", "layers.18.attn.wo_b.scales": "model-00020-of-00046.safetensors", @@ -44727,8 +44560,6 @@ "layers.18.ffn.experts.99.w3.qweight": "model-00020-of-00046.safetensors", "layers.18.ffn.experts.99.w3.qzeros": "model-00020-of-00046.safetensors", "layers.18.ffn.experts.99.w3.scales": "model-00020-of-00046.safetensors", - "layers.18.ffn.gate.bias": "model-00020-of-00046.safetensors", - "layers.18.ffn.gate.weight": "model-00020-of-00046.safetensors", "layers.18.ffn.shared_experts.w1.qweight": "model-00020-of-00046.safetensors", "layers.18.ffn.shared_experts.w1.qzeros": "model-00020-of-00046.safetensors", "layers.18.ffn.shared_experts.w1.scales": "model-00020-of-00046.safetensors", @@ -44745,23 +44576,23 @@ "layers.18.hc_ffn_base": "model-00020-of-00046.safetensors", "layers.18.hc_ffn_fn": "model-00020-of-00046.safetensors", "layers.18.hc_ffn_scale": "model-00020-of-00046.safetensors", + "layers.18.attn.compressor.norm.weight": "model-00020-of-00046.safetensors", + "layers.18.attn.compressor.wgate.weight": "model-00020-of-00046.safetensors", + "layers.18.attn.compressor.wkv.weight": "model-00020-of-00046.safetensors", + "layers.18.attn.indexer.compressor.norm.weight": "model-00020-of-00046.safetensors", + "layers.18.attn.indexer.compressor.wgate.weight": "model-00020-of-00046.safetensors", + "layers.18.attn.indexer.compressor.wkv.weight": "model-00020-of-00046.safetensors", + "layers.18.attn.indexer.weights_proj.weight": "model-00020-of-00046.safetensors", + "layers.18.ffn.gate.bias": "model-00020-of-00046.safetensors", + "layers.18.ffn.gate.weight": "model-00020-of-00046.safetensors", "layers.19.attn.attn_sink": "model-00021-of-00046.safetensors", "layers.19.attn.compressor.ape": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.norm.weight": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.wgate.qweight": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.wgate.qzeros": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.wgate.scales": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.wkv.qweight": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.wkv.qzeros": "model-00021-of-00046.safetensors", - "layers.19.attn.compressor.wkv.scales": "model-00021-of-00046.safetensors", "layers.19.attn.kv_norm.weight": "model-00021-of-00046.safetensors", "layers.19.attn.q_norm.weight": "model-00021-of-00046.safetensors", "layers.19.attn.wkv.qweight": "model-00021-of-00046.safetensors", "layers.19.attn.wkv.qzeros": "model-00021-of-00046.safetensors", "layers.19.attn.wkv.scales": "model-00021-of-00046.safetensors", - "layers.19.attn.wo_a.qweight": "model-00021-of-00046.safetensors", - "layers.19.attn.wo_a.qzeros": "model-00021-of-00046.safetensors", - "layers.19.attn.wo_a.scales": "model-00021-of-00046.safetensors", + "layers.19.attn.wo_a.weight": "model-00021-of-00046.safetensors", "layers.19.attn.wo_b.qweight": "model-00021-of-00046.safetensors", "layers.19.attn.wo_b.qzeros": "model-00021-of-00046.safetensors", "layers.19.attn.wo_b.scales": "model-00021-of-00046.safetensors", @@ -47076,8 +46907,6 @@ "layers.19.ffn.experts.99.w3.qweight": "model-00021-of-00046.safetensors", "layers.19.ffn.experts.99.w3.qzeros": "model-00021-of-00046.safetensors", "layers.19.ffn.experts.99.w3.scales": "model-00021-of-00046.safetensors", - "layers.19.ffn.gate.bias": "model-00021-of-00046.safetensors", - "layers.19.ffn.gate.weight": "model-00021-of-00046.safetensors", "layers.19.ffn.shared_experts.w1.qweight": "model-00021-of-00046.safetensors", "layers.19.ffn.shared_experts.w1.qzeros": "model-00021-of-00046.safetensors", "layers.19.ffn.shared_experts.w1.scales": "model-00021-of-00046.safetensors", @@ -47094,26 +46923,14 @@ "layers.19.hc_ffn_base": "model-00021-of-00046.safetensors", "layers.19.hc_ffn_fn": "model-00021-of-00046.safetensors", "layers.19.hc_ffn_scale": "model-00021-of-00046.safetensors", + "layers.19.attn.compressor.norm.weight": "model-00021-of-00046.safetensors", + "layers.19.attn.compressor.wgate.weight": "model-00021-of-00046.safetensors", + "layers.19.attn.compressor.wkv.weight": "model-00021-of-00046.safetensors", + "layers.19.ffn.gate.bias": "model-00021-of-00046.safetensors", + "layers.19.ffn.gate.weight": "model-00021-of-00046.safetensors", "layers.20.attn.attn_sink": "model-00022-of-00046.safetensors", "layers.20.attn.compressor.ape": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.norm.weight": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.wgate.qweight": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.wgate.qzeros": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.wgate.scales": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.wkv.qweight": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.wkv.qzeros": "model-00022-of-00046.safetensors", - "layers.20.attn.compressor.wkv.scales": "model-00022-of-00046.safetensors", "layers.20.attn.indexer.compressor.ape": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.norm.weight": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.wgate.qweight": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.wgate.qzeros": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.wgate.scales": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.wkv.qweight": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.wkv.qzeros": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.compressor.wkv.scales": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.weights_proj.qweight": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.weights_proj.qzeros": "model-00022-of-00046.safetensors", - "layers.20.attn.indexer.weights_proj.scales": "model-00022-of-00046.safetensors", "layers.20.attn.indexer.wq_b.qweight": "model-00022-of-00046.safetensors", "layers.20.attn.indexer.wq_b.qzeros": "model-00022-of-00046.safetensors", "layers.20.attn.indexer.wq_b.scales": "model-00022-of-00046.safetensors", @@ -47122,9 +46939,7 @@ "layers.20.attn.wkv.qweight": "model-00022-of-00046.safetensors", "layers.20.attn.wkv.qzeros": "model-00022-of-00046.safetensors", "layers.20.attn.wkv.scales": "model-00022-of-00046.safetensors", - "layers.20.attn.wo_a.qweight": "model-00022-of-00046.safetensors", - "layers.20.attn.wo_a.qzeros": "model-00022-of-00046.safetensors", - "layers.20.attn.wo_a.scales": "model-00022-of-00046.safetensors", + "layers.20.attn.wo_a.weight": "model-00022-of-00046.safetensors", "layers.20.attn.wo_b.qweight": "model-00022-of-00046.safetensors", "layers.20.attn.wo_b.qzeros": "model-00022-of-00046.safetensors", "layers.20.attn.wo_b.scales": "model-00022-of-00046.safetensors", @@ -49439,8 +49254,6 @@ "layers.20.ffn.experts.99.w3.qweight": "model-00022-of-00046.safetensors", "layers.20.ffn.experts.99.w3.qzeros": "model-00022-of-00046.safetensors", "layers.20.ffn.experts.99.w3.scales": "model-00022-of-00046.safetensors", - "layers.20.ffn.gate.bias": "model-00022-of-00046.safetensors", - "layers.20.ffn.gate.weight": "model-00022-of-00046.safetensors", "layers.20.ffn.shared_experts.w1.qweight": "model-00022-of-00046.safetensors", "layers.20.ffn.shared_experts.w1.qzeros": "model-00022-of-00046.safetensors", "layers.20.ffn.shared_experts.w1.scales": "model-00022-of-00046.safetensors", @@ -49457,23 +49270,23 @@ "layers.20.hc_ffn_base": "model-00022-of-00046.safetensors", "layers.20.hc_ffn_fn": "model-00022-of-00046.safetensors", "layers.20.hc_ffn_scale": "model-00022-of-00046.safetensors", + "layers.20.attn.compressor.norm.weight": "model-00022-of-00046.safetensors", + "layers.20.attn.compressor.wgate.weight": "model-00022-of-00046.safetensors", + "layers.20.attn.compressor.wkv.weight": "model-00022-of-00046.safetensors", + "layers.20.attn.indexer.compressor.norm.weight": "model-00022-of-00046.safetensors", + "layers.20.attn.indexer.compressor.wgate.weight": "model-00022-of-00046.safetensors", + "layers.20.attn.indexer.compressor.wkv.weight": "model-00022-of-00046.safetensors", + "layers.20.attn.indexer.weights_proj.weight": "model-00022-of-00046.safetensors", + "layers.20.ffn.gate.bias": "model-00022-of-00046.safetensors", + "layers.20.ffn.gate.weight": "model-00022-of-00046.safetensors", "layers.21.attn.attn_sink": "model-00023-of-00046.safetensors", "layers.21.attn.compressor.ape": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.norm.weight": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.wgate.qweight": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.wgate.qzeros": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.wgate.scales": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.wkv.qweight": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.wkv.qzeros": "model-00023-of-00046.safetensors", - "layers.21.attn.compressor.wkv.scales": "model-00023-of-00046.safetensors", "layers.21.attn.kv_norm.weight": "model-00023-of-00046.safetensors", "layers.21.attn.q_norm.weight": "model-00023-of-00046.safetensors", "layers.21.attn.wkv.qweight": "model-00023-of-00046.safetensors", "layers.21.attn.wkv.qzeros": "model-00023-of-00046.safetensors", "layers.21.attn.wkv.scales": "model-00023-of-00046.safetensors", - "layers.21.attn.wo_a.qweight": "model-00023-of-00046.safetensors", - "layers.21.attn.wo_a.qzeros": "model-00023-of-00046.safetensors", - "layers.21.attn.wo_a.scales": "model-00023-of-00046.safetensors", + "layers.21.attn.wo_a.weight": "model-00023-of-00046.safetensors", "layers.21.attn.wo_b.qweight": "model-00023-of-00046.safetensors", "layers.21.attn.wo_b.qzeros": "model-00023-of-00046.safetensors", "layers.21.attn.wo_b.scales": "model-00023-of-00046.safetensors", @@ -51788,8 +51601,6 @@ "layers.21.ffn.experts.99.w3.qweight": "model-00023-of-00046.safetensors", "layers.21.ffn.experts.99.w3.qzeros": "model-00023-of-00046.safetensors", "layers.21.ffn.experts.99.w3.scales": "model-00023-of-00046.safetensors", - "layers.21.ffn.gate.bias": "model-00023-of-00046.safetensors", - "layers.21.ffn.gate.weight": "model-00023-of-00046.safetensors", "layers.21.ffn.shared_experts.w1.qweight": "model-00023-of-00046.safetensors", "layers.21.ffn.shared_experts.w1.qzeros": "model-00023-of-00046.safetensors", "layers.21.ffn.shared_experts.w1.scales": "model-00023-of-00046.safetensors", @@ -51806,26 +51617,14 @@ "layers.21.hc_ffn_base": "model-00023-of-00046.safetensors", "layers.21.hc_ffn_fn": "model-00023-of-00046.safetensors", "layers.21.hc_ffn_scale": "model-00023-of-00046.safetensors", + "layers.21.attn.compressor.norm.weight": "model-00023-of-00046.safetensors", + "layers.21.attn.compressor.wgate.weight": "model-00023-of-00046.safetensors", + "layers.21.attn.compressor.wkv.weight": "model-00023-of-00046.safetensors", + "layers.21.ffn.gate.bias": "model-00023-of-00046.safetensors", + "layers.21.ffn.gate.weight": "model-00023-of-00046.safetensors", "layers.22.attn.attn_sink": "model-00024-of-00046.safetensors", "layers.22.attn.compressor.ape": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.norm.weight": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.wgate.qweight": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.wgate.qzeros": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.wgate.scales": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.wkv.qweight": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.wkv.qzeros": "model-00024-of-00046.safetensors", - "layers.22.attn.compressor.wkv.scales": "model-00024-of-00046.safetensors", "layers.22.attn.indexer.compressor.ape": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.norm.weight": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.wgate.qweight": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.wgate.qzeros": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.wgate.scales": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.wkv.qweight": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.wkv.qzeros": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.compressor.wkv.scales": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.weights_proj.qweight": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.weights_proj.qzeros": "model-00024-of-00046.safetensors", - "layers.22.attn.indexer.weights_proj.scales": "model-00024-of-00046.safetensors", "layers.22.attn.indexer.wq_b.qweight": "model-00024-of-00046.safetensors", "layers.22.attn.indexer.wq_b.qzeros": "model-00024-of-00046.safetensors", "layers.22.attn.indexer.wq_b.scales": "model-00024-of-00046.safetensors", @@ -51834,9 +51633,7 @@ "layers.22.attn.wkv.qweight": "model-00024-of-00046.safetensors", "layers.22.attn.wkv.qzeros": "model-00024-of-00046.safetensors", "layers.22.attn.wkv.scales": "model-00024-of-00046.safetensors", - "layers.22.attn.wo_a.qweight": "model-00024-of-00046.safetensors", - "layers.22.attn.wo_a.qzeros": "model-00024-of-00046.safetensors", - "layers.22.attn.wo_a.scales": "model-00024-of-00046.safetensors", + "layers.22.attn.wo_a.weight": "model-00024-of-00046.safetensors", "layers.22.attn.wo_b.qweight": "model-00024-of-00046.safetensors", "layers.22.attn.wo_b.qzeros": "model-00024-of-00046.safetensors", "layers.22.attn.wo_b.scales": "model-00024-of-00046.safetensors", @@ -54151,8 +53948,6 @@ "layers.22.ffn.experts.99.w3.qweight": "model-00024-of-00046.safetensors", "layers.22.ffn.experts.99.w3.qzeros": "model-00024-of-00046.safetensors", "layers.22.ffn.experts.99.w3.scales": "model-00024-of-00046.safetensors", - "layers.22.ffn.gate.bias": "model-00024-of-00046.safetensors", - "layers.22.ffn.gate.weight": "model-00024-of-00046.safetensors", "layers.22.ffn.shared_experts.w1.qweight": "model-00024-of-00046.safetensors", "layers.22.ffn.shared_experts.w1.qzeros": "model-00024-of-00046.safetensors", "layers.22.ffn.shared_experts.w1.scales": "model-00024-of-00046.safetensors", @@ -54169,23 +53964,23 @@ "layers.22.hc_ffn_base": "model-00024-of-00046.safetensors", "layers.22.hc_ffn_fn": "model-00024-of-00046.safetensors", "layers.22.hc_ffn_scale": "model-00024-of-00046.safetensors", + "layers.22.attn.compressor.norm.weight": "model-00024-of-00046.safetensors", + "layers.22.attn.compressor.wgate.weight": "model-00024-of-00046.safetensors", + "layers.22.attn.compressor.wkv.weight": "model-00024-of-00046.safetensors", + "layers.22.attn.indexer.compressor.norm.weight": "model-00024-of-00046.safetensors", + "layers.22.attn.indexer.compressor.wgate.weight": "model-00024-of-00046.safetensors", + "layers.22.attn.indexer.compressor.wkv.weight": "model-00024-of-00046.safetensors", + "layers.22.attn.indexer.weights_proj.weight": "model-00024-of-00046.safetensors", + "layers.22.ffn.gate.bias": "model-00024-of-00046.safetensors", + "layers.22.ffn.gate.weight": "model-00024-of-00046.safetensors", "layers.23.attn.attn_sink": "model-00025-of-00046.safetensors", "layers.23.attn.compressor.ape": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.norm.weight": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.wgate.qweight": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.wgate.qzeros": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.wgate.scales": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.wkv.qweight": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.wkv.qzeros": "model-00025-of-00046.safetensors", - "layers.23.attn.compressor.wkv.scales": "model-00025-of-00046.safetensors", "layers.23.attn.kv_norm.weight": "model-00025-of-00046.safetensors", "layers.23.attn.q_norm.weight": "model-00025-of-00046.safetensors", "layers.23.attn.wkv.qweight": "model-00025-of-00046.safetensors", "layers.23.attn.wkv.qzeros": "model-00025-of-00046.safetensors", "layers.23.attn.wkv.scales": "model-00025-of-00046.safetensors", - "layers.23.attn.wo_a.qweight": "model-00025-of-00046.safetensors", - "layers.23.attn.wo_a.qzeros": "model-00025-of-00046.safetensors", - "layers.23.attn.wo_a.scales": "model-00025-of-00046.safetensors", + "layers.23.attn.wo_a.weight": "model-00025-of-00046.safetensors", "layers.23.attn.wo_b.qweight": "model-00025-of-00046.safetensors", "layers.23.attn.wo_b.qzeros": "model-00025-of-00046.safetensors", "layers.23.attn.wo_b.scales": "model-00025-of-00046.safetensors", @@ -56500,8 +56295,6 @@ "layers.23.ffn.experts.99.w3.qweight": "model-00025-of-00046.safetensors", "layers.23.ffn.experts.99.w3.qzeros": "model-00025-of-00046.safetensors", "layers.23.ffn.experts.99.w3.scales": "model-00025-of-00046.safetensors", - "layers.23.ffn.gate.bias": "model-00025-of-00046.safetensors", - "layers.23.ffn.gate.weight": "model-00025-of-00046.safetensors", "layers.23.ffn.shared_experts.w1.qweight": "model-00025-of-00046.safetensors", "layers.23.ffn.shared_experts.w1.qzeros": "model-00025-of-00046.safetensors", "layers.23.ffn.shared_experts.w1.scales": "model-00025-of-00046.safetensors", @@ -56518,26 +56311,14 @@ "layers.23.hc_ffn_base": "model-00025-of-00046.safetensors", "layers.23.hc_ffn_fn": "model-00025-of-00046.safetensors", "layers.23.hc_ffn_scale": "model-00025-of-00046.safetensors", + "layers.23.attn.compressor.norm.weight": "model-00025-of-00046.safetensors", + "layers.23.attn.compressor.wgate.weight": "model-00025-of-00046.safetensors", + "layers.23.attn.compressor.wkv.weight": "model-00025-of-00046.safetensors", + "layers.23.ffn.gate.bias": "model-00025-of-00046.safetensors", + "layers.23.ffn.gate.weight": "model-00025-of-00046.safetensors", "layers.24.attn.attn_sink": "model-00026-of-00046.safetensors", "layers.24.attn.compressor.ape": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.norm.weight": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.wgate.qweight": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.wgate.qzeros": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.wgate.scales": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.wkv.qweight": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.wkv.qzeros": "model-00026-of-00046.safetensors", - "layers.24.attn.compressor.wkv.scales": "model-00026-of-00046.safetensors", "layers.24.attn.indexer.compressor.ape": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.norm.weight": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.wgate.qweight": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.wgate.qzeros": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.wgate.scales": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.wkv.qweight": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.wkv.qzeros": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.compressor.wkv.scales": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.weights_proj.qweight": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.weights_proj.qzeros": "model-00026-of-00046.safetensors", - "layers.24.attn.indexer.weights_proj.scales": "model-00026-of-00046.safetensors", "layers.24.attn.indexer.wq_b.qweight": "model-00026-of-00046.safetensors", "layers.24.attn.indexer.wq_b.qzeros": "model-00026-of-00046.safetensors", "layers.24.attn.indexer.wq_b.scales": "model-00026-of-00046.safetensors", @@ -56546,9 +56327,7 @@ "layers.24.attn.wkv.qweight": "model-00026-of-00046.safetensors", "layers.24.attn.wkv.qzeros": "model-00026-of-00046.safetensors", "layers.24.attn.wkv.scales": "model-00026-of-00046.safetensors", - "layers.24.attn.wo_a.qweight": "model-00026-of-00046.safetensors", - "layers.24.attn.wo_a.qzeros": "model-00026-of-00046.safetensors", - "layers.24.attn.wo_a.scales": "model-00026-of-00046.safetensors", + "layers.24.attn.wo_a.weight": "model-00026-of-00046.safetensors", "layers.24.attn.wo_b.qweight": "model-00026-of-00046.safetensors", "layers.24.attn.wo_b.qzeros": "model-00026-of-00046.safetensors", "layers.24.attn.wo_b.scales": "model-00026-of-00046.safetensors", @@ -58863,8 +58642,6 @@ "layers.24.ffn.experts.99.w3.qweight": "model-00026-of-00046.safetensors", "layers.24.ffn.experts.99.w3.qzeros": "model-00026-of-00046.safetensors", "layers.24.ffn.experts.99.w3.scales": "model-00026-of-00046.safetensors", - "layers.24.ffn.gate.bias": "model-00026-of-00046.safetensors", - "layers.24.ffn.gate.weight": "model-00026-of-00046.safetensors", "layers.24.ffn.shared_experts.w1.qweight": "model-00026-of-00046.safetensors", "layers.24.ffn.shared_experts.w1.qzeros": "model-00026-of-00046.safetensors", "layers.24.ffn.shared_experts.w1.scales": "model-00026-of-00046.safetensors", @@ -58881,23 +58658,23 @@ "layers.24.hc_ffn_base": "model-00026-of-00046.safetensors", "layers.24.hc_ffn_fn": "model-00026-of-00046.safetensors", "layers.24.hc_ffn_scale": "model-00026-of-00046.safetensors", + "layers.24.attn.compressor.norm.weight": "model-00026-of-00046.safetensors", + "layers.24.attn.compressor.wgate.weight": "model-00026-of-00046.safetensors", + "layers.24.attn.compressor.wkv.weight": "model-00026-of-00046.safetensors", + "layers.24.attn.indexer.compressor.norm.weight": "model-00026-of-00046.safetensors", + "layers.24.attn.indexer.compressor.wgate.weight": "model-00026-of-00046.safetensors", + "layers.24.attn.indexer.compressor.wkv.weight": "model-00026-of-00046.safetensors", + "layers.24.attn.indexer.weights_proj.weight": "model-00026-of-00046.safetensors", + "layers.24.ffn.gate.bias": "model-00026-of-00046.safetensors", + "layers.24.ffn.gate.weight": "model-00026-of-00046.safetensors", "layers.25.attn.attn_sink": "model-00027-of-00046.safetensors", "layers.25.attn.compressor.ape": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.norm.weight": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.wgate.qweight": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.wgate.qzeros": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.wgate.scales": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.wkv.qweight": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.wkv.qzeros": "model-00027-of-00046.safetensors", - "layers.25.attn.compressor.wkv.scales": "model-00027-of-00046.safetensors", "layers.25.attn.kv_norm.weight": "model-00027-of-00046.safetensors", "layers.25.attn.q_norm.weight": "model-00027-of-00046.safetensors", "layers.25.attn.wkv.qweight": "model-00027-of-00046.safetensors", "layers.25.attn.wkv.qzeros": "model-00027-of-00046.safetensors", "layers.25.attn.wkv.scales": "model-00027-of-00046.safetensors", - "layers.25.attn.wo_a.qweight": "model-00027-of-00046.safetensors", - "layers.25.attn.wo_a.qzeros": "model-00027-of-00046.safetensors", - "layers.25.attn.wo_a.scales": "model-00027-of-00046.safetensors", + "layers.25.attn.wo_a.weight": "model-00027-of-00046.safetensors", "layers.25.attn.wo_b.qweight": "model-00027-of-00046.safetensors", "layers.25.attn.wo_b.qzeros": "model-00027-of-00046.safetensors", "layers.25.attn.wo_b.scales": "model-00027-of-00046.safetensors", @@ -61212,8 +60989,6 @@ "layers.25.ffn.experts.99.w3.qweight": "model-00027-of-00046.safetensors", "layers.25.ffn.experts.99.w3.qzeros": "model-00027-of-00046.safetensors", "layers.25.ffn.experts.99.w3.scales": "model-00027-of-00046.safetensors", - "layers.25.ffn.gate.bias": "model-00027-of-00046.safetensors", - "layers.25.ffn.gate.weight": "model-00027-of-00046.safetensors", "layers.25.ffn.shared_experts.w1.qweight": "model-00027-of-00046.safetensors", "layers.25.ffn.shared_experts.w1.qzeros": "model-00027-of-00046.safetensors", "layers.25.ffn.shared_experts.w1.scales": "model-00027-of-00046.safetensors", @@ -61230,26 +61005,14 @@ "layers.25.hc_ffn_base": "model-00027-of-00046.safetensors", "layers.25.hc_ffn_fn": "model-00027-of-00046.safetensors", "layers.25.hc_ffn_scale": "model-00027-of-00046.safetensors", + "layers.25.attn.compressor.norm.weight": "model-00027-of-00046.safetensors", + "layers.25.attn.compressor.wgate.weight": "model-00027-of-00046.safetensors", + "layers.25.attn.compressor.wkv.weight": "model-00027-of-00046.safetensors", + "layers.25.ffn.gate.bias": "model-00027-of-00046.safetensors", + "layers.25.ffn.gate.weight": "model-00027-of-00046.safetensors", "layers.26.attn.attn_sink": "model-00028-of-00046.safetensors", "layers.26.attn.compressor.ape": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.norm.weight": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.wgate.qweight": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.wgate.qzeros": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.wgate.scales": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.wkv.qweight": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.wkv.qzeros": "model-00028-of-00046.safetensors", - "layers.26.attn.compressor.wkv.scales": "model-00028-of-00046.safetensors", "layers.26.attn.indexer.compressor.ape": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.norm.weight": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.wgate.qweight": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.wgate.qzeros": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.wgate.scales": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.wkv.qweight": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.wkv.qzeros": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.compressor.wkv.scales": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.weights_proj.qweight": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.weights_proj.qzeros": "model-00028-of-00046.safetensors", - "layers.26.attn.indexer.weights_proj.scales": "model-00028-of-00046.safetensors", "layers.26.attn.indexer.wq_b.qweight": "model-00028-of-00046.safetensors", "layers.26.attn.indexer.wq_b.qzeros": "model-00028-of-00046.safetensors", "layers.26.attn.indexer.wq_b.scales": "model-00028-of-00046.safetensors", @@ -61258,9 +61021,7 @@ "layers.26.attn.wkv.qweight": "model-00028-of-00046.safetensors", "layers.26.attn.wkv.qzeros": "model-00028-of-00046.safetensors", "layers.26.attn.wkv.scales": "model-00028-of-00046.safetensors", - "layers.26.attn.wo_a.qweight": "model-00028-of-00046.safetensors", - "layers.26.attn.wo_a.qzeros": "model-00028-of-00046.safetensors", - "layers.26.attn.wo_a.scales": "model-00028-of-00046.safetensors", + "layers.26.attn.wo_a.weight": "model-00028-of-00046.safetensors", "layers.26.attn.wo_b.qweight": "model-00028-of-00046.safetensors", "layers.26.attn.wo_b.qzeros": "model-00028-of-00046.safetensors", "layers.26.attn.wo_b.scales": "model-00028-of-00046.safetensors", @@ -63575,8 +63336,6 @@ "layers.26.ffn.experts.99.w3.qweight": "model-00028-of-00046.safetensors", "layers.26.ffn.experts.99.w3.qzeros": "model-00028-of-00046.safetensors", "layers.26.ffn.experts.99.w3.scales": "model-00028-of-00046.safetensors", - "layers.26.ffn.gate.bias": "model-00028-of-00046.safetensors", - "layers.26.ffn.gate.weight": "model-00028-of-00046.safetensors", "layers.26.ffn.shared_experts.w1.qweight": "model-00028-of-00046.safetensors", "layers.26.ffn.shared_experts.w1.qzeros": "model-00028-of-00046.safetensors", "layers.26.ffn.shared_experts.w1.scales": "model-00028-of-00046.safetensors", @@ -63593,23 +63352,23 @@ "layers.26.hc_ffn_base": "model-00028-of-00046.safetensors", "layers.26.hc_ffn_fn": "model-00028-of-00046.safetensors", "layers.26.hc_ffn_scale": "model-00028-of-00046.safetensors", + "layers.26.attn.compressor.norm.weight": "model-00028-of-00046.safetensors", + "layers.26.attn.compressor.wgate.weight": "model-00028-of-00046.safetensors", + "layers.26.attn.compressor.wkv.weight": "model-00028-of-00046.safetensors", + "layers.26.attn.indexer.compressor.norm.weight": "model-00028-of-00046.safetensors", + "layers.26.attn.indexer.compressor.wgate.weight": "model-00028-of-00046.safetensors", + "layers.26.attn.indexer.compressor.wkv.weight": "model-00028-of-00046.safetensors", + "layers.26.attn.indexer.weights_proj.weight": "model-00028-of-00046.safetensors", + "layers.26.ffn.gate.bias": "model-00028-of-00046.safetensors", + "layers.26.ffn.gate.weight": "model-00028-of-00046.safetensors", "layers.27.attn.attn_sink": "model-00029-of-00046.safetensors", "layers.27.attn.compressor.ape": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.norm.weight": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.wgate.qweight": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.wgate.qzeros": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.wgate.scales": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.wkv.qweight": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.wkv.qzeros": "model-00029-of-00046.safetensors", - "layers.27.attn.compressor.wkv.scales": "model-00029-of-00046.safetensors", "layers.27.attn.kv_norm.weight": "model-00029-of-00046.safetensors", "layers.27.attn.q_norm.weight": "model-00029-of-00046.safetensors", "layers.27.attn.wkv.qweight": "model-00029-of-00046.safetensors", "layers.27.attn.wkv.qzeros": "model-00029-of-00046.safetensors", "layers.27.attn.wkv.scales": "model-00029-of-00046.safetensors", - "layers.27.attn.wo_a.qweight": "model-00029-of-00046.safetensors", - "layers.27.attn.wo_a.qzeros": "model-00029-of-00046.safetensors", - "layers.27.attn.wo_a.scales": "model-00029-of-00046.safetensors", + "layers.27.attn.wo_a.weight": "model-00029-of-00046.safetensors", "layers.27.attn.wo_b.qweight": "model-00029-of-00046.safetensors", "layers.27.attn.wo_b.qzeros": "model-00029-of-00046.safetensors", "layers.27.attn.wo_b.scales": "model-00029-of-00046.safetensors", @@ -65924,8 +65683,6 @@ "layers.27.ffn.experts.99.w3.qweight": "model-00029-of-00046.safetensors", "layers.27.ffn.experts.99.w3.qzeros": "model-00029-of-00046.safetensors", "layers.27.ffn.experts.99.w3.scales": "model-00029-of-00046.safetensors", - "layers.27.ffn.gate.bias": "model-00029-of-00046.safetensors", - "layers.27.ffn.gate.weight": "model-00029-of-00046.safetensors", "layers.27.ffn.shared_experts.w1.qweight": "model-00029-of-00046.safetensors", "layers.27.ffn.shared_experts.w1.qzeros": "model-00029-of-00046.safetensors", "layers.27.ffn.shared_experts.w1.scales": "model-00029-of-00046.safetensors", @@ -65942,26 +65699,14 @@ "layers.27.hc_ffn_base": "model-00029-of-00046.safetensors", "layers.27.hc_ffn_fn": "model-00029-of-00046.safetensors", "layers.27.hc_ffn_scale": "model-00029-of-00046.safetensors", + "layers.27.attn.compressor.norm.weight": "model-00029-of-00046.safetensors", + "layers.27.attn.compressor.wgate.weight": "model-00029-of-00046.safetensors", + "layers.27.attn.compressor.wkv.weight": "model-00029-of-00046.safetensors", + "layers.27.ffn.gate.bias": "model-00029-of-00046.safetensors", + "layers.27.ffn.gate.weight": "model-00029-of-00046.safetensors", "layers.28.attn.attn_sink": "model-00030-of-00046.safetensors", "layers.28.attn.compressor.ape": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.norm.weight": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.wgate.qweight": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.wgate.qzeros": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.wgate.scales": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.wkv.qweight": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.wkv.qzeros": "model-00030-of-00046.safetensors", - "layers.28.attn.compressor.wkv.scales": "model-00030-of-00046.safetensors", "layers.28.attn.indexer.compressor.ape": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.norm.weight": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.wgate.qweight": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.wgate.qzeros": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.wgate.scales": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.wkv.qweight": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.wkv.qzeros": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.compressor.wkv.scales": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.weights_proj.qweight": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.weights_proj.qzeros": "model-00030-of-00046.safetensors", - "layers.28.attn.indexer.weights_proj.scales": "model-00030-of-00046.safetensors", "layers.28.attn.indexer.wq_b.qweight": "model-00030-of-00046.safetensors", "layers.28.attn.indexer.wq_b.qzeros": "model-00030-of-00046.safetensors", "layers.28.attn.indexer.wq_b.scales": "model-00030-of-00046.safetensors", @@ -65970,9 +65715,7 @@ "layers.28.attn.wkv.qweight": "model-00030-of-00046.safetensors", "layers.28.attn.wkv.qzeros": "model-00030-of-00046.safetensors", "layers.28.attn.wkv.scales": "model-00030-of-00046.safetensors", - "layers.28.attn.wo_a.qweight": "model-00030-of-00046.safetensors", - "layers.28.attn.wo_a.qzeros": "model-00030-of-00046.safetensors", - "layers.28.attn.wo_a.scales": "model-00030-of-00046.safetensors", + "layers.28.attn.wo_a.weight": "model-00030-of-00046.safetensors", "layers.28.attn.wo_b.qweight": "model-00030-of-00046.safetensors", "layers.28.attn.wo_b.qzeros": "model-00030-of-00046.safetensors", "layers.28.attn.wo_b.scales": "model-00030-of-00046.safetensors", @@ -68287,8 +68030,6 @@ "layers.28.ffn.experts.99.w3.qweight": "model-00030-of-00046.safetensors", "layers.28.ffn.experts.99.w3.qzeros": "model-00030-of-00046.safetensors", "layers.28.ffn.experts.99.w3.scales": "model-00030-of-00046.safetensors", - "layers.28.ffn.gate.bias": "model-00030-of-00046.safetensors", - "layers.28.ffn.gate.weight": "model-00030-of-00046.safetensors", "layers.28.ffn.shared_experts.w1.qweight": "model-00030-of-00046.safetensors", "layers.28.ffn.shared_experts.w1.qzeros": "model-00030-of-00046.safetensors", "layers.28.ffn.shared_experts.w1.scales": "model-00030-of-00046.safetensors", @@ -68305,23 +68046,23 @@ "layers.28.hc_ffn_base": "model-00030-of-00046.safetensors", "layers.28.hc_ffn_fn": "model-00030-of-00046.safetensors", "layers.28.hc_ffn_scale": "model-00030-of-00046.safetensors", + "layers.28.attn.compressor.norm.weight": "model-00030-of-00046.safetensors", + "layers.28.attn.compressor.wgate.weight": "model-00030-of-00046.safetensors", + "layers.28.attn.compressor.wkv.weight": "model-00030-of-00046.safetensors", + "layers.28.attn.indexer.compressor.norm.weight": "model-00030-of-00046.safetensors", + "layers.28.attn.indexer.compressor.wgate.weight": "model-00030-of-00046.safetensors", + "layers.28.attn.indexer.compressor.wkv.weight": "model-00030-of-00046.safetensors", + "layers.28.attn.indexer.weights_proj.weight": "model-00030-of-00046.safetensors", + "layers.28.ffn.gate.bias": "model-00030-of-00046.safetensors", + "layers.28.ffn.gate.weight": "model-00030-of-00046.safetensors", "layers.29.attn.attn_sink": "model-00031-of-00046.safetensors", "layers.29.attn.compressor.ape": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.norm.weight": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.wgate.qweight": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.wgate.qzeros": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.wgate.scales": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.wkv.qweight": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.wkv.qzeros": "model-00031-of-00046.safetensors", - "layers.29.attn.compressor.wkv.scales": "model-00031-of-00046.safetensors", "layers.29.attn.kv_norm.weight": "model-00031-of-00046.safetensors", "layers.29.attn.q_norm.weight": "model-00031-of-00046.safetensors", "layers.29.attn.wkv.qweight": "model-00031-of-00046.safetensors", "layers.29.attn.wkv.qzeros": "model-00031-of-00046.safetensors", "layers.29.attn.wkv.scales": "model-00031-of-00046.safetensors", - "layers.29.attn.wo_a.qweight": "model-00031-of-00046.safetensors", - "layers.29.attn.wo_a.qzeros": "model-00031-of-00046.safetensors", - "layers.29.attn.wo_a.scales": "model-00031-of-00046.safetensors", + "layers.29.attn.wo_a.weight": "model-00031-of-00046.safetensors", "layers.29.attn.wo_b.qweight": "model-00031-of-00046.safetensors", "layers.29.attn.wo_b.qzeros": "model-00031-of-00046.safetensors", "layers.29.attn.wo_b.scales": "model-00031-of-00046.safetensors", @@ -70636,8 +70377,6 @@ "layers.29.ffn.experts.99.w3.qweight": "model-00031-of-00046.safetensors", "layers.29.ffn.experts.99.w3.qzeros": "model-00031-of-00046.safetensors", "layers.29.ffn.experts.99.w3.scales": "model-00031-of-00046.safetensors", - "layers.29.ffn.gate.bias": "model-00031-of-00046.safetensors", - "layers.29.ffn.gate.weight": "model-00031-of-00046.safetensors", "layers.29.ffn.shared_experts.w1.qweight": "model-00031-of-00046.safetensors", "layers.29.ffn.shared_experts.w1.qzeros": "model-00031-of-00046.safetensors", "layers.29.ffn.shared_experts.w1.scales": "model-00031-of-00046.safetensors", @@ -70654,26 +70393,14 @@ "layers.29.hc_ffn_base": "model-00031-of-00046.safetensors", "layers.29.hc_ffn_fn": "model-00031-of-00046.safetensors", "layers.29.hc_ffn_scale": "model-00031-of-00046.safetensors", + "layers.29.attn.compressor.norm.weight": "model-00031-of-00046.safetensors", + "layers.29.attn.compressor.wgate.weight": "model-00031-of-00046.safetensors", + "layers.29.attn.compressor.wkv.weight": "model-00031-of-00046.safetensors", + "layers.29.ffn.gate.bias": "model-00031-of-00046.safetensors", + "layers.29.ffn.gate.weight": "model-00031-of-00046.safetensors", "layers.30.attn.attn_sink": "model-00032-of-00046.safetensors", "layers.30.attn.compressor.ape": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.norm.weight": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.wgate.qweight": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.wgate.qzeros": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.wgate.scales": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.wkv.qweight": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.wkv.qzeros": "model-00032-of-00046.safetensors", - "layers.30.attn.compressor.wkv.scales": "model-00032-of-00046.safetensors", "layers.30.attn.indexer.compressor.ape": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.norm.weight": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.wgate.qweight": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.wgate.qzeros": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.wgate.scales": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.wkv.qweight": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.wkv.qzeros": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.compressor.wkv.scales": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.weights_proj.qweight": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.weights_proj.qzeros": "model-00032-of-00046.safetensors", - "layers.30.attn.indexer.weights_proj.scales": "model-00032-of-00046.safetensors", "layers.30.attn.indexer.wq_b.qweight": "model-00032-of-00046.safetensors", "layers.30.attn.indexer.wq_b.qzeros": "model-00032-of-00046.safetensors", "layers.30.attn.indexer.wq_b.scales": "model-00032-of-00046.safetensors", @@ -70682,9 +70409,7 @@ "layers.30.attn.wkv.qweight": "model-00032-of-00046.safetensors", "layers.30.attn.wkv.qzeros": "model-00032-of-00046.safetensors", "layers.30.attn.wkv.scales": "model-00032-of-00046.safetensors", - "layers.30.attn.wo_a.qweight": "model-00032-of-00046.safetensors", - "layers.30.attn.wo_a.qzeros": "model-00032-of-00046.safetensors", - "layers.30.attn.wo_a.scales": "model-00032-of-00046.safetensors", + "layers.30.attn.wo_a.weight": "model-00032-of-00046.safetensors", "layers.30.attn.wo_b.qweight": "model-00032-of-00046.safetensors", "layers.30.attn.wo_b.qzeros": "model-00032-of-00046.safetensors", "layers.30.attn.wo_b.scales": "model-00032-of-00046.safetensors", @@ -72999,8 +72724,6 @@ "layers.30.ffn.experts.99.w3.qweight": "model-00032-of-00046.safetensors", "layers.30.ffn.experts.99.w3.qzeros": "model-00032-of-00046.safetensors", "layers.30.ffn.experts.99.w3.scales": "model-00032-of-00046.safetensors", - "layers.30.ffn.gate.bias": "model-00032-of-00046.safetensors", - "layers.30.ffn.gate.weight": "model-00032-of-00046.safetensors", "layers.30.ffn.shared_experts.w1.qweight": "model-00032-of-00046.safetensors", "layers.30.ffn.shared_experts.w1.qzeros": "model-00032-of-00046.safetensors", "layers.30.ffn.shared_experts.w1.scales": "model-00032-of-00046.safetensors", @@ -73017,23 +72740,23 @@ "layers.30.hc_ffn_base": "model-00032-of-00046.safetensors", "layers.30.hc_ffn_fn": "model-00032-of-00046.safetensors", "layers.30.hc_ffn_scale": "model-00032-of-00046.safetensors", + "layers.30.attn.compressor.norm.weight": "model-00032-of-00046.safetensors", + "layers.30.attn.compressor.wgate.weight": "model-00032-of-00046.safetensors", + "layers.30.attn.compressor.wkv.weight": "model-00032-of-00046.safetensors", + "layers.30.attn.indexer.compressor.norm.weight": "model-00032-of-00046.safetensors", + "layers.30.attn.indexer.compressor.wgate.weight": "model-00032-of-00046.safetensors", + "layers.30.attn.indexer.compressor.wkv.weight": "model-00032-of-00046.safetensors", + "layers.30.attn.indexer.weights_proj.weight": "model-00032-of-00046.safetensors", + "layers.30.ffn.gate.bias": "model-00032-of-00046.safetensors", + "layers.30.ffn.gate.weight": "model-00032-of-00046.safetensors", "layers.31.attn.attn_sink": "model-00033-of-00046.safetensors", "layers.31.attn.compressor.ape": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.norm.weight": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.wgate.qweight": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.wgate.qzeros": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.wgate.scales": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.wkv.qweight": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.wkv.qzeros": "model-00033-of-00046.safetensors", - "layers.31.attn.compressor.wkv.scales": "model-00033-of-00046.safetensors", "layers.31.attn.kv_norm.weight": "model-00033-of-00046.safetensors", "layers.31.attn.q_norm.weight": "model-00033-of-00046.safetensors", "layers.31.attn.wkv.qweight": "model-00033-of-00046.safetensors", "layers.31.attn.wkv.qzeros": "model-00033-of-00046.safetensors", "layers.31.attn.wkv.scales": "model-00033-of-00046.safetensors", - "layers.31.attn.wo_a.qweight": "model-00033-of-00046.safetensors", - "layers.31.attn.wo_a.qzeros": "model-00033-of-00046.safetensors", - "layers.31.attn.wo_a.scales": "model-00033-of-00046.safetensors", + "layers.31.attn.wo_a.weight": "model-00033-of-00046.safetensors", "layers.31.attn.wo_b.qweight": "model-00033-of-00046.safetensors", "layers.31.attn.wo_b.qzeros": "model-00033-of-00046.safetensors", "layers.31.attn.wo_b.scales": "model-00033-of-00046.safetensors", @@ -75348,8 +75071,6 @@ "layers.31.ffn.experts.99.w3.qweight": "model-00033-of-00046.safetensors", "layers.31.ffn.experts.99.w3.qzeros": "model-00033-of-00046.safetensors", "layers.31.ffn.experts.99.w3.scales": "model-00033-of-00046.safetensors", - "layers.31.ffn.gate.bias": "model-00033-of-00046.safetensors", - "layers.31.ffn.gate.weight": "model-00033-of-00046.safetensors", "layers.31.ffn.shared_experts.w1.qweight": "model-00033-of-00046.safetensors", "layers.31.ffn.shared_experts.w1.qzeros": "model-00033-of-00046.safetensors", "layers.31.ffn.shared_experts.w1.scales": "model-00033-of-00046.safetensors", @@ -75366,26 +75087,14 @@ "layers.31.hc_ffn_base": "model-00033-of-00046.safetensors", "layers.31.hc_ffn_fn": "model-00033-of-00046.safetensors", "layers.31.hc_ffn_scale": "model-00033-of-00046.safetensors", + "layers.31.attn.compressor.norm.weight": "model-00033-of-00046.safetensors", + "layers.31.attn.compressor.wgate.weight": "model-00033-of-00046.safetensors", + "layers.31.attn.compressor.wkv.weight": "model-00033-of-00046.safetensors", + "layers.31.ffn.gate.bias": "model-00033-of-00046.safetensors", + "layers.31.ffn.gate.weight": "model-00033-of-00046.safetensors", "layers.32.attn.attn_sink": "model-00034-of-00046.safetensors", "layers.32.attn.compressor.ape": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.norm.weight": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.wgate.qweight": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.wgate.qzeros": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.wgate.scales": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.wkv.qweight": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.wkv.qzeros": "model-00034-of-00046.safetensors", - "layers.32.attn.compressor.wkv.scales": "model-00034-of-00046.safetensors", "layers.32.attn.indexer.compressor.ape": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.norm.weight": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.wgate.qweight": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.wgate.qzeros": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.wgate.scales": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.wkv.qweight": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.wkv.qzeros": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.compressor.wkv.scales": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.weights_proj.qweight": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.weights_proj.qzeros": "model-00034-of-00046.safetensors", - "layers.32.attn.indexer.weights_proj.scales": "model-00034-of-00046.safetensors", "layers.32.attn.indexer.wq_b.qweight": "model-00034-of-00046.safetensors", "layers.32.attn.indexer.wq_b.qzeros": "model-00034-of-00046.safetensors", "layers.32.attn.indexer.wq_b.scales": "model-00034-of-00046.safetensors", @@ -75394,9 +75103,7 @@ "layers.32.attn.wkv.qweight": "model-00034-of-00046.safetensors", "layers.32.attn.wkv.qzeros": "model-00034-of-00046.safetensors", "layers.32.attn.wkv.scales": "model-00034-of-00046.safetensors", - "layers.32.attn.wo_a.qweight": "model-00034-of-00046.safetensors", - "layers.32.attn.wo_a.qzeros": "model-00034-of-00046.safetensors", - "layers.32.attn.wo_a.scales": "model-00034-of-00046.safetensors", + "layers.32.attn.wo_a.weight": "model-00034-of-00046.safetensors", "layers.32.attn.wo_b.qweight": "model-00034-of-00046.safetensors", "layers.32.attn.wo_b.qzeros": "model-00034-of-00046.safetensors", "layers.32.attn.wo_b.scales": "model-00034-of-00046.safetensors", @@ -77711,8 +77418,6 @@ "layers.32.ffn.experts.99.w3.qweight": "model-00034-of-00046.safetensors", "layers.32.ffn.experts.99.w3.qzeros": "model-00034-of-00046.safetensors", "layers.32.ffn.experts.99.w3.scales": "model-00034-of-00046.safetensors", - "layers.32.ffn.gate.bias": "model-00034-of-00046.safetensors", - "layers.32.ffn.gate.weight": "model-00034-of-00046.safetensors", "layers.32.ffn.shared_experts.w1.qweight": "model-00034-of-00046.safetensors", "layers.32.ffn.shared_experts.w1.qzeros": "model-00034-of-00046.safetensors", "layers.32.ffn.shared_experts.w1.scales": "model-00034-of-00046.safetensors", @@ -77729,23 +77434,23 @@ "layers.32.hc_ffn_base": "model-00034-of-00046.safetensors", "layers.32.hc_ffn_fn": "model-00034-of-00046.safetensors", "layers.32.hc_ffn_scale": "model-00034-of-00046.safetensors", + "layers.32.attn.compressor.norm.weight": "model-00034-of-00046.safetensors", + "layers.32.attn.compressor.wgate.weight": "model-00034-of-00046.safetensors", + "layers.32.attn.compressor.wkv.weight": "model-00034-of-00046.safetensors", + "layers.32.attn.indexer.compressor.norm.weight": "model-00034-of-00046.safetensors", + "layers.32.attn.indexer.compressor.wgate.weight": "model-00034-of-00046.safetensors", + "layers.32.attn.indexer.compressor.wkv.weight": "model-00034-of-00046.safetensors", + "layers.32.attn.indexer.weights_proj.weight": "model-00034-of-00046.safetensors", + "layers.32.ffn.gate.bias": "model-00034-of-00046.safetensors", + "layers.32.ffn.gate.weight": "model-00034-of-00046.safetensors", "layers.33.attn.attn_sink": "model-00035-of-00046.safetensors", "layers.33.attn.compressor.ape": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.norm.weight": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.wgate.qweight": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.wgate.qzeros": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.wgate.scales": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.wkv.qweight": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.wkv.qzeros": "model-00035-of-00046.safetensors", - "layers.33.attn.compressor.wkv.scales": "model-00035-of-00046.safetensors", "layers.33.attn.kv_norm.weight": "model-00035-of-00046.safetensors", "layers.33.attn.q_norm.weight": "model-00035-of-00046.safetensors", "layers.33.attn.wkv.qweight": "model-00035-of-00046.safetensors", "layers.33.attn.wkv.qzeros": "model-00035-of-00046.safetensors", "layers.33.attn.wkv.scales": "model-00035-of-00046.safetensors", - "layers.33.attn.wo_a.qweight": "model-00035-of-00046.safetensors", - "layers.33.attn.wo_a.qzeros": "model-00035-of-00046.safetensors", - "layers.33.attn.wo_a.scales": "model-00035-of-00046.safetensors", + "layers.33.attn.wo_a.weight": "model-00035-of-00046.safetensors", "layers.33.attn.wo_b.qweight": "model-00035-of-00046.safetensors", "layers.33.attn.wo_b.qzeros": "model-00035-of-00046.safetensors", "layers.33.attn.wo_b.scales": "model-00035-of-00046.safetensors", @@ -80060,8 +79765,6 @@ "layers.33.ffn.experts.99.w3.qweight": "model-00035-of-00046.safetensors", "layers.33.ffn.experts.99.w3.qzeros": "model-00035-of-00046.safetensors", "layers.33.ffn.experts.99.w3.scales": "model-00035-of-00046.safetensors", - "layers.33.ffn.gate.bias": "model-00035-of-00046.safetensors", - "layers.33.ffn.gate.weight": "model-00035-of-00046.safetensors", "layers.33.ffn.shared_experts.w1.qweight": "model-00035-of-00046.safetensors", "layers.33.ffn.shared_experts.w1.qzeros": "model-00035-of-00046.safetensors", "layers.33.ffn.shared_experts.w1.scales": "model-00035-of-00046.safetensors", @@ -80078,26 +79781,14 @@ "layers.33.hc_ffn_base": "model-00035-of-00046.safetensors", "layers.33.hc_ffn_fn": "model-00035-of-00046.safetensors", "layers.33.hc_ffn_scale": "model-00035-of-00046.safetensors", + "layers.33.attn.compressor.norm.weight": "model-00035-of-00046.safetensors", + "layers.33.attn.compressor.wgate.weight": "model-00035-of-00046.safetensors", + "layers.33.attn.compressor.wkv.weight": "model-00035-of-00046.safetensors", + "layers.33.ffn.gate.bias": "model-00035-of-00046.safetensors", + "layers.33.ffn.gate.weight": "model-00035-of-00046.safetensors", "layers.34.attn.attn_sink": "model-00036-of-00046.safetensors", "layers.34.attn.compressor.ape": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.norm.weight": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.wgate.qweight": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.wgate.qzeros": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.wgate.scales": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.wkv.qweight": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.wkv.qzeros": "model-00036-of-00046.safetensors", - "layers.34.attn.compressor.wkv.scales": "model-00036-of-00046.safetensors", "layers.34.attn.indexer.compressor.ape": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.norm.weight": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.wgate.qweight": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.wgate.qzeros": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.wgate.scales": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.wkv.qweight": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.wkv.qzeros": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.compressor.wkv.scales": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.weights_proj.qweight": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.weights_proj.qzeros": "model-00036-of-00046.safetensors", - "layers.34.attn.indexer.weights_proj.scales": "model-00036-of-00046.safetensors", "layers.34.attn.indexer.wq_b.qweight": "model-00036-of-00046.safetensors", "layers.34.attn.indexer.wq_b.qzeros": "model-00036-of-00046.safetensors", "layers.34.attn.indexer.wq_b.scales": "model-00036-of-00046.safetensors", @@ -80106,9 +79797,7 @@ "layers.34.attn.wkv.qweight": "model-00036-of-00046.safetensors", "layers.34.attn.wkv.qzeros": "model-00036-of-00046.safetensors", "layers.34.attn.wkv.scales": "model-00036-of-00046.safetensors", - "layers.34.attn.wo_a.qweight": "model-00036-of-00046.safetensors", - "layers.34.attn.wo_a.qzeros": "model-00036-of-00046.safetensors", - "layers.34.attn.wo_a.scales": "model-00036-of-00046.safetensors", + "layers.34.attn.wo_a.weight": "model-00036-of-00046.safetensors", "layers.34.attn.wo_b.qweight": "model-00036-of-00046.safetensors", "layers.34.attn.wo_b.qzeros": "model-00036-of-00046.safetensors", "layers.34.attn.wo_b.scales": "model-00036-of-00046.safetensors", @@ -82423,8 +82112,6 @@ "layers.34.ffn.experts.99.w3.qweight": "model-00036-of-00046.safetensors", "layers.34.ffn.experts.99.w3.qzeros": "model-00036-of-00046.safetensors", "layers.34.ffn.experts.99.w3.scales": "model-00036-of-00046.safetensors", - "layers.34.ffn.gate.bias": "model-00036-of-00046.safetensors", - "layers.34.ffn.gate.weight": "model-00036-of-00046.safetensors", "layers.34.ffn.shared_experts.w1.qweight": "model-00036-of-00046.safetensors", "layers.34.ffn.shared_experts.w1.qzeros": "model-00036-of-00046.safetensors", "layers.34.ffn.shared_experts.w1.scales": "model-00036-of-00046.safetensors", @@ -82441,23 +82128,23 @@ "layers.34.hc_ffn_base": "model-00036-of-00046.safetensors", "layers.34.hc_ffn_fn": "model-00036-of-00046.safetensors", "layers.34.hc_ffn_scale": "model-00036-of-00046.safetensors", + "layers.34.attn.compressor.norm.weight": "model-00036-of-00046.safetensors", + "layers.34.attn.compressor.wgate.weight": "model-00036-of-00046.safetensors", + "layers.34.attn.compressor.wkv.weight": "model-00036-of-00046.safetensors", + "layers.34.attn.indexer.compressor.norm.weight": "model-00036-of-00046.safetensors", + "layers.34.attn.indexer.compressor.wgate.weight": "model-00036-of-00046.safetensors", + "layers.34.attn.indexer.compressor.wkv.weight": "model-00036-of-00046.safetensors", + "layers.34.attn.indexer.weights_proj.weight": "model-00036-of-00046.safetensors", + "layers.34.ffn.gate.bias": "model-00036-of-00046.safetensors", + "layers.34.ffn.gate.weight": "model-00036-of-00046.safetensors", "layers.35.attn.attn_sink": "model-00037-of-00046.safetensors", "layers.35.attn.compressor.ape": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.norm.weight": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.wgate.qweight": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.wgate.qzeros": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.wgate.scales": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.wkv.qweight": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.wkv.qzeros": "model-00037-of-00046.safetensors", - "layers.35.attn.compressor.wkv.scales": "model-00037-of-00046.safetensors", "layers.35.attn.kv_norm.weight": "model-00037-of-00046.safetensors", "layers.35.attn.q_norm.weight": "model-00037-of-00046.safetensors", "layers.35.attn.wkv.qweight": "model-00037-of-00046.safetensors", "layers.35.attn.wkv.qzeros": "model-00037-of-00046.safetensors", "layers.35.attn.wkv.scales": "model-00037-of-00046.safetensors", - "layers.35.attn.wo_a.qweight": "model-00037-of-00046.safetensors", - "layers.35.attn.wo_a.qzeros": "model-00037-of-00046.safetensors", - "layers.35.attn.wo_a.scales": "model-00037-of-00046.safetensors", + "layers.35.attn.wo_a.weight": "model-00037-of-00046.safetensors", "layers.35.attn.wo_b.qweight": "model-00037-of-00046.safetensors", "layers.35.attn.wo_b.qzeros": "model-00037-of-00046.safetensors", "layers.35.attn.wo_b.scales": "model-00037-of-00046.safetensors", @@ -84772,8 +84459,6 @@ "layers.35.ffn.experts.99.w3.qweight": "model-00037-of-00046.safetensors", "layers.35.ffn.experts.99.w3.qzeros": "model-00037-of-00046.safetensors", "layers.35.ffn.experts.99.w3.scales": "model-00037-of-00046.safetensors", - "layers.35.ffn.gate.bias": "model-00037-of-00046.safetensors", - "layers.35.ffn.gate.weight": "model-00037-of-00046.safetensors", "layers.35.ffn.shared_experts.w1.qweight": "model-00037-of-00046.safetensors", "layers.35.ffn.shared_experts.w1.qzeros": "model-00037-of-00046.safetensors", "layers.35.ffn.shared_experts.w1.scales": "model-00037-of-00046.safetensors", @@ -84790,26 +84475,14 @@ "layers.35.hc_ffn_base": "model-00037-of-00046.safetensors", "layers.35.hc_ffn_fn": "model-00037-of-00046.safetensors", "layers.35.hc_ffn_scale": "model-00037-of-00046.safetensors", + "layers.35.attn.compressor.norm.weight": "model-00037-of-00046.safetensors", + "layers.35.attn.compressor.wgate.weight": "model-00037-of-00046.safetensors", + "layers.35.attn.compressor.wkv.weight": "model-00037-of-00046.safetensors", + "layers.35.ffn.gate.bias": "model-00037-of-00046.safetensors", + "layers.35.ffn.gate.weight": "model-00037-of-00046.safetensors", "layers.36.attn.attn_sink": "model-00038-of-00046.safetensors", "layers.36.attn.compressor.ape": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.norm.weight": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.wgate.qweight": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.wgate.qzeros": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.wgate.scales": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.wkv.qweight": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.wkv.qzeros": "model-00038-of-00046.safetensors", - "layers.36.attn.compressor.wkv.scales": "model-00038-of-00046.safetensors", "layers.36.attn.indexer.compressor.ape": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.norm.weight": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.wgate.qweight": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.wgate.qzeros": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.wgate.scales": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.wkv.qweight": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.wkv.qzeros": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.compressor.wkv.scales": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.weights_proj.qweight": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.weights_proj.qzeros": "model-00038-of-00046.safetensors", - "layers.36.attn.indexer.weights_proj.scales": "model-00038-of-00046.safetensors", "layers.36.attn.indexer.wq_b.qweight": "model-00038-of-00046.safetensors", "layers.36.attn.indexer.wq_b.qzeros": "model-00038-of-00046.safetensors", "layers.36.attn.indexer.wq_b.scales": "model-00038-of-00046.safetensors", @@ -84818,9 +84491,7 @@ "layers.36.attn.wkv.qweight": "model-00038-of-00046.safetensors", "layers.36.attn.wkv.qzeros": "model-00038-of-00046.safetensors", "layers.36.attn.wkv.scales": "model-00038-of-00046.safetensors", - "layers.36.attn.wo_a.qweight": "model-00038-of-00046.safetensors", - "layers.36.attn.wo_a.qzeros": "model-00038-of-00046.safetensors", - "layers.36.attn.wo_a.scales": "model-00038-of-00046.safetensors", + "layers.36.attn.wo_a.weight": "model-00038-of-00046.safetensors", "layers.36.attn.wo_b.qweight": "model-00038-of-00046.safetensors", "layers.36.attn.wo_b.qzeros": "model-00038-of-00046.safetensors", "layers.36.attn.wo_b.scales": "model-00038-of-00046.safetensors", @@ -87135,8 +86806,6 @@ "layers.36.ffn.experts.99.w3.qweight": "model-00038-of-00046.safetensors", "layers.36.ffn.experts.99.w3.qzeros": "model-00038-of-00046.safetensors", "layers.36.ffn.experts.99.w3.scales": "model-00038-of-00046.safetensors", - "layers.36.ffn.gate.bias": "model-00038-of-00046.safetensors", - "layers.36.ffn.gate.weight": "model-00038-of-00046.safetensors", "layers.36.ffn.shared_experts.w1.qweight": "model-00038-of-00046.safetensors", "layers.36.ffn.shared_experts.w1.qzeros": "model-00038-of-00046.safetensors", "layers.36.ffn.shared_experts.w1.scales": "model-00038-of-00046.safetensors", @@ -87153,23 +86822,23 @@ "layers.36.hc_ffn_base": "model-00038-of-00046.safetensors", "layers.36.hc_ffn_fn": "model-00038-of-00046.safetensors", "layers.36.hc_ffn_scale": "model-00038-of-00046.safetensors", + "layers.36.attn.compressor.norm.weight": "model-00038-of-00046.safetensors", + "layers.36.attn.compressor.wgate.weight": "model-00038-of-00046.safetensors", + "layers.36.attn.compressor.wkv.weight": "model-00038-of-00046.safetensors", + "layers.36.attn.indexer.compressor.norm.weight": "model-00038-of-00046.safetensors", + "layers.36.attn.indexer.compressor.wgate.weight": "model-00038-of-00046.safetensors", + "layers.36.attn.indexer.compressor.wkv.weight": "model-00038-of-00046.safetensors", + "layers.36.attn.indexer.weights_proj.weight": "model-00038-of-00046.safetensors", + "layers.36.ffn.gate.bias": "model-00038-of-00046.safetensors", + "layers.36.ffn.gate.weight": "model-00038-of-00046.safetensors", "layers.37.attn.attn_sink": "model-00039-of-00046.safetensors", "layers.37.attn.compressor.ape": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.norm.weight": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.wgate.qweight": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.wgate.qzeros": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.wgate.scales": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.wkv.qweight": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.wkv.qzeros": "model-00039-of-00046.safetensors", - "layers.37.attn.compressor.wkv.scales": "model-00039-of-00046.safetensors", "layers.37.attn.kv_norm.weight": "model-00039-of-00046.safetensors", "layers.37.attn.q_norm.weight": "model-00039-of-00046.safetensors", "layers.37.attn.wkv.qweight": "model-00039-of-00046.safetensors", "layers.37.attn.wkv.qzeros": "model-00039-of-00046.safetensors", "layers.37.attn.wkv.scales": "model-00039-of-00046.safetensors", - "layers.37.attn.wo_a.qweight": "model-00039-of-00046.safetensors", - "layers.37.attn.wo_a.qzeros": "model-00039-of-00046.safetensors", - "layers.37.attn.wo_a.scales": "model-00039-of-00046.safetensors", + "layers.37.attn.wo_a.weight": "model-00039-of-00046.safetensors", "layers.37.attn.wo_b.qweight": "model-00039-of-00046.safetensors", "layers.37.attn.wo_b.qzeros": "model-00039-of-00046.safetensors", "layers.37.attn.wo_b.scales": "model-00039-of-00046.safetensors", @@ -89484,8 +89153,6 @@ "layers.37.ffn.experts.99.w3.qweight": "model-00039-of-00046.safetensors", "layers.37.ffn.experts.99.w3.qzeros": "model-00039-of-00046.safetensors", "layers.37.ffn.experts.99.w3.scales": "model-00039-of-00046.safetensors", - "layers.37.ffn.gate.bias": "model-00039-of-00046.safetensors", - "layers.37.ffn.gate.weight": "model-00039-of-00046.safetensors", "layers.37.ffn.shared_experts.w1.qweight": "model-00039-of-00046.safetensors", "layers.37.ffn.shared_experts.w1.qzeros": "model-00039-of-00046.safetensors", "layers.37.ffn.shared_experts.w1.scales": "model-00039-of-00046.safetensors", @@ -89502,26 +89169,14 @@ "layers.37.hc_ffn_base": "model-00039-of-00046.safetensors", "layers.37.hc_ffn_fn": "model-00039-of-00046.safetensors", "layers.37.hc_ffn_scale": "model-00039-of-00046.safetensors", + "layers.37.attn.compressor.norm.weight": "model-00039-of-00046.safetensors", + "layers.37.attn.compressor.wgate.weight": "model-00039-of-00046.safetensors", + "layers.37.attn.compressor.wkv.weight": "model-00039-of-00046.safetensors", + "layers.37.ffn.gate.bias": "model-00039-of-00046.safetensors", + "layers.37.ffn.gate.weight": "model-00039-of-00046.safetensors", "layers.38.attn.attn_sink": "model-00040-of-00046.safetensors", "layers.38.attn.compressor.ape": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.norm.weight": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.wgate.qweight": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.wgate.qzeros": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.wgate.scales": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.wkv.qweight": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.wkv.qzeros": "model-00040-of-00046.safetensors", - "layers.38.attn.compressor.wkv.scales": "model-00040-of-00046.safetensors", "layers.38.attn.indexer.compressor.ape": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.norm.weight": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.wgate.qweight": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.wgate.qzeros": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.wgate.scales": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.wkv.qweight": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.wkv.qzeros": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.compressor.wkv.scales": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.weights_proj.qweight": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.weights_proj.qzeros": "model-00040-of-00046.safetensors", - "layers.38.attn.indexer.weights_proj.scales": "model-00040-of-00046.safetensors", "layers.38.attn.indexer.wq_b.qweight": "model-00040-of-00046.safetensors", "layers.38.attn.indexer.wq_b.qzeros": "model-00040-of-00046.safetensors", "layers.38.attn.indexer.wq_b.scales": "model-00040-of-00046.safetensors", @@ -89530,9 +89185,7 @@ "layers.38.attn.wkv.qweight": "model-00040-of-00046.safetensors", "layers.38.attn.wkv.qzeros": "model-00040-of-00046.safetensors", "layers.38.attn.wkv.scales": "model-00040-of-00046.safetensors", - "layers.38.attn.wo_a.qweight": "model-00040-of-00046.safetensors", - "layers.38.attn.wo_a.qzeros": "model-00040-of-00046.safetensors", - "layers.38.attn.wo_a.scales": "model-00040-of-00046.safetensors", + "layers.38.attn.wo_a.weight": "model-00040-of-00046.safetensors", "layers.38.attn.wo_b.qweight": "model-00040-of-00046.safetensors", "layers.38.attn.wo_b.qzeros": "model-00040-of-00046.safetensors", "layers.38.attn.wo_b.scales": "model-00040-of-00046.safetensors", @@ -91847,8 +91500,6 @@ "layers.38.ffn.experts.99.w3.qweight": "model-00040-of-00046.safetensors", "layers.38.ffn.experts.99.w3.qzeros": "model-00040-of-00046.safetensors", "layers.38.ffn.experts.99.w3.scales": "model-00040-of-00046.safetensors", - "layers.38.ffn.gate.bias": "model-00040-of-00046.safetensors", - "layers.38.ffn.gate.weight": "model-00040-of-00046.safetensors", "layers.38.ffn.shared_experts.w1.qweight": "model-00040-of-00046.safetensors", "layers.38.ffn.shared_experts.w1.qzeros": "model-00040-of-00046.safetensors", "layers.38.ffn.shared_experts.w1.scales": "model-00040-of-00046.safetensors", @@ -91865,23 +91516,23 @@ "layers.38.hc_ffn_base": "model-00040-of-00046.safetensors", "layers.38.hc_ffn_fn": "model-00040-of-00046.safetensors", "layers.38.hc_ffn_scale": "model-00040-of-00046.safetensors", + "layers.38.attn.compressor.norm.weight": "model-00040-of-00046.safetensors", + "layers.38.attn.compressor.wgate.weight": "model-00040-of-00046.safetensors", + "layers.38.attn.compressor.wkv.weight": "model-00040-of-00046.safetensors", + "layers.38.attn.indexer.compressor.norm.weight": "model-00040-of-00046.safetensors", + "layers.38.attn.indexer.compressor.wgate.weight": "model-00040-of-00046.safetensors", + "layers.38.attn.indexer.compressor.wkv.weight": "model-00040-of-00046.safetensors", + "layers.38.attn.indexer.weights_proj.weight": "model-00040-of-00046.safetensors", + "layers.38.ffn.gate.bias": "model-00040-of-00046.safetensors", + "layers.38.ffn.gate.weight": "model-00040-of-00046.safetensors", "layers.39.attn.attn_sink": "model-00041-of-00046.safetensors", "layers.39.attn.compressor.ape": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.norm.weight": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.wgate.qweight": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.wgate.qzeros": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.wgate.scales": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.wkv.qweight": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.wkv.qzeros": "model-00041-of-00046.safetensors", - "layers.39.attn.compressor.wkv.scales": "model-00041-of-00046.safetensors", "layers.39.attn.kv_norm.weight": "model-00041-of-00046.safetensors", "layers.39.attn.q_norm.weight": "model-00041-of-00046.safetensors", "layers.39.attn.wkv.qweight": "model-00041-of-00046.safetensors", "layers.39.attn.wkv.qzeros": "model-00041-of-00046.safetensors", "layers.39.attn.wkv.scales": "model-00041-of-00046.safetensors", - "layers.39.attn.wo_a.qweight": "model-00041-of-00046.safetensors", - "layers.39.attn.wo_a.qzeros": "model-00041-of-00046.safetensors", - "layers.39.attn.wo_a.scales": "model-00041-of-00046.safetensors", + "layers.39.attn.wo_a.weight": "model-00041-of-00046.safetensors", "layers.39.attn.wo_b.qweight": "model-00041-of-00046.safetensors", "layers.39.attn.wo_b.qzeros": "model-00041-of-00046.safetensors", "layers.39.attn.wo_b.scales": "model-00041-of-00046.safetensors", @@ -94196,8 +93847,6 @@ "layers.39.ffn.experts.99.w3.qweight": "model-00041-of-00046.safetensors", "layers.39.ffn.experts.99.w3.qzeros": "model-00041-of-00046.safetensors", "layers.39.ffn.experts.99.w3.scales": "model-00041-of-00046.safetensors", - "layers.39.ffn.gate.bias": "model-00041-of-00046.safetensors", - "layers.39.ffn.gate.weight": "model-00041-of-00046.safetensors", "layers.39.ffn.shared_experts.w1.qweight": "model-00041-of-00046.safetensors", "layers.39.ffn.shared_experts.w1.qzeros": "model-00041-of-00046.safetensors", "layers.39.ffn.shared_experts.w1.scales": "model-00041-of-00046.safetensors", @@ -94214,26 +93863,14 @@ "layers.39.hc_ffn_base": "model-00041-of-00046.safetensors", "layers.39.hc_ffn_fn": "model-00041-of-00046.safetensors", "layers.39.hc_ffn_scale": "model-00041-of-00046.safetensors", + "layers.39.attn.compressor.norm.weight": "model-00041-of-00046.safetensors", + "layers.39.attn.compressor.wgate.weight": "model-00041-of-00046.safetensors", + "layers.39.attn.compressor.wkv.weight": "model-00041-of-00046.safetensors", + "layers.39.ffn.gate.bias": "model-00041-of-00046.safetensors", + "layers.39.ffn.gate.weight": "model-00041-of-00046.safetensors", "layers.40.attn.attn_sink": "model-00042-of-00046.safetensors", "layers.40.attn.compressor.ape": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.norm.weight": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.wgate.qweight": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.wgate.qzeros": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.wgate.scales": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.wkv.qweight": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.wkv.qzeros": "model-00042-of-00046.safetensors", - "layers.40.attn.compressor.wkv.scales": "model-00042-of-00046.safetensors", "layers.40.attn.indexer.compressor.ape": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.norm.weight": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.wgate.qweight": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.wgate.qzeros": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.wgate.scales": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.wkv.qweight": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.wkv.qzeros": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.compressor.wkv.scales": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.weights_proj.qweight": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.weights_proj.qzeros": "model-00042-of-00046.safetensors", - "layers.40.attn.indexer.weights_proj.scales": "model-00042-of-00046.safetensors", "layers.40.attn.indexer.wq_b.qweight": "model-00042-of-00046.safetensors", "layers.40.attn.indexer.wq_b.qzeros": "model-00042-of-00046.safetensors", "layers.40.attn.indexer.wq_b.scales": "model-00042-of-00046.safetensors", @@ -94242,9 +93879,7 @@ "layers.40.attn.wkv.qweight": "model-00042-of-00046.safetensors", "layers.40.attn.wkv.qzeros": "model-00042-of-00046.safetensors", "layers.40.attn.wkv.scales": "model-00042-of-00046.safetensors", - "layers.40.attn.wo_a.qweight": "model-00042-of-00046.safetensors", - "layers.40.attn.wo_a.qzeros": "model-00042-of-00046.safetensors", - "layers.40.attn.wo_a.scales": "model-00042-of-00046.safetensors", + "layers.40.attn.wo_a.weight": "model-00042-of-00046.safetensors", "layers.40.attn.wo_b.qweight": "model-00042-of-00046.safetensors", "layers.40.attn.wo_b.qzeros": "model-00042-of-00046.safetensors", "layers.40.attn.wo_b.scales": "model-00042-of-00046.safetensors", @@ -96559,8 +96194,6 @@ "layers.40.ffn.experts.99.w3.qweight": "model-00042-of-00046.safetensors", "layers.40.ffn.experts.99.w3.qzeros": "model-00042-of-00046.safetensors", "layers.40.ffn.experts.99.w3.scales": "model-00042-of-00046.safetensors", - "layers.40.ffn.gate.bias": "model-00042-of-00046.safetensors", - "layers.40.ffn.gate.weight": "model-00042-of-00046.safetensors", "layers.40.ffn.shared_experts.w1.qweight": "model-00042-of-00046.safetensors", "layers.40.ffn.shared_experts.w1.qzeros": "model-00042-of-00046.safetensors", "layers.40.ffn.shared_experts.w1.scales": "model-00042-of-00046.safetensors", @@ -96577,23 +96210,23 @@ "layers.40.hc_ffn_base": "model-00042-of-00046.safetensors", "layers.40.hc_ffn_fn": "model-00042-of-00046.safetensors", "layers.40.hc_ffn_scale": "model-00042-of-00046.safetensors", + "layers.40.attn.compressor.norm.weight": "model-00042-of-00046.safetensors", + "layers.40.attn.compressor.wgate.weight": "model-00042-of-00046.safetensors", + "layers.40.attn.compressor.wkv.weight": "model-00042-of-00046.safetensors", + "layers.40.attn.indexer.compressor.norm.weight": "model-00042-of-00046.safetensors", + "layers.40.attn.indexer.compressor.wgate.weight": "model-00042-of-00046.safetensors", + "layers.40.attn.indexer.compressor.wkv.weight": "model-00042-of-00046.safetensors", + "layers.40.attn.indexer.weights_proj.weight": "model-00042-of-00046.safetensors", + "layers.40.ffn.gate.bias": "model-00042-of-00046.safetensors", + "layers.40.ffn.gate.weight": "model-00042-of-00046.safetensors", "layers.41.attn.attn_sink": "model-00043-of-00046.safetensors", "layers.41.attn.compressor.ape": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.norm.weight": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.wgate.qweight": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.wgate.qzeros": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.wgate.scales": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.wkv.qweight": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.wkv.qzeros": "model-00043-of-00046.safetensors", - "layers.41.attn.compressor.wkv.scales": "model-00043-of-00046.safetensors", "layers.41.attn.kv_norm.weight": "model-00043-of-00046.safetensors", "layers.41.attn.q_norm.weight": "model-00043-of-00046.safetensors", "layers.41.attn.wkv.qweight": "model-00043-of-00046.safetensors", "layers.41.attn.wkv.qzeros": "model-00043-of-00046.safetensors", "layers.41.attn.wkv.scales": "model-00043-of-00046.safetensors", - "layers.41.attn.wo_a.qweight": "model-00043-of-00046.safetensors", - "layers.41.attn.wo_a.qzeros": "model-00043-of-00046.safetensors", - "layers.41.attn.wo_a.scales": "model-00043-of-00046.safetensors", + "layers.41.attn.wo_a.weight": "model-00043-of-00046.safetensors", "layers.41.attn.wo_b.qweight": "model-00043-of-00046.safetensors", "layers.41.attn.wo_b.qzeros": "model-00043-of-00046.safetensors", "layers.41.attn.wo_b.scales": "model-00043-of-00046.safetensors", @@ -98908,8 +98541,6 @@ "layers.41.ffn.experts.99.w3.qweight": "model-00043-of-00046.safetensors", "layers.41.ffn.experts.99.w3.qzeros": "model-00043-of-00046.safetensors", "layers.41.ffn.experts.99.w3.scales": "model-00043-of-00046.safetensors", - "layers.41.ffn.gate.bias": "model-00043-of-00046.safetensors", - "layers.41.ffn.gate.weight": "model-00043-of-00046.safetensors", "layers.41.ffn.shared_experts.w1.qweight": "model-00043-of-00046.safetensors", "layers.41.ffn.shared_experts.w1.qzeros": "model-00043-of-00046.safetensors", "layers.41.ffn.shared_experts.w1.scales": "model-00043-of-00046.safetensors", @@ -98926,26 +98557,14 @@ "layers.41.hc_ffn_base": "model-00043-of-00046.safetensors", "layers.41.hc_ffn_fn": "model-00043-of-00046.safetensors", "layers.41.hc_ffn_scale": "model-00043-of-00046.safetensors", + "layers.41.attn.compressor.norm.weight": "model-00043-of-00046.safetensors", + "layers.41.attn.compressor.wgate.weight": "model-00043-of-00046.safetensors", + "layers.41.attn.compressor.wkv.weight": "model-00043-of-00046.safetensors", + "layers.41.ffn.gate.bias": "model-00043-of-00046.safetensors", + "layers.41.ffn.gate.weight": "model-00043-of-00046.safetensors", "layers.42.attn.attn_sink": "model-00044-of-00046.safetensors", "layers.42.attn.compressor.ape": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.norm.weight": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.wgate.qweight": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.wgate.qzeros": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.wgate.scales": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.wkv.qweight": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.wkv.qzeros": "model-00044-of-00046.safetensors", - "layers.42.attn.compressor.wkv.scales": "model-00044-of-00046.safetensors", "layers.42.attn.indexer.compressor.ape": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.norm.weight": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.wgate.qweight": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.wgate.qzeros": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.wgate.scales": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.wkv.qweight": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.wkv.qzeros": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.compressor.wkv.scales": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.weights_proj.qweight": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.weights_proj.qzeros": "model-00044-of-00046.safetensors", - "layers.42.attn.indexer.weights_proj.scales": "model-00044-of-00046.safetensors", "layers.42.attn.indexer.wq_b.qweight": "model-00044-of-00046.safetensors", "layers.42.attn.indexer.wq_b.qzeros": "model-00044-of-00046.safetensors", "layers.42.attn.indexer.wq_b.scales": "model-00044-of-00046.safetensors", @@ -98954,9 +98573,7 @@ "layers.42.attn.wkv.qweight": "model-00044-of-00046.safetensors", "layers.42.attn.wkv.qzeros": "model-00044-of-00046.safetensors", "layers.42.attn.wkv.scales": "model-00044-of-00046.safetensors", - "layers.42.attn.wo_a.qweight": "model-00044-of-00046.safetensors", - "layers.42.attn.wo_a.qzeros": "model-00044-of-00046.safetensors", - "layers.42.attn.wo_a.scales": "model-00044-of-00046.safetensors", + "layers.42.attn.wo_a.weight": "model-00044-of-00046.safetensors", "layers.42.attn.wo_b.qweight": "model-00044-of-00046.safetensors", "layers.42.attn.wo_b.qzeros": "model-00044-of-00046.safetensors", "layers.42.attn.wo_b.scales": "model-00044-of-00046.safetensors", @@ -101271,8 +100888,6 @@ "layers.42.ffn.experts.99.w3.qweight": "model-00044-of-00046.safetensors", "layers.42.ffn.experts.99.w3.qzeros": "model-00044-of-00046.safetensors", "layers.42.ffn.experts.99.w3.scales": "model-00044-of-00046.safetensors", - "layers.42.ffn.gate.bias": "model-00044-of-00046.safetensors", - "layers.42.ffn.gate.weight": "model-00044-of-00046.safetensors", "layers.42.ffn.shared_experts.w1.qweight": "model-00044-of-00046.safetensors", "layers.42.ffn.shared_experts.w1.qzeros": "model-00044-of-00046.safetensors", "layers.42.ffn.shared_experts.w1.scales": "model-00044-of-00046.safetensors", @@ -101289,20 +100904,27 @@ "layers.42.hc_ffn_base": "model-00044-of-00046.safetensors", "layers.42.hc_ffn_fn": "model-00044-of-00046.safetensors", "layers.42.hc_ffn_scale": "model-00044-of-00046.safetensors", + "layers.42.attn.compressor.norm.weight": "model-00044-of-00046.safetensors", + "layers.42.attn.compressor.wgate.weight": "model-00044-of-00046.safetensors", + "layers.42.attn.compressor.wkv.weight": "model-00044-of-00046.safetensors", + "layers.42.attn.indexer.compressor.norm.weight": "model-00044-of-00046.safetensors", + "layers.42.attn.indexer.compressor.wgate.weight": "model-00044-of-00046.safetensors", + "layers.42.attn.indexer.compressor.wkv.weight": "model-00044-of-00046.safetensors", + "layers.42.attn.indexer.weights_proj.weight": "model-00044-of-00046.safetensors", + "layers.42.ffn.gate.bias": "model-00044-of-00046.safetensors", + "layers.42.ffn.gate.weight": "model-00044-of-00046.safetensors", "hc_head_base": "model-00045-of-00046.safetensors", "hc_head_fn": "model-00045-of-00046.safetensors", "hc_head_scale": "model-00045-of-00046.safetensors", - "head.weight": "model-00045-of-00046.safetensors", "norm.weight": "model-00045-of-00046.safetensors", + "head.weight": "model-00045-of-00046.safetensors", "mtp.0.attn.attn_sink": "model-00046-of-00046.safetensors", "mtp.0.attn.kv_norm.weight": "model-00046-of-00046.safetensors", "mtp.0.attn.q_norm.weight": "model-00046-of-00046.safetensors", "mtp.0.attn.wkv.qweight": "model-00046-of-00046.safetensors", "mtp.0.attn.wkv.qzeros": "model-00046-of-00046.safetensors", "mtp.0.attn.wkv.scales": "model-00046-of-00046.safetensors", - "mtp.0.attn.wo_a.qweight": "model-00046-of-00046.safetensors", - "mtp.0.attn.wo_a.qzeros": "model-00046-of-00046.safetensors", - "mtp.0.attn.wo_a.scales": "model-00046-of-00046.safetensors", + "mtp.0.attn.wo_a.weight": "model-00046-of-00046.safetensors", "mtp.0.attn.wo_b.qweight": "model-00046-of-00046.safetensors", "mtp.0.attn.wo_b.qzeros": "model-00046-of-00046.safetensors", "mtp.0.attn.wo_b.scales": "model-00046-of-00046.safetensors", @@ -103621,8 +103243,6 @@ "mtp.0.ffn.experts.99.w3.qweight": "model-00046-of-00046.safetensors", "mtp.0.ffn.experts.99.w3.qzeros": "model-00046-of-00046.safetensors", "mtp.0.ffn.experts.99.w3.scales": "model-00046-of-00046.safetensors", - "mtp.0.ffn.gate.bias": "model-00046-of-00046.safetensors", - "mtp.0.ffn.gate.weight": "model-00046-of-00046.safetensors", "mtp.0.ffn.shared_experts.w1.qweight": "model-00046-of-00046.safetensors", "mtp.0.ffn.shared_experts.w1.qzeros": "model-00046-of-00046.safetensors", "mtp.0.ffn.shared_experts.w1.scales": "model-00046-of-00046.safetensors", @@ -103646,6 +103266,8 @@ "mtp.0.hc_head_fn": "model-00046-of-00046.safetensors", "mtp.0.hc_head_scale": "model-00046-of-00046.safetensors", "mtp.0.hnorm.weight": "model-00046-of-00046.safetensors", - "mtp.0.norm.weight": "model-00046-of-00046.safetensors" + "mtp.0.norm.weight": "model-00046-of-00046.safetensors", + "mtp.0.ffn.gate.bias": "model-00046-of-00046.safetensors", + "mtp.0.ffn.gate.weight": "model-00046-of-00046.safetensors" } } \ No newline at end of file diff --git a/quantization_config.json b/quantization_config.json index bce2c1b5c3227fb812b3e9fafb3dd6a2da4f98f5..79533c578162be46325cd3d0508da74b0e196b18 100644 --- a/quantization_config.json +++ b/quantization_config.json @@ -7,11 +7,950 @@ "data_type": "int", "iters": 0, "model_free": true, - "autoround_version": "0.13.0", + "autoround_version": "0.14.0", "extra_config": { + "wo_a": { + "bits": 16 + }, + "embed": { + "bits": 16, + "data_type": "float" + }, + "layers.0.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.0.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.1.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.1.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.2.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.2.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.3.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.3.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.3.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.3.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.4.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.4.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.5.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.5.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.5.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.5.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.6.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.6.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.7.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.7.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.7.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.7.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.8.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.8.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.9.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.9.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.9.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.9.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.10.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.10.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.11.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.11.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.11.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.11.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.12.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.12.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.13.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.13.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.13.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.13.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.14.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.14.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.15.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.15.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.15.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.15.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.16.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.16.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.17.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.17.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.17.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.17.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.18.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.18.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.19.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.19.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.19.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.19.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.20.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.20.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.21.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.21.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.21.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.21.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.22.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.22.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.23.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.23.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.23.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.23.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.24.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.24.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.25.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.25.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.25.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.25.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.26.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.26.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.27.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.27.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.27.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.27.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.28.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.28.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.29.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.29.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.29.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.29.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.30.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.30.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.31.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.31.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.31.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.31.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.32.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.32.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.33.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.33.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.33.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.33.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.34.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.34.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.35.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.35.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.35.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.35.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.36.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.36.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.37.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.37.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.37.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.37.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.38.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.38.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.39.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.39.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.39.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.39.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.40.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.40.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.41.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.41.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.41.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.41.ffn.gate": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.indexer.compressor.wgate": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.indexer.compressor.wkv": { + "bits": 16, + "data_type": "float" + }, + "layers.42.attn.indexer.weights_proj": { + "bits": 16, + "data_type": "float" + }, + "layers.42.ffn.gate": { + "bits": 16, + "data_type": "float" + }, "head": { "bits": 16, "data_type": "float" + }, + "mtp.0.attn.wo_a": { + "bits": 16, + "data_type": "float" + }, + "mtp.0.ffn.gate": { + "bits": 16, + "data_type": "float" } } } \ No newline at end of file