Upload folder using huggingface_hub
Browse files- .gitattributes +3 -10
- README.md +67 -96
- assets/logo.png +3 -0
- assets/logo2.jpeg +3 -0
- assets/pipe.png +3 -0
- diffusion_pytorch_model.safetensors.index.json +0 -0
- quant_models/dit_model_map_int8.json +1 -0
- quant_models/quantization_map_fp8_FusionX.json +1 -0
- quant_models/quantization_map_int8_FusionX.json +1 -0
- quant_models/t5_map_fp8.json +1 -0
- quant_models/t5_map_int8.json +1 -0
.gitattributes
CHANGED
|
@@ -33,13 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
eval/ArtificialAnalysis-2025-02-26.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 40 |
-
eval/TTS_Arena-2025-02-26.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 41 |
-
eval/TTS_Spaces_Arena-2025-02-26.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 42 |
-
samples/HEARME.wav filter=lfs diff=lfs merge=lfs -text
|
| 43 |
-
samples/af_heart_0.wav filter=lfs diff=lfs merge=lfs -text
|
| 44 |
-
samples/af_heart_1.wav filter=lfs diff=lfs merge=lfs -text
|
| 45 |
-
samples/af_heart_2.wav filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/logo.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/logo2.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/pipe.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -2,116 +2,87 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
|
|
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
-
**Kokoro** is an open-weight TTS model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost-efficient. With Apache-licensed weights, Kokoro can be deployed anywhere from production environments to personal projects.
|
| 10 |
-
|
| 11 |
-
<audio controls><source src="https://huggingface.co/hexgrad/Kokoro-82M/resolve/main/samples/HEARME.wav" type="audio/wav"></audio>
|
| 12 |
-
|
| 13 |
-
🐈 **GitHub**: https://github.com/hexgrad/kokoro
|
| 14 |
-
|
| 15 |
-
🚀 **Demo**: https://hf.co/spaces/hexgrad/Kokoro-TTS
|
| 16 |
-
|
| 17 |
-
> [!NOTE]
|
| 18 |
-
> As of April 2025, the market rate of Kokoro served over API is **under $1 per million characters of text input**, or under $0.06 per hour of audio output. (On average, 1000 characters of input is about 1 minute of output.) Sources: [ArtificialAnalysis/Replicate at 65 cents per M chars](https://artificialanalysis.ai/text-to-speech/model-family/kokoro#price) and [DeepInfra at 80 cents per M chars](https://deepinfra.com/hexgrad/Kokoro-82M).
|
| 19 |
-
>
|
| 20 |
-
> This is an Apache-licensed model, and Kokoro has been deployed in numerous projects and commercial APIs. We welcome the deployment of the model in real use cases.
|
| 21 |
-
|
| 22 |
-
> [!CAUTION]
|
| 23 |
-
> Fake websites like kokorottsai_com (snapshot: https://archive.ph/nRRnk) and kokorotts_net (snapshot: https://archive.ph/60opa) are likely scams masquerading under the banner of a popular model.
|
| 24 |
-
>
|
| 25 |
-
> Any website containing "kokoro" in its root domain (e.g. kokorottsai_com, kokorotts_net) is **NOT owned by and NOT affiliated with this model page or its author**, and attempts to imply otherwise are red flags.
|
| 26 |
-
|
| 27 |
-
- [Releases](#releases)
|
| 28 |
-
- [Usage](#usage)
|
| 29 |
-
- [EVAL.md](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/EVAL.md) ↗️
|
| 30 |
-
- [SAMPLES.md](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/SAMPLES.md) ↗️
|
| 31 |
-
- [VOICES.md](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/VOICES.md) ↗️
|
| 32 |
-
- [Model Facts](#model-facts)
|
| 33 |
-
- [Training Details](#training-details)
|
| 34 |
-
- [Creative Commons Attribution](#creative-commons-attribution)
|
| 35 |
-
- [Acknowledgements](#acknowledgements)
|
| 36 |
-
|
| 37 |
-
### Releases
|
| 38 |
-
|
| 39 |
-
| Model | Published | Training Data | Langs & Voices | SHA256 |
|
| 40 |
-
| ----- | --------- | ------------- | -------------- | ------ |
|
| 41 |
-
| **v1.0** | **2025 Jan 27** | **Few hundred hrs** | [**8 & 54**](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/VOICES.md) | `496dba11` |
|
| 42 |
-
| [v0.19](https://huggingface.co/hexgrad/kLegacy/tree/main/v0.19) | 2024 Dec 25 | <100 hrs | 1 & 10 | `3b0c392f` |
|
| 43 |
-
|
| 44 |
-
| Training Costs | v0.19 | v1.0 | **Total** |
|
| 45 |
-
| -------------- | ----- | ---- | ----- |
|
| 46 |
-
| in A100 80GB GPU hours | 500 | 500 | **1000** |
|
| 47 |
-
| average hourly rate | $0.80/h | $1.20/h | **$1/h** |
|
| 48 |
-
| in USD | $400 | $600 | **$1000** |
|
| 49 |
-
|
| 50 |
-
### Usage
|
| 51 |
-
You can run this basic cell on [Google Colab](https://colab.research.google.com/). [Listen to samples](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/SAMPLES.md). For more languages and details, see [Advanced Usage](https://github.com/hexgrad/kokoro?tab=readme-ov-file#advanced-usage).
|
| 52 |
-
```py
|
| 53 |
-
!pip install -q kokoro>=0.9.2 soundfile
|
| 54 |
-
!apt-get -qq -y install espeak-ng > /dev/null 2>&1
|
| 55 |
-
from kokoro import KPipeline
|
| 56 |
-
from IPython.display import display, Audio
|
| 57 |
-
import soundfile as sf
|
| 58 |
-
import torch
|
| 59 |
-
pipeline = KPipeline(lang_code='a')
|
| 60 |
-
text = '''
|
| 61 |
-
[Kokoro](/kˈOkəɹO/) is an open-weight TTS model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost-efficient. With Apache-licensed weights, [Kokoro](/kˈOkəɹO/) can be deployed anywhere from production environments to personal projects.
|
| 62 |
-
'''
|
| 63 |
-
generator = pipeline(text, voice='af_heart')
|
| 64 |
-
for i, (gs, ps, audio) in enumerate(generator):
|
| 65 |
-
print(i, gs, ps)
|
| 66 |
-
display(Audio(data=audio, rate=24000, autoplay=i==0))
|
| 67 |
-
sf.write(f'{i}.wav', audio, 24000)
|
| 68 |
-
```
|
| 69 |
-
Under the hood, `kokoro` uses [`misaki`](https://pypi.org/project/misaki/), a G2P library at https://github.com/hexgrad/misaki
|
| 70 |
|
| 71 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
-
**
|
| 74 |
-
- StyleTTS 2: https://arxiv.org/abs/2306.07691
|
| 75 |
-
- ISTFTNet: https://arxiv.org/abs/2203.02395
|
| 76 |
-
- Decoder only: no diffusion, no encoder release
|
| 77 |
|
| 78 |
-
**Architected by:** Li et al @ https://github.com/yl4579/StyleTTS2
|
| 79 |
|
| 80 |
-
**Trained by**: `@rzvzn` on Discord
|
| 81 |
|
| 82 |
-
**Languages:** Multiple
|
| 83 |
|
| 84 |
-
**Model SHA256 Hash:** `496dba118d1a58f5f3db2efc88dbdc216e0483fc89fe6e47ee1f2c53f18ad1e4`
|
| 85 |
|
| 86 |
-
### Training Details
|
| 87 |
|
| 88 |
-
|
| 89 |
-
-
|
| 90 |
-
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
[2] No synthetic audio from open TTS models or "custom voice clones"
|
| 94 |
|
| 95 |
-
|
| 96 |
|
| 97 |
-
**Total Training Cost:** About $1000 for 1000 hours of A100 80GB vRAM
|
| 98 |
|
| 99 |
-
### Creative Commons Attribution
|
| 100 |
|
| 101 |
-
The following CC BY audio was part of the dataset used to train Kokoro v1.0.
|
| 102 |
|
| 103 |
-
| Audio Data | Duration Used | License | Added to Training Set After |
|
| 104 |
-
| ---------- | ------------- | ------- | --------------------------- |
|
| 105 |
-
| [Koniwa](https://github.com/koniwa/koniwa) `tnc` | <1h | [CC BY 3.0](https://creativecommons.org/licenses/by/3.0/deed.ja) | v0.19 / 22 Nov 2024 |
|
| 106 |
-
| [SIWIS](https://datashare.ed.ac.uk/handle/10283/2353) | <11h | [CC BY 4.0](https://datashare.ed.ac.uk/bitstream/handle/10283/2353/license_text) | v0.19 / 22 Nov 2024 |
|
| 107 |
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
-
- 🛠️ [@yl4579](https://huggingface.co/yl4579) for architecting StyleTTS 2.
|
| 111 |
-
- 🏆 [@Pendrokar](https://huggingface.co/Pendrokar) for adding Kokoro as a contender in the TTS Spaces Arena.
|
| 112 |
-
- 📊 Thank you to everyone who contributed synthetic training data.
|
| 113 |
-
- ❤️ Special thanks to all compute sponsors.
|
| 114 |
-
- 👾 Discord server: https://discord.gg/QuGxSWBfQy
|
| 115 |
-
- 🪽 Kokoro is a Japanese word that translates to "heart" or "spirit". It is also the name of an [AI in the Terminator franchise](https://terminator.fandom.com/wiki/Kokoro).
|
| 116 |
|
| 117 |
-
<img src="https://static0.gamerantimages.com/wordpress/wp-content/uploads/2024/08/terminator-zero-41-1.jpg" width="400" alt="kokoro" />
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
+
- zh
|
| 6 |
+
tags:
|
| 7 |
+
- video generation
|
| 8 |
+
- conversational video generation
|
| 9 |
+
- talking human video generation
|
| 10 |
+
pipeline_tag: image-to-video
|
| 11 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
+
<p align="center">
|
| 14 |
+
<img src="assets/logo2.jpeg" alt="MultiTalk" width="300"/>
|
| 15 |
+
</p>
|
| 16 |
+
|
| 17 |
+
# MeiGen-MultiTalk • Audio-Driven Multi-Person Conversational Video Generation
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
<p align="left">
|
| 22 |
+
<a href="https://meigen-ai.github.io/multi-talk/">
|
| 23 |
+
<img
|
| 24 |
+
src="https://img.shields.io/badge/MultiTalk-Website-0A66C2?logo=safari&logoColor=white" style="display: inline-block; vertical-align: middle;"
|
| 25 |
+
alt="MultiTalk Website"
|
| 26 |
+
/>
|
| 27 |
+
</a>
|
| 28 |
+
<a href="https://arxiv.org/abs/2505.22647">
|
| 29 |
+
<img
|
| 30 |
+
src="https://img.shields.io/badge/MultiTalk-Paper-red?logo=arxiv&logoColor=red" style="display: inline-block; vertical-align: middle;"
|
| 31 |
+
alt="MultiTalk Paper on arXiv"
|
| 32 |
+
/>
|
| 33 |
+
</a>
|
| 34 |
+
<a href="https://github.com/MeiGen-AI/MultiTalk" target="_blank" style="margin: 2px;">
|
| 35 |
+
<img
|
| 36 |
+
alt="Github" src="https://img.shields.io/badge/MultiTalk-Codebase-536af5?color=536af5&logo=github" style="display: inline-block; vertical-align: middle;"
|
| 37 |
+
alt="MultiTalk Codebase"
|
| 38 |
+
/>
|
| 39 |
+
</a>
|
| 40 |
+
|
| 41 |
+
</p>
|
| 42 |
+
|
| 43 |
+
> We present **MultiTalk**, an open-source audio-driven multi-person conversational video generation model with the state-of-the-art lip synchronization accuracy.
|
| 44 |
+
> Key features:
|
| 45 |
+
> - 💬 Realistic Conversations - Supports single & multi-person generation
|
| 46 |
+
> - 👥 Interactive Character Control - Direct virtual humans via prompts
|
| 47 |
+
> - 🎤 Generalization Performances - Supports the generation of cartoon character and singing
|
| 48 |
+
> - 📺 Resolution Flexibility: 480p & 720p output at arbitrary aspect ratios
|
| 49 |
+
> - ⏱️ **Long Video Generation**: Support video generation up to 15 seconds
|
| 50 |
+
|
| 51 |
|
| 52 |
+
This repository hosts the model weights for **MultiTalk**. For installation, usage instructions, and further documentation, please visit our [GitHub repository](https://github.com/MeiGen-AI/MultiTalk).
|
|
|
|
|
|
|
|
|
|
| 53 |
|
|
|
|
| 54 |
|
|
|
|
| 55 |
|
|
|
|
| 56 |
|
|
|
|
| 57 |
|
|
|
|
| 58 |
|
| 59 |
+
## Method
|
| 60 |
+
We propose a novel framework, MultiTalk, for audio-driven multi-person conversational video generation. We investigate several schemes for audio injection and introduce
|
| 61 |
+
the Label Rotary Position Embedding (L-RoPE) method. By assigning identical labels to audio embeddings and video latents, it effectively activates specific regions within the audio cross-attention
|
| 62 |
+
map, thereby resolving incorrect binding issues. To localize the region of the specified person, we introduce the adaptive person localization by computing the similarity
|
| 63 |
+
between the features of the given region of a person in the reference image and all the features of the whole video.
|
|
|
|
| 64 |
|
| 65 |
+
<p align="left"><img src="assets/pipe.png" width="80%"></p>
|
| 66 |
|
|
|
|
| 67 |
|
|
|
|
| 68 |
|
|
|
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
+
## Citation
|
| 72 |
+
If you find our work helpful, please cite us.
|
| 73 |
+
|
| 74 |
+
```
|
| 75 |
+
@article{kong2025let,
|
| 76 |
+
title={Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation},
|
| 77 |
+
author={Kong, Zhe and Gao, Feng and Zhang, Yong and Kang, Zhuoliang and Wei, Xiaoming and Cai, Xunliang and Chen, Guanying and Luo, Wenhan},
|
| 78 |
+
journal={arXiv preprint arXiv:2505.22647},
|
| 79 |
+
year={2025}
|
| 80 |
+
}
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
## License Agreement
|
| 86 |
+
The models in this repository are licensed under the Apache 2.0 License. We claim no rights over the your generated contents, granting you the freedom to use them while ensuring that your usage complies with the provisions of this license. You are fully accountable for your use of the models, which must not involve sharing any content that violates applicable laws, causes harm to individuals or groups, disseminates personal information intended for harm, spreads misinformation, or targets vulnerable populations.
|
| 87 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
|
|
assets/logo.png
ADDED
|
Git LFS Details
|
assets/logo2.jpeg
ADDED
|
Git LFS Details
|
assets/pipe.png
ADDED
|
Git LFS Details
|
diffusion_pytorch_model.safetensors.index.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
quant_models/dit_model_map_int8.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"text_embedding.0": {"weights": "qint8", "activations": "none"}, "text_embedding.2": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.0.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.0.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.0.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.1.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.1.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.1.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.2.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.2.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.2.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.3.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.3.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.3.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.4.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.4.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.4.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.5.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.5.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.5.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.6.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.6.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.6.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.7.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.7.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.7.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.8.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.8.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.8.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.9.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.9.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.9.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.10.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.10.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.10.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.11.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.11.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.11.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.12.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.12.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.12.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.13.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.13.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.13.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.14.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.14.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.14.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.15.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.15.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.15.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.16.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.16.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.16.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.17.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.17.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.17.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.18.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.18.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.18.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.19.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.19.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.19.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.20.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.20.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.20.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.21.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.21.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.21.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.22.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.22.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.22.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.23.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.23.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.23.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.24.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.24.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.24.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.24.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.24.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.25.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.25.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.25.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.25.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.25.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.26.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.26.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.26.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.26.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.26.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.27.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.27.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.27.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.27.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.27.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.28.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.28.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.28.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.28.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.28.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.29.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.29.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.29.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.29.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.29.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.30.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.30.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.30.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.30.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.30.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.31.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.31.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.31.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.31.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.31.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.32.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.32.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.32.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.32.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.32.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.33.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.33.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.33.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.33.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.33.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.34.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.34.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.34.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.34.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.34.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.35.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.35.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.35.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.35.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.35.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.36.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.36.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.36.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.36.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.36.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.37.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.37.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.37.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.37.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.37.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.38.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.38.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.38.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.38.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.38.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.39.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.39.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.39.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.39.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.39.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "audio_proj.proj1": {"weights": "qint8", "activations": "none"}, "audio_proj.proj1_vf": {"weights": "qint8", "activations": "none"}, "audio_proj.proj2": {"weights": "qint8", "activations": "none"}, "audio_proj.proj3": {"weights": "qint8", "activations": "none"}}
|
quant_models/quantization_map_fp8_FusionX.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"text_embedding.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "text_embedding.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.24.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.25.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.26.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.27.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.28.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.29.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.30.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.31.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.32.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.33.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.34.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.35.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.36.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.37.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.38.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.self_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.self_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.self_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.self_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.cross_attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.cross_attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.cross_attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.cross_attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.cross_attn.k_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.cross_attn.v_img": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.ffn.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.ffn.2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.audio_cross_attn.q_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.audio_cross_attn.proj": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.39.audio_cross_attn.kv_linear": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "audio_proj.proj1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "audio_proj.proj1_vf": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "audio_proj.proj2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "audio_proj.proj3": {"weights": "qfloat8_e4m3fn", "activations": "none"}}
|
quant_models/quantization_map_int8_FusionX.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"text_embedding.0": {"weights": "qint8", "activations": "none"}, "text_embedding.2": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.0.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.0.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.0.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.0.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.0.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.1.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.1.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.1.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.1.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.1.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.2.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.2.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.2.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.2.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.2.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.3.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.3.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.3.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.3.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.3.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.4.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.4.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.4.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.4.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.4.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.5.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.5.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.5.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.5.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.5.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.6.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.6.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.6.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.6.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.6.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.7.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.7.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.7.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.7.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.7.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.8.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.8.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.8.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.8.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.8.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.9.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.9.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.9.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.9.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.9.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.10.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.10.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.10.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.10.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.10.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.11.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.11.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.11.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.11.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.11.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.12.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.12.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.12.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.12.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.12.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.13.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.13.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.13.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.13.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.13.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.14.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.14.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.14.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.14.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.14.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.15.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.15.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.15.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.15.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.15.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.16.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.16.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.16.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.16.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.16.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.17.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.17.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.17.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.17.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.17.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.18.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.18.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.18.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.18.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.18.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.19.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.19.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.19.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.19.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.19.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.20.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.20.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.20.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.20.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.20.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.21.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.21.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.21.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.21.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.21.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.22.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.22.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.22.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.22.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.22.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.23.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.23.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.23.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.23.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.23.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.24.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.24.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.24.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.24.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.24.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.24.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.24.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.25.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.25.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.25.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.25.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.25.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.25.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.25.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.26.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.26.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.26.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.26.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.26.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.26.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.26.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.27.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.27.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.27.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.27.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.27.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.27.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.27.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.28.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.28.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.28.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.28.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.28.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.28.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.28.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.29.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.29.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.29.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.29.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.29.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.29.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.29.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.30.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.30.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.30.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.30.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.30.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.30.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.30.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.31.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.31.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.31.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.31.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.31.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.31.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.31.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.32.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.32.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.32.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.32.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.32.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.32.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.32.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.33.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.33.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.33.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.33.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.33.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.33.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.33.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.34.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.34.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.34.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.34.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.34.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.34.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.34.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.35.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.35.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.35.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.35.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.35.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.35.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.35.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.36.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.36.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.36.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.36.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.36.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.36.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.36.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.37.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.37.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.37.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.37.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.37.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.37.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.37.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.38.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.38.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.38.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.38.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.38.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.38.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.38.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.39.self_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.q": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.k": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.v": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.o": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.k_img": {"weights": "qint8", "activations": "none"}, "blocks.39.cross_attn.v_img": {"weights": "qint8", "activations": "none"}, "blocks.39.ffn.0": {"weights": "qint8", "activations": "none"}, "blocks.39.ffn.2": {"weights": "qint8", "activations": "none"}, "blocks.39.audio_cross_attn.q_linear": {"weights": "qint8", "activations": "none"}, "blocks.39.audio_cross_attn.proj": {"weights": "qint8", "activations": "none"}, "blocks.39.audio_cross_attn.kv_linear": {"weights": "qint8", "activations": "none"}, "audio_proj.proj1": {"weights": "qint8", "activations": "none"}, "audio_proj.proj1_vf": {"weights": "qint8", "activations": "none"}, "audio_proj.proj2": {"weights": "qint8", "activations": "none"}, "audio_proj.proj3": {"weights": "qint8", "activations": "none"}}
|
quant_models/t5_map_fp8.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"blocks.0.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.0.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.1.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.2.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.3.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.4.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.5.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.6.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.7.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.8.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.9.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.10.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.11.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.12.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.13.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.14.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.15.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.16.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.17.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.18.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.19.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.20.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.21.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.22.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.attn.q": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.attn.k": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.attn.v": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.attn.o": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.ffn.gate.0": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.ffn.fc1": {"weights": "qfloat8_e4m3fn", "activations": "none"}, "blocks.23.ffn.fc2": {"weights": "qfloat8_e4m3fn", "activations": "none"}}
|
quant_models/t5_map_int8.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"blocks.0.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.0.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.0.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.0.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.0.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.1.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.1.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.1.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.1.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.1.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.2.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.2.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.2.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.2.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.2.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.3.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.3.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.3.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.3.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.3.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.4.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.4.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.4.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.4.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.4.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.5.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.5.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.5.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.5.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.5.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.6.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.6.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.6.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.6.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.6.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.7.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.7.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.7.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.7.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.7.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.8.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.8.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.8.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.8.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.8.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.9.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.9.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.9.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.9.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.9.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.10.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.10.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.10.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.10.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.10.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.11.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.11.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.11.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.11.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.11.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.12.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.12.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.12.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.12.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.12.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.13.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.13.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.13.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.13.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.13.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.14.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.14.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.14.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.14.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.14.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.15.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.15.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.15.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.15.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.15.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.16.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.16.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.16.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.16.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.16.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.17.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.17.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.17.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.17.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.17.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.18.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.18.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.18.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.18.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.18.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.19.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.19.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.19.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.19.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.19.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.20.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.20.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.20.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.20.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.20.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.21.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.21.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.21.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.21.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.21.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.22.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.22.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.22.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.22.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.22.ffn.fc2": {"weights": "qint8", "activations": "none"}, "blocks.23.attn.q": {"weights": "qint8", "activations": "none"}, "blocks.23.attn.k": {"weights": "qint8", "activations": "none"}, "blocks.23.attn.v": {"weights": "qint8", "activations": "none"}, "blocks.23.attn.o": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.gate.0": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.fc1": {"weights": "qint8", "activations": "none"}, "blocks.23.ffn.fc2": {"weights": "qint8", "activations": "none"}}
|