Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/benchmarks/design_arena.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmarks/design_arena2.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmarks/ideogram_benchmark.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmarks/lmarena_benchmark.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmarks/opensource.png filter=lfs diff=lfs merge=lfs -text
+assets/benchmarks/opensource2.png filter=lfs diff=lfs merge=lfs -text
+assets/samples/collage_landscape.jpg filter=lfs diff=lfs merge=lfs -text
+assets/comparison_matrix.webp filter=lfs diff=lfs merge=lfs -text

LICENSE.md

@@ -0,0 +1,55 @@
+Ideogram Non-Commercial Model Agreement
+
+Last Updated: June 3, 2026
+
+This Ideogram Non-Commercial Model Agreement (“Agreement”) is a legal agreement between you and Ideogram, Inc. (“Company”, “we” or “us”). This Agreement governs your access to and use of the Models (as defined below) that we make available to you under this Agreement. You expressly agree that if you accept the terms of this Agreement or if you otherwise download, access, use or Distribute any portion of the Model or Model Derivatives, then you agree to be bound by this Agreement. Company may designate certain materials that are designed to be used with the Models (such as related inference code) under a separate license or agreement, and nothing in this Agreement will be deemed to restrict or limit any other rights or licenses granted by Company in such other materials.
+
+If you are entering into this Agreement on behalf of an entity, then “you” means that entity, and you represent and warrant that (a) you are an authorized representative of the entity with the authority to bind the entity to this Agreement and (b) you agree to this Agreement on the entity’s behalf.
+
+1. Definitions.
+
+(a) “Distribution” or “Distribute” means any distribution, making available, transmission, publication or other sharing of the Model or any Model Derivative with or to a third party, including by providing or making the Model or its functionality available as a hosted service via API, web access or any other electronic or remote means (“Hosted Service”), and including by incorporating the Model into products or services that you make available to third parties.
+
+(b) “Model” means the machine learning model, software, algorithms, trained model weights, parameters and documentation that Company designates as subject to this Agreement. For clarity, the following are considered Models subject to this Agreement: Ideogram 4.
+
+(c) “Model Derivative” means all (i) modifications to, and fine-tuned versions of, the Model, (ii) any other machine learning model which is created by transferring the weights and parameters of the Model (or patterns thereof) to such model in a manner that causes it to perform similar to the Model, or (iii) any other derivatives of the Model. For clarity, an Output is not a Model Derivative.
+
+(d) “Non-Commercial Purposes” means activity or use that fits in any of the following categories: (i) use that does not directly or indirectly generate revenue and is not otherwise intended for or directed towards commercial advantage or monetary compensation, (ii) use by a for-profit entity solely for testing, evaluation, or research and development in a “non-production environment” (an environment that is not deployed in live systems, customer-facing applications or any other environment beyond internal development, testing or prototyping), (iii) personal use for research, experimentation, testing purposes as part of a personal study, private entertainment or hobby project, or (iv) use by a charitable organization for charitable purposes. Without limiting the foregoing, any use that involves training, fine tuning, or distilling AI models for commercial use or that involves generating Output to include in, or to advertise or promote, revenue-generating products or services, in each case, is not a Non-Commercial Purpose.
+
+(e) “Output” means any content or other output generated by the inference operation of the Model or any Model Derivative, in response to an input or prompt provided by the user. For the avoidance of doubt, Outputs do not include any components of a Model, such as any fine-tuned versions of the Model, the weights, or parameters.
+
+2. Grant of Rights.
+
+We hereby permit you to use, reproduce, Distribute, copy, create derivative works of (including Model Derivatives), and make modifications to the Model for Non-Commercial Purposes subject to the terms of this Agreement, including any use restrictions set forth below. For clarity, you are only authorized to exercise the rights under this Agreement for Non-Commercial Purposes only, and may not exercise any of the rights under this Agreement for other purposes unless or until Company otherwise expressly grants you such rights in a separate agreement, which Company may grant or not grant in its sole discretion.
+
+3. Redistribution and Use.
+
+You may reproduce or Distribute copies of the Model or Model Derivatives only if you meet all of the following conditions: (i) all permitted use of the reproduced and re-Distributed Model or Model Derivatives must be on terms that are no less restrictive than those set forth in this Agreement for the Model (e.g., including the same standards for Non-Commercial Purposes as set forth in Section 2 above and at least the same use restrictions set forth in Section 4 below), (ii) you provide all third party recipients of the Model or Model Derivative a copy of this Agreement; (iii) you retain in all copies of the Model or Model Derivatives that you Distribute the following attribution notice within a “Notice” text file that accompanies such copy: “Ideogram 4 is provided under and subject to the Ideogram Non-Commercial Model Agreement available at https://github.com/ideogram-oss/ideogram-4/model_licenses/LICENSE-IDEOGRAM-4-NON-COMMERCIAL. All rights reserved. Copyright © Ideogram, Inc.”; (iv) you cause any modified files to carry prominent notices stating that you modified the files; (v) any terms and conditions you impose on third-party recipients must include a disclaimer of warranties and limitation of liability provisions that are at least as protective of Company as those set forth herein; and (vi) you must not misrepresent or imply that any Model Derivative made by you is an official product of Company or has been endorsed, approved or validated by Company.
+
+4. Use Restrictions.
+
+Your use of the Model and any Model Derivative must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy available at https://ideogram.ai/legal/usage-policy, which is hereby incorporated by reference into this Agreement. Without limiting the foregoing, you will not (and will not permit or enable any third party to) use the Model or any Model Derivative: (a) for military purposes or purposes of surveillance, including any research or development relating to surveillance; (b) for biometric processing; (c) in any manner that infringes, misappropriates, or otherwise violates any third party’s legal rights, including rights of publicity; (d) to generate unlawful content, including child sexual abuse material or non-consensual intimate images; (e) in any manner that violates any applicable privacy or data protection laws; or (f) to make automated decisions in domains that affect material or individual rights or well-being (e.g., finance, legal, employment, healthcare, housing, insurance and social welfare) or otherwise in a manner that poses a significant risk of harm to the health, safety or fundamental rights of persons, including to influence any “consequential decision” under applicable law or for any other use case that is categorized as “high risk” under applicable law (“High Risk Use Cases”). You are responsible for implementing appropriate safety measures, including content filters and human oversight, suitable for your use case and to prevent the creation, display, generation or reproduction of unlawful or infringing content, and for including any legally required disclosure (or other provenance measures) to disclose that the Output was generated or modified using artificial intelligence technologies. However, we may also implement certain safety measures, content protections and other technological measures for the Model, including content filters and watermarking, and you agree that you will not circumvent, remove, alter, deactivate, degrade or thwart any such measures. To the extent required under applicable law, you must ensure any Output includes disclosure or other indication that the Output was generated or modified using artificial intelligence technologies.
+
+5. Disclaimer of Warranty.
+
+UNLESS REQUIRED BY APPLICABLE LAW, THE MODEL, ANY MODEL DERIVATIVE AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND WE DISCLAIM ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF, AND ASSUME ALL RISKS ASSOCIATED WITH, USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING THE MODEL, ANY MODEL DERIVATIVE OR ANY OUTPUT. YOU AGREE THAT THE MODEL AND ANY MODEL DERIVATIVE ARE NOT DESIGNED OR INTENDED FOR HIGH RISK USE CASES OR AS PART OF HIGH RISK AI SYSTEMS (AS DEFINED UNDER APPLICABLE LAWS), AND ANY SUCH USE IS IN VIOLATION OF THIS AGREEMENT AND AT YOUR OWN RISK.
+
+6. Limitation of Liability.
+
+IN NO EVENT WILL COMPANY OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR DAMAGES OF ANY KIND, INCLUDING DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES OR LOST PROFITS OF ANY KIND, EVEN IF COMPANY OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+
+7. Intellectual Property.
+
+Company reserves any rights not expressly granted herein. Nothing in this Agreement grants you any rights to use Company’s trademarks, trade names, or logos (except as required for the attribution described above). Subject to Company’s ownership of the Model and Model Derivatives made by or for Company, with respect to any Model Derivatives that are made by you, as between you and Company, you own any incremental rights in such Model Derivatives beyond Company’s rights in the underlying Model or its own Model Derivatives. We claim no rights in outputs you generate using the Model. You are responsible for outputs and their subsequent uses. You may not use any Output to develop, train, fine-tune or distill a model or other product or services that is competitive with the Model or any of Company’s other products or services.
+
+8. Indemnification.
+
+You will indemnify and hold harmless the Company and its affiliates and each of their respective shareholders, directors, officers, employees, agents, successors, and assigns (collectively, the “Company Parties”) from and against any losses, liabilities, damages, fines, penalties, and expenses (including reasonable attorneys’ fees) incurred by any Company Party in connection with any claim, demand, allegation, lawsuit, proceeding, or investigation arising out of or related to your use, access, hosting or Distribution of the Model or Model Derivatives or any Outputs or your breach of this Agreement.
+
+9. Term and Termination.
+
+This Agreement continues in full force and effect until terminated in accordance with the terms and conditions herein. We may terminate this Agreement at any time upon notice to you. We may also terminate this Agreement if you are in breach of any term or condition of this Agreement. If you institute litigation or other proceedings against Company or any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Model, Model Derivative or output, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. Upon termination of this Agreement, you shall delete and cease use or Distribution of the Model and any Model Derivative. Sections 5–10 shall survive the termination of this Agreement.
+
+10. General.
+
+This Agreement will be governed and construed under the laws of the State of New York without regard to conflicts of law provisions. If any provision or part of a provision of this Agreement is unlawful, void or unenforceable, that provision or part of the provision is deemed severed from this Agreement, and will not affect the validity and enforceability of any remaining provisions. The failure of Company to exercise or enforce any right or provision of this Agreement will not operate as a waiver of such right or provision. This Agreement does not confer any third-party beneficiary rights upon any other person or entity. This Agreement contains the entire understanding between you and Company regarding the subject matter of this Agreement, and supersedes all other written or oral agreements and understandings between you and Company regarding such subject matter. You may not assign or transfer this Agreement, including any of your rights or obligations hereunder, without the prior written consent of Company. Any purported assignment not in accordance with this Section will be null and void. We may modify this Agreement from time to time in which case we will update the “Last Updated” date at the top of these Terms. It is your sole responsibility to review this Agreement from time to time to view any such changes. The updated Agreement will be effective as of the time of posting, or such later date as may be specified in the updated Agreement. Your continued access or use of the Model or any Model Derivatives after the modifications have become effective will be deemed your acceptance of the modified Agreement.

README.md

@@ -0,0 +1,119 @@
+---
+license: other
+license_name: ideogram-4-non-commercial
+base_model: ideogram-ai/ideogram-4-fp8
+pipeline_tag: text-to-image
+tags:
+- ideogram
+- text-to-image
+- sdnq
+- uint4
+- diffusion
+- typography
+---
+
+# Ideogram 4 FP8 -> SDNQ UInt4
+
+This is an experimental SDNQ UInt4 conversion of `ideogram-ai/ideogram-4-fp8`. It is intended for local research and non-commercial use under the upstream Ideogram 4 license. The conversion was made from the FP8 checkpoint, materializing FP8 linears back to bf16 and then applying static SDNQ `uint4` component-by-component.
+
+The model includes SDNQ-compressed `text_encoder`, `transformer`, `unconditional_transformer`, and `vae` components. The official `ideogram4` loader does not know how to instantiate SDNQ-packed custom transformers, so this repository includes `ideogram4_sdnq_pipeline.py`.
+
+## Usage
+
+```python
+import torch
+from ideogram4 import PRESETS
+from ideogram4_sdnq_pipeline import Ideogram4SDNQPipeline
+
+pipe = Ideogram4SDNQPipeline.from_pretrained(
+    "WaveCut/ideogram-4-sdnq-uint4",
+    device="cuda",
+    dtype=torch.bfloat16,
+)
+
+preset = PRESETS["V4_DEFAULT_20"]
+image = pipe(
+    "a typographic poster reading HELLO WORLD",
+    height=1024,
+    width=1024,
+    num_steps=preset.num_steps,
+    guidance_schedule=preset.guidance_schedule,
+    mu=preset.mu,
+    std=preset.std,
+    seed=4101,
+    raise_on_caption_issues=False,
+)[0]
+image.save("out.png")
+```
+
+Install requirements:
+
+```bash
+pip install git+https://github.com/ideogram-oss/ideogram4 sdnq safetensors transformers accelerate pillow
+```
+
+## Component Structure
+
+Upstream FP8 structure:
+
+- `text_encoder`: Qwen3-VL text path used in text-only mode. Hidden states from 13 layers are concatenated for the DiT.
+- `transformer`: conditional 34-layer single-stream DiT.
+- `unconditional_transformer`: image-only negative branch used for asymmetric CFG.
+- `vae`: Flux2-style KL autoencoder decoder.
+- `tokenizer` and `scheduler`: copied from upstream.
+
+## Quantization
+
+| Component | Source materialized MB | SDNQ state MB | Quantize s | Quant peak nvidia MB |
+| --- | --- | --- | --- | --- |
+| transformer | 17698.84 | 4979.66 | 112.64 | 36525.00 |
+| unconditional_transformer | 17698.84 | 4979.66 | 108.68 | 36525.00 |
+| text_encoder | 14435.59 | 4097.53 | 102.32 | 24477.00 |
+| vae | 160.31 | 50.19 | 2.68 | 861.00 |
+
+## Benchmark
+
+Hardware: RunPod NVIDIA RTX PRO 6000 Blackwell Server Edition, single process, concurrency 1. Generation used 10 structured JSON prompts at 1024x1024 with `V4_DEFAULT_20`.
+The FP8 baseline was loaded through the upstream `ideogram4` `Ideogram4Pipeline.from_pretrained` recipe with `weights_repo="ideogram-ai/ideogram-4-fp8"`; magic-prompt expansion was disabled because the prompts are already structured captions.
+
+| Variant | Load s | Load peak reserved MB | Load peak nvidia MB | Cold request s | Hot mean s | Gen peak reserved MB | Gen peak nvidia MB |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| original | 267.83 | 28198.00 | 28759.00 | 17.90 | 17.51 | 34430.00 | 35099.00 |
+| sdnq | 239.46 | 14558.00 | 15109.00 | 18.56 | 16.52 | 21650.00 | 22321.00 |
+
+## Example Matrix
+
+The matrix below contains the 10 original FP8 generations followed by the 10 SDNQ UInt4 generations. It is a square WebP at quality 95.
+
+![Comparison matrix](assets/comparison_matrix.webp)
+
+## Prompt Set
+
+| # | id | summary |
+| --- | --- | --- |
+| 1 | `editorial_watch_photo` | A photorealistic editorial product photograph of a transparent mechanical wristwatch resting on a wet black stone slab, with tiny engraved labels visible on the dial. |
+| 2 | `risograph_botanical_poster` | A layered risograph botanical exhibition poster with bold overprint textures and clean typographic hierarchy. |
+| 3 | `cyrillic_cafe_menu` | A cozy Moscow cafe menu board photographed straight-on, testing clean Cyrillic typography in chalk and printed labels. |
+| 4 | `brutalist_architecture` | A cinematic architectural photograph of a brutalist library atrium with tiny wayfinding signs and people for scale. |
+| 5 | `ink_manga_rain` | A dramatic black-and-white manga splash page of a courier cycling through rain, with sound effects and shop signage. |
+| 6 | `museum_clay_render` | A polished 3D clay render of a museum diorama showing a future Arctic research station with labeled miniature modules. |
+| 7 | `food_packaging_label` | A realistic premium chocolate bar packaging mockup with layered foil, embossed typography, and ingredient microcopy. |
+| 8 | `fantasy_map_typography` | A hand-painted fantasy map on parchment with readable place names, compass ornament, and coastal illustrations. |
+| 9 | `streetwear_lookbook` | A fashion lookbook cover photograph for a streetwear collection, with crisp cover typography and realistic fabric textures. |
+| 10 | `scientific_cutaway` | A detailed scientific cutaway illustration of a compact fusion battery prototype with annotated parts and clean technical typography. |
+
+## Files
+
+- `prompts.json`: the 10 structured prompts used for the comparison.
+- `assets/comparison_matrix.webp`: square WebP comparison matrix, quality 95.
+- `benchmark/`: raw benchmark JSONL/CSV files and `summary.json`.
+- `quantization_manifest.json`: component-level quantization timings, storage, and VRAM peaks.
+- `ideogram4_sdnq_pipeline.py`: loader helper for the SDNQ custom transformer components.
+
+## Follow-up
+
+A separate follow-up run will compare this SDNQ UInt4 checkpoint against the official `ideogram-ai/ideogram-4-nf4` checkpoint on an RTX 3090/4090-class pod and append the full-pipeline results here.
+
+## License
+
+This checkpoint is derived from `ideogram-ai/ideogram-4-fp8` and follows the upstream Ideogram 4 non-commercial license. See `LICENSE.md`.

benchmark/original_metrics.csv

@@ -0,0 +1,12 @@
+name,elapsed_seconds,gpu_before_mb,gpu_after_mb,gpu_peak_mb,torch_peak_allocated_mb,torch_peak_reserved_mb,variant,case_id,case_index,seed,height,width,preset,request_temperature,image
+original_load,267.8321040520095,561,28759,28759,26578.111328125,28198.0,original,,,,,,,,
+original_generate,17.904404125991277,27409,35055,35055,30214.517578125,34386.0,original,editorial_watch_photo,0,4101,1024,1024,V4_DEFAULT_20,cold,/workspace/ideogram4_lab/results/original/images/01_editorial_watch_photo_original.png
+original_generate,17.435602533019846,27549,35039,35039,30206.37060546875,34370.0,original,risograph_botanical_poster,1,4102,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/02_risograph_botanical_poster_original.png
+original_generate,17.781690142001025,27549,35099,35099,30225.720703125,34430.0,original,cyrillic_cafe_menu,2,4103,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/03_cyrillic_cafe_menu_original.png
+original_generate,17.494729810015997,27549,35053,35053,30213.90673828125,34384.0,original,brutalist_architecture,3,4104,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/04_brutalist_architecture_original.png
+original_generate,17.477317612007027,27549,35039,35039,30206.37060546875,34370.0,original,ink_manga_rain,4,4105,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/05_ink_manga_rain_original.png
+original_generate,17.449652091017924,27549,35041,35041,30209.64306640625,34372.0,original,museum_clay_render,5,4106,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/06_museum_clay_render_original.png
+original_generate,17.48920578099205,27549,35041,35041,30207.7958984375,34372.0,original,food_packaging_label,6,4107,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/07_food_packaging_label_original.png
+original_generate,17.515618185978383,27549,35069,35069,30217.66455078125,34400.0,original,fantasy_map_typography,7,4108,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/08_fantasy_map_typography_original.png
+original_generate,17.44910855300259,27549,35033,35033,30204.537109375,34364.0,original,streetwear_lookbook,8,4109,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/09_streetwear_lookbook_original.png
+original_generate,17.518164148001233,27549,35055,35055,30214.31396484375,34386.0,original,scientific_cutaway,9,4110,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/original/images/10_scientific_cutaway_original.png

benchmark/original_metrics.jsonl

@@ -0,0 +1,11 @@
+{"name": "original_load", "elapsed_seconds": 267.8321040520095, "gpu_before_mb": 561, "gpu_after_mb": 28759, "gpu_peak_mb": 28759, "torch_peak_allocated_mb": 26578.111328125, "torch_peak_reserved_mb": 28198.0, "variant": "original"}
+{"name": "original_generate", "elapsed_seconds": 17.904404125991277, "gpu_before_mb": 27409, "gpu_after_mb": 35055, "gpu_peak_mb": 35055, "torch_peak_allocated_mb": 30214.517578125, "torch_peak_reserved_mb": 34386.0, "variant": "original", "case_id": "editorial_watch_photo", "case_index": 0, "seed": 4101, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "cold", "image": "/workspace/ideogram4_lab/results/original/images/01_editorial_watch_photo_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.435602533019846, "gpu_before_mb": 27549, "gpu_after_mb": 35039, "gpu_peak_mb": 35039, "torch_peak_allocated_mb": 30206.37060546875, "torch_peak_reserved_mb": 34370.0, "variant": "original", "case_id": "risograph_botanical_poster", "case_index": 1, "seed": 4102, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/02_risograph_botanical_poster_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.781690142001025, "gpu_before_mb": 27549, "gpu_after_mb": 35099, "gpu_peak_mb": 35099, "torch_peak_allocated_mb": 30225.720703125, "torch_peak_reserved_mb": 34430.0, "variant": "original", "case_id": "cyrillic_cafe_menu", "case_index": 2, "seed": 4103, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/03_cyrillic_cafe_menu_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.494729810015997, "gpu_before_mb": 27549, "gpu_after_mb": 35053, "gpu_peak_mb": 35053, "torch_peak_allocated_mb": 30213.90673828125, "torch_peak_reserved_mb": 34384.0, "variant": "original", "case_id": "brutalist_architecture", "case_index": 3, "seed": 4104, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/04_brutalist_architecture_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.477317612007027, "gpu_before_mb": 27549, "gpu_after_mb": 35039, "gpu_peak_mb": 35039, "torch_peak_allocated_mb": 30206.37060546875, "torch_peak_reserved_mb": 34370.0, "variant": "original", "case_id": "ink_manga_rain", "case_index": 4, "seed": 4105, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/05_ink_manga_rain_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.449652091017924, "gpu_before_mb": 27549, "gpu_after_mb": 35041, "gpu_peak_mb": 35041, "torch_peak_allocated_mb": 30209.64306640625, "torch_peak_reserved_mb": 34372.0, "variant": "original", "case_id": "museum_clay_render", "case_index": 5, "seed": 4106, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/06_museum_clay_render_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.48920578099205, "gpu_before_mb": 27549, "gpu_after_mb": 35041, "gpu_peak_mb": 35041, "torch_peak_allocated_mb": 30207.7958984375, "torch_peak_reserved_mb": 34372.0, "variant": "original", "case_id": "food_packaging_label", "case_index": 6, "seed": 4107, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/07_food_packaging_label_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.515618185978383, "gpu_before_mb": 27549, "gpu_after_mb": 35069, "gpu_peak_mb": 35069, "torch_peak_allocated_mb": 30217.66455078125, "torch_peak_reserved_mb": 34400.0, "variant": "original", "case_id": "fantasy_map_typography", "case_index": 7, "seed": 4108, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/08_fantasy_map_typography_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.44910855300259, "gpu_before_mb": 27549, "gpu_after_mb": 35033, "gpu_peak_mb": 35033, "torch_peak_allocated_mb": 30204.537109375, "torch_peak_reserved_mb": 34364.0, "variant": "original", "case_id": "streetwear_lookbook", "case_index": 8, "seed": 4109, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/09_streetwear_lookbook_original.png"}
+{"name": "original_generate", "elapsed_seconds": 17.518164148001233, "gpu_before_mb": 27549, "gpu_after_mb": 35055, "gpu_peak_mb": 35055, "torch_peak_allocated_mb": 30214.31396484375, "torch_peak_reserved_mb": 34386.0, "variant": "original", "case_id": "scientific_cutaway", "case_index": 9, "seed": 4110, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/original/images/10_scientific_cutaway_original.png"}

benchmark/sdnq_metrics.csv

@@ -0,0 +1,12 @@
+name,elapsed_seconds,gpu_before_mb,gpu_after_mb,gpu_peak_mb,torch_peak_allocated_mb,torch_peak_reserved_mb,variant,case_id,case_index,seed,height,width,preset,request_temperature,image
+sdnq_load,239.45547024699044,561,15119,15109,14377.78173828125,14558.0,sdnq,,,,,,,,
+sdnq_generate,18.559326965012588,15119,22259,22259,18335.75439453125,21588.0,sdnq,editorial_watch_photo,0,4101,1024,1024,V4_DEFAULT_20,cold,/workspace/ideogram4_lab/results/sdnq/images/01_editorial_watch_photo_sdnq.png
+sdnq_generate,16.76757296800497,15357,22273,22273,18326.81640625,21602.0,sdnq,risograph_botanical_poster,1,4102,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/02_risograph_botanical_poster_sdnq.png
+sdnq_generate,16.798510612017708,15357,22321,22321,18346.47216796875,21650.0,sdnq,cyrillic_cafe_menu,2,4103,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/03_cyrillic_cafe_menu_sdnq.png
+sdnq_generate,16.49672631698195,15357,22289,22289,18335.4169921875,21618.0,sdnq,brutalist_architecture,3,4104,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/04_brutalist_architecture_sdnq.png
+sdnq_generate,15.973647239006823,15357,22245,22245,18326.81640625,21574.0,sdnq,ink_manga_rain,4,4105,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/05_ink_manga_rain_sdnq.png
+sdnq_generate,16.52093323500594,15357,22275,22275,18331.3173828125,21604.0,sdnq,museum_clay_render,5,4106,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/06_museum_clay_render_sdnq.png
+sdnq_generate,16.54349116497906,15357,22275,22275,18328.56787109375,21604.0,sdnq,food_packaging_label,6,4107,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/07_food_packaging_label_sdnq.png
+sdnq_generate,16.577632450003875,15357,22295,22295,18339.3388671875,21624.0,sdnq,fantasy_map_typography,7,4108,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/08_fantasy_map_typography_sdnq.png
+sdnq_generate,16.463748395995935,15357,22267,22267,18325.27783203125,21596.0,sdnq,streetwear_lookbook,8,4109,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/09_streetwear_lookbook_sdnq.png
+sdnq_generate,16.568265846988652,15357,22289,22289,18335.5888671875,21618.0,sdnq,scientific_cutaway,9,4110,1024,1024,V4_DEFAULT_20,hot,/workspace/ideogram4_lab/results/sdnq/images/10_scientific_cutaway_sdnq.png

benchmark/sdnq_metrics.jsonl

@@ -0,0 +1,11 @@
+{"name": "sdnq_load", "elapsed_seconds": 239.45547024699044, "gpu_before_mb": 561, "gpu_after_mb": 15119, "gpu_peak_mb": 15109, "torch_peak_allocated_mb": 14377.78173828125, "torch_peak_reserved_mb": 14558.0, "variant": "sdnq"}
+{"name": "sdnq_generate", "elapsed_seconds": 18.559326965012588, "gpu_before_mb": 15119, "gpu_after_mb": 22259, "gpu_peak_mb": 22259, "torch_peak_allocated_mb": 18335.75439453125, "torch_peak_reserved_mb": 21588.0, "variant": "sdnq", "case_id": "editorial_watch_photo", "case_index": 0, "seed": 4101, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "cold", "image": "/workspace/ideogram4_lab/results/sdnq/images/01_editorial_watch_photo_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.76757296800497, "gpu_before_mb": 15357, "gpu_after_mb": 22273, "gpu_peak_mb": 22273, "torch_peak_allocated_mb": 18326.81640625, "torch_peak_reserved_mb": 21602.0, "variant": "sdnq", "case_id": "risograph_botanical_poster", "case_index": 1, "seed": 4102, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/02_risograph_botanical_poster_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.798510612017708, "gpu_before_mb": 15357, "gpu_after_mb": 22321, "gpu_peak_mb": 22321, "torch_peak_allocated_mb": 18346.47216796875, "torch_peak_reserved_mb": 21650.0, "variant": "sdnq", "case_id": "cyrillic_cafe_menu", "case_index": 2, "seed": 4103, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/03_cyrillic_cafe_menu_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.49672631698195, "gpu_before_mb": 15357, "gpu_after_mb": 22289, "gpu_peak_mb": 22289, "torch_peak_allocated_mb": 18335.4169921875, "torch_peak_reserved_mb": 21618.0, "variant": "sdnq", "case_id": "brutalist_architecture", "case_index": 3, "seed": 4104, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/04_brutalist_architecture_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 15.973647239006823, "gpu_before_mb": 15357, "gpu_after_mb": 22245, "gpu_peak_mb": 22245, "torch_peak_allocated_mb": 18326.81640625, "torch_peak_reserved_mb": 21574.0, "variant": "sdnq", "case_id": "ink_manga_rain", "case_index": 4, "seed": 4105, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/05_ink_manga_rain_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.52093323500594, "gpu_before_mb": 15357, "gpu_after_mb": 22275, "gpu_peak_mb": 22275, "torch_peak_allocated_mb": 18331.3173828125, "torch_peak_reserved_mb": 21604.0, "variant": "sdnq", "case_id": "museum_clay_render", "case_index": 5, "seed": 4106, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/06_museum_clay_render_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.54349116497906, "gpu_before_mb": 15357, "gpu_after_mb": 22275, "gpu_peak_mb": 22275, "torch_peak_allocated_mb": 18328.56787109375, "torch_peak_reserved_mb": 21604.0, "variant": "sdnq", "case_id": "food_packaging_label", "case_index": 6, "seed": 4107, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/07_food_packaging_label_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.577632450003875, "gpu_before_mb": 15357, "gpu_after_mb": 22295, "gpu_peak_mb": 22295, "torch_peak_allocated_mb": 18339.3388671875, "torch_peak_reserved_mb": 21624.0, "variant": "sdnq", "case_id": "fantasy_map_typography", "case_index": 7, "seed": 4108, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/08_fantasy_map_typography_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.463748395995935, "gpu_before_mb": 15357, "gpu_after_mb": 22267, "gpu_peak_mb": 22267, "torch_peak_allocated_mb": 18325.27783203125, "torch_peak_reserved_mb": 21596.0, "variant": "sdnq", "case_id": "streetwear_lookbook", "case_index": 8, "seed": 4109, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/09_streetwear_lookbook_sdnq.png"}
+{"name": "sdnq_generate", "elapsed_seconds": 16.568265846988652, "gpu_before_mb": 15357, "gpu_after_mb": 22289, "gpu_peak_mb": 22289, "torch_peak_allocated_mb": 18335.5888671875, "torch_peak_reserved_mb": 21618.0, "variant": "sdnq", "case_id": "scientific_cutaway", "case_index": 9, "seed": 4110, "height": 1024, "width": 1024, "preset": "V4_DEFAULT_20", "request_temperature": "hot", "image": "/workspace/ideogram4_lab/results/sdnq/images/10_scientific_cutaway_sdnq.png"}

benchmark/summary.json

@@ -0,0 +1,30 @@
+[
+  {
+    "variant": "original",
+    "load_seconds": 267.8321040520095,
+    "load_peak_reserved_mb": 28198.0,
+    "load_peak_nvidia_mb": 28759,
+    "cold_request_seconds": 17.904404125991277,
+    "cold_request_peak_reserved_mb": 34386.0,
+    "cold_request_peak_nvidia_mb": 35055,
+    "hot_request_mean_seconds": 17.51234320622623,
+    "hot_request_max_seconds": 17.781690142001025,
+    "generation_peak_reserved_mb": 34430.0,
+    "generation_peak_nvidia_mb": 35099.0,
+    "generation_gpu_after_max_mb": 35099.0
+  },
+  {
+    "variant": "sdnq",
+    "load_seconds": 239.45547024699044,
+    "load_peak_reserved_mb": 14558.0,
+    "load_peak_nvidia_mb": 15109,
+    "cold_request_seconds": 18.559326965012588,
+    "cold_request_peak_reserved_mb": 21588.0,
+    "cold_request_peak_nvidia_mb": 22259,
+    "hot_request_mean_seconds": 16.523392025442767,
+    "hot_request_max_seconds": 16.798510612017708,
+    "generation_peak_reserved_mb": 21650.0,
+    "generation_peak_nvidia_mb": 22321.0,
+    "generation_gpu_after_max_mb": 22321.0
+  }
+]

ideogram4_sdnq_pipeline.py

@@ -0,0 +1,250 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Optional
+
+import torch
+from accelerate import init_empty_weights
+from huggingface_hub import snapshot_download
+from safetensors.torch import load_file
+from transformers import AutoConfig, AutoTokenizer, Qwen3VLTextModel
+
+from ideogram4.autoencoder import AutoEncoder, AutoEncoderParams
+from ideogram4.modeling_ideogram4 import Ideogram4Config, Ideogram4Transformer
+from ideogram4.pipeline_ideogram4 import Ideogram4Pipeline, Ideogram4PipelineConfig
+from sdnq import sdnq_post_load_quant
+from sdnq.loader import apply_sdnq_options_to_model, load_files, post_process_model
+from sdnq.utils import get_quant_args_from_config
+
+
+def _resolve_model_dir(model_id_or_path: str, revision: str | None = None) -> Path:
+  path = Path(model_id_or_path)
+  if path.exists():
+    return path
+  return Path(
+    snapshot_download(
+      model_id_or_path,
+      revision=revision,
+      allow_patterns=[
+        "model_index.json",
+        "scheduler/*",
+        "tokenizer/*",
+        "text_encoder/*",
+        "transformer/*",
+        "unconditional_transformer/*",
+        "vae/*",
+      ],
+    )
+  )
+
+
+def _read_json(path: Path) -> dict:
+  with path.open("r", encoding="utf-8") as f:
+    return json.load(f)
+
+
+def _ideogram_config_from_diffusers_config(config_path: Path) -> Ideogram4Config:
+  cfg = _read_json(config_path)
+  num_heads = int(cfg.get("num_attention_heads", cfg.get("num_heads", 18)))
+  head_dim = int(cfg.get("attention_head_dim", 256))
+  return Ideogram4Config(
+    emb_dim=num_heads * head_dim,
+    num_layers=int(cfg.get("num_layers", 34)),
+    num_heads=num_heads,
+    intermediate_size=int(cfg.get("intermediate_size", 12288)),
+    adanln_dim=int(cfg.get("adaln_dim", 512)),
+    in_channels=int(cfg.get("in_channels", 128)),
+    llm_features_dim=int(cfg.get("llm_features_dim", 53248)),
+    rope_theta=int(cfg.get("rope_theta", 5_000_000)),
+    mrope_section=tuple(cfg.get("mrope_section", [24, 20, 20])),
+    norm_eps=float(cfg.get("norm_eps", 1e-5)),
+  )
+
+
+def _load_single_or_indexed_safetensors(folder: Path, basename: str) -> dict[str, torch.Tensor]:
+  index_path = folder / f"{basename}.safetensors.index.json"
+  if not index_path.exists():
+    return load_file(folder / f"{basename}.safetensors")
+  index = _read_json(index_path)
+  state: dict[str, torch.Tensor] = {}
+  for shard_name in sorted(set(index["weight_map"].values())):
+    state.update(load_file(folder / shard_name))
+  return state
+
+
+def _set_compute_dtype(model: torch.nn.Module, dtype: torch.dtype) -> torch.nn.Module:
+  for module in model.modules():
+    if hasattr(module, "sdnq_dequantizer"):
+      module.compute_dtype = dtype
+  return model
+
+
+def _load_sdnq_transformer(
+  folder: Path,
+  device: torch.device,
+  dtype: torch.dtype,
+  use_quantized_matmul: bool,
+  dequantize_fp32: bool,
+) -> Ideogram4Transformer:
+  config = _ideogram_config_from_diffusers_config(folder / "config.json")
+  quant_config = _read_json(folder / "quantization_config.json")
+  model = Ideogram4Transformer(config)
+  model = sdnq_post_load_quant(
+    model,
+    torch_dtype=dtype,
+    add_skip_keys=False,
+    use_dynamic_quantization=False,
+    **get_quant_args_from_config(quant_config),
+  )
+  state = _load_single_or_indexed_safetensors(folder, "diffusion_pytorch_model")
+  model.load_state_dict(state, assign=True)
+  del state
+  model = apply_sdnq_options_to_model(
+    model,
+    dtype=dtype,
+    dequantize_fp32=dequantize_fp32,
+    use_quantized_matmul=use_quantized_matmul,
+  )
+  _set_compute_dtype(model, dtype)
+  model.to(device)
+  model.eval()
+  return model
+
+
+def _load_sdnq_autoencoder(
+  folder: Path,
+  device: torch.device,
+  dtype: torch.dtype,
+  use_quantized_matmul: bool,
+  dequantize_fp32: bool,
+) -> AutoEncoder:
+  quant_config = _read_json(folder / "quantization_config.json")
+  model = AutoEncoder(AutoEncoderParams())
+  model = sdnq_post_load_quant(
+    model,
+    torch_dtype=dtype,
+    add_skip_keys=False,
+    use_dynamic_quantization=False,
+    **get_quant_args_from_config(quant_config),
+  )
+  state = _load_single_or_indexed_safetensors(folder, "diffusion_pytorch_model")
+  model.load_state_dict(state, assign=True)
+  del state
+  model = apply_sdnq_options_to_model(
+    model,
+    dtype=dtype,
+    dequantize_fp32=dequantize_fp32,
+    use_quantized_matmul=use_quantized_matmul,
+  )
+  _set_compute_dtype(model, dtype)
+  model.to(device)
+  model.eval()
+  return model
+
+
+def _load_text_encoder(
+  folder: Path,
+  device: torch.device,
+  dtype: torch.dtype,
+  use_quantized_matmul: bool,
+  dequantize_fp32: bool,
+):
+  quant_config = _read_json(folder / "quantization_config.json")
+  with init_empty_weights():
+    config = AutoConfig.from_pretrained(folder)
+    language_model = Qwen3VLTextModel(config)
+    language_model = sdnq_post_load_quant(
+      language_model,
+      torch_dtype=dtype,
+      add_skip_keys=False,
+      use_dynamic_quantization=False,
+      **get_quant_args_from_config(quant_config),
+    )
+  files = sorted(str(path) for path in folder.glob("*.safetensors"))
+  state = load_files(
+    files,
+    key_mapping=getattr(language_model, "_checkpoint_conversion_mapping", None),
+    device=device,
+    method="safetensors",
+  )
+  language_model.load_state_dict(state, assign=True)
+  del state
+  language_model = post_process_model(language_model)
+  language_model = apply_sdnq_options_to_model(
+    language_model,
+    dtype=dtype,
+    dequantize_fp32=dequantize_fp32,
+    use_quantized_matmul=use_quantized_matmul,
+  )
+  _set_compute_dtype(language_model, dtype)
+  language_model.eval()
+  return SimpleNamespace(language_model=language_model)
+
+
+class Ideogram4SDNQPipeline(Ideogram4Pipeline):
+  @classmethod
+  def from_pretrained(
+    cls,
+    model_id_or_path: str,
+    *,
+    revision: Optional[str] = None,
+    device: str | torch.device = "cuda",
+    dtype: torch.dtype = torch.bfloat16,
+    use_quantized_matmul: bool = False,
+    dequantize_fp32: bool = False,
+  ) -> "Ideogram4SDNQPipeline":
+    root = _resolve_model_dir(model_id_or_path, revision=revision)
+    device = torch.device(device)
+
+    config = Ideogram4PipelineConfig(weights_repo=str(root))
+    tokenizer = AutoTokenizer.from_pretrained(root / "tokenizer")
+    text_encoder = _load_text_encoder(
+      root / "text_encoder",
+      device,
+      dtype,
+      use_quantized_matmul=use_quantized_matmul,
+      dequantize_fp32=dequantize_fp32,
+    )
+    conditional_transformer = _load_sdnq_transformer(
+      root / "transformer",
+      device,
+      dtype,
+      use_quantized_matmul=use_quantized_matmul,
+      dequantize_fp32=dequantize_fp32,
+    )
+    unconditional_transformer = _load_sdnq_transformer(
+      root / "unconditional_transformer",
+      device,
+      dtype,
+      use_quantized_matmul=use_quantized_matmul,
+      dequantize_fp32=dequantize_fp32,
+    )
+
+    vae_dir = root / "vae"
+    if (vae_dir / "quantization_config.json").exists():
+      autoencoder = _load_sdnq_autoencoder(
+        vae_dir,
+        device,
+        dtype,
+        use_quantized_matmul=use_quantized_matmul,
+        dequantize_fp32=dequantize_fp32,
+      )
+    else:
+      from ideogram4.pipeline_ideogram4 import _load_autoencoder
+
+      autoencoder = _load_autoencoder(
+        str(vae_dir / "diffusion_pytorch_model.safetensors"), device, dtype
+      )
+
+    return cls(
+      conditional_transformer=conditional_transformer,
+      unconditional_transformer=unconditional_transformer,
+      text_encoder=text_encoder,
+      text_tokenizer=tokenizer,
+      autoencoder=autoencoder,
+      config=config,
+      device=device,
+      dtype=dtype,
+    )

model_index.json

@@ -0,0 +1,34 @@
+{
+  "_class_name": "Ideogram4SDNQPipeline",
+  "_diffusers_version": "0.39.0.dev0",
+  "_name_or_path": "ideogram-ai/debug-ideogram-v4",
+  "scheduler": [
+    "diffusers",
+    "FlowMatchEulerDiscreteScheduler"
+  ],
+  "text_encoder": [
+    "transformers",
+    "Qwen3VLTextModel"
+  ],
+  "tokenizer": [
+    "transformers",
+    "Qwen2Tokenizer"
+  ],
+  "transformer": [
+    "sdnq",
+    "Ideogram4Transformer"
+  ],
+  "unconditional_transformer": [
+    "sdnq",
+    "Ideogram4Transformer"
+  ],
+  "vae": [
+    "diffusers",
+    "AutoencoderKLFlux2"
+  ],
+  "quantization": {
+    "method": "SDNQ",
+    "weights_dtype": "uint4",
+    "source_repo": "ideogram-ai/ideogram-4-fp8"
+  }
+}

prompts.json

@@ -0,0 +1,246 @@
+[
+  {
+    "id": "editorial_watch_photo",
+    "seed": 4101,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A photorealistic editorial product photograph of a transparent mechanical wristwatch resting on a wet black stone slab, with tiny engraved labels visible on the dial.",
+      "style_description": {
+        "aesthetics": "luxury, precise, tactile, high contrast, realistic micro-detail",
+        "lighting": "large softbox from upper left, thin rim light, controlled glossy reflections",
+        "photo": "100mm macro lens, f/8, focus stacked, Hasselblad medium-format look",
+        "medium": "photograph",
+        "color_palette": ["#070707", "#D9D1BD", "#9CC7CF", "#E6E6E6", "#2B2B2B"]
+      },
+      "compositional_deconstruction": {
+        "background": "A dark studio setup with a wet black stone slab, fine water droplets, and soft reflections fading into a neutral charcoal backdrop.",
+        "elements": [
+          {"type": "obj", "bbox": [170, 170, 800, 820], "desc": "A transparent mechanical wristwatch with visible gears, brushed titanium case, sapphire crystal, realistic refractions, and shallow scratches on the metal."},
+          {"type": "text", "bbox": [395, 400, 475, 605], "text": "OPEN WORKS", "desc": "Tiny engraved uppercase dial text following the curve of the watch face, crisp but naturally printed on metal."},
+          {"type": "text", "bbox": [720, 240, 780, 760], "text": "CALIBRE 04", "desc": "Small technical label etched into the wet stone reflection, subtle and readable."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "risograph_botanical_poster",
+    "seed": 4102,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A layered risograph botanical exhibition poster with bold overprint textures and clean typographic hierarchy.",
+      "style_description": {
+        "aesthetics": "graphic, tactile, imperfect registration, gallery poster, bold negative space",
+        "lighting": "flat scanned-paper lighting with visible ink grain",
+        "medium": "graphic_design",
+        "art_style": "two-color risograph print, rough halftone grain, offset ink overlap",
+        "color_palette": ["#0E3B2E", "#F05A28", "#F3E9CF", "#1F1F1F"]
+      },
+      "compositional_deconstruction": {
+        "background": "Warm cream paper with visible fibers, subtle scanner shadow at the edges, and a dark green border.",
+        "elements": [
+          {"type": "obj", "bbox": [210, 120, 760, 870], "desc": "A dense arrangement of stylized fern leaves and seed pods printed in dark green with orange overprint shadows."},
+          {"type": "text", "bbox": [70, 95, 185, 905], "text": "NIGHT GARDEN", "desc": "Large condensed uppercase title across the top, dark green ink with slight orange misregistration."},
+          {"type": "text", "bbox": [820, 180, 900, 820], "text": "BOTANICAL STUDIES 2026", "desc": "Small footer caption in neat monospaced type, aligned to the bottom margin."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "cyrillic_cafe_menu",
+    "seed": 4103,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A cozy Moscow cafe menu board photographed straight-on, testing clean Cyrillic typography in chalk and printed labels.",
+      "style_description": {
+        "aesthetics": "warm, realistic, inviting, handcrafted, legible Cyrillic signage",
+        "lighting": "soft morning window light, gentle tungsten fill, mild chalkboard glare",
+        "photo": "35mm documentary photograph, straight-on composition, natural lens distortion",
+        "medium": "photograph",
+        "color_palette": ["#2B2118", "#F4E6C8", "#C47A3C", "#FFFFFF", "#5F7A61"]
+      },
+      "compositional_deconstruction": {
+        "background": "A small cafe wall with dark wood shelves, ceramic cups, a chalkboard menu, and a few paper pastry labels pinned below.",
+        "elements": [
+          {"type": "text", "bbox": [105, 130, 205, 870], "text": "УТРО В ГОРОДЕ", "desc": "Large Cyrillic chalk title, all caps, centered on the board, readable and slightly imperfect."},
+          {"type": "text", "bbox": [280, 170, 610, 840], "text": "кофе 220\nсырники 360\nкаша 290", "desc": "Three-line Cyrillic menu in white chalk, aligned left, with prices clearly separated."},
+          {"type": "text", "bbox": [735, 210, 815, 800], "text": "сегодня: вишнёвый пирог", "desc": "Small handwritten Cyrillic paper label under the board, readable but naturally casual."},
+          {"type": "obj", "bbox": [650, 90, 930, 930], "desc": "Wooden counter edge with a cappuccino, a linen napkin, and a slice of cherry pie on a ceramic plate."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "brutalist_architecture",
+    "seed": 4104,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A cinematic architectural photograph of a brutalist library atrium with tiny wayfinding signs and people for scale.",
+      "style_description": {
+        "aesthetics": "monumental, quiet, realistic, geometric, concrete texture",
+        "lighting": "late afternoon sun shafts through skylights, cool shadows, warm highlights",
+        "photo": "24mm tilt-shift architectural lens, high dynamic range, crisp verticals",
+        "medium": "photograph",
+        "color_palette": ["#B8B0A1", "#3D4142", "#E0C06A", "#6E7C87", "#1C1C1C"]
+      },
+      "compositional_deconstruction": {
+        "background": "A vast concrete atrium with suspended walkways, angular skylights, book stacks, and polished stone floors.",
+        "elements": [
+          {"type": "obj", "bbox": [80, 80, 910, 930], "desc": "Layered brutalist balconies and concrete ribs forming a deep central perspective."},
+          {"type": "text", "bbox": [390, 150, 455, 315], "text": "READING HALL", "desc": "Small black wayfinding sign with clean white uppercase type mounted on concrete."},
+          {"type": "text", "bbox": [600, 700, 650, 875], "text": "LEVEL 04", "desc": "Tiny yellow floor label on a distant balcony, legible but integrated."},
+          {"type": "obj", "bbox": [665, 400, 850, 620], "desc": "Three small visitors walking across the floor, casting long shadows."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "ink_manga_rain",
+    "seed": 4105,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A dramatic black-and-white manga splash page of a courier cycling through rain, with sound effects and shop signage.",
+      "style_description": {
+        "aesthetics": "kinetic, high contrast, rain-soaked, expressive, detailed linework",
+        "lighting": "night street lighting rendered as stark white highlights and deep ink shadows",
+        "medium": "illustration",
+        "art_style": "manga ink drawing, screentone gradients, speed lines, hand-lettered effects",
+        "color_palette": ["#FFFFFF", "#111111", "#777777", "#D7D7D7"]
+      },
+      "compositional_deconstruction": {
+        "background": "A dense urban alley in heavy rain, storefront awnings, puddles, electrical wires, and diagonal speed lines.",
+        "elements": [
+          {"type": "obj", "bbox": [220, 190, 875, 780], "desc": "A raincoat-wearing bicycle courier leaning hard into a turn, wheels spraying water, dynamic foreshortening."},
+          {"type": "text", "bbox": [90, 90, 220, 350], "text": "WHOOSH", "desc": "Large hand-lettered sound effect integrated into the rain and speed lines."},
+          {"type": "text", "bbox": [160, 650, 245, 910], "text": "OPEN 24H", "desc": "Small glowing shop sign in block letters, partly reflected in puddles."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "museum_clay_render",
+    "seed": 4106,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A polished 3D clay render of a museum diorama showing a future Arctic research station with labeled miniature modules.",
+      "style_description": {
+        "aesthetics": "clean, miniature, educational, soft, premium exhibition design",
+        "lighting": "large overhead museum softbox, ambient occlusion, gentle contact shadows",
+        "medium": "3d_render",
+        "art_style": "matte clay render, isometric diorama, subtle bevels, toy-like scale",
+        "color_palette": ["#F0F0E8", "#B9D7E8", "#F2B84B", "#394B59", "#FFFFFF"]
+      },
+      "compositional_deconstruction": {
+        "background": "A square museum plinth with snowy terrain, ice ridges, and a glass display cover implied by faint reflections.",
+        "elements": [
+          {"type": "obj", "bbox": [230, 210, 760, 820], "desc": "A modular Arctic research station with rounded white pods, solar panels, antennae, and tiny tracked vehicles."},
+          {"type": "text", "bbox": [120, 145, 185, 470], "text": "ARCTIC NODE", "desc": "Museum label title printed on the front edge of the plinth, dark gray sans-serif."},
+          {"type": "text", "bbox": [785, 560, 850, 845], "text": "SOLAR ARRAY", "desc": "Tiny technical callout label with a thin pointer line toward the solar panels."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "food_packaging_label",
+    "seed": 4107,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A realistic premium chocolate bar packaging mockup with layered foil, embossed typography, and ingredient microcopy.",
+      "style_description": {
+        "aesthetics": "premium, appetizing, tactile, elegant, realistic packaging",
+        "lighting": "warm studio strip lights creating controlled foil highlights",
+        "photo": "70mm product photography, three-quarter angle, crisp shadows, commercial retouching",
+        "medium": "photograph",
+        "color_palette": ["#4A1F16", "#D7A94B", "#F7EFE2", "#1E1A18", "#8C3A2B"]
+      },
+      "compositional_deconstruction": {
+        "background": "A warm stone tabletop with cocoa powder dust, roasted hazelnuts, and a folded piece of gold foil.",
+        "elements": [
+          {"type": "obj", "bbox": [210, 160, 820, 835], "desc": "A dark chocolate bar wrapper, partly opened, with embossed gold foil and visible chocolate squares."},
+          {"type": "text", "bbox": [300, 255, 470, 760], "text": "NOIR 72%", "desc": "Large embossed serif product name in metallic gold on the wrapper."},
+          {"type": "text", "bbox": [530, 310, 620, 720], "text": "hazelnut • sea salt • cacao nib", "desc": "Small ingredient line in cream ink, readable and aligned under the title."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "fantasy_map_typography",
+    "seed": 4108,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A hand-painted fantasy map on parchment with readable place names, compass ornament, and coastal illustrations.",
+      "style_description": {
+        "aesthetics": "adventurous, ornate, aged, cartographic, storybook",
+        "lighting": "flat archival scan with slight parchment waviness and warm edge darkening",
+        "medium": "painting",
+        "art_style": "watercolor and ink fantasy cartography, fine calligraphy, engraved coastline marks",
+        "color_palette": ["#E9D7A6", "#4E6B4C", "#2F4B6B", "#8A5A2B", "#1F1712"]
+      },
+      "compositional_deconstruction": {
+        "background": "Aged parchment with hand-drawn mountains, forests, rivers, dotted travel routes, and inked coastlines.",
+        "elements": [
+          {"type": "text", "bbox": [220, 315, 305, 650], "text": "Elder Coast", "desc": "Elegant map calligraphy following the curve of a bay, readable and inked in dark brown."},
+          {"type": "text", "bbox": [530, 560, 610, 790], "text": "Moon Gate", "desc": "Small calligraphic city label beside a walled port symbol."},
+          {"type": "text", "bbox": [760, 100, 850, 330], "text": "North", "desc": "Compass rose label beside an ornate compass star."},
+          {"type": "obj", "bbox": [300, 130, 780, 870], "desc": "Mountains, forests, a serpentine river, a ship illustration, and dotted trade route marks."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "streetwear_lookbook",
+    "seed": 4109,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A fashion lookbook cover photograph for a streetwear collection, with crisp cover typography and realistic fabric textures.",
+      "style_description": {
+        "aesthetics": "urban, editorial, confident, modern, magazine cover",
+        "lighting": "overcast daylight with soft fill, wet pavement reflections",
+        "photo": "50mm fashion editorial lens, full-body portrait, muted city colors",
+        "medium": "photograph",
+        "color_palette": ["#101820", "#EDEBE3", "#BFC7C9", "#7A1E2C", "#4C5B61"]
+      },
+      "compositional_deconstruction": {
+        "background": "A quiet city side street after rain, concrete walls, steel shutters, and reflective asphalt.",
+        "elements": [
+          {"type": "obj", "bbox": [180, 330, 890, 710], "desc": "A model wearing an oversized cream technical jacket, black cargo trousers, red knit cap, and layered accessories."},
+          {"type": "text", "bbox": [80, 80, 190, 930], "text": "URBAN WEATHER", "desc": "Large magazine cover title in clean condensed uppercase letters across the top."},
+          {"type": "text", "bbox": [835, 120, 900, 880], "text": "LOOKBOOK 04", "desc": "Small footer text spaced widely, aligned along the bottom edge."}
+        ]
+      }
+    }
+  },
+  {
+    "id": "scientific_cutaway",
+    "seed": 4110,
+    "height": 1024,
+    "width": 1024,
+    "caption": {
+      "high_level_description": "A detailed scientific cutaway illustration of a compact fusion battery prototype with annotated parts and clean technical typography.",
+      "style_description": {
+        "aesthetics": "precise, futuristic, educational, clean, high-detail engineering",
+        "lighting": "neutral white studio lighting with subtle blue glow from the device core",
+        "medium": "illustration",
+        "art_style": "technical cutaway illustration, vector-like linework with soft 3D shading",
+        "color_palette": ["#F7F8FA", "#1E2A33", "#2FA7C9", "#F0C74A", "#707A83"]
+      },
+      "compositional_deconstruction": {
+        "background": "A clean off-white technical poster background with faint grid lines and small registration marks.",
+        "elements": [
+          {"type": "obj", "bbox": [210, 210, 780, 780], "desc": "A cylindrical compact fusion battery shown in cutaway, with glowing blue core, layered shielding, coolant channels, and brass connector rings."},
+          {"type": "text", "bbox": [90, 115, 175, 600], "text": "COMPACT FUSION CELL", "desc": "Main technical title in crisp uppercase sans-serif type."},
+          {"type": "text", "bbox": [330, 705, 390, 940], "text": "coolant loop", "desc": "Small annotation label with a thin leader line pointing to blue channels."},
+          {"type": "text", "bbox": [610, 80, 675, 330], "text": "field coil", "desc": "Small annotation label with a leader line pointing to the copper coil."}
+        ]
+      }
+    }
+  }
+]

quantization_manifest.json

@@ -0,0 +1,73 @@
+{
+  "source_repo": "ideogram-ai/ideogram-4-fp8",
+  "target_repo": "WaveCut/ideogram-4-sdnq-uint4",
+  "quantization": {
+    "weights_dtype": "uint4",
+    "group_size": 0,
+    "use_svd": false,
+    "use_dynamic_quantization": false,
+    "use_stochastic_rounding": false,
+    "dequantize_fp32": false,
+    "add_skip_keys": false
+  },
+  "components": {
+    "transformer": {
+      "file": "model/transformer/diffusion_pytorch_model.safetensors",
+      "storage_mb": 4979.658447265625,
+      "num_state_tensors": 880,
+      "source_materialized_storage_mb": 17698.838134765625,
+      "name": "quantize_transformer",
+      "elapsed_seconds": 112.6355704489979,
+      "gpu_before_mb": 561,
+      "gpu_after_mb": 649,
+      "gpu_peak_mb": 36525,
+      "torch_peak_allocated_mb": 0.0,
+      "torch_peak_reserved_mb": 0.0,
+      "component": "transformer"
+    },
+    "unconditional_transformer": {
+      "file": "model/unconditional_transformer/diffusion_pytorch_model.safetensors",
+      "storage_mb": 4979.658447265625,
+      "num_state_tensors": 880,
+      "source_materialized_storage_mb": 17698.838134765625,
+      "name": "quantize_unconditional_transformer",
+      "elapsed_seconds": 108.67946223300532,
+      "gpu_before_mb": 649,
+      "gpu_after_mb": 649,
+      "gpu_peak_mb": 36525,
+      "torch_peak_allocated_mb": 0.0,
+      "torch_peak_reserved_mb": 0.0,
+      "component": "unconditional_transformer"
+    },
+    "text_encoder": {
+      "storage_mb": 4097.525390625,
+      "source_materialized_storage_mb": 14435.587890625,
+      "num_state_tensors": 904,
+      "name": "quantize_text_encoder",
+      "elapsed_seconds": 102.32456034698407,
+      "gpu_before_mb": 649,
+      "gpu_after_mb": 649,
+      "gpu_peak_mb": 24477,
+      "torch_peak_allocated_mb": 0.0,
+      "torch_peak_reserved_mb": 0.0,
+      "component": "text_encoder"
+    },
+    "vae": {
+      "file": "model/vae/diffusion_pytorch_model.safetensors",
+      "storage_mb": 50.18652153015137,
+      "num_state_tensors": 395,
+      "source_materialized_storage_mb": 160.30573844909668,
+      "name": "quantize_vae",
+      "elapsed_seconds": 2.675335832987912,
+      "gpu_before_mb": 649,
+      "gpu_after_mb": 649,
+      "gpu_peak_mb": 861,
+      "torch_peak_allocated_mb": 0.0,
+      "torch_peak_reserved_mb": 0.0,
+      "component": "vae"
+    }
+  },
+  "scaffolding": {
+    "snapshot": "/root/.cache/huggingface/hub/models--ideogram-ai--ideogram-4-fp8/snapshots/ee79a7237b519f1402ceacf952f30c8a31ec5073"
+  }
+}

scheduler/scheduler_config.json

@@ -0,0 +1,18 @@
+{
+  "_class_name": "FlowMatchEulerDiscreteScheduler",
+  "_diffusers_version": "0.39.0.dev0",
+  "base_image_seq_len": 256,
+  "base_shift": 0.5,
+  "invert_sigmas": false,
+  "max_image_seq_len": 4096,
+  "max_shift": 1.15,
+  "num_train_timesteps": 1000,
+  "shift": 1.0,
+  "shift_terminal": null,
+  "stochastic_sampling": false,
+  "time_shift_type": "exponential",
+  "use_beta_sigmas": false,
+  "use_dynamic_shifting": false,
+  "use_exponential_sigmas": false,
+  "use_karras_sigmas": false
+}

text_encoder/config.json

@@ -0,0 +1,213 @@
+{
+  "architectures": [
+    "Qwen3VLTextModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "max_position_embeddings": 262144,
+  "model_type": "qwen3_vl_text",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 8,
+  "pad_token_id": null,
+  "quantization_config": {
+    "add_skip_keys": false,
+    "dequantize_fp32": false,
+    "dynamic_loss_threshold": null,
+    "group_size": 0,
+    "hadamard_group_size": 128,
+    "is_integer": true,
+    "is_training": false,
+    "modules_dtype_dict": {},
+    "modules_quant_config": {},
+    "modules_to_not_convert": [
+      "layers.0.self_attn.q_norm.weight",
+      "layers.0.self_attn.k_norm.weight",
+      "layers.0.input_layernorm.weight",
+      "layers.0.post_attention_layernorm.weight",
+      "layers.1.self_attn.q_norm.weight",
+      "layers.1.self_attn.k_norm.weight",
+      "layers.1.input_layernorm.weight",
+      "layers.1.post_attention_layernorm.weight",
+      "layers.2.self_attn.q_norm.weight",
+      "layers.2.self_attn.k_norm.weight",
+      "layers.2.input_layernorm.weight",
+      "layers.2.post_attention_layernorm.weight",
+      "layers.3.self_attn.q_norm.weight",
+      "layers.3.self_attn.k_norm.weight",
+      "layers.3.input_layernorm.weight",
+      "layers.3.post_attention_layernorm.weight",
+      "layers.4.self_attn.q_norm.weight",
+      "layers.4.self_attn.k_norm.weight",
+      "layers.4.input_layernorm.weight",
+      "layers.4.post_attention_layernorm.weight",
+      "layers.5.self_attn.q_norm.weight",
+      "layers.5.self_attn.k_norm.weight",
+      "layers.5.input_layernorm.weight",
+      "layers.5.post_attention_layernorm.weight",
+      "layers.6.self_attn.q_norm.weight",
+      "layers.6.self_attn.k_norm.weight",
+      "layers.6.input_layernorm.weight",
+      "layers.6.post_attention_layernorm.weight",
+      "layers.7.self_attn.q_norm.weight",
+      "layers.7.self_attn.k_norm.weight",
+      "layers.7.input_layernorm.weight",
+      "layers.7.post_attention_layernorm.weight",
+      "layers.8.self_attn.q_norm.weight",
+      "layers.8.self_attn.k_norm.weight",
+      "layers.8.input_layernorm.weight",
+      "layers.8.post_attention_layernorm.weight",
+      "layers.9.self_attn.q_norm.weight",
+      "layers.9.self_attn.k_norm.weight",
+      "layers.9.input_layernorm.weight",
+      "layers.9.post_attention_layernorm.weight",
+      "layers.10.self_attn.q_norm.weight",
+      "layers.10.self_attn.k_norm.weight",
+      "layers.10.input_layernorm.weight",
+      "layers.10.post_attention_layernorm.weight",
+      "layers.11.self_attn.q_norm.weight",
+      "layers.11.self_attn.k_norm.weight",
+      "layers.11.input_layernorm.weight",
+      "layers.11.post_attention_layernorm.weight",
+      "layers.12.self_attn.q_norm.weight",
+      "layers.12.self_attn.k_norm.weight",
+      "layers.12.input_layernorm.weight",
+      "layers.12.post_attention_layernorm.weight",
+      "layers.13.self_attn.q_norm.weight",
+      "layers.13.self_attn.k_norm.weight",
+      "layers.13.input_layernorm.weight",
+      "layers.13.post_attention_layernorm.weight",
+      "layers.14.self_attn.q_norm.weight",
+      "layers.14.self_attn.k_norm.weight",
+      "layers.14.input_layernorm.weight",
+      "layers.14.post_attention_layernorm.weight",
+      "layers.15.self_attn.q_norm.weight",
+      "layers.15.self_attn.k_norm.weight",
+      "layers.15.input_layernorm.weight",
+      "layers.15.post_attention_layernorm.weight",
+      "layers.16.self_attn.q_norm.weight",
+      "layers.16.self_attn.k_norm.weight",
+      "layers.16.input_layernorm.weight",
+      "layers.16.post_attention_layernorm.weight",
+      "layers.17.self_attn.q_norm.weight",
+      "layers.17.self_attn.k_norm.weight",
+      "layers.17.input_layernorm.weight",
+      "layers.17.post_attention_layernorm.weight",
+      "layers.18.self_attn.q_norm.weight",
+      "layers.18.self_attn.k_norm.weight",
+      "layers.18.input_layernorm.weight",
+      "layers.18.post_attention_layernorm.weight",
+      "layers.19.self_attn.q_norm.weight",
+      "layers.19.self_attn.k_norm.weight",
+      "layers.19.input_layernorm.weight",
+      "layers.19.post_attention_layernorm.weight",
+      "layers.20.self_attn.q_norm.weight",
+      "layers.20.self_attn.k_norm.weight",
+      "layers.20.input_layernorm.weight",
+      "layers.20.post_attention_layernorm.weight",
+      "layers.21.self_attn.q_norm.weight",
+      "layers.21.self_attn.k_norm.weight",
+      "layers.21.input_layernorm.weight",
+      "layers.21.post_attention_layernorm.weight",
+      "layers.22.self_attn.q_norm.weight",
+      "layers.22.self_attn.k_norm.weight",
+      "layers.22.input_layernorm.weight",
+      "layers.22.post_attention_layernorm.weight",
+      "layers.23.self_attn.q_norm.weight",
+      "layers.23.self_attn.k_norm.weight",
+      "layers.23.input_layernorm.weight",
+      "layers.23.post_attention_layernorm.weight",
+      "layers.24.self_attn.q_norm.weight",
+      "layers.24.self_attn.k_norm.weight",
+      "layers.24.input_layernorm.weight",
+      "layers.24.post_attention_layernorm.weight",
+      "layers.25.self_attn.q_norm.weight",
+      "layers.25.self_attn.k_norm.weight",
+      "layers.25.input_layernorm.weight",
+      "layers.25.post_attention_layernorm.weight",
+      "layers.26.self_attn.q_norm.weight",
+      "layers.26.self_attn.k_norm.weight",
+      "layers.26.input_layernorm.weight",
+      "layers.26.post_attention_layernorm.weight",
+      "layers.27.self_attn.q_norm.weight",
+      "layers.27.self_attn.k_norm.weight",
+      "layers.27.input_layernorm.weight",
+      "layers.27.post_attention_layernorm.weight",
+      "layers.28.self_attn.q_norm.weight",
+      "layers.28.self_attn.k_norm.weight",
+      "layers.28.input_layernorm.weight",
+      "layers.28.post_attention_layernorm.weight",
+      "layers.29.self_attn.q_norm.weight",
+      "layers.29.self_attn.k_norm.weight",
+      "layers.29.input_layernorm.weight",
+      "layers.29.post_attention_layernorm.weight",
+      "layers.30.self_attn.q_norm.weight",
+      "layers.30.self_attn.k_norm.weight",
+      "layers.30.input_layernorm.weight",
+      "layers.30.post_attention_layernorm.weight",
+      "layers.31.self_attn.q_norm.weight",
+      "layers.31.self_attn.k_norm.weight",
+      "layers.31.input_layernorm.weight",
+      "layers.31.post_attention_layernorm.weight",
+      "layers.32.self_attn.q_norm.weight",
+      "layers.32.self_attn.k_norm.weight",
+      "layers.32.input_layernorm.weight",
+      "layers.32.post_attention_layernorm.weight",
+      "layers.33.self_attn.q_norm.weight",
+      "layers.33.self_attn.k_norm.weight",
+      "layers.33.input_layernorm.weight",
+      "layers.33.post_attention_layernorm.weight",
+      "layers.34.self_attn.q_norm.weight",
+      "layers.34.self_attn.k_norm.weight",
+      "layers.34.input_layernorm.weight",
+      "layers.34.post_attention_layernorm.weight",
+      "layers.35.self_attn.q_norm.weight",
+      "layers.35.self_attn.k_norm.weight",
+      "layers.35.input_layernorm.weight",
+      "layers.35.post_attention_layernorm.weight",
+      "norm.weight"
+    ],
+    "modules_to_not_use_matmul": [],
+    "non_blocking": false,
+    "quant_conv": false,
+    "quant_embedding": true,
+    "quant_method": "sdnq",
+    "quantization_device": null,
+    "quantized_matmul_dtype": null,
+    "return_device": null,
+    "sdnq_version": "0.1.9",
+    "svd_rank": 32,
+    "svd_steps": 8,
+    "use_dynamic_quantization": false,
+    "use_grad_ckpt": true,
+    "use_hadamard": false,
+    "use_quantized_matmul": false,
+    "use_quantized_matmul_conv": false,
+    "use_static_quantization": true,
+    "use_stochastic_rounding": false,
+    "use_svd": false,
+    "weights_dtype": "uint4"
+  },
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "mrope_interleaved": true,
+    "mrope_section": [
+      24,
+      20,
+      20
+    ],
+    "rope_theta": 5000000,
+    "rope_type": "default"
+  },
+  "transformers_version": "5.10.1",
+  "use_cache": true,
+  "vocab_size": 151936
+}

text_encoder/model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd5552a390cd4e6f8e3c70e3270fd1372b87409832c4741d62128e96b1a5f2f6
+size 3994609744

text_encoder/model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e80201da79c9e870ae78f71cebdabb85b28ee9ed97b619370b17e75ba2d09904
+size 302054848

text_encoder/model.safetensors.index.json

@@ -0,0 +1,912 @@
+{
+  "metadata": {
+    "total_parameters": 4040307712,
+    "total_size": 4296566784
+  },
+  "weight_map": {
+    "embed_tokens.scale": "model-00001-of-00002.safetensors",
+    "embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "embed_tokens.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.23.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.24.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.25.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.26.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.27.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.28.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.29.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.30.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.31.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.32.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.up_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.up_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.33.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.up_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.up_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.k_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.k_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.q_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.q_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.v_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.v_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.up_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.up_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.k_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.k_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.q_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.q_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.v_proj.scale": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.v_proj.zero_point": "model-00002-of-00002.safetensors",
+    "layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.up_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.up_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.k_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.k_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.q_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.q_proj.zero_point": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.v_proj.scale": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.v_proj.zero_point": "model-00001-of-00002.safetensors",
+    "norm.weight": "model-00002-of-00002.safetensors"
+  }
+}

text_encoder/quantization_config.json

@@ -0,0 +1,178 @@
+{
+  "add_skip_keys": false,
+  "dequantize_fp32": false,
+  "dynamic_loss_threshold": null,
+  "group_size": 0,
+  "hadamard_group_size": 128,
+  "is_integer": true,
+  "is_training": false,
+  "modules_dtype_dict": {},
+  "modules_quant_config": {},
+  "modules_to_not_convert": [
+    "layers.0.self_attn.q_norm.weight",
+    "layers.0.self_attn.k_norm.weight",
+    "layers.0.input_layernorm.weight",
+    "layers.0.post_attention_layernorm.weight",
+    "layers.1.self_attn.q_norm.weight",
+    "layers.1.self_attn.k_norm.weight",
+    "layers.1.input_layernorm.weight",
+    "layers.1.post_attention_layernorm.weight",
+    "layers.2.self_attn.q_norm.weight",
+    "layers.2.self_attn.k_norm.weight",
+    "layers.2.input_layernorm.weight",
+    "layers.2.post_attention_layernorm.weight",
+    "layers.3.self_attn.q_norm.weight",
+    "layers.3.self_attn.k_norm.weight",
+    "layers.3.input_layernorm.weight",
+    "layers.3.post_attention_layernorm.weight",
+    "layers.4.self_attn.q_norm.weight",
+    "layers.4.self_attn.k_norm.weight",
+    "layers.4.input_layernorm.weight",
+    "layers.4.post_attention_layernorm.weight",
+    "layers.5.self_attn.q_norm.weight",
+    "layers.5.self_attn.k_norm.weight",
+    "layers.5.input_layernorm.weight",
+    "layers.5.post_attention_layernorm.weight",
+    "layers.6.self_attn.q_norm.weight",
+    "layers.6.self_attn.k_norm.weight",
+    "layers.6.input_layernorm.weight",
+    "layers.6.post_attention_layernorm.weight",
+    "layers.7.self_attn.q_norm.weight",
+    "layers.7.self_attn.k_norm.weight",
+    "layers.7.input_layernorm.weight",
+    "layers.7.post_attention_layernorm.weight",
+    "layers.8.self_attn.q_norm.weight",
+    "layers.8.self_attn.k_norm.weight",
+    "layers.8.input_layernorm.weight",
+    "layers.8.post_attention_layernorm.weight",
+    "layers.9.self_attn.q_norm.weight",
+    "layers.9.self_attn.k_norm.weight",
+    "layers.9.input_layernorm.weight",
+    "layers.9.post_attention_layernorm.weight",
+    "layers.10.self_attn.q_norm.weight",
+    "layers.10.self_attn.k_norm.weight",
+    "layers.10.input_layernorm.weight",
+    "layers.10.post_attention_layernorm.weight",
+    "layers.11.self_attn.q_norm.weight",
+    "layers.11.self_attn.k_norm.weight",
+    "layers.11.input_layernorm.weight",
+    "layers.11.post_attention_layernorm.weight",
+    "layers.12.self_attn.q_norm.weight",
+    "layers.12.self_attn.k_norm.weight",
+    "layers.12.input_layernorm.weight",
+    "layers.12.post_attention_layernorm.weight",
+    "layers.13.self_attn.q_norm.weight",
+    "layers.13.self_attn.k_norm.weight",
+    "layers.13.input_layernorm.weight",
+    "layers.13.post_attention_layernorm.weight",
+    "layers.14.self_attn.q_norm.weight",
+    "layers.14.self_attn.k_norm.weight",
+    "layers.14.input_layernorm.weight",
+    "layers.14.post_attention_layernorm.weight",
+    "layers.15.self_attn.q_norm.weight",
+    "layers.15.self_attn.k_norm.weight",
+    "layers.15.input_layernorm.weight",
+    "layers.15.post_attention_layernorm.weight",
+    "layers.16.self_attn.q_norm.weight",
+    "layers.16.self_attn.k_norm.weight",
+    "layers.16.input_layernorm.weight",
+    "layers.16.post_attention_layernorm.weight",
+    "layers.17.self_attn.q_norm.weight",
+    "layers.17.self_attn.k_norm.weight",
+    "layers.17.input_layernorm.weight",
+    "layers.17.post_attention_layernorm.weight",
+    "layers.18.self_attn.q_norm.weight",
+    "layers.18.self_attn.k_norm.weight",
+    "layers.18.input_layernorm.weight",
+    "layers.18.post_attention_layernorm.weight",
+    "layers.19.self_attn.q_norm.weight",
+    "layers.19.self_attn.k_norm.weight",
+    "layers.19.input_layernorm.weight",
+    "layers.19.post_attention_layernorm.weight",
+    "layers.20.self_attn.q_norm.weight",
+    "layers.20.self_attn.k_norm.weight",
+    "layers.20.input_layernorm.weight",
+    "layers.20.post_attention_layernorm.weight",
+    "layers.21.self_attn.q_norm.weight",
+    "layers.21.self_attn.k_norm.weight",
+    "layers.21.input_layernorm.weight",
+    "layers.21.post_attention_layernorm.weight",
+    "layers.22.self_attn.q_norm.weight",
+    "layers.22.self_attn.k_norm.weight",
+    "layers.22.input_layernorm.weight",
+    "layers.22.post_attention_layernorm.weight",
+    "layers.23.self_attn.q_norm.weight",
+    "layers.23.self_attn.k_norm.weight",
+    "layers.23.input_layernorm.weight",
+    "layers.23.post_attention_layernorm.weight",
+    "layers.24.self_attn.q_norm.weight",
+    "layers.24.self_attn.k_norm.weight",
+    "layers.24.input_layernorm.weight",
+    "layers.24.post_attention_layernorm.weight",
+    "layers.25.self_attn.q_norm.weight",
+    "layers.25.self_attn.k_norm.weight",
+    "layers.25.input_layernorm.weight",
+    "layers.25.post_attention_layernorm.weight",
+    "layers.26.self_attn.q_norm.weight",
+    "layers.26.self_attn.k_norm.weight",
+    "layers.26.input_layernorm.weight",
+    "layers.26.post_attention_layernorm.weight",
+    "layers.27.self_attn.q_norm.weight",
+    "layers.27.self_attn.k_norm.weight",
+    "layers.27.input_layernorm.weight",
+    "layers.27.post_attention_layernorm.weight",
+    "layers.28.self_attn.q_norm.weight",
+    "layers.28.self_attn.k_norm.weight",
+    "layers.28.input_layernorm.weight",
+    "layers.28.post_attention_layernorm.weight",
+    "layers.29.self_attn.q_norm.weight",
+    "layers.29.self_attn.k_norm.weight",
+    "layers.29.input_layernorm.weight",
+    "layers.29.post_attention_layernorm.weight",
+    "layers.30.self_attn.q_norm.weight",
+    "layers.30.self_attn.k_norm.weight",
+    "layers.30.input_layernorm.weight",
+    "layers.30.post_attention_layernorm.weight",
+    "layers.31.self_attn.q_norm.weight",
+    "layers.31.self_attn.k_norm.weight",
+    "layers.31.input_layernorm.weight",
+    "layers.31.post_attention_layernorm.weight",
+    "layers.32.self_attn.q_norm.weight",
+    "layers.32.self_attn.k_norm.weight",
+    "layers.32.input_layernorm.weight",
+    "layers.32.post_attention_layernorm.weight",
+    "layers.33.self_attn.q_norm.weight",
+    "layers.33.self_attn.k_norm.weight",
+    "layers.33.input_layernorm.weight",
+    "layers.33.post_attention_layernorm.weight",
+    "layers.34.self_attn.q_norm.weight",
+    "layers.34.self_attn.k_norm.weight",
+    "layers.34.input_layernorm.weight",
+    "layers.34.post_attention_layernorm.weight",
+    "layers.35.self_attn.q_norm.weight",
+    "layers.35.self_attn.k_norm.weight",
+    "layers.35.input_layernorm.weight",
+    "layers.35.post_attention_layernorm.weight",
+    "norm.weight"
+  ],
+  "modules_to_not_use_matmul": [],
+  "non_blocking": false,
+  "quant_conv": false,
+  "quant_embedding": true,
+  "quant_method": "sdnq",
+  "quantization_device": null,
+  "quantized_matmul_dtype": null,
+  "return_device": null,
+  "sdnq_version": "0.1.9",
+  "svd_rank": 32,
+  "svd_steps": 8,
+  "use_dynamic_quantization": false,
+  "use_grad_ckpt": true,
+  "use_hadamard": false,
+  "use_quantized_matmul": false,
+  "use_quantized_matmul_conv": false,
+  "use_static_quantization": true,
+  "use_stochastic_rounding": false,
+  "use_svd": false,
+  "weights_dtype": "uint4"
+}

tokenizer/chat_template.jinja

@@ -0,0 +1,120 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' }}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- for message in messages %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content_item in message.content %}
+                {%- if 'text' in content_item %}
+                    {{- content_item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and message.content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

tokenizer/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

transformer/config.json

@@ -0,0 +1,19 @@
+{
+  "_class_name": "Ideogram4Transformer2DModel",
+  "_diffusers_version": "0.39.0.dev0",
+  "_name_or_path": "/home/jinli/.cache/huggingface/hub/models--ideogram-ai--debug-ideogram-v4/snapshots/41af6183c9fd9b6254864b0720319ef984535bfc/transformer",
+  "adaln_dim": 512,
+  "attention_head_dim": 256,
+  "in_channels": 128,
+  "intermediate_size": 12288,
+  "llm_features_dim": 53248,
+  "mrope_section": [
+    24,
+    20,
+    20
+  ],
+  "norm_eps": 1e-05,
+  "num_attention_heads": 18,
+  "num_layers": 34,
+  "rope_theta": 5000000
+}

transformer/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c43f8ea4d035a29eb8bd6fe3bd6a9bbe3c6f31e16fe5a0cba02bd89134307cb
+size 5221643880

transformer/quantization_config.json

@@ -0,0 +1,239 @@
+{
+  "add_skip_keys": false,
+  "dequantize_fp32": false,
+  "dynamic_loss_threshold": null,
+  "group_size": 0,
+  "hadamard_group_size": 128,
+  "is_integer": true,
+  "is_training": false,
+  "modules_dtype_dict": {},
+  "modules_quant_config": {},
+  "modules_to_not_convert": [
+    "llm_cond_norm.weight",
+    "embed_image_indicator.weight",
+    "layers.0.attention.norm_q.weight",
+    "layers.0.attention.norm_k.weight",
+    "layers.0.attention_norm1.weight",
+    "layers.0.ffn_norm1.weight",
+    "layers.0.attention_norm2.weight",
+    "layers.0.ffn_norm2.weight",
+    "layers.1.attention.norm_q.weight",
+    "layers.1.attention.norm_k.weight",
+    "layers.1.attention_norm1.weight",
+    "layers.1.ffn_norm1.weight",
+    "layers.1.attention_norm2.weight",
+    "layers.1.ffn_norm2.weight",
+    "layers.2.attention.norm_q.weight",
+    "layers.2.attention.norm_k.weight",
+    "layers.2.attention_norm1.weight",
+    "layers.2.ffn_norm1.weight",
+    "layers.2.attention_norm2.weight",
+    "layers.2.ffn_norm2.weight",
+    "layers.3.attention.norm_q.weight",
+    "layers.3.attention.norm_k.weight",
+    "layers.3.attention_norm1.weight",
+    "layers.3.ffn_norm1.weight",
+    "layers.3.attention_norm2.weight",
+    "layers.3.ffn_norm2.weight",
+    "layers.4.attention.norm_q.weight",
+    "layers.4.attention.norm_k.weight",
+    "layers.4.attention_norm1.weight",
+    "layers.4.ffn_norm1.weight",
+    "layers.4.attention_norm2.weight",
+    "layers.4.ffn_norm2.weight",
+    "layers.5.attention.norm_q.weight",
+    "layers.5.attention.norm_k.weight",
+    "layers.5.attention_norm1.weight",
+    "layers.5.ffn_norm1.weight",
+    "layers.5.attention_norm2.weight",
+    "layers.5.ffn_norm2.weight",
+    "layers.6.attention.norm_q.weight",
+    "layers.6.attention.norm_k.weight",
+    "layers.6.attention_norm1.weight",
+    "layers.6.ffn_norm1.weight",
+    "layers.6.attention_norm2.weight",
+    "layers.6.ffn_norm2.weight",
+    "layers.7.attention.norm_q.weight",
+    "layers.7.attention.norm_k.weight",
+    "layers.7.attention_norm1.weight",
+    "layers.7.ffn_norm1.weight",
+    "layers.7.attention_norm2.weight",
+    "layers.7.ffn_norm2.weight",
+    "layers.8.attention.norm_q.weight",
+    "layers.8.attention.norm_k.weight",
+    "layers.8.attention_norm1.weight",
+    "layers.8.ffn_norm1.weight",
+    "layers.8.attention_norm2.weight",
+    "layers.8.ffn_norm2.weight",
+    "layers.9.attention.norm_q.weight",
+    "layers.9.attention.norm_k.weight",
+    "layers.9.attention_norm1.weight",
+    "layers.9.ffn_norm1.weight",
+    "layers.9.attention_norm2.weight",
+    "layers.9.ffn_norm2.weight",
+    "layers.10.attention.norm_q.weight",
+    "layers.10.attention.norm_k.weight",
+    "layers.10.attention_norm1.weight",
+    "layers.10.ffn_norm1.weight",
+    "layers.10.attention_norm2.weight",
+    "layers.10.ffn_norm2.weight",
+    "layers.11.attention.norm_q.weight",
+    "layers.11.attention.norm_k.weight",
+    "layers.11.attention_norm1.weight",
+    "layers.11.ffn_norm1.weight",
+    "layers.11.attention_norm2.weight",
+    "layers.11.ffn_norm2.weight",
+    "layers.12.attention.norm_q.weight",
+    "layers.12.attention.norm_k.weight",
+    "layers.12.attention_norm1.weight",
+    "layers.12.ffn_norm1.weight",
+    "layers.12.attention_norm2.weight",
+    "layers.12.ffn_norm2.weight",
+    "layers.13.attention.norm_q.weight",
+    "layers.13.attention.norm_k.weight",
+    "layers.13.attention_norm1.weight",
+    "layers.13.ffn_norm1.weight",
+    "layers.13.attention_norm2.weight",
+    "layers.13.ffn_norm2.weight",
+    "layers.14.attention.norm_q.weight",
+    "layers.14.attention.norm_k.weight",
+    "layers.14.attention_norm1.weight",
+    "layers.14.ffn_norm1.weight",
+    "layers.14.attention_norm2.weight",
+    "layers.14.ffn_norm2.weight",
+    "layers.15.attention.norm_q.weight",
+    "layers.15.attention.norm_k.weight",
+    "layers.15.attention_norm1.weight",
+    "layers.15.ffn_norm1.weight",
+    "layers.15.attention_norm2.weight",
+    "layers.15.ffn_norm2.weight",
+    "layers.16.attention.norm_q.weight",
+    "layers.16.attention.norm_k.weight",
+    "layers.16.attention_norm1.weight",
+    "layers.16.ffn_norm1.weight",
+    "layers.16.attention_norm2.weight",
+    "layers.16.ffn_norm2.weight",
+    "layers.17.attention.norm_q.weight",
+    "layers.17.attention.norm_k.weight",
+    "layers.17.attention_norm1.weight",
+    "layers.17.ffn_norm1.weight",
+    "layers.17.attention_norm2.weight",
+    "layers.17.ffn_norm2.weight",
+    "layers.18.attention.norm_q.weight",
+    "layers.18.attention.norm_k.weight",
+    "layers.18.attention_norm1.weight",
+    "layers.18.ffn_norm1.weight",
+    "layers.18.attention_norm2.weight",
+    "layers.18.ffn_norm2.weight",
+    "layers.19.attention.norm_q.weight",
+    "layers.19.attention.norm_k.weight",
+    "layers.19.attention_norm1.weight",
+    "layers.19.ffn_norm1.weight",
+    "layers.19.attention_norm2.weight",
+    "layers.19.ffn_norm2.weight",
+    "layers.20.attention.norm_q.weight",
+    "layers.20.attention.norm_k.weight",
+    "layers.20.attention_norm1.weight",
+    "layers.20.ffn_norm1.weight",
+    "layers.20.attention_norm2.weight",
+    "layers.20.ffn_norm2.weight",
+    "layers.21.attention.norm_q.weight",
+    "layers.21.attention.norm_k.weight",
+    "layers.21.attention_norm1.weight",
+    "layers.21.ffn_norm1.weight",
+    "layers.21.attention_norm2.weight",
+    "layers.21.ffn_norm2.weight",
+    "layers.22.attention.norm_q.weight",
+    "layers.22.attention.norm_k.weight",
+    "layers.22.attention_norm1.weight",
+    "layers.22.ffn_norm1.weight",
+    "layers.22.attention_norm2.weight",
+    "layers.22.ffn_norm2.weight",
+    "layers.23.attention.norm_q.weight",
+    "layers.23.attention.norm_k.weight",
+    "layers.23.attention_norm1.weight",
+    "layers.23.ffn_norm1.weight",
+    "layers.23.attention_norm2.weight",
+    "layers.23.ffn_norm2.weight",
+    "layers.24.attention.norm_q.weight",
+    "layers.24.attention.norm_k.weight",
+    "layers.24.attention_norm1.weight",
+    "layers.24.ffn_norm1.weight",
+    "layers.24.attention_norm2.weight",
+    "layers.24.ffn_norm2.weight",
+    "layers.25.attention.norm_q.weight",
+    "layers.25.attention.norm_k.weight",
+    "layers.25.attention_norm1.weight",
+    "layers.25.ffn_norm1.weight",
+    "layers.25.attention_norm2.weight",
+    "layers.25.ffn_norm2.weight",
+    "layers.26.attention.norm_q.weight",
+    "layers.26.attention.norm_k.weight",
+    "layers.26.attention_norm1.weight",
+    "layers.26.ffn_norm1.weight",
+    "layers.26.attention_norm2.weight",
+    "layers.26.ffn_norm2.weight",
+    "layers.27.attention.norm_q.weight",
+    "layers.27.attention.norm_k.weight",
+    "layers.27.attention_norm1.weight",
+    "layers.27.ffn_norm1.weight",
+    "layers.27.attention_norm2.weight",
+    "layers.27.ffn_norm2.weight",
+    "layers.28.attention.norm_q.weight",
+    "layers.28.attention.norm_k.weight",
+    "layers.28.attention_norm1.weight",
+    "layers.28.ffn_norm1.weight",
+    "layers.28.attention_norm2.weight",
+    "layers.28.ffn_norm2.weight",
+    "layers.29.attention.norm_q.weight",
+    "layers.29.attention.norm_k.weight",
+    "layers.29.attention_norm1.weight",
+    "layers.29.ffn_norm1.weight",
+    "layers.29.attention_norm2.weight",
+    "layers.29.ffn_norm2.weight",
+    "layers.30.attention.norm_q.weight",
+    "layers.30.attention.norm_k.weight",
+    "layers.30.attention_norm1.weight",
+    "layers.30.ffn_norm1.weight",
+    "layers.30.attention_norm2.weight",
+    "layers.30.ffn_norm2.weight",
+    "layers.31.attention.norm_q.weight",
+    "layers.31.attention.norm_k.weight",
+    "layers.31.attention_norm1.weight",
+    "layers.31.ffn_norm1.weight",
+    "layers.31.attention_norm2.weight",
+    "layers.31.ffn_norm2.weight",
+    "layers.32.attention.norm_q.weight",
+    "layers.32.attention.norm_k.weight",
+    "layers.32.attention_norm1.weight",
+    "layers.32.ffn_norm1.weight",
+    "layers.32.attention_norm2.weight",
+    "layers.32.ffn_norm2.weight",
+    "layers.33.attention.norm_q.weight",
+    "layers.33.attention.norm_k.weight",
+    "layers.33.attention_norm1.weight",
+    "layers.33.ffn_norm1.weight",
+    "layers.33.attention_norm2.weight",
+    "layers.33.ffn_norm2.weight"
+  ],
+  "modules_to_not_use_matmul": [],
+  "non_blocking": false,
+  "quant_conv": false,
+  "quant_embedding": false,
+  "quant_method": "sdnq",
+  "quantization_device": "cuda",
+  "quantized_matmul_dtype": null,
+  "return_device": "cpu",
+  "sdnq_version": "0.1.9",
+  "svd_rank": 32,
+  "svd_steps": 8,
+  "use_dynamic_quantization": false,
+  "use_grad_ckpt": true,
+  "use_hadamard": false,
+  "use_quantized_matmul": false,
+  "use_quantized_matmul_conv": false,
+  "use_static_quantization": true,
+  "use_stochastic_rounding": false,
+  "use_svd": false,
+  "weights_dtype": "uint4"
+}

unconditional_transformer/config.json

@@ -0,0 +1,19 @@
+{
+  "_class_name": "Ideogram4Transformer2DModel",
+  "_diffusers_version": "0.39.0.dev0",
+  "_name_or_path": "/home/jinli/.cache/huggingface/hub/models--ideogram-ai--debug-ideogram-v4/snapshots/41af6183c9fd9b6254864b0720319ef984535bfc/unconditional_transformer",
+  "adaln_dim": 512,
+  "attention_head_dim": 256,
+  "in_channels": 128,
+  "intermediate_size": 12288,
+  "llm_features_dim": 53248,
+  "mrope_section": [
+    24,
+    20,
+    20
+  ],
+  "norm_eps": 1e-05,
+  "num_attention_heads": 18,
+  "num_layers": 34,
+  "rope_theta": 5000000
+}

unconditional_transformer/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:032d92062f43bec5122be61c71cec7edd83c04cd57f8cb6280f9a6a58ca9ef92
+size 5221643880

unconditional_transformer/quantization_config.json

@@ -0,0 +1,239 @@
+{
+  "add_skip_keys": false,
+  "dequantize_fp32": false,
+  "dynamic_loss_threshold": null,
+  "group_size": 0,
+  "hadamard_group_size": 128,
+  "is_integer": true,
+  "is_training": false,
+  "modules_dtype_dict": {},
+  "modules_quant_config": {},
+  "modules_to_not_convert": [
+    "llm_cond_norm.weight",
+    "embed_image_indicator.weight",
+    "layers.0.attention.norm_q.weight",
+    "layers.0.attention.norm_k.weight",
+    "layers.0.attention_norm1.weight",
+    "layers.0.ffn_norm1.weight",
+    "layers.0.attention_norm2.weight",
+    "layers.0.ffn_norm2.weight",
+    "layers.1.attention.norm_q.weight",
+    "layers.1.attention.norm_k.weight",
+    "layers.1.attention_norm1.weight",
+    "layers.1.ffn_norm1.weight",
+    "layers.1.attention_norm2.weight",
+    "layers.1.ffn_norm2.weight",
+    "layers.2.attention.norm_q.weight",
+    "layers.2.attention.norm_k.weight",
+    "layers.2.attention_norm1.weight",
+    "layers.2.ffn_norm1.weight",
+    "layers.2.attention_norm2.weight",
+    "layers.2.ffn_norm2.weight",
+    "layers.3.attention.norm_q.weight",
+    "layers.3.attention.norm_k.weight",
+    "layers.3.attention_norm1.weight",
+    "layers.3.ffn_norm1.weight",
+    "layers.3.attention_norm2.weight",
+    "layers.3.ffn_norm2.weight",
+    "layers.4.attention.norm_q.weight",
+    "layers.4.attention.norm_k.weight",
+    "layers.4.attention_norm1.weight",
+    "layers.4.ffn_norm1.weight",
+    "layers.4.attention_norm2.weight",
+    "layers.4.ffn_norm2.weight",
+    "layers.5.attention.norm_q.weight",
+    "layers.5.attention.norm_k.weight",
+    "layers.5.attention_norm1.weight",
+    "layers.5.ffn_norm1.weight",
+    "layers.5.attention_norm2.weight",
+    "layers.5.ffn_norm2.weight",
+    "layers.6.attention.norm_q.weight",
+    "layers.6.attention.norm_k.weight",
+    "layers.6.attention_norm1.weight",
+    "layers.6.ffn_norm1.weight",
+    "layers.6.attention_norm2.weight",
+    "layers.6.ffn_norm2.weight",
+    "layers.7.attention.norm_q.weight",
+    "layers.7.attention.norm_k.weight",
+    "layers.7.attention_norm1.weight",
+    "layers.7.ffn_norm1.weight",
+    "layers.7.attention_norm2.weight",
+    "layers.7.ffn_norm2.weight",
+    "layers.8.attention.norm_q.weight",
+    "layers.8.attention.norm_k.weight",
+    "layers.8.attention_norm1.weight",
+    "layers.8.ffn_norm1.weight",
+    "layers.8.attention_norm2.weight",
+    "layers.8.ffn_norm2.weight",
+    "layers.9.attention.norm_q.weight",
+    "layers.9.attention.norm_k.weight",
+    "layers.9.attention_norm1.weight",
+    "layers.9.ffn_norm1.weight",
+    "layers.9.attention_norm2.weight",
+    "layers.9.ffn_norm2.weight",
+    "layers.10.attention.norm_q.weight",
+    "layers.10.attention.norm_k.weight",
+    "layers.10.attention_norm1.weight",
+    "layers.10.ffn_norm1.weight",
+    "layers.10.attention_norm2.weight",
+    "layers.10.ffn_norm2.weight",
+    "layers.11.attention.norm_q.weight",
+    "layers.11.attention.norm_k.weight",
+    "layers.11.attention_norm1.weight",
+    "layers.11.ffn_norm1.weight",
+    "layers.11.attention_norm2.weight",
+    "layers.11.ffn_norm2.weight",
+    "layers.12.attention.norm_q.weight",
+    "layers.12.attention.norm_k.weight",
+    "layers.12.attention_norm1.weight",
+    "layers.12.ffn_norm1.weight",
+    "layers.12.attention_norm2.weight",
+    "layers.12.ffn_norm2.weight",
+    "layers.13.attention.norm_q.weight",
+    "layers.13.attention.norm_k.weight",
+    "layers.13.attention_norm1.weight",
+    "layers.13.ffn_norm1.weight",
+    "layers.13.attention_norm2.weight",
+    "layers.13.ffn_norm2.weight",
+    "layers.14.attention.norm_q.weight",
+    "layers.14.attention.norm_k.weight",
+    "layers.14.attention_norm1.weight",
+    "layers.14.ffn_norm1.weight",
+    "layers.14.attention_norm2.weight",
+    "layers.14.ffn_norm2.weight",
+    "layers.15.attention.norm_q.weight",
+    "layers.15.attention.norm_k.weight",
+    "layers.15.attention_norm1.weight",
+    "layers.15.ffn_norm1.weight",
+    "layers.15.attention_norm2.weight",
+    "layers.15.ffn_norm2.weight",
+    "layers.16.attention.norm_q.weight",
+    "layers.16.attention.norm_k.weight",
+    "layers.16.attention_norm1.weight",
+    "layers.16.ffn_norm1.weight",
+    "layers.16.attention_norm2.weight",
+    "layers.16.ffn_norm2.weight",
+    "layers.17.attention.norm_q.weight",
+    "layers.17.attention.norm_k.weight",
+    "layers.17.attention_norm1.weight",
+    "layers.17.ffn_norm1.weight",
+    "layers.17.attention_norm2.weight",
+    "layers.17.ffn_norm2.weight",
+    "layers.18.attention.norm_q.weight",
+    "layers.18.attention.norm_k.weight",
+    "layers.18.attention_norm1.weight",
+    "layers.18.ffn_norm1.weight",
+    "layers.18.attention_norm2.weight",
+    "layers.18.ffn_norm2.weight",
+    "layers.19.attention.norm_q.weight",
+    "layers.19.attention.norm_k.weight",
+    "layers.19.attention_norm1.weight",
+    "layers.19.ffn_norm1.weight",
+    "layers.19.attention_norm2.weight",
+    "layers.19.ffn_norm2.weight",
+    "layers.20.attention.norm_q.weight",
+    "layers.20.attention.norm_k.weight",
+    "layers.20.attention_norm1.weight",
+    "layers.20.ffn_norm1.weight",
+    "layers.20.attention_norm2.weight",
+    "layers.20.ffn_norm2.weight",
+    "layers.21.attention.norm_q.weight",
+    "layers.21.attention.norm_k.weight",
+    "layers.21.attention_norm1.weight",
+    "layers.21.ffn_norm1.weight",
+    "layers.21.attention_norm2.weight",
+    "layers.21.ffn_norm2.weight",
+    "layers.22.attention.norm_q.weight",
+    "layers.22.attention.norm_k.weight",
+    "layers.22.attention_norm1.weight",
+    "layers.22.ffn_norm1.weight",
+    "layers.22.attention_norm2.weight",
+    "layers.22.ffn_norm2.weight",
+    "layers.23.attention.norm_q.weight",
+    "layers.23.attention.norm_k.weight",
+    "layers.23.attention_norm1.weight",
+    "layers.23.ffn_norm1.weight",
+    "layers.23.attention_norm2.weight",
+    "layers.23.ffn_norm2.weight",
+    "layers.24.attention.norm_q.weight",
+    "layers.24.attention.norm_k.weight",
+    "layers.24.attention_norm1.weight",
+    "layers.24.ffn_norm1.weight",
+    "layers.24.attention_norm2.weight",
+    "layers.24.ffn_norm2.weight",
+    "layers.25.attention.norm_q.weight",
+    "layers.25.attention.norm_k.weight",
+    "layers.25.attention_norm1.weight",
+    "layers.25.ffn_norm1.weight",
+    "layers.25.attention_norm2.weight",
+    "layers.25.ffn_norm2.weight",
+    "layers.26.attention.norm_q.weight",
+    "layers.26.attention.norm_k.weight",
+    "layers.26.attention_norm1.weight",
+    "layers.26.ffn_norm1.weight",
+    "layers.26.attention_norm2.weight",
+    "layers.26.ffn_norm2.weight",
+    "layers.27.attention.norm_q.weight",
+    "layers.27.attention.norm_k.weight",
+    "layers.27.attention_norm1.weight",
+    "layers.27.ffn_norm1.weight",
+    "layers.27.attention_norm2.weight",
+    "layers.27.ffn_norm2.weight",
+    "layers.28.attention.norm_q.weight",
+    "layers.28.attention.norm_k.weight",
+    "layers.28.attention_norm1.weight",
+    "layers.28.ffn_norm1.weight",
+    "layers.28.attention_norm2.weight",
+    "layers.28.ffn_norm2.weight",
+    "layers.29.attention.norm_q.weight",
+    "layers.29.attention.norm_k.weight",
+    "layers.29.attention_norm1.weight",
+    "layers.29.ffn_norm1.weight",
+    "layers.29.attention_norm2.weight",
+    "layers.29.ffn_norm2.weight",
+    "layers.30.attention.norm_q.weight",
+    "layers.30.attention.norm_k.weight",
+    "layers.30.attention_norm1.weight",
+    "layers.30.ffn_norm1.weight",
+    "layers.30.attention_norm2.weight",
+    "layers.30.ffn_norm2.weight",
+    "layers.31.attention.norm_q.weight",
+    "layers.31.attention.norm_k.weight",
+    "layers.31.attention_norm1.weight",
+    "layers.31.ffn_norm1.weight",
+    "layers.31.attention_norm2.weight",
+    "layers.31.ffn_norm2.weight",
+    "layers.32.attention.norm_q.weight",
+    "layers.32.attention.norm_k.weight",
+    "layers.32.attention_norm1.weight",
+    "layers.32.ffn_norm1.weight",
+    "layers.32.attention_norm2.weight",
+    "layers.32.ffn_norm2.weight",
+    "layers.33.attention.norm_q.weight",
+    "layers.33.attention.norm_k.weight",
+    "layers.33.attention_norm1.weight",
+    "layers.33.ffn_norm1.weight",
+    "layers.33.attention_norm2.weight",
+    "layers.33.ffn_norm2.weight"
+  ],
+  "modules_to_not_use_matmul": [],
+  "non_blocking": false,
+  "quant_conv": false,
+  "quant_embedding": false,
+  "quant_method": "sdnq",
+  "quantization_device": "cuda",
+  "quantized_matmul_dtype": null,
+  "return_device": "cpu",
+  "sdnq_version": "0.1.9",
+  "svd_rank": 32,
+  "svd_steps": 8,
+  "use_dynamic_quantization": false,
+  "use_grad_ckpt": true,
+  "use_hadamard": false,
+  "use_quantized_matmul": false,
+  "use_quantized_matmul_conv": false,
+  "use_static_quantization": true,
+  "use_stochastic_rounding": false,
+  "use_svd": false,
+  "weights_dtype": "uint4"
+}

vae/config.json

@@ -0,0 +1,41 @@
+{
+  "_class_name": "AutoencoderKLFlux2",
+  "_diffusers_version": "0.39.0.dev0",
+  "_name_or_path": "/home/jinli/.cache/huggingface/hub/models--ideogram-ai--debug-ideogram-v4/snapshots/41af6183c9fd9b6254864b0720319ef984535bfc/vae",
+  "act_fn": "silu",
+  "batch_norm_eps": 0.0001,
+  "batch_norm_momentum": 0.1,
+  "block_out_channels": [
+    128,
+    256,
+    512,
+    512
+  ],
+  "decoder_block_out_channels": null,
+  "down_block_types": [
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D",
+    "DownEncoderBlock2D"
+  ],
+  "force_upcast": true,
+  "in_channels": 3,
+  "latent_channels": 32,
+  "layers_per_block": 2,
+  "mid_block_add_attention": true,
+  "norm_num_groups": 32,
+  "out_channels": 3,
+  "patch_size": [
+    2,
+    2
+  ],
+  "sample_size": 1024,
+  "up_block_types": [
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D",
+    "UpDecoderBlock2D"
+  ],
+  "use_post_quant_conv": true,
+  "use_quant_conv": true
+}

vae/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81efe2a79ca70a1c25772f85cd9c2c2e609c0723b00bbe9486666826f6c72144
+size 52665102

vae/quantization_config.json

@@ -0,0 +1,85 @@
+{
+  "add_skip_keys": false,
+  "dequantize_fp32": false,
+  "dynamic_loss_threshold": null,
+  "group_size": 0,
+  "hadamard_group_size": 128,
+  "is_integer": true,
+  "is_training": false,
+  "modules_dtype_dict": {},
+  "modules_quant_config": {},
+  "modules_to_not_convert": [
+    "encoder.down.0.block.0.norm1.weight",
+    "encoder.down.0.block.0.norm2.weight",
+    "encoder.down.0.block.1.norm1.weight",
+    "encoder.down.0.block.1.norm2.weight",
+    "encoder.down.1.block.0.norm1.weight",
+    "encoder.down.1.block.0.norm2.weight",
+    "encoder.down.1.block.1.norm1.weight",
+    "encoder.down.1.block.1.norm2.weight",
+    "encoder.down.2.block.0.norm1.weight",
+    "encoder.down.2.block.0.norm2.weight",
+    "encoder.down.2.block.1.norm1.weight",
+    "encoder.down.2.block.1.norm2.weight",
+    "encoder.down.3.block.0.norm1.weight",
+    "encoder.down.3.block.0.norm2.weight",
+    "encoder.down.3.block.1.norm1.weight",
+    "encoder.down.3.block.1.norm2.weight",
+    "encoder.mid.block_1.norm1.weight",
+    "encoder.mid.block_1.norm2.weight",
+    "encoder.mid.attn_1.norm.weight",
+    "encoder.mid.block_2.norm1.weight",
+    "encoder.mid.block_2.norm2.weight",
+    "encoder.norm_out.weight",
+    "decoder.mid.block_1.norm1.weight",
+    "decoder.mid.block_1.norm2.weight",
+    "decoder.mid.attn_1.norm.weight",
+    "decoder.mid.block_2.norm1.weight",
+    "decoder.mid.block_2.norm2.weight",
+    "decoder.up.0.block.0.norm1.weight",
+    "decoder.up.0.block.0.norm2.weight",
+    "decoder.up.0.block.1.norm1.weight",
+    "decoder.up.0.block.1.norm2.weight",
+    "decoder.up.0.block.2.norm1.weight",
+    "decoder.up.0.block.2.norm2.weight",
+    "decoder.up.1.block.0.norm1.weight",
+    "decoder.up.1.block.0.norm2.weight",
+    "decoder.up.1.block.1.norm1.weight",
+    "decoder.up.1.block.1.norm2.weight",
+    "decoder.up.1.block.2.norm1.weight",
+    "decoder.up.1.block.2.norm2.weight",
+    "decoder.up.2.block.0.norm1.weight",
+    "decoder.up.2.block.0.norm2.weight",
+    "decoder.up.2.block.1.norm1.weight",
+    "decoder.up.2.block.1.norm2.weight",
+    "decoder.up.2.block.2.norm1.weight",
+    "decoder.up.2.block.2.norm2.weight",
+    "decoder.up.3.block.0.norm1.weight",
+    "decoder.up.3.block.0.norm2.weight",
+    "decoder.up.3.block.1.norm1.weight",
+    "decoder.up.3.block.1.norm2.weight",
+    "decoder.up.3.block.2.norm1.weight",
+    "decoder.up.3.block.2.norm2.weight",
+    "decoder.norm_out.weight"
+  ],
+  "modules_to_not_use_matmul": [],
+  "non_blocking": false,
+  "quant_conv": true,
+  "quant_embedding": false,
+  "quant_method": "sdnq",
+  "quantization_device": "cuda",
+  "quantized_matmul_dtype": null,
+  "return_device": "cpu",
+  "sdnq_version": "0.1.9",
+  "svd_rank": 32,
+  "svd_steps": 8,
+  "use_dynamic_quantization": false,
+  "use_grad_ckpt": true,
+  "use_hadamard": false,
+  "use_quantized_matmul": false,
+  "use_quantized_matmul_conv": false,
+  "use_static_quantization": true,
+  "use_stochastic_rounding": false,
+  "use_svd": false,
+  "weights_dtype": "uint4"
+}