AhmadRH commited on
Commit
a81dc00
·
verified ·
1 Parent(s): fef8d2b

Release PACT SD3.5 and LTX paper adapters, model cards and upstream licenses

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ ltx/ACCEPTABLE_USE_POLICY.pdf filter=lfs diff=lfs merge=lfs -text
LICENSE.md ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # License scope
2
+
3
+ This model repository contains two PACT adapters with separate governing licenses. There is no single permissive license covering the entire repository.
4
+
5
+ - **`sd35/`**: the [Stability AI Community License Agreement](sd35/LICENSE), July 5, 2024, governs the SD3.5 derivative adapter, including the PACT authors' LoRA/conditioner modifications. Preserve its [Notice](sd35/Notice), display **Powered by Stability AI**, and comply with the incorporated [Acceptable Use Policy](https://stability.ai/use-policy), including its updates. A readable policy snapshot is included in `sd35/ACCEPTABLE_USE_POLICY.txt`.
6
+ - **`ltx/`**: the [LTX-2 Community License Agreement](ltx/LICENSE), January 5, 2026, governs the LTX derivative adapter, including the PACT authors' modifications. Its full Attachment A, use restrictions, acceptable-use obligations and redistribution terms apply. Preserve [NOTICE](ltx/NOTICE) and the included `ltx/ACCEPTABLE_USE_POLICY.pdf`.
7
+
8
+ These copies match the LICENSE files in the pinned base-model repositories used by this release. LTX's pinned README also links a mutable upstream `LICENSE-2_x` page containing a newer agreement; we preserve the pinned repository's actual LICENSE instead of silently substituting that linked text. The included official LTX acceptable-use policy was retrieved on October 12, 2026; its URL is explicitly identified in the newer upstream agreement. The pinned January agreement requires an acceptable-use policy but supplies no separate URL.
9
+
10
+ To the extent the PACT authors control rights in the released LoRA and conditioning weights, they permit their use, reproduction, modification and redistribution subject to the corresponding model agreement identified above and included in that adapter's folder. This permission covers only the authors' adapter modifications; it does not replace the upstream grants or confer rights in third-party models, datasets or outputs. The same permission is recorded in each adapter's notice so it accompanies downloads of either folder alone.
11
+
12
+ Commercial conditions differ by model and may require upstream registration or a separate agreement. Training-source rights are separate: VidProM and TIP-I2V declare CC-BY-NC-4.0; DiffusionDB declares CC0-1.0; the inspected Pick-a-Pic v1 card has no declared license field. This release provides source attribution, not blanket clearance for commercial use of the training sources.
13
+
14
+ Base weights, Gemma, training data and third-party software are not distributed in this repository. Obtain them separately under their own terms. The accompanying PACT source code's original-code license remains a separate decision; no code-wide license is assigned by this model repository.
15
+
16
+ Copyright © 2026 PACT authors for their adapter modifications and original model-card text, subject to the governing upstream agreements above. Copies of these cards and notices may be redistributed with the adapters under the corresponding agreement. Neither upstream company endorses this release.
README.md ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ license_name: pact-per-adapter-community-licenses
6
+ license_link: LICENSE.md
7
+ base_model:
8
+ - stabilityai/stable-diffusion-3.5-medium
9
+ - Lightricks/LTX-2.3
10
+ base_model_relation: adapter
11
+ library_name: pytorch
12
+ tags:
13
+ - pact
14
+ - flow-matching
15
+ - distillation
16
+ - lora
17
+ - text-to-image
18
+ - text-to-video
19
+ - image-to-video
20
+ - audio-video
21
+ ---
22
+
23
+ # PACT: Preserving Accurate Continuous Trajectories via Flow-Map Distillation
24
+
25
+ PACT trains adapters on an offline bank of teacher trajectories to predict interval-average velocity, enabling few-step image and audiovisual generation while preserving teacher correspondence. This repository contains the two selected **paper inference adapters**, including their learned destination-time and guidance conditioners. Base models are separate downloads.
26
+
27
+ | Adapter | Base model | Paper setting | Download size | Details |
28
+ | --- | --- | --- | --- | --- |
29
+ | [`sd35/`](sd35/) | Stable Diffusion 3.5 Medium, 2.5B | 4 steps, 1024×1024; raw step 190,000 | 471 MB | [Model card and usage](sd35/README.md) |
30
+ | [`ltx/`](ltx/) | LTX-2.3 22B dev, joint audio/video | 8 steps, 768×512, 121 frames at 24 fps; raw step 30,000 | 15.0 GB | [Model card and usage](ltx/README.md) |
31
+
32
+ Both use rank/alpha 384. **Use the PACT implementation and load the complete adapter folder.** Generic Diffusers or upstream LTX LoRA loaders omit required PACT conditioners and do not implement its flow-map sampling path. These packages contain inference weights, not optimizer state for exact training resumption.
33
+
34
+ ## Download
35
+
36
+ With the Hugging Face CLI installed, choose one or both:
37
+
38
+ ```bash
39
+ hf download AhmadRH/PACT --include 'sd35/*' --local-dir ./pact-checkpoints
40
+ hf download AhmadRH/PACT --include 'ltx/*' --local-dir ./pact-checkpoints
41
+ ```
42
+
43
+ Read the corresponding model card for base-model pins and inference commands. Source code publication on GitHub is pending; inference requires the accompanying PACT release source tree (`SD35-FlowBank-Distillation/` or `LTX-FlowBank-Distillation/`). The weights alone are not a standalone pipeline. Public paper and code links will be added when available.
44
+
45
+ ## Validation and limitations
46
+
47
+ The release adapters were tested with the PACT loaders and representative few-step generations. Small preparation, trajectory-bank, 500-update training, checkpoint-reload and evaluation checks passed for both implementations. These are functional release checks, not a rerun of the paper benchmarks or a guarantee of quality for every prompt. No subjective audio-quality claim follows from the audio integrity checks. See each card for data provenance and reproducibility limitations.
48
+
49
+ ## Licenses and attribution
50
+
51
+ The two adapters have different governing agreements: [SD3.5 Stability AI Community License](sd35/LICENSE) and [LTX-2 Community License](ltx/LICENSE). Read the [license scope](LICENSE.md), [SD3.5 Notice](sd35/Notice) and [LTX NOTICE](ltx/NOTICE) before use. These notices include the authors' permission for their controlled adapter modifications under the corresponding agreement. This repository does not grant rights to the separate base models, text encoders or datasets. The LTX training sources declare noncommercial terms; no blanket commercial-data clearance is asserted.
52
+
53
+ **Powered by Stability AI.** SD3.5 is based on Stability AI's Stable Diffusion 3.5 Medium. The audiovisual adapter is a modified derivative of Lightricks' LTX-2.3, using a separate Google Gemma text encoder. These adapters are provided by the PACT authors; upstream attribution does not imply endorsement.
SHA256SUMS ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ a455965a1628f948fc055c3e577302ba4c54403e4a6e06d7c98e7f3e8e4ac160 LICENSE.md
2
+ 77477b18dc5dcb819b1c89048cac0489ca998aa97f2a08b10e04dce1ab20b797 README.md
3
+ 054b9e6e9d4b8632dd13abe64faded682c344cf9e514813b8d262d0ea94329f1 ltx/ACCEPTABLE_USE_POLICY.pdf
4
+ e28ef68d90e17b06308e4c5cba5c9a967b70c3d67f87ddc3dfc46a312cb6a134 ltx/LICENSE
5
+ f8762b60bc5b5383e01000895fde153d4205750c197882e10b5b76d75879f38a ltx/NOTICE
6
+ 9e7b1bf0b3baddb32f848d33cc4efa6788c65e5183758a114caa2918faeb45c7 ltx/README.md
7
+ d65e69a4da372cd79aaa5e822437cc4b0a8ce29501e5bbb17c24e60ed1aa8277 ltx/SHA256SUMS
8
+ 93af29000f56ba7871b7776db7d7e1702aaebb6e8e1a5fd9db248c23c645ea78 ltx/adapter.yaml
9
+ 64c7d40d520f7f9ec6132c2282d5b9ea521d1eeb5ed609b407cbc1028c88c0e5 ltx/inference.json
10
+ d4b8626e01f84880976183a2fbc061ba808576a25dfe858f0b84c5aba209867e ltx/lora_weights_step_30000.safetensors
11
+ 667673ea095deb42d7c143b50c61ff7ef941179fc48baf53c1abe41a8a6699da sd35/ACCEPTABLE_USE_POLICY.txt
12
+ 9b60b9d0411f54d4087484ad4acf28ca6fd9f6a84c30ddacb04de2c623254749 sd35/LICENSE
13
+ 212c56d85f751980f710fbe199fe10a047d50782ab3fa3844a30644a339db158 sd35/Notice
14
+ e4ad455da3df7e54cd28835034a25e24570b59e0e08ae911f2d5cf99a935b78a sd35/README.md
15
+ fd1dad23636c52f955a58c3ced65ba083688dc3613ef2aec9283c17806a72e70 sd35/SHA256SUMS
16
+ 1ebd2ef7643e6bbfd8c3e123c2f3f262c986b2a6f9e5fac0e107902bdeb3f60b sd35/export_validation.json
17
+ 84bab29fdcf94e206a305015c78c2adbfa269a92af01c1aa9a68cb6815323ba9 sd35/flowmap_conditioner.safetensors
18
+ abd3ccf966654fabe8aadc0c093414c6b9143d2b31840cf84c205b4e6f26c1e8 sd35/flowmap_lora.safetensors
19
+ 17fe4f9f35c0dbf60873fa5349b4f94d1ff58674c0c5e807908dc4c900bf7e0f sd35/flowmap_manifest.json
20
+ 64913aa6ba554a3a644dee9e9a6c5a065efc4c1f832c64302ef3f111f522ce38 sd35/paper_schedules.json
21
+ 9f8c529eb81803154971047ac058a15312c069104ff7f4893a398137ceda9f65 sd35/training_config.yaml
ltx/ACCEPTABLE_USE_POLICY.pdf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:054b9e6e9d4b8632dd13abe64faded682c344cf9e514813b8d262d0ea94329f1
3
+ size 110423
ltx/LICENSE ADDED
@@ -0,0 +1,349 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ LTX-2 Community License Agreement
2
+ License date: January 5, 2026
3
+
4
+
5
+ By using or distributing any portion or element of LTX-2, you agree
6
+ to be bound by this Agreement.
7
+
8
+ 1. Definitions.
9
+
10
+ "Agreement" means the terms and conditions for the license, use,
11
+ reproduction, and distribution of LTX-2 and the Complementary
12
+ Materials, as specified in this document.
13
+
14
+ "Control" means the direct or indirect ownership of fifty percent
15
+ (50%) or more of the voting securities or other ownership
16
+ interests, or the power to direct the management and policies of
17
+ such Entity through voting rights, contract, or otherwise.
18
+
19
+ "Data" means a collection of information and/or content extracted
20
+ from the dataset used with LTX-2, including to train, pretrain,
21
+ or otherwise evaluate LTX-2. The Data is not licensed under this
22
+ Agreement.
23
+
24
+ "Derivatives of LTX-2" means all modifications to LTX-2, works
25
+ based on LTX-2, or any other model which is created or initialized
26
+ by transfer of patterns of the weights, parameters, activations or
27
+ output of LTX-2, to the other model, in order to cause the other
28
+ model to perform similarly to LTX-2, including – but not limited
29
+ to - distillation methods entailing the use of intermediate data
30
+ representations or methods based on the generation of synthetic
31
+ data by LTX-2 for training the other model. For clarity, Derivatives
32
+ of LTX-2 include: (i) any fine-tuned or adapted weights, parameters,
33
+ or checkpoints derived from LTX-2; (ii) derivative model architectures
34
+ that incorporate or are based upon LTX-2's architecture; and
35
+ (iii) any modified or extended versions of the Complementary
36
+ Materials. All intellectual property rights in Derivatives of LTX-2
37
+ shall be subject to the terms of this Agreement, and you may not
38
+ claim exclusive ownership rights in any Derivatives of LTX-2 that
39
+ would restrict the rights granted herein.
40
+ "Entity" means any individual, corporation, partnership, limited
41
+ liability company, or other legal entity. For purposes of this
42
+ Agreement, an Entity shall be deemed to include, on an aggregative
43
+ basis, all subsidiaries, affiliates, and other companies under
44
+ common Control with such Entity. When determining whether an Entity
45
+ meets any threshold under this Agreement (including revenue
46
+ thresholds), all subsidiaries, affiliates, and companies under
47
+ common Control shall be considered collectively.
48
+ "Harm" includes but is not limited to physical, mental,
49
+ psychological, financial and reputational damage, pain, or loss.
50
+ "Licensor" or "Lightricks" means the owner that is granting the
51
+ license under this Agreement. For the purposes of this Agreement,
52
+ the Licensor is Lightricks Ltd.
53
+ "LTX-2" means the large language models, text/image/video/audio/3D
54
+ generation models, and multimodal large language models and their
55
+ software and algorithms, including trained model weights, parameters
56
+ (including optimizer states), machine-learning model code,
57
+ inference-enabling code, training-enabling code, fine-tuning
58
+ enabling code, accompanying source code, scripts, documentation,
59
+ tutorials, examples, and all other elements of the foregoing
60
+ distributed and made publicly available by Lightricks (including,
61
+ for example, at https://github.com/Lightricks/LTX-2) for the LTX-2
62
+ model released on January 5, 2026. This license is applicable to
63
+ all LTX-2 versions released since January 5, 2026, and all future
64
+ releases of LTX-2 under this license.
65
+ "Output" means the results of operating LTX-2 as embodied in
66
+ informational content resulting therefrom.
67
+ "you" (or "your") means an individual or legal Entity licensing
68
+ LTX-2 in accordance with this Agreement and/or making use of LTX-2
69
+ for whichever purpose and in any field of use, including usage of
70
+ LTX-2 in an end-use application - e.g. chatbot, translator, image
71
+ generator.
72
+ 2. Grant of License. Subject to the terms and conditions of this
73
+ Agreement, you are granted a non-exclusive, worldwide,
74
+ non-transferable and royalty-free limited license under Licensor's
75
+ intellectual property or other rights owned by Licensor embodied
76
+ in LTX-2 to use, reproduce, prepare, distribute, publicly display,
77
+ publicly perform, sublicense, copy, create derivative works of,
78
+ and make modifications to LTX-2, for any purpose, subject to the
79
+ restrictions set forth in Attachment A; provided however, that
80
+ Entities with annual revenues of at least $10,000,000 (the
81
+ "Commercial Entities") are required to obtain a paid commercial
82
+ use license in order to use LTX-2 and Derivatives of LTX-2,
83
+ subject to the terms and provisions of a different license (the
84
+ "Commercial Use Agreement"), as will be provided by the Licensor.
85
+ Commercial Entities interested in such a commercial license are
86
+ required to contact Licensor. Any commercial use of LTX-2 or
87
+ Derivatives of LTX-2 by the Commercial Entities not in accordance
88
+ with this Agreement and/or the Commercial Use Agreement is strictly
89
+ prohibited and shall be deemed a material breach of this Agreement.
90
+ Such material breach will be subject, in addition to any license
91
+ fees owed to Licensor for the period such Commercial Entity used
92
+ LTX-2 (as will be determined by Licensor), to liquidated damages,
93
+ which will be paid to Licensor immediately upon demand, in an
94
+ amount equal to double the amount that would otherwise have been
95
+ paid by you for the relevant period of time. Such amount reflects
96
+ a reasonable estimation of the losses and administrative costs
97
+ incurred due to such breach. You agree and understand that this
98
+ remedy does not limit the Licensor's right to pursue other remedies
99
+ available at law or equity.
100
+ 3. Distribution and Redistribution. You may host for third parties
101
+ remote access purposes (e.g. software-as-a-service), reproduce
102
+ and distribute copies of LTX-2 or Derivatives of LTX-2 thereof in
103
+ any medium, with or without modifications, provided that you meet
104
+ the following conditions:
105
+ (a) Use-based restrictions as referenced in paragraph 4 and all
106
+ provisions of Attachment A MUST be included as an enforceable
107
+ provision by you in any type of legal agreement (e.g. a
108
+ license) governing the use and/or distribution of LTX-2 or
109
+ Derivatives of LTX-2, and you shall give notice to subsequent
110
+ users you distribute to, that LTX-2 or Derivatives of LTX-2
111
+ are subject to paragraph 4 and Attachment A in their entirety,
112
+ including all use restrictions and acceptable use policies;
113
+ (b) You must provide any third party recipients of LTX-2 or
114
+ Derivatives of LTX-2 a copy of this Agreement, including all
115
+ attachments and use policies. Any Derivative of LTX-2 (as
116
+ defined in Section 1, including but not limited to fine-tuned
117
+ weights, modified training code, models trained on Outputs, or
118
+ any other derivative) must be distributed exclusively under
119
+ the terms of this Agreement with a complete copy of this
120
+ license included;
121
+ (c) You must cause any modified files to carry prominent notices
122
+ stating that you changed the files;
123
+ (d) You must retain all copyright, patent, trademark, and
124
+ attribution notices excluding those notices that do not
125
+ pertain to any part of LTX-2, Derivatives of LTX-2.
126
+ You may add your own copyright statement to your modifications and
127
+ may provide additional or different license terms and conditions -
128
+ respecting paragraph 3(a) - for use, reproduction, or distribution
129
+ of your modifications, or for any such Derivatives of LTX-2 as a
130
+ whole, provided your use, reproduction, and distribution of LTX-2
131
+ otherwise complies with the conditions stated in this Agreement,
132
+ and you provide a complete copy of this Agreement with any such
133
+ use, reproduction and distribution of LTX-2 and any Derivatives
134
+ thereof.
135
+ 4. Use-based restrictions. The restrictions set forth in Attachment A
136
+ are considered Use-based restrictions. Therefore, you cannot use
137
+ LTX-2 and the Derivatives of LTX-2 in violation of the specified
138
+ restricted uses. You may use LTX-2 subject to this Agreement,
139
+ including only for lawful purposes and in accordance with the
140
+ Agreement. "Use" may include creating any content with, fine-tuning,
141
+ updating, running, training, evaluating and/or re-parametrizing
142
+ LTX-2. You shall require all of your users who use LTX-2 or a
143
+ Derivative of LTX-2 to comply with the terms of this paragraph 4.
144
+ 5. The Output You Generate. Except as set forth herein, Licensor
145
+ claims no rights in the Output you generate using LTX-2. You are
146
+ accountable for input you insert into LTX-2, the Output you
147
+ generate and its subsequent uses. No use of the Output can
148
+ contravene any provision as stated in the Agreement.
149
+ 6. Updates and Runtime Restrictions. To the maximum extent permitted
150
+ by law, Licensor reserves the right to restrict (remotely or
151
+ otherwise) usage of LTX-2 in violation of this Agreement, update
152
+ LTX-2 through electronic means, or modify the Output of LTX-2
153
+ based on updates. You shall undertake reasonable efforts to use
154
+ the latest version of LTX-2. Any use of the non-current version
155
+ of LTX-2 is done solely at your risk.
156
+ 7. Export Controls and Sanctions Compliance. You acknowledge that
157
+ LTX-2, Derivatives of LTX-2 may be subject to export control laws
158
+ and regulations, including but not limited to the U.S. Export
159
+ Administration Regulations and sanctions programs administered by
160
+ the Office of Foreign Assets Control (OFAC). You represent and
161
+ warrant that you and any users of LTX-2 are not (i) located in,
162
+ organized under the laws of, or ordinarily resident in any country
163
+ or territory subject to comprehensive sanctions; (ii) identified
164
+ on any U.S. government restricted party list, including the
165
+ Specially Designated Nationals and Blocked Persons List; or
166
+ (iii) otherwise prohibited from receiving LTX-2 under applicable
167
+ law. You shall not export, re-export, or transfer LTX-2, directly
168
+ or indirectly, in violation of any applicable export control or
169
+ sanctions laws or regulations. You agree to comply with all
170
+ applicable trade control laws and shall indemnify and hold
171
+ Licensor harmless from any claims arising from your failure to
172
+ comply with such laws.
173
+ 8. Trademarks and related. Nothing in this Agreement permits you to
174
+ make use of Licensor's trademarks, trade names, logos or to
175
+ otherwise suggest endorsement or misrepresent the relationship
176
+ between the parties; and any rights not expressly granted herein
177
+ are reserved by the Licensor.
178
+
179
+ 9. Disclaimer of Warranty. Unless required by applicable law or
180
+ agreed to in writing, Licensor provides LTX-2 on an "AS IS" BASIS,
181
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
182
+ implied, including, without limitation, any warranties or
183
+ conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS
184
+ FOR A PARTICULAR PURPOSE. You are solely responsible for
185
+ determining the appropriateness of using or redistributing LTX-2
186
+ and Derivatives of LTX-2 and assume any risks associated with
187
+ your exercise of permissions under this Agreement.
188
+
189
+ 10. Limitation of Liability. In no event and under no legal theory,
190
+ whether in tort (including negligence), contract, or otherwise,
191
+ unless required by applicable law (such as deliberate and grossly
192
+ negligent acts) or agreed to in writing, shall Licensor be liable
193
+ to you for damages, including any direct, indirect, special,
194
+ incidental, or consequential damages of any character arising as
195
+ a result of this Agreement or out of the use or inability to use
196
+ LTX-2 (including but not limited to damages for loss of goodwill,
197
+ work stoppage, computer failure or malfunction, or any and all
198
+ other commercial damages or losses), even if Licensor has been
199
+ advised of the possibility of such damages.
200
+
201
+ 11. Accepting Warranty or Additional Liability. While redistributing
202
+ LTX-2 and Derivatives of LTX-2, you may, provided you do not
203
+ violate the terms of this Agreement, choose to offer and charge
204
+ a fee for, acceptance of support, warranty, indemnity, or other
205
+ liability obligations. However, in accepting such obligations,
206
+ you may act only on your own behalf and on your sole
207
+ responsibility, not on behalf of Licensor, and only if you agree
208
+ to indemnify, defend, and hold Licensor harmless for any liability
209
+ incurred by, or claims asserted against Licensor, by reason of
210
+ your accepting any such warranty or additional liability.
211
+
212
+ 12. Governing Law. This Agreement and all relations, disputes, claims
213
+ and other matters arising hereunder (including non-contractual
214
+ disputes or claims) will be governed exclusively by, and construed
215
+ exclusively in accordance with, the laws of the State of New York.
216
+ To the extent permitted by law, choice of laws rules and the
217
+ United Nations Convention on Contracts for the International Sale
218
+ of Goods will not apply. For the purposes of adjudicating any
219
+ action or proceeding to enforce the terms of this Agreement, you
220
+ hereby irrevocably consent to the exclusive jurisdiction of, and
221
+ venue in, the federal and state courts located in the County of
222
+ New York within the State of New York. The prevailing party in
223
+ any claim or dispute between the parties under this Agreement
224
+ will be entitled to reimbursement of its reasonable attorneys'
225
+ fees and costs. You hereby waive the right to a trial by jury,
226
+ to participate in a class or representative action (including in
227
+ arbitration), or to combine individual proceedings in court or
228
+ in arbitration without the consent of all parties.
229
+ 13. Term and Termination. This Agreement is effective upon your
230
+ acceptance and continues until terminated. Licensor may terminate
231
+ this Agreement immediately upon written notice to you if you
232
+ breach any provision of this Agreement, including but not limited
233
+ to violations of the use restrictions in Attachment A or
234
+ unauthorized commercial use. Upon termination: (a) all rights
235
+ granted to you under this Agreement will immediately cease;
236
+ (b) you must immediately cease all use of LTX-2 and Derivatives
237
+ of LTX-2; (c) you must delete or destroy all copies of LTX-2
238
+ and Derivatives of LTX-2 in your possession or control; and
239
+ (d) you must notify any third parties to whom you distributed
240
+ LTX-2 or Derivatives of LTX-2 of the termination. Sections 8-13,
241
+ and Section 15 shall survive termination of this Agreement.
242
+ Termination does not relieve you of any obligations incurred
243
+ prior to termination, including payment obligations under
244
+ Section 2. In addition, if You commence a lawsuit or other
245
+ proceedings (including a cross-claim or counterclaim in a lawsuit)
246
+ against Licensor or any person or entity alleging that LTX-2 or
247
+ any Output, or any portion of any of the foregoing, infringe any
248
+ intellectual property or other right owned or licensable by you,
249
+ then all licenses granted to you under this Agreement shall
250
+ terminate as of the date such lawsuit or other proceeding is filed.
251
+ 14. Disputes and Arbitration. All disputes arising in connection with
252
+ this Agreement shall be finally settled by arbitration under the
253
+ Rules of Arbitration of the International Chamber of Commerce
254
+ ("ICC Rules"), by one (1) arbitrator appointed in accordance with
255
+ the ICC Rules. The seat of arbitration shall be New York, NY, USA,
256
+ and the proceedings shall be conducted in English. The arbitrator
257
+ shall be empowered to grant any relief that a court could grant.
258
+ Judgment on the arbitration award may be entered by any court
259
+ having jurisdiction thereof. Each party waives its right to a
260
+ trial by jury and to participate in any class or representative
261
+ action.
262
+ 15. If any provision of this Agreement is held to be
263
+ invalid, illegal
264
+ or unenforceable, the remaining provisions shall be unaffected
265
+ thereby and remain valid as if such provision had not been set
266
+ forth herein.
267
+ END OF TERMS AND CONDITIONS
268
+ ATTACHMENT A: Use Restrictions
269
+ When using the Outputs, LTX-2 and any Derivatives thereof, you
270
+ will comply with the Acceptable Use Policy. In addition, you
271
+ agree not to use the Outputs, LTX-2 or its Derivatives in any
272
+ of the following ways:
273
+ 1. In any way that violates any applicable national, federal,
274
+ state, local or international law or regulation;
275
+ 2. For the purpose of exploiting, Harming or attempting to
276
+ exploit or Harm minors in any way;
277
+ 3. To generate or disseminate false information and/or content
278
+ with the purpose of Harming others;
279
+ 4. To generate or disseminate personal identifiable information
280
+ that can be used to Harm an individual;
281
+ 5. To generate or disseminate information and/or content (e.g.
282
+ images, code, posts, articles), and place the information
283
+ and/or content in any context (e.g. bot generating tweets)
284
+ without expressly and intelligibly disclaiming that the
285
+ information and/or content is machine generated;
286
+ 6. To defame, disparage or otherwise harass others;
287
+ 7. To impersonate or attempt to impersonate (e.g. deepfakes)
288
+ others without their consent;
289
+ 8. For fully automated decision making that adversely impacts an
290
+ individual's legal rights or otherwise creates or modifies a
291
+ binding, enforceable obligation;
292
+
293
+ 9. For any use intended to or which has the effect of
294
+ discriminating against or Harming individuals or groups based
295
+ on online or offline social behavior or known or predicted
296
+ personal or personality characteristics;
297
+
298
+ 10. To exploit any of the vulnerabilities of a specific group of
299
+ persons based on their age, social, physical or mental
300
+ characteristics, in order to materially distort the behavior
301
+ of a person pertaining to that group in a manner that causes
302
+ or is likely to cause that person or another person physical
303
+ or psychological Harm;
304
+
305
+ 11. For any use intended to or which has the effect of
306
+ discriminating against individuals or groups based on legally
307
+ protected characteristics or categories;
308
+
309
+ 12. To provide medical advice and medical results interpretation;
310
+
311
+ 13. To generate or disseminate information for the purpose to be
312
+ used for administration of justice, law enforcement,
313
+ immigration or asylum processes, such as predicting an
314
+ individual will commit fraud/crime commitment (e.g. by text
315
+ profiling, drawing causal relationships between assertions
316
+ made in documents, indiscriminate and arbitrarily-targeted use);
317
+
318
+ 14. To generate and/or disseminate malware (including – but not
319
+ limited to – ransomware) or any other content to be used for
320
+ the purpose of harming electronic systems;
321
+
322
+ 15. To engage in, promote, incite, or facilitate discrimination
323
+ or other unlawful or harmful conduct in the provision of
324
+ employment, employment benefits, credit, housing, or other
325
+ essential goods and services;
326
+
327
+ 16. To engage in, promote, incite, or facilitate the harassment,
328
+ abuse, threatening, or bullying of individuals or groups of
329
+ individuals;
330
+
331
+ 17. For military, warfare, nuclear industries or applications,
332
+ weapons development, or any use in connection with activities
333
+ that may cause death, personal injury, or severe physical or
334
+ environmental damage;
335
+
336
+ 18. For commercial use only: To train, improve, or fine-tune any
337
+ other machine learning model, artificial intelligence system,
338
+ or competing model, except for Derivatives of LTX-2 as
339
+ expressly permitted under this Agreement;
340
+
341
+ 19. To circumvent, disable, or interfere with any technical
342
+ limitations, safety features, content filters, or use
343
+ restrictions implemented in LTX-2 by Licensor;
344
+
345
+ 20. To use LTX-2 or Derivatives of LTX-2 in any product, service,
346
+ or application that directly competes with Licensor's
347
+ commercial products or services, or is designed to replace or
348
+ substitute Licensor's offerings in the market, without
349
+ obtaining a separate commercial license from Licensor.
ltx/NOTICE ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ PACT audiovisual adapter — modified derivative of Lightricks LTX-2.3.
2
+ LTX-2 and its upstream materials are provided by Lightricks Ltd.
3
+ PACT modifications: Copyright © 2026 PACT authors.
4
+
5
+ MODIFIED FILE: lora_weights_step_30000.safetensors
6
+ This file contains learned rank/alpha-384 LoRA changes for video, audio and
7
+ cross-modal attention/FFN projections and new video/audio destination-time
8
+ and guidance conditioners. Training uses cached teacher trajectories and
9
+ equal video/audio velocity loss weights. The selected model is raw step
10
+ 30,000. Export sanitizes private training metadata while preserving the
11
+ original tensor payload. adapter.yaml and inference.json describe these
12
+ modifications and their exact inference contract. This is not an official
13
+ Lightricks release.
14
+
15
+ These derivatives and the PACT modifications are distributed under the
16
+ included LTX-2 Community License Agreement, January 5, 2026, including
17
+ paragraph 4 and Attachment A in their entirety. All use restrictions and
18
+ acceptable-use requirements apply to subsequent users and redistribution.
19
+ Retain the full agreement, policies and this modification notice.
20
+
21
+ To the extent the PACT authors control rights in
22
+ lora_weights_step_30000.safetensors, they permit use, reproduction,
23
+ modification and redistribution of those adapter modifications subject to
24
+ the accompanying LTX-2 Community License Agreement, including paragraph 4
25
+ and Attachment A in their entirety. This permission does not replace
26
+ Lightricks' grant or confer rights in third-party models, datasets or
27
+ outputs. It does not license the separate PACT source code. Keep this
28
+ file-specific modification notice alongside the adapter on redistribution.
29
+
30
+ Base: Lightricks/LTX-2.3, ltx-2.3-22b-dev.safetensors
31
+ Revision: 3c6a4e66e5d0a684231950b9c74dd4ded7b6fadc
32
+ License source: https://huggingface.co/Lightricks/LTX-2.3/blob/3c6a4e66e5d0a684231950b9c74dd4ded7b6fadc/LICENSE
33
+ ACCEPTABLE_USE_POLICY.pdf is the official LTX policy last updated March 30,
34
+ 2026, retrieved October 12, 2026:
35
+ https://static.lightricks.com/legal/ltx-acceptable-use-policy.pdf
36
+ The policy URL is identified by the newer upstream LICENSE-2_x; the pinned
37
+ January agreement requires an AUP without specifying its separate URL.
38
+ See the root LICENSE.md for the pinned README/license-link discrepancy.
39
+
40
+ Separate text encoder: Google Gemma 3 12B IT
41
+ https://huggingface.co/google/gemma-3-12b-it
42
+ https://ai.google.dev/gemma/terms
43
+ Gemma weights are not distributed here.
44
+
45
+ Training-source attribution (source data not redistributed):
46
+ - VidProM — Wenhao Wang and Yi Yang.
47
+ https://huggingface.co/datasets/WenhaoWang/VidProM
48
+ Verified release snapshot: 0c0900e6a037b058b1903bc1de9bcb6037d9fc24
49
+ - TIP-I2V — Wenhao Wang and Yi Yang.
50
+ https://huggingface.co/datasets/WenhaoWang/TIP-I2V
51
+ Verified release snapshot: 958f51df0ba725f53e2387bc19011a001d5ce7a4
52
+ Both source cards declare Creative Commons Attribution-NonCommercial 4.0:
53
+ https://creativecommons.org/licenses/by-nc/4.0/
54
+ Preparation filtered and clustered prompt/image cases, replaced unavailable
55
+ images within clusters, and enhanced prompts with Gemma. Reference images
56
+ remain subject to their original source rights. The public source pins
57
+ identify release snapshots, not recovered historical unpinned revisions.
58
+ This release is intended for noncommercial research and evaluation and
59
+ makes no commercial-data clearance claim. Model license scope and dataset
60
+ license scope remain distinct. Consult source cards for full citations.
ltx/README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ license_name: ltx-2-community-license
6
+ license_link: LICENSE
7
+ base_model: Lightricks/LTX-2.3
8
+ base_model_relation: adapter
9
+ library_name: pytorch
10
+ pipeline_tag: text-to-video
11
+ tags: [pact, flow-matching, distillation, lora, image-to-video, audio-video]
12
+ ---
13
+
14
+ # PACT for LTX-2.3 audiovisual generation
15
+
16
+ Inference adapter for **PACT: Preserving Accurate Continuous Trajectories via Flow-Map Distillation**, released by the PACT authors. This is the **raw step-30,000**, rank/alpha-384 joint audiovisual checkpoint, trained with **equal video and audio loss weights**. It is a modified derivative of Lightricks' LTX-2.3 22B dev model, not an official Lightricks release.
17
+
18
+ ## Base and contents
19
+
20
+ Download separately:
21
+
22
+ - [`Lightricks/LTX-2.3`](https://huggingface.co/Lightricks/LTX-2.3), file `ltx-2.3-22b-dev.safetensors`, revision `3c6a4e66e5d0a684231950b9c74dd4ded7b6fadc`.
23
+ - [`google/gemma-3-12b-it`](https://huggingface.co/google/gemma-3-12b-it), verified release revision `96b6f1eccf38110c56df3a15bffe176da04bfd80`. Accept Google's access and license terms. The original historical Gemma download was unpinned; this pin identifies the tested release snapshot.
24
+
25
+ `lora_weights_step_30000.safetensors` contains AV/cross-modal LoRA plus **both** destination/guidance conditioners. `adapter.yaml` supplies construction metadata; `inference.json` records the exact sigma schedule, base/adapter hashes and sanitized-export receipt. The tensor payload is unchanged from the selected paper checkpoint. `SHA256SUMS` covers this complete folder. No optimizer or full training-resume state is included.
26
+
27
+ The paper setting uses **eight joint evaluations, learned video CFG 3, no external guidance, 768×512, 121 frames, 24 fps, and 48 kHz stereo audio**. The adapter supports text-to-video and first-frame image-to-video. Generic upstream LoRA inference omits PACT conditioning and is incompatible. The approximately 15 GB FP32 adapter requires substantial GPU memory in addition to the base model; release validation used GH200 GPUs.
28
+
29
+ ## Usage
30
+
31
+ Install all three packages from the accompanying PACT `LTX-FlowBank-Distillation/` source tree following its README (source publication is pending). Use its CUDA environment and FFmpeg; keep it separate from the SD3.5 environment. From the LTX source directory, after downloading the base and Gemma:
32
+
33
+ ```bash
34
+ export PACT_DATA=/absolute/path/to/pact-ltx-data
35
+ hf download AhmadRH/PACT --include 'ltx/*' --local-dir "$PACT_DATA/checkpoints"
36
+ python scripts/configure.py \
37
+ --data-root "$PACT_DATA" \
38
+ --base "$PACT_DATA/models/ltx-2.3/ltx-2.3-22b-dev.safetensors" \
39
+ --gemma "$PACT_DATA/models/gemma-3-12b" \
40
+ --adapter "$PACT_DATA/checkpoints/ltx/lora_weights_step_30000.safetensors" \
41
+ --adapter-config "$PACT_DATA/checkpoints/ltx/adapter.yaml" \
42
+ --output "$PACT_DATA/configs"
43
+ cat > "$PACT_DATA/prompts.jsonl" <<'JSONL'
44
+ {"prompt":"A red fox walks through a snowy pine forest. The camera tracks smoothly beside it as soft snow crunches under its paws and a quiet wind blows through the trees.","seed":42}
45
+ JSONL
46
+ python scripts/generate.py --config "$PACT_DATA/configs/vbench_student.yaml" \
47
+ --prompts "$PACT_DATA/prompts.jsonl" --output "$PACT_DATA/samples" --cache-only
48
+ python scripts/generate.py --config "$PACT_DATA/configs/vbench_student.yaml" \
49
+ --prompts "$PACT_DATA/prompts.jsonl" --output "$PACT_DATA/samples"
50
+ ```
51
+
52
+ For I2V, add an `"image":"/absolute/path/reference.png"` field to a prompt record. Generation creates MP4s with audio and JSON receipts. The separate cache step releases text-encoder memory before loading the audiovisual transformer. Training is unnecessary.
53
+
54
+ ## Training data and method
55
+
56
+ The paper uses 25,000 [VidProM](https://huggingface.co/datasets/WenhaoWang/VidProM) text prompts and 25,000 [TIP-I2V](https://huggingface.co/datasets/WenhaoWang/TIP-I2V) image/prompt cases, with 10% of each task held out. Preparation filters, embeds, clusters, selects and enhances conditions using Gemma. The teacher stores 30-step joint trajectories; PACT learns average velocity with separate masked video/audio losses of equal weight. Training data and Gemma/base weights are not redistributed here.
57
+
58
+ Both dataset cards declare **CC-BY-NC-4.0**. See [NOTICE](NOTICE) for source attribution and preparation changes. This release is intended for noncommercial research and evaluation and does not assert clearance for commercial use of its training sources. The datasets' terms are distinct from the model agreement; training on them does not by itself establish that every output or adapter inherits a Creative Commons license.
59
+
60
+ ## Evaluation and limitations
61
+
62
+ Release checks include paper-adapter T2V/I2V generation and visual review, audiovisual integrity checks, and bounded evaluation execution. A separate fresh 500-update smoke test reduced total loss from 0.548355 to 0.060541; tiny held-out errors increased, so this is not evidence of generalization. Visual VBench metrics do not measure audio fidelity or audiovisual synchronization; no subjective listening evaluation is claimed. Full paper benchmarks were not rerun during packaging.
63
+
64
+ Generated motion, fine details and composition may differ from the teacher, particularly for difficult image-conditioned dynamics. The adapter inherits its base model's limitations and biases. Preserve the complete AV conditioners, image masks, paired sigma schedule and learned guidance when using it.
65
+
66
+ ## License
67
+
68
+ The adapter is distributed under the included **[LTX-2 Community License Agreement](LICENSE), dated January 5, 2026**, exactly as present in the pinned base-model repository. Its full Attachment A, acceptable-use requirements, commercial restrictions and redistribution obligations apply. [NOTICE](NOTICE) contains prominent modification and upstream attribution notices, plus the PACT authors' permission for their controlled adapter modifications subject to that agreement. The separate Gemma encoder is governed by [Gemma terms](https://ai.google.dev/gemma/terms), and dataset licenses remain applicable. This release is not a blanket commercial-use clearance.
ltx/SHA256SUMS ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ 054b9e6e9d4b8632dd13abe64faded682c344cf9e514813b8d262d0ea94329f1 ACCEPTABLE_USE_POLICY.pdf
2
+ e28ef68d90e17b06308e4c5cba5c9a967b70c3d67f87ddc3dfc46a312cb6a134 LICENSE
3
+ f8762b60bc5b5383e01000895fde153d4205750c197882e10b5b76d75879f38a NOTICE
4
+ 9e7b1bf0b3baddb32f848d33cc4efa6788c65e5183758a114caa2918faeb45c7 README.md
5
+ 93af29000f56ba7871b7776db7d7e1702aaebb6e8e1a5fd9db248c23c645ea78 adapter.yaml
6
+ 64c7d40d520f7f9ec6132c2282d5b9ea521d1eeb5ed609b407cbc1028c88c0e5 inference.json
7
+ d4b8626e01f84880976183a2fbc061ba808576a25dfe858f0b84c5aba209867e lora_weights_step_30000.safetensors
ltx/adapter.yaml ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ lora:
2
+ rank: 384
3
+ alpha: 384
4
+ dropout: 0.0
5
+ target_modules:
6
+ - to_q
7
+ - to_k
8
+ - to_v
9
+ - to_out.0
10
+ - ff.net.0.proj
11
+ - ff.net.2
12
+ training_strategy:
13
+ name: flowmap
14
+ selection_mode: all_video_train
15
+ trajectory_count: 45000
16
+ pairs_per_trajectory: 1
17
+ heldout_pairs_per_trajectory: 0
18
+ min_delta_sigma: 0.05
19
+ pair_sampling_mode: delta_sigma_bucket
20
+ delta_sigma_bucket_boundaries:
21
+ - 0.0
22
+ - 0.25
23
+ - 0.5
24
+ - 0.75
25
+ - 1.0
26
+ delta_sigma_bucket_weights:
27
+ - 0.3
28
+ - 0.3
29
+ - 0.3
30
+ - 0.1
31
+ jump_gated_lora: false
32
+ cached_validation_interval: 500
33
+ cached_validation_trajectory_count: 32
34
+ cached_validation_pairs_per_bucket: 1
35
+ audio_enabled: true
36
+ audio_loss_weight: 1.0
ltx/inference.json ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "PACT LTX-2.3 joint AV",
3
+ "weights": "raw",
4
+ "step": 30000,
5
+ "rank": 384,
6
+ "alpha": 384,
7
+ "base_model": "Lightricks/LTX-2.3",
8
+ "base_file": "ltx-2.3-22b-dev.safetensors",
9
+ "base_sha256": "7ab7225325bc403448ea84b6db2269811a880e5118cd2ee2b6282a93d585016f",
10
+ "adapter_file": "lora_weights_step_30000.safetensors",
11
+ "source_archive_sha256": "6b1a6bf635c5497e7f807e587a673da98573853ac456d6719726c9fe240015ea",
12
+ "adapter_sha256": "d4b8626e01f84880976183a2fbc061ba808576a25dfe858f0b84c5aba209867e",
13
+ "tensor_payload_sha256": "66f7a04755a7b7733a404485f6745accd8e382c5f4000e89813a5c4dd5e6b0c8",
14
+ "tensor_payload_equal": true,
15
+ "tensor_count": 2704,
16
+ "required_tensor_groups": {
17
+ "video_flowmap_conditioner": 8,
18
+ "audio_flowmap_conditioner": 8,
19
+ "audio_attn1": 384,
20
+ "audio_attn2": 384,
21
+ "audio_to_video_attn": 384,
22
+ "video_to_audio_attn": 384,
23
+ "lora_A": 1344,
24
+ "lora_B": 1344
25
+ },
26
+ "checkpoint_metadata": {
27
+ "flowmap_conditioning_version": "2",
28
+ "flowmap_fixed_teacher_guidance": "{\"audio\": {\"cfg_scale\": 7.0, \"modality_scale\": 3.0, \"rescale_scale\": 0.7, \"skip_step\": 0, \"stg_blocks\": [28], \"stg_scale\": 1.0}, \"video_modality_scale\": 3.0, \"video_rescale\": 0.7, \"video_stg\": 1.0, \"video_stg_blocks\": [28]}",
29
+ "flowmap_lora_scope": "audio_video_attention_ffn_cross_modal",
30
+ "flowmap_sigma_order": "sigma_src>=sigma_dst",
31
+ "lora_dropout": "0.0",
32
+ "flowmap_jump_gated_lora": "False",
33
+ "flowmap_cfg_reference": "1.0",
34
+ "lora_target_modules": "to_q,to_k,to_v,to_out.0,ff.net.0.proj,ff.net.2",
35
+ "lora_rank": "384",
36
+ "lora_alpha": "384",
37
+ "flowmap_output": "average_velocity",
38
+ "flowmap_teacher_checkpoint_sha256": "7ab7225325bc403448ea84b6db2269811a880e5118cd2ee2b6282a93d585016f",
39
+ "flowmap_audio_endpoint": "audio_z_dst=audio_z_src+(sigma_dst-sigma_src)*audio_u",
40
+ "flowmap_bank_schema": "ltx-flowmap-trajectory-av-v2",
41
+ "flowmap_ti2v_mask": "stored_video_denoise_mask",
42
+ "flowmap_endpoint": "z_dst=z_src+(sigma_dst-sigma_src)*u",
43
+ "flowmap_lora_targets": "to_q,to_k,to_v,to_out.0,ff.net.0.proj,ff.net.2",
44
+ "flowmap_audio_guidance_fields": "video_text_cfg;audio_guidance_fixed_by_bank",
45
+ "flowmap_guidance_fields": "video_text_cfg",
46
+ "flowmap_selection_mode": "all_video_train",
47
+ "flowmap_audio_loss_weight": "1.0",
48
+ "flowmap_audio_enabled": "True",
49
+ "flowmap_lora_gate": "none"
50
+ },
51
+ "video_cfg_scale": 3.0,
52
+ "external_guidance": false,
53
+ "sigma_schedule": [
54
+ 1.0,
55
+ 0.9816284775733948,
56
+ 0.9572813510894775,
57
+ 0.9234784841537476,
58
+ 0.8733806610107422,
59
+ 0.79146808385849,
60
+ 0.6333323121070862,
61
+ 0.19999998807907104,
62
+ 0.0
63
+ ],
64
+ "width": 768,
65
+ "height": 512,
66
+ "frames": 121,
67
+ "fps": 24,
68
+ "audio_sample_rate": 48000
69
+ }
ltx/lora_weights_step_30000.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4b8626e01f84880976183a2fbc061ba808576a25dfe858f0b84c5aba209867e
3
+ size 14990938152
sd35/ACCEPTABLE_USE_POLICY.txt ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Source: https://stability.ai/use-policy
2
+ Retrieved: 2026-10-12
3
+ Plain-text rendering of linked policy; the exact HTML snapshot is preserved alongside this file.
4
+
5
+ Acceptable Use Policy
6
+ Effective: September 30, 2026 (see previous version)
7
+ Our Acceptable Use Policy is intended to promote the safe and responsible use of Stability AI (“Stability”) models, products and services, application programming interfaces (“APIs”), software, or other materials (“Stability Technology”).
8
+ If you access, use, or distribute any Stability AI Technology, you agree to this Acceptable Use Policy (“Policy”). This includes non-commercial, research, and commercial uses of Stability AI Technology, including self-hosting Stability AI code or weights under the terms of our Non-Commercial Research, Community or Enterprise Licenses and includes accessing Stability Technology via third-party hosted platforms. If we learn that you have violated this Policy, we may suspend or terminate your access to our products and services.
9
+ We Prohibit Violations of Law or Others’ Rights.
10
+ This includes using Stability Technology to facilitate:
11
+ violations of law or others’ rights, including intellectual property and privacy rights.
12
+ violations of AI laws, such as by:
13
+ using subliminal, manipulative, or deceptive techniques that can distort a person’s ability to make informed decisions and is likely to cause harm.
14
+ exploiting vulnerabilities due to age, disability, or socio-economic situations.
15
+ evaluating or classifying persons based on their social behavior, personal characteristics, or the use of social scoring leading to detrimental or unfavorable treatment.
16
+ assessing or predicting the risk of a person committing a crime, based solely on profiling or personal traits.
17
+ creating or expanding facial recognition databases without consent.
18
+ inferring emotions in the workplace or education institution, except for medical or safety reasons.
19
+ categorizing people based on their biometric data to infer their race, political opinion, trade union membership, religious or philosophical beliefs, sex life or sexual orientation.
20
+ using real-time biometric identification systems in public spaces for law enforcement purposes.
21
+ sharing of personal information without consent.
22
+ provision of advice on essential services, including in the medical or health field, without review by a qualified professional and disclosure of the use of AI assistance and its potential limitations.
23
+ We Prohibit Harm to or Exploitation against Children.
24
+ This includes using Stability Technology to facilitate:
25
+ child sexual abuse material (CSAM), sexual exploitation, grooming, or trafficking of minors.
26
+ exploiting or abusing minors, including sextortion or any impersonation or enticement of minors for exploitative purposes.
27
+ pedophilic behavior, such as through suggestive depictions of minors.
28
+ We will report CSAM to the appropriate authorities.
29
+ We Prohibit Sexually Explicit Content.
30
+ This includes using Stability Technology to facilitate:
31
+ non-consensual intimate imagery (NCII).
32
+ illegal pornographic content.
33
+ content relating to sexual intercourse, sexual acts, or sexual violence.
34
+ We Prohibit Emotional or Physical Harm to Others or Self.
35
+ This includes using Stability Technology to facilitate:
36
+ encouragement or instructions related to self harm or harm to others.
37
+ discrimination, threats, or promotion of violence or hateful content based on protected attributes.
38
+ support for organizations or individuals associated with terrorism or hate.
39
+ content relating to human trafficking or exploitation.
40
+ extreme gore, such as content involving bodily destruction, mutilation, torture or animal abuse.
41
+ the development or manufacturing of any illegal or regulated weapons.
42
+ We Prohibit Circumvention of Our Safeguards.
43
+ This includes facilitating any of the following:
44
+ intentional bypassing of product safeguards or restrictions established within Stability Technology.
45
+ intentional bypassing of an account ban, such as by creating new accounts.
46
+ malicious code, malware, computer viruses, or any activity that could interfere with the integrity of a website or system.
47
+ We Prohibit Deceiving or Misleading Others.
48
+ This includes using Stability Technology to facilitate:
49
+ misinformation or disinformation.
50
+ impersonation of others without consent or legal right, including defamatory content.
51
+ misleading end users about the nature of outputs from Stability AI Technology (such as pretending it was made by a human) or failing to appropriately disclose when someone is interacting with AI where it is not apparent.
52
+ content that may disrupt democratic or judicial processes, including content that discourages participation in elections.
sd35/LICENSE ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ STABILITY AI COMMUNITY LICENSE AGREEMENT
2
+ Last Updated: July 5, 2024
3
+
4
+
5
+ I. INTRODUCTION
6
+
7
+ This Agreement applies to any individual person or entity ("You", "Your" or "Licensee") that uses or distributes any portion or element of the Stability AI Materials or Derivative Works thereof for any Research & Non-Commercial or Commercial purpose. Capitalized terms not otherwise defined herein are defined in Section V below.
8
+
9
+
10
+ This Agreement is intended to allow research, non-commercial, and limited commercial uses of the Models free of charge. In order to ensure that certain limited commercial uses of the Models continue to be allowed, this Agreement preserves free access to the Models for people or organizations generating annual revenue of less than US $1,000,000 (or local currency equivalent).
11
+
12
+
13
+ By clicking "I Accept" or by using or distributing or using any portion or element of the Stability Materials or Derivative Works, You agree that You have read, understood and are bound by the terms of this Agreement. If You are acting on behalf of a company, organization or other entity, then "You" includes you and that entity, and You agree that You: (i) are an authorized representative of such entity with the authority to bind such entity to this Agreement, and (ii) You agree to the terms of this Agreement on that entity's behalf.
14
+
15
+ II. RESEARCH & NON-COMMERCIAL USE LICENSE
16
+
17
+ Subject to the terms of this Agreement, Stability AI grants You a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable and royalty-free limited license under Stability AI's intellectual property or other rights owned by Stability AI embodied in the Stability AI Materials to use, reproduce, distribute, and create Derivative Works of, and make modifications to, the Stability AI Materials for any Research or Non-Commercial Purpose. "Research Purpose" means academic or scientific advancement, and in each case, is not primarily intended for commercial advantage or monetary compensation to You or others. "Non-Commercial Purpose" means any purpose other than a Research Purpose that is not primarily intended for commercial advantage or monetary compensation to You or others, such as personal use (i.e., hobbyist) or evaluation and testing.
18
+
19
+ III. COMMERCIAL USE LICENSE
20
+
21
+ Subject to the terms of this Agreement (including the remainder of this Section III), Stability AI grants You a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable and royalty-free limited license under Stability AI's intellectual property or other rights owned by Stability AI embodied in the Stability AI Materials to use, reproduce, distribute, and create Derivative Works of, and make modifications to, the Stability AI Materials for any Commercial Purpose. "Commercial Purpose" means any purpose other than a Research Purpose or Non-Commercial Purpose that is primarily intended for commercial advantage or monetary compensation to You or others, including but not limited to, (i) creating, modifying, or distributing Your product or service, including via a hosted service or application programming interface, and (ii) for Your business's or organization's internal operations.
22
+ If You are using or distributing the Stability AI Materials for a Commercial Purpose, You must register with Stability AI at (https://stability.ai/community-license). If at any time You or Your Affiliate(s), either individually or in aggregate, generate more than USD $1,000,000 in annual revenue (or the equivalent thereof in Your local currency), regardless of whether that revenue is generated directly or indirectly from the Stability AI Materials or Derivative Works, any licenses granted to You under this Agreement shall terminate as of such date. You must request a license from Stability AI at (https://stability.ai/enterprise) , which Stability AI may grant to You in its sole discretion. If you receive Stability AI Materials, or any Derivative Works thereof, from a Licensee as part of an integrated end user product, then Section III of this Agreement will not apply to you.
23
+
24
+ IV. GENERAL TERMS
25
+
26
+ Your Research, Non-Commercial, and Commercial License(s) under this Agreement are subject to the following terms.
27
+ a. Distribution & Attribution. If You distribute or make available the Stability AI Materials or a Derivative Work to a third party, or a product or service that uses any portion of them, You shall: (i) provide a copy of this Agreement to that third party, (ii) retain the following attribution notice within a "Notice" text file distributed as a part of such copies: "This Stability AI Model is licensed under the Stability AI Community License, Copyright © Stability AI Ltd. All Rights Reserved", and (iii) prominently display "Powered by Stability AI" on a related website, user interface, blogpost, about page, or product documentation. If You create a Derivative Work, You may add your own attribution notice(s) to the "Notice" text file included with that Derivative Work, provided that You clearly indicate which attributions apply to the Stability AI Materials and state in the "Notice" text file that You changed the Stability AI Materials and how it was modified.
28
+ b. Use Restrictions. Your use of the Stability AI Materials and Derivative Works, including any output or results of the Stability AI Materials or Derivative Works, must comply with applicable laws and regulations (including Trade Control Laws and equivalent regulations) and adhere to the Documentation and Stability AI's AUP, which is hereby incorporated by reference. Furthermore, You will not use the Stability AI Materials or Derivative Works, or any output or results of the Stability AI Materials or Derivative Works, to create or improve any foundational generative AI model (excluding the Models or Derivative Works).
29
+ c. Intellectual Property.
30
+ (i) Trademark License. No trademark licenses are granted under this Agreement, and in connection with the Stability AI Materials or Derivative Works, You may not use any name or mark owned by or associated with Stability AI or any of its Affiliates, except as required under Section IV(a) herein.
31
+ (ii) Ownership of Derivative Works. As between You and Stability AI, You are the owner of Derivative Works You create, subject to Stability AI's ownership of the Stability AI Materials and any Derivative Works made by or for Stability AI.
32
+ (iii) Ownership of Outputs. As between You and Stability AI, You own any outputs generated from the Models or Derivative Works to the extent permitted by applicable law.
33
+ (iv) Disputes. If You or Your Affiliate(s) institute litigation or other proceedings against Stability AI (including a cross-claim or counterclaim in a lawsuit) alleging that the Stability AI Materials, Derivative Works or associated outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by You, then any licenses granted to You under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Stability AI from and against any claim by any third party arising out of or related to Your use or distribution of the Stability AI Materials or Derivative Works in violation of this Agreement.
34
+ (v) Feedback. From time to time, You may provide Stability AI with verbal and/or written suggestions, comments or other feedback related to Stability AI's existing or prospective technology, products or services (collectively, "Feedback"). You are not obligated to provide Stability AI with Feedback, but to the extent that You do, You hereby grant Stability AI a perpetual, irrevocable, royalty-free, fully-paid, sub-licensable, transferable, non-exclusive, worldwide right and license to exploit the Feedback in any manner without restriction. Your Feedback is provided "AS IS" and You make no warranties whatsoever about any Feedback.
35
+ d. Disclaimer Of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE STABILITY AI MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OR LAWFULNESS OF USING OR REDISTRIBUTING THE STABILITY AI MATERIALS, DERIVATIVE WORKS OR ANY OUTPUT OR RESULTS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE STABILITY AI MATERIALS, DERIVATIVE WORKS AND ANY OUTPUT AND RESULTS.
36
+ e. Limitation Of Liability. IN NO EVENT WILL STABILITY AI OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF STABILITY AI OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
37
+ f. Term And Termination. The term of this Agreement will commence upon Your acceptance of this Agreement or access to the Stability AI Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Stability AI may terminate this Agreement if You are in breach of any term or condition of this Agreement. Upon termination of this Agreement, You shall delete and cease use of any Stability AI Materials or Derivative Works. Section IV(d), (e), and (g) shall survive the termination of this Agreement.
38
+ g. Governing Law. This Agreement will be governed by and constructed in accordance with the laws of the United States and the State of California without regard to choice of law principles, and the UN Convention on Contracts for International Sale of Goods does not apply to this Agreement.
39
+
40
+ V. DEFINITIONS
41
+
42
+ "Affiliate(s)" means any entity that directly or indirectly controls, is controlled by, or is under common control with the subject entity; for purposes of this definition, "control" means direct or indirect ownership or control of more than 50% of the voting interests of the subject entity.
43
+ "Agreement" means this Stability AI Community License Agreement.
44
+ "AUP" means the Stability AI Acceptable Use Policy available at https://stability.ai/use-policy, as may be updated from time to time.
45
+ "Derivative Work(s)" means (a) any derivative work of the Stability AI Materials as recognized by U.S. copyright laws and (b) any modifications to a Model, and any other model created which is based on or derived from the Model or the Model's output, including"fine tune" and "low-rank adaptation" models derived from a Model or a Model's output, but do not include the output of any Model.
46
+ "Documentation" means any specifications, manuals, documentation, and other written information provided by Stability AI related to the Software or Models.
47
+ "Model(s)" means, collectively, Stability AI's proprietary models and algorithms, including machine-learning models, trained model weights and other elements of the foregoing listed on Stability's Core Models Webpage available at, https://stability.ai/core-models, as may be updated from time to time.
48
+ "Stability AI" or "we" means Stability AI Ltd. and its Affiliates.
49
+ "Software" means Stability AI's proprietary software made available under this Agreement now or in the future.
50
+ "Stability AI Materials" means, collectively, Stability's proprietary Models, Software and Documentation (and any portion or combination thereof) made available under this Agreement.
51
+ "Trade Control Laws" means any applicable U.S. and non-U.S. export control and trade sanctions laws and regulations.
sd35/Notice ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ This Stability AI Model is licensed under the Stability AI Community License, Copyright © Stability AI Ltd. All Rights Reserved
2
+
3
+ Powered by Stability AI
4
+
5
+ PACT modifications — Copyright © 2026 PACT authors.
6
+ We changed Stable Diffusion 3.5 Medium by training rank/alpha-384 LoRA on
7
+ image/context attention projections and new destination-time/guidance
8
+ conditioning. The released model predicts interval-average velocity from
9
+ cached guided teacher trajectories. flowmap_lora.safetensors contains the
10
+ learned LoRA changes; flowmap_conditioner.safetensors contains the new
11
+ conditioner weights. The selected adapter uses raw training step 190,000.
12
+ These modifications are distributed under the accompanying Stability AI
13
+ Community License, subject to its conditions. This is not an official
14
+ Stability AI release.
15
+
16
+ To the extent the PACT authors control rights in flowmap_lora.safetensors
17
+ and flowmap_conditioner.safetensors, they permit use, reproduction,
18
+ modification and redistribution of those adapter modifications subject to
19
+ the accompanying Stability AI Community License Agreement. This permission
20
+ does not replace Stability AI's grant or confer rights in third-party models,
21
+ datasets or outputs. It does not license the separate PACT source code.
22
+ Retain this notice and the complete agreement with redistributed copies.
23
+
24
+ Base: stabilityai/stable-diffusion-3.5-medium
25
+ Revision: b940f670f0eda2d07fbb75229e779da1ad11eb80
26
+ License source: https://huggingface.co/stabilityai/stable-diffusion-3.5-medium/blob/b940f670f0eda2d07fbb75229e779da1ad11eb80/LICENSE.md
27
+ AUP source: https://stability.ai/use-policy
28
+ ACCEPTABLE_USE_POLICY.txt is a readable rendering of the policy effective
29
+ September 30, 2026, retrieved October 12, 2026. The authoritative policy,
30
+ including subsequent updates as incorporated by the license, governs use.
31
+
32
+ Training-source attribution (data not redistributed):
33
+ - DiffusionDB — Zijie J. Wang, Evan Montoya, David Munechika, Haoyang Yang,
34
+ Benjamin Hoover, and Duen Horng Chau, as cited in the pinned dataset card:
35
+ https://huggingface.co/datasets/poloclub/diffusiondb
36
+ Release snapshot: fb620fbe49fa4420e0734bd9c0df11f51176b61f
37
+ Declared terms: CC0-1.0.
38
+ - Pick-a-Pic — the Pick-a-Pic authors and dataset contributors.
39
+ https://huggingface.co/datasets/pickapic-anonymous/pickapic_v1
40
+ Release snapshot: 8e69b58d729339cb7771133c70ee18b527fd4ae3
41
+ No license field was declared in the inspected source card; do not infer
42
+ a permissive license from this adapter release.
43
+ - The additional DiffusionDB prompt selection was obtained from DP-DMD:
44
+ https://github.com/Multimedia-Analytics-Laboratory/dpdmd
45
+ Source revision: 9c7bab8ce99bb7157ad62762ad05f340b95cb8b1
46
+ Earlier prompt preparation filtered, normalized, deduplicated, embedded and
47
+ clustered raw prompts; it did not enhance them. The later DP-DMD selection
48
+ retains repeated physical prompt rows. The exact historical earlier 50k manifest
49
+ is unavailable. Release snapshots do not imply byte-identical recovery of
50
+ all historical prompt data. Consult source cards for citation and rights.
sd35/README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ license_name: stabilityai-community-license
6
+ license_link: LICENSE
7
+ base_model: stabilityai/stable-diffusion-3.5-medium
8
+ base_model_relation: adapter
9
+ library_name: pytorch
10
+ pipeline_tag: text-to-image
11
+ tags: [pact, flow-matching, distillation, lora]
12
+ ---
13
+
14
+ # PACT for Stable Diffusion 3.5 Medium
15
+
16
+ **Powered by Stability AI.** Inference adapter for **PACT: Preserving Accurate Continuous Trajectories via Flow-Map Distillation**, released by the PACT authors. This is the **raw step-190,000** paper checkpoint, with rank/alpha 384 attention LoRA and the learned destination-time/guidance conditioner. It is a modified derivative of Stable Diffusion 3.5 Medium, not an official Stability AI release.
17
+
18
+ ## Base and contents
19
+
20
+ Download the base separately after accepting its access terms:
21
+ [`stabilityai/stable-diffusion-3.5-medium`](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium), revision `b940f670f0eda2d07fbb75229e779da1ad11eb80`.
22
+
23
+ `flowmap_lora.safetensors` and `flowmap_conditioner.safetensors` are both required. `flowmap_manifest.json` records architecture and tensor-file hashes; `training_config.yaml` preserves the portable final recipe; `paper_schedules.json` stores the exact schedules. All 390 trainable tensors passed bitwise zero/reload export verification, recorded in `export_validation.json`. `SHA256SUMS` covers the complete published folder.
24
+
25
+ The main setting is **four student evaluations at 1024×1024, learned CFG 4.5, and no additional external CFG**. Use the PACT loader; generic LoRA loading is insufficient.
26
+
27
+ ## Usage
28
+
29
+ Install the accompanying PACT `SD35-FlowBank-Distillation/` source tree following its README (source publication is pending), and authenticate for the gated base with `hf auth login`. Training is unnecessary. From that source directory:
30
+
31
+ ```bash
32
+ export SD35_DATA=/absolute/path/to/pact-sd35-data
33
+ hf download AhmadRH/PACT --include 'sd35/*' --local-dir "$SD35_DATA/pact-checkpoints"
34
+ export SD35_ADAPTER="$SD35_DATA/pact-checkpoints/sd35"
35
+ python scripts/infer.py \
36
+ --checkpoint "$SD35_ADAPTER" \
37
+ --prompt 'A wooden rowboat on a turquoise lake beneath snowy peaks.' \
38
+ --seed 42 --nfe 4 --output "$SD35_DATA/rowboat.png"
39
+ ```
40
+
41
+ The source README documents CUDA/PyTorch and pinned Diffusers/Transformers dependencies. Use a separate environment for this image implementation and the LTX implementation. These adapters do not contain the base weights or optimizer/RNG/EMA state.
42
+
43
+ ## Training data and method
44
+
45
+ The final method uses raw prompts from [DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb) (CC0-1.0) and [Pick-a-Pic v1](https://huggingface.co/datasets/pickapic-anonymous/pickapic_v1) (no license field in the inspected card). Prompts are filtered, clustered and selected without enhancement. A frozen teacher constructs 28-step trajectories; PACT trains destination/guidance conditioning and rank-384 attention LoRA on interval-average velocity plus endpoint MSE. See [Notice](Notice) for pinned source attribution.
46
+
47
+ The exact earlier 50k prompt manifest was not preserved in the release. The code reconstructs the selection method without claiming byte-identical historical data. Training data are not redistributed here.
48
+
49
+ ## Evaluation and limitations
50
+
51
+ The raw paper adapter passed release inference and visual checks. A separate 500-update fresh-training smoke test reduced mean loss from 0.124115 to 0.013757. These small checks establish execution and loading; they do not reproduce the full paper's image-quality, diversity or teacher-similarity benchmarks. The code provides the paper evaluation entrypoints with their original metric backbones.
52
+
53
+ The adapter inherits base-model biases and limitations. Few-step outputs may differ from the teacher in fine detail, color and composition. Research and evaluation are the intended uses; users must comply with upstream restrictions and applicable rights in inputs/outputs.
54
+
55
+ ## License
56
+
57
+ The adapter is distributed under the included [Stability AI Community License](LICENSE), dated July 5, 2024, with its incorporated [Acceptable Use Policy](https://stability.ai/use-policy). The license covers LoRA derivatives, requires its attribution and notices on redistribution, and imposes commercial registration/revenue conditions. Read the full terms; this release does not waive them or clear dataset rights. [Notice](Notice) includes the PACT authors' permission for the adapter modifications whose rights they control, subject to that agreement. The separate original PACT code license is not assigned by this model card.
sd35/SHA256SUMS ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ 667673ea095deb42d7c143b50c61ff7ef941179fc48baf53c1abe41a8a6699da ACCEPTABLE_USE_POLICY.txt
2
+ 9b60b9d0411f54d4087484ad4acf28ca6fd9f6a84c30ddacb04de2c623254749 LICENSE
3
+ 212c56d85f751980f710fbe199fe10a047d50782ab3fa3844a30644a339db158 Notice
4
+ e4ad455da3df7e54cd28835034a25e24570b59e0e08ae911f2d5cf99a935b78a README.md
5
+ 1ebd2ef7643e6bbfd8c3e123c2f3f262c986b2a6f9e5fac0e107902bdeb3f60b export_validation.json
6
+ 84bab29fdcf94e206a305015c78c2adbfa269a92af01c1aa9a68cb6815323ba9 flowmap_conditioner.safetensors
7
+ abd3ccf966654fabe8aadc0c093414c6b9143d2b31840cf84c205b4e6f26c1e8 flowmap_lora.safetensors
8
+ 17fe4f9f35c0dbf60873fa5349b4f94d1ff58674c0c5e807908dc4c900bf7e0f flowmap_manifest.json
9
+ 64913aa6ba554a3a644dee9e9a6c5a065efc4c1f832c64302ef3f111f522ce38 paper_schedules.json
10
+ 9f8c529eb81803154971047ac058a15312c069104ff7f4893a398137ceda9f65 training_config.yaml
sd35/export_validation.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "passed": true,
3
+ "source_checkpoint_sha256": "6041855f7b2c871044802df13486bbd290d8c256b2c3f6ed4af2281a94671141",
4
+ "trainable_tensors_bitwise_verified": 390,
5
+ "format": "flowmap_manifest.json + flowmap_lora.safetensors + flowmap_conditioner.safetensors",
6
+ "weights": "raw"
7
+ }
sd35/flowmap_conditioner.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84bab29fdcf94e206a305015c78c2adbfa269a92af01c1aa9a68cb6815323ba9
3
+ size 20473304
sd35/flowmap_lora.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abd3ccf966654fabe8aadc0c093414c6b9143d2b31840cf84c205b4e6f26c1e8
3
+ size 450672752
sd35/flowmap_manifest.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_model": {
3
+ "fine_tuning_mode": "lora",
4
+ "freeze_text_encoders": true,
5
+ "freeze_transformer_base": true,
6
+ "freeze_vae": true,
7
+ "model_id": "stabilityai/stable-diffusion-3.5-medium",
8
+ "revision": "b940f670f0eda2d07fbb75229e779da1ad11eb80",
9
+ "training_load_mode": "transformer_only"
10
+ },
11
+ "base_revision": "b940f670f0eda2d07fbb75229e779da1ad11eb80",
12
+ "checkpoint_weights": "raw",
13
+ "destination_conditioner": {
14
+ "enabled": true,
15
+ "guidance": {
16
+ "enabled": true,
17
+ "reference_scale": 1.0,
18
+ "scale_normalizer": 1.0
19
+ },
20
+ "zero_initialized": true
21
+ },
22
+ "files": {
23
+ "conditioner": {
24
+ "name": "flowmap_conditioner.safetensors",
25
+ "sha256": "84bab29fdcf94e206a305015c78c2adbfa269a92af01c1aa9a68cb6815323ba9"
26
+ },
27
+ "lora": {
28
+ "name": "flowmap_lora.safetensors",
29
+ "sha256": "abd3ccf966654fabe8aadc0c093414c6b9143d2b31840cf84c205b4e6f26c1e8"
30
+ }
31
+ },
32
+ "format_version": 1,
33
+ "lora": {
34
+ "alpha": 384,
35
+ "dropout": 0.0,
36
+ "enabled": true,
37
+ "rank": 384,
38
+ "target_policy": "sd35_joint_attention_image_and_context_projections"
39
+ },
40
+ "lora_serialization": "peft",
41
+ "method": "PACT",
42
+ "output_parameterization": "average_velocity",
43
+ "sigma_convention": "post_shift_normalized",
44
+ "source_checkpoint_sha256": "6041855f7b2c871044802df13486bbd290d8c256b2c3f6ed4af2281a94671141"
45
+ }
sd35/paper_schedules.json ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema_version": "sd35-flowmap-paper-schedules-v1",
3
+ "coordinate": "actual_post_shift_normalized_sigma",
4
+ "best_by_nfe": {
5
+ "4": {
6
+ "name": "high_noise_power_2.75",
7
+ "sigmas": [
8
+ 1.0,
9
+ 0.977902889251709,
10
+ 0.8513491153717041,
11
+ 0.5466657876968384,
12
+ 0.0
13
+ ]
14
+ },
15
+ "6": {
16
+ "name": "high_noise_power_2.5",
17
+ "sigmas": [
18
+ 1.0,
19
+ 0.9886597394943237,
20
+ 0.9358499646186829,
21
+ 0.8232232928276062,
22
+ 0.6371126770973206,
23
+ 0.36606186628341675,
24
+ 0.0
25
+ ]
26
+ },
27
+ "8": {
28
+ "name": "high_noise_power_2",
29
+ "sigmas": [
30
+ 1.0,
31
+ 0.984375,
32
+ 0.9375,
33
+ 0.859375,
34
+ 0.75,
35
+ 0.609375,
36
+ 0.4375,
37
+ 0.234375,
38
+ 0.0
39
+ ]
40
+ },
41
+ "12": {
42
+ "name": "high_noise_power_1.5",
43
+ "sigmas": [
44
+ 1.0,
45
+ 0.9759437441825867,
46
+ 0.9319586157798767,
47
+ 0.875,
48
+ 0.8075498938560486,
49
+ 0.7310428023338318,
50
+ 0.6464465856552124,
51
+ 0.5544718503952026,
52
+ 0.45566898584365845,
53
+ 0.35048097372055054,
54
+ 0.239274263381958,
55
+ 0.12235844135284424,
56
+ 0.0
57
+ ]
58
+ },
59
+ "16": {
60
+ "name": "native",
61
+ "sigmas": [
62
+ 1.0,
63
+ 0.9768170714378357,
64
+ 0.9513797760009766,
65
+ 0.9233425259590149,
66
+ 0.8922852873802185,
67
+ 0.857692301273346,
68
+ 0.8189233541488647,
69
+ 0.7751736044883728,
70
+ 0.72541743516922,
71
+ 0.6683266758918762,
72
+ 0.602150559425354,
73
+ 0.5245327353477478,
74
+ 0.4322250783443451,
75
+ 0.32062146067619324,
76
+ 0.18296529352664948,
77
+ 0.008928571827709675,
78
+ 0.0
79
+ ]
80
+ },
81
+ "20": {
82
+ "name": "linear",
83
+ "sigmas": [
84
+ 1.0,
85
+ 0.949999988079071,
86
+ 0.8999999761581421,
87
+ 0.8500000238418579,
88
+ 0.800000011920929,
89
+ 0.75,
90
+ 0.699999988079071,
91
+ 0.6499999761581421,
92
+ 0.6000000238418579,
93
+ 0.550000011920929,
94
+ 0.5,
95
+ 0.45000001788139343,
96
+ 0.4000000059604645,
97
+ 0.3499999940395355,
98
+ 0.30000001192092896,
99
+ 0.25,
100
+ 0.20000000298023224,
101
+ 0.15000000596046448,
102
+ 0.10000000149011612,
103
+ 0.05000000074505806,
104
+ 0.0
105
+ ]
106
+ }
107
+ }
108
+ }
sd35/training_config.yaml ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Paper scientific recipe; default launcher: four GPUs, accumulation four.
2
+ base_model:
3
+ model_id: stabilityai/stable-diffusion-3.5-medium
4
+ revision: b940f670f0eda2d07fbb75229e779da1ad11eb80
5
+ freeze_transformer_base: true
6
+ freeze_text_encoders: true
7
+ freeze_vae: true
8
+ training_load_mode: transformer_only
9
+ fine_tuning_mode: lora
10
+ data:
11
+ bank_root: data/bank-packed
12
+ schema_version: sd35-flowbank-v1
13
+ split: train
14
+ validation_split: validation
15
+ pairs_per_trajectory_load: 8
16
+ min_index_gap: 1
17
+ max_index_gap: null
18
+ min_delta_sigma: 0.05
19
+ jump_sampling:
20
+ mode: delta_sigma_bucket
21
+ boundaries:
22
+ - 0.0
23
+ - 0.25
24
+ - 0.5
25
+ - 0.75
26
+ - 1.0
27
+ weights:
28
+ - 0.3
29
+ - 0.3
30
+ - 0.3
31
+ - 0.1
32
+ pair_sampling_seed: 35101
33
+ streaming:
34
+ enabled: true
35
+ num_workers: 2
36
+ prefetch_factor: 4
37
+ persistent_workers: false
38
+ multiprocessing_context: spawn
39
+ evaluation_pair_count: 128
40
+ verify_checksums: false
41
+ require_validation: true
42
+ model:
43
+ output_parameterization: average_velocity
44
+ sigma_coordinate: actual_post_shift_normalized_sigma
45
+ destination_conditioner:
46
+ enabled: true
47
+ zero_initialized: true
48
+ guidance:
49
+ enabled: true
50
+ reference_scale: 1.0
51
+ scale_normalizer: 1.0
52
+ lora:
53
+ enabled: true
54
+ rank: 384
55
+ alpha: 384
56
+ dropout: 0.0
57
+ target_policy: sd35_joint_attention_image_and_context_projections
58
+ optimization:
59
+ learning_rate: 0.00015
60
+ optimizer: adamw
61
+ weight_decay: 0.01
62
+ max_train_steps: 190000
63
+ stop_steps: 190000
64
+ per_device_batch_size: 2
65
+ gradient_accumulation_steps: 4
66
+ gradient_checkpointing: false
67
+ mixed_precision: bf16
68
+ loss_dtype: fp32
69
+ loss: mse
70
+ velocity_weight: 1.0
71
+ endpoint_weight: 1.0
72
+ grad_clip_norm: 1.0
73
+ seed: 35102
74
+ schedule:
75
+ name: warmup_cosine
76
+ warmup_steps: 500
77
+ horizon_steps: 190000
78
+ min_lr_ratio: 0.0
79
+ torch_compile: false
80
+ torch_compile_mode: reduce-overhead
81
+ torch_compile_backend: inductor
82
+ stream_prefetch_depth: 0
83
+ non_blocking_transfers: true
84
+ validation:
85
+ every_steps: 1000
86
+ jump_buckets: true
87
+ report_velocity_mse: true
88
+ report_endpoint_mse: true
89
+ rollout_nfes:
90
+ - 1
91
+ - 2
92
+ - 4
93
+ - 8
94
+ checkpointing:
95
+ output_dir: outputs/training
96
+ every_steps: 5000
97
+ save_trainable_only: true
98
+ ema:
99
+ enabled: false
100
+ decay: 0.9999
101
+ telemetry:
102
+ enabled: false
103
+ required: false
104
+ wandb_project: pact-sd35
105
+ tags:
106
+ - sd35
107
+ - pact
108
+ - paper
109
+ factory: sd35_flowmap.factories:training_factory
110
+ distributed:
111
+ enabled: true
112
+ expected_world_size: 4
113
+ global_batch_size: 32