Add accuracy recovery adapters

accuracy_recovery_adapters/.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

accuracy_recovery_adapters/README.md

@@ -0,0 +1,26 @@
+---
+tags:
+- accuracy recovery adapter
+- AI Toolkit
+---
+# Accuracy Recovery Adapters
+
+This repo contains various accuracy recovery adapters (ARAs) that I have trained, primarialy for use with [AI Toolkit](https://github.com/ostris/ai-toolkit).
+An ARA is a LoRA that is trained via student teacher training with the student being quantized down to a low precision and the teacher having a high precision.
+The goal is to have a side chain LoRA, at bfloat16, that runs parallel to highly quantized layers in a network to compensate for the loss in precision that happens
+when weights are quantized. The training is done on a per layer basis in order to match the parent output as much as possible. 
+
+While this can be used on inference, my primary goal is to make large models finetunable on consumer grade hardware. With the 3bit Qwen Image adapter, it
+is now possible to train a LoRA on top of it, with 1 MP images, on a 24 GB GPU, such as a 3090/4090. 
+
+I have found the sweet spot, at least for [Qwen-Image](https://huggingface.co/Qwen/Qwen-Image), is 3 bit quantization with a rank 16 adapter.
+
+More info, examples, links, training scripts, AI Toolkit example configs, and adapters to some soon.
+
+## License
+
+All adapters inherit the parent model license. Apache 2.0 for Apache 2.0, BFL License for BFL License, etc.
+
+## Qwen-Image 3 bit quantization
+
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/643cb43e6eeb746f5ad81c26/omdVOGwi3H8P83o8d6nKm.jpeg)

accuracy_recovery_adapters/flux1_dev_kontext_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eefe8672057d111ad8dd8367a08da1cceb191b909d98dad33a9a34a8e3c5b56c
+size 173623624

accuracy_recovery_adapters/hidream_i1_full_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b89cbd7c1e22e554d64f062bbbf0bac5378a46d0c95c2b981f7b711914f3215
+size 306532608

accuracy_recovery_adapters/qwen_image_2512_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3b456a335cf60b5ef9a531a40fce5be5f03a69c2667a544794551f4669a54ac
+size 295146200

accuracy_recovery_adapters/qwen_image_2512_torchao_uint4.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f50363dc04c49ef3bedf2bfa975e5a31a8dcaaa9031e99d307df7e6f516363ed
+size 295146200

accuracy_recovery_adapters/qwen_image_edit_2509_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:914e1952f450c809a916fff3613542f124b0a6395c92accc782675024d066c3d
+size 296159384

accuracy_recovery_adapters/qwen_image_edit_2511_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c22d6c4d22e5f80a6099c83e1a98942e27eae134891268b21a5bbee5f598afa8
+size 296159376

accuracy_recovery_adapters/qwen_image_edit_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6592df2b461a4e8fca4919fbff39999015f56fec554ebda805f91af82f9f696a
+size 296159368

accuracy_recovery_adapters/qwen_image_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1791fea0181da84e411c8ed8ff00c00cc5c6cdd32b1a26e2f590ece6e5fa8186
+size 296157792

accuracy_recovery_adapters/wan22_14b_i2v_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1708e008e8ba36e8618b8a6effd62fccbf8431420e4341dc5b56e29ad05d92c1
+size 311804040

accuracy_recovery_adapters/wan22_14b_i2v_torchao_uint4.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5c5cd2d232ff37fbd866d20eac632067f3a6bf5d4eb5b125b6c09479fd851bd
+size 311804040

accuracy_recovery_adapters/wan22_14b_t2i_torchao_uint3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8b9d26ddd919e380ecfffb8d027d20f2f3ee20fbceb19592ac7c233b3d0d30b
+size 311804032

accuracy_recovery_adapters/wan22_14b_t2i_torchao_uint4.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3baad095fc0b56ad88dc5f2c92b12f1a0c4229c13034a9ae9e92c2c6a7ebf74c
+size 311804032