Upload folder using huggingface_hub

README.md

@@ -0,0 +1,52 @@
+---
+license: apache-2.0
+tags:
+- coreml
+- text-to-image
+- wan
+- dit
+- apple-neural-engine
+base_model: kpsss34/Walkyrie-1.3B-v2.0
+pipeline_tag: text-to-image
+---
+
+# Walkyrie-1.3B-v2.0 Core ML (Unquantized)
+
+This repository contains the first native Apple Silicon Core ML conversion of the **Walkyrie-1.3B-v2.0** core transformer brain, an image model built on top of the Wan 2.1 Diffusion Transformer (DiT) framework.
+
+
+## Repository Layout
+* `Walkyrie_1.3B_v2.0_float16.mlpackage`: The complete 30-block core DiT transformer layer, fully optimized to execute on the Apple Neural Engine (ANE) and Apple Graphics Processor (GPU).
+
+## Implementation & Pipeline Notes
+This asset contains **only the core transformer block**. To build a complete text-to-image pipeline inside a native Swift application, you will need to pair this core package with a text tokenizer and a VAE decoder:
+1. **Text Encoder (UMT5-XXL):** Because compiling an 11B parameter text encoder directly to a static Core ML graph triggers high memory overhead during compilation on 16GB machines, it is highly recommended to run the UMT5 text layer as a raw weight array processed on the CPU/GPU via libraries like `swift-tokenizers` or `mlx-swift`.
+2. **VAE Decoder:** Can be mapped natively via standard Core ML convolutional upsampling to translate the finished transformer latents into viewable RGB images.
+
+---
+
+## 🛠️ Replication & Conversion Process
+
+If you want to re-compile or modify this setup from scratch using the `silicon-alloy` converter or direct `coremltools` tracing, you must bypass several legacy architectural structural mismatches hardcoded into older diffusion conversion scripts. 
+
+The original codebase must be patched with the following workflow modifications:
+
+### 1. Alignment with Modern Diffusers Layer Naming
+The newer Wan 2.1 architecture uses updated property names. Legacy scripts searching for sub-modules will throw immediate `AttributeErrors` unless mapped to the following properties:
+* Change `.transformer_blocks` references to `.blocks`
+* Change `.patch_embed` references to `.patch_embedding`
+
+### 2. Migrating to the Unified Condition Embedder
+Older models process prompt token arrays and timesteps via isolated `.text_embed()` and `.time_embed()` functions. Wan 2.1 consolidates these into a single unified block. 
+* Remove the standalone text and time embedding calls.
+* Call the unified module directly: `temb, timestep_proj, encoder_hidden_states, _ = self.model.condition_embedder(timestep, encoder_hidden_states, None)`
+* Unflatten the resulting projection matrix into its multi-head layout before passing it along: `timestep_proj = timestep_proj.unflatten(1, (6, -1))`
+
+### 3. Spatial Tensor Flattening vs. 5D RoPE Tracking
+The patch embedding layer outputs a 5D spatial video matrix structured as `[Batch, Hidden_Dim, Frames, Height, Width]`. The transformer blocks, however, expect a flattened 3D sequence token vector `[Batch, Sequence_Length, Hidden_Dim]`. Crucially, the Rotary Position Embedding (`.rope`) module still requires the 5D spatial layout to calculate coordinates.
+* **The correct execution sequence:** Pass the 5D spatial matrix into the `.rope()` module *first* to extract your rotary embedding parameters: `image_rotary_emb = self.model.rope(hidden_states_5d)`
+* Flatten and transpose the spatial matrix into sequence tokens *second*, right before launching your core transformer blocks loop: `hidden_states = hidden_states_5d.flatten(2).transpose(1, 2)`
+
+## Acknowledgements
+* Original model weights trained and released by [kpsss34](https://huggingface.co/kpsss34).
+* Core ML compilation achieved via the `silicon-alloy` framework.

Walkyrie_1.3B_v2.0_Float16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8140fd2ac50ccc6dc2320d7ce704079266729e9b8e5e1c5b67bff28f8eb366b1
+size 1086061

Walkyrie_1.3B_v2.0_Float16.mlpackage/Data/com.apple.CoreML/weights/0-weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:874cb4613dacae5dd59108cab44542f9987367fc8c8d3e2ca228f0e2f655bbd3
+size 1444858944

Walkyrie_1.3B_v2.0_Float16.mlpackage/Data/com.apple.CoreML/weights/1-weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0143af7b5d2bc14bbcf50cad1cb0c6728aea4aef425511eafc2f8164ad058580
+size 1393706240

Walkyrie_1.3B_v2.0_Float16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "7F0A0DF2-B8B0-4AFB-A834-C6BB686174A4": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "ED98EB8D-7B7E-4D61-9248-72B31AEDE7A9": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "ED98EB8D-7B7E-4D61-9248-72B31AEDE7A9"
+}

text_encoder/config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "UMT5EncoderModel"
+  ],
+  "classifier_dropout": 0.0,
+  "d_ff": 10240,
+  "d_kv": 64,
+  "d_model": 4096,
+  "decoder_start_token_id": 0,
+  "dense_act_fn": "gelu_new",
+  "dropout_rate": 0.1,
+  "dtype": "float32",
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "initializer_factor": 1.0,
+  "is_decoder": false,
+  "is_encoder_decoder": true,
+  "is_gated_act": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "umt5",
+  "num_decoder_layers": 24,
+  "num_heads": 64,
+  "num_layers": 1,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_max_distance": 128,
+  "relative_attention_num_buckets": 32,
+  "scalable_attention": true,
+  "tie_word_embeddings": true,
+  "tokenizer_class": "T5Tokenizer",
+  "transformers_version": "5.8.0",
+  "use_cache": true,
+  "vocab_size": 256384
+}

text_encoder/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8acd5f1bbd6d5872c3614f05a494bfe9532aaeebc9d94e8a3d100e9bf5d57296
+size 2486203808

vae/config.json

@@ -0,0 +1,64 @@
+{
+  "_class_name": "AutoencoderKLWan",
+  "_diffusers_version": "0.39.0.dev0",
+  "_name_or_path": "kpsss34/Walkyrie-1.3B-v1.0",
+  "attn_scales": [],
+  "base_dim": 96,
+  "decoder_base_dim": null,
+  "dim_mult": [
+    1,
+    2,
+    4,
+    4
+  ],
+  "dropout": 0.0,
+  "in_channels": 3,
+  "is_residual": false,
+  "latents_mean": [
+    -0.7571,
+    -0.7089,
+    -0.9113,
+    0.1075,
+    -0.1745,
+    0.9653,
+    -0.1517,
+    1.5508,
+    0.4134,
+    -0.0715,
+    0.5517,
+    -0.3632,
+    -0.1922,
+    -0.9497,
+    0.2503,
+    -0.2921
+  ],
+  "latents_std": [
+    2.8184,
+    1.4541,
+    2.3275,
+    2.6558,
+    1.2196,
+    1.7708,
+    2.6052,
+    2.0743,
+    3.2687,
+    2.1526,
+    2.8652,
+    1.5579,
+    1.6382,
+    1.1253,
+    2.8251,
+    1.916
+  ],
+  "num_res_blocks": 2,
+  "out_channels": 3,
+  "patch_size": null,
+  "scale_factor_spatial": 8,
+  "scale_factor_temporal": 4,
+  "temperal_downsample": [
+    false,
+    true,
+    true
+  ],
+  "z_dim": 16
+}

vae/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8da83431220b1e9e55e04f3f84a6d5974d1de614b24fad99a3d0c399b0efd70a
+size 507591892