Add files using upload-large-folder tool

README.md

@@ -0,0 +1,38 @@
+---
+base_model: oceanicity/Qwen3-4B-Instruct-2507
+library_name: coreml
+tags:
+- text-generation
+- coreml
+- apple-silicon
+- 8-bit
+- quantized
+- qwen
+---
+
+# Qwen3-4B-Instruct - CoreML (8-Bit Quantised)
+
+This is an 8-bit quantised CoreML conversion of the [oceanicity/Qwen3-4B-Instruct-2507](https://huggingface.co/oceanicity/Qwen3-4B-Instruct-2507) model. It has been heavily optimised for fast, efficient, and low-memory inference on Apple Silicon using the Apple Neural Engine (ANE).
+
+Conversion and quantisation work was performed using a customised version of [0seba's coremlmodels tool](https://github.com/0seba/coremlmodels).
+
+## Model Details
+- **Architecture:** Qwen3 (4 Billion Parameters)
+- **Precision:** 8-bit Weights (Linear Symmetric, Per-Channel) / 16-bit Activations
+- **Context Length:** 8,192 Tokens (KV Cache)
+- **Format:** CoreML `.mlpackage` chunks
+
+## Optimisations Applied
+- **Linear-to-Conv2d Patching:** Transformer linear layers were patched into 1x1 convolutions to better align with the Neural Engine backend.
+- **RMSNorm Fusion:** Layer normalisation layers were fused using CoreMLTools graph passes to prevent FP16 overflow.
+- **Chunking:** The model was split into 8 chunks to safely bypass the Neural Engine's hardware memory limits per segment.
+- **Vocabulary Chunking:** The massive LM head was exported as a standalone chunked model to bypass the ~16,384 dimension limit on Apple Silicon.
+- **Pre-computed Position Embeddings:** RoPE embeddings were computed statically during tracing to avoid precision loss and runtime math overhead.
+
+## Files Included
+- `chunk_0.mlpackage` through `chunk_7.mlpackage`: The core transformer layers.
+- `lm_head.mlpackage`: The chunked vocabulary output head.
+- `embeddings.npy`: The standalone token embedding weights.
+
+## Usage
+This model is ready to be used in CoreML inference pipelines that support multi-chunked stateful transformers. Ensure that your inference engine stitches the chunks together sequentially and routes the KV cache states appropriately.

chunk_0.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_0.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:255d391aaaf105393b676ebbbcda59a43df5167018850879a821abf8447ca1a9
+size 507276352

chunk_0.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "3C4C4B24-4AF8-4585-9C1C-1FAF868F6B73": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "4516D45A-F658-4BD4-9011-E4E1AB57D907": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "4516D45A-F658-4BD4-9011-E4E1AB57D907"
+}

chunk_1.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_1.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:093e67b8f6a6384b3c7fe1050ba65eff6473308ba7fb65aaa8fcf695ae0d15a9
+size 507276352

chunk_1.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "2DED1C0D-D0AF-4295-B9B4-465E71FA03ED": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "9F73F851-3EB1-43D6-A746-CD0549B07C95": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "2DED1C0D-D0AF-4295-B9B4-465E71FA03ED"
+}

chunk_2.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_2.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4950dff882094e67d5d310b58bd37b9d74eb542142e20ebab6e889ade934707
+size 507276352

chunk_2.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "041B62D5-F52F-49BE-B3F3-1013EBE4AA88": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "C1697977-BD4C-4505-BB62-435B3CB3A6DC": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "C1697977-BD4C-4505-BB62-435B3CB3A6DC"
+}

chunk_3.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_3.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c5ec16b0114f82e4aec89e7885357955eb5326b6fef8095220a2f281db7b022
+size 507276352

chunk_3.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "796D866D-730A-4FDB-BF2E-793D38FA912C": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "EBC31B9E-711A-4636-93E7-C8DF2982D06C": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "EBC31B9E-711A-4636-93E7-C8DF2982D06C"
+}

chunk_4.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_4.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf27297aa5b3a5e993ca70fb61fe071d2524da271216b8406f8e9766be9c9218
+size 507276352

chunk_4.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "33FDED6E-DFB7-4361-9EA6-6DBB90E15854": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "93A3AB51-6C0B-4EC2-82DE-321E98959477": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "93A3AB51-6C0B-4EC2-82DE-321E98959477"
+}

chunk_5.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_5.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c81e21b64a3ccd29a9d167eaca9bda85f0460a4d0bc942e829894f02ae62617
+size 507276352

chunk_5.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "8DADA1E5-0366-448D-A905-F9F278E1D51B": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "EC62FA5D-38DF-4A17-A9CC-2C72695B734C": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "8DADA1E5-0366-448D-A905-F9F278E1D51B"
+}

chunk_6.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72b99cd9ee893ab92b3a2a07af8e32961d81d5aeae2453fa1117a7770d7b96f
+size 234631

chunk_6.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55ba91df21326e7b13f37da146102abcc19402a303fe23c06ebf2e8af499f924
+size 507276352

chunk_6.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "4C357D2C-464D-4675-ABB1-1F9CA995B9FD": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "EED91872-DAF6-49EB-A3AE-89CE8E8E2023": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "4C357D2C-464D-4675-ABB1-1F9CA995B9FD"
+}

chunk_7.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd44c2fa723ef5de11f6b66b07fa1029819491516b3030452ccc7bff862c09d4
+size 53810

chunk_7.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a53675dc8f61b8c3e6f4d2a8236d2140d47ce3604da045a42a284263be0b06f3
+size 103248512

chunk_7.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "019FD19E-3365-40D8-A130-605BCB688D60": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "B5F08179-FEC7-4498-A56C-CD5B5B336697": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "019FD19E-3365-40D8-A130-605BCB688D60"
+}

embeddings.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00e192df9c1acace5c016ba051ede3f4a036a147d1dafef2a5569195c9f2aa34
+size 777912448

lm_head.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dca5ea47e7b4aed28987d6ae50fe358fa981b26c11cb4f9aaf3ec70b4bc9ac2a
+size 65009

lm_head.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfcc26df9482d5bcbf7adca59e48b959b9dfd9957db61e3e52e580a441ba00bd
+size 777913984

lm_head.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "1C815E65-E745-4338-922A-7CB18BFE0BFE": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "1D9BB6FC-0A5D-4CA5-B744-8CFA3CF532D1": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "1D9BB6FC-0A5D-4CA5-B744-8CFA3CF532D1"
+}