Add Audio8-ASR 0.1B Core ML hybrid package

README.md

@@ -0,0 +1,95 @@
+---
+license: cc-by-nc-4.0
+library_name: coremltools
+pipeline_tag: automatic-speech-recognition
+base_model: Audio8/Audio8-ASR-0.1B
+language:
+  - en
+  - zh
+  - fr
+  - de
+  - ja
+  - ko
+  - yue
+tags:
+  - coreml
+  - apple-silicon
+  - automatic-speech-recognition
+  - multilingual
+  - macos
+---
+
+# Audio8-ASR 0.1B — Core ML for Apple Silicon
+
+A ready-to-use hybrid Core ML conversion of [Audio8-ASR-0.1B](https://huggingface.co/Audio8/Audio8-ASR-0.1B) for fast, private, multilingual speech transcription on Apple Silicon.
+
+This model is integrated into [Hugging Mac](https://github.com/devilyouwei/hugging-mac), where you can run it locally and build macOS voice apps, transcription services, plugins, and agents.
+
+> **Non-commercial license:** the model is provided under CC-BY-NC-4.0 and may not be used commercially without separate permission from the upstream rights holder.
+
+## Architecture
+
+This is an explicitly hybrid runtime rather than a fully Core ML autoregressive model:
+
+- The audio encoder and MLP tower run with Core ML on the Apple Neural Engine.
+- The cached Qwen2 decoder runs with PyTorch MPS on the GPU.
+- A multifunction ML Program provides optimized 5, 10, and 30-second audio buckets.
+- Audio-tower weights use INT8 linear quantization with FP16 compute.
+
+The package includes the Core ML audio tower, decoder and projector weights, processor configuration, and reproducible conversion metadata. The tokenizer is downloaded from the pinned upstream model by Hugging Mac.
+
+## Model details
+
+| Property | Value |
+|---|---|
+| Languages | Chinese, English, French, German, Japanese, Korean, Cantonese |
+| Audio | 16 kHz mono, up to 30 seconds |
+| Parameters | 0.324B end-to-end; 0.104B language model |
+| Core ML target | macOS 15 or later |
+| Package size | 429 MB |
+| Decoding | Greedy autoregressive decoding with KV cache |
+
+## Core ML functions
+
+| Function | `audios` input | `attn_mask` input | `hidden` output |
+|---|---|---|---|
+| `tower_5s` | `1 × 128 × 500` FP32 | `1 × 1 × 65 × 65` FP32 | `65 × 1024` FP16 |
+| `tower_10s` | `1 × 128 × 1000` FP32 | `1 × 1 × 130 × 130` FP32 | `130 × 1024` FP16 |
+| `tower_30s` | `1 × 128 × 3000` FP32 | `1 × 1 × 390 × 390` FP32 | `390 × 1024` FP16 |
+
+Inputs are 128-bin log-Mel features plus an additive attention mask. The default Core ML function is `tower_30s`; Hugging Mac automatically selects the smallest suitable bucket.
+
+## Use with Hugging Mac
+
+Install the ASR dependencies and run the Hugging Mac application:
+
+```bash
+git clone https://github.com/devilyouwei/hugging-mac.git
+cd hugging-mac
+uv sync --all-packages
+uv run hugging-mac-web
+```
+
+Open the Models page, install **Audio8-ASR 0.1B Core ML**, then use Live Transcription or build against the `SpeechTranscription` capability.
+
+For the full runtime API and implementation, see the [Audio8-ASR SDK package](https://github.com/devilyouwei/hugging-mac/tree/main/packages/hugging_mac_sdk/src/hugging_mac_sdk/models/audio8_asr).
+
+## Provenance
+
+- Upstream model: [Audio8/Audio8-ASR-0.1B](https://huggingface.co/Audio8/Audio8-ASR-0.1B)
+- Upstream revision: `8487da63d581fa4fc9b5c60444cb57c3a523d7aa`
+- Conversion format: `audio8-asr-coreml-hybrid`
+- Conversion metadata: included in `audio8-asr-coreml/conversion.json`
+- No upstream remote Python code is included or executed by Hugging Mac.
+
+## Integrity
+
+Directory SHA-256:
+
+```text
+6721e90590aee02b6091ec79e3ea451ae1997399e18fcf451f079535a44fce48
+```
+
+## License and attribution
+
+The converted model retains the upstream **CC-BY-NC-4.0** license, attribution requirements, and non-commercial restriction. Hugging Mac is an independent open-source project and is not affiliated with or endorsed by Audio8 or AutoArk.

audio8-asr-coreml.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f38444a41e7406fb69c3a03c65c5edc8d1451c3014b712696be9cb345238842
+size 429392896

audio8-asr-coreml/audio_tower.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f94745a4fe6efd39dc8c64c8a5c55bce9dfe66429e00a9f34a66775f91f89a4
+size 715194

audio8-asr-coreml/audio_tower.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:130e99fdcb02b4d07896e8471e4b7dd739ad3836f38bee27ffa446a38619865e
+size 220589056

audio8-asr-coreml/audio_tower.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "38964EC0-C564-40C3-A9E1-BAA5B803935C": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "BDC345DC-3D40-4DF7-8F7D-C8A2305F7BDE": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "38964EC0-C564-40C3-A9E1-BAA5B803935C"
+}

audio8-asr-coreml/config.json

@@ -0,0 +1,286 @@
+{
+  "adapter_type": "qwen3_asr_mlp_tower",
+  "architectures": [
+    "ArkasrForConditionalGeneration"
+  ],
+  "attention_dropout": 0.0,
+  "audio_token_id": 151646,
+  "auto_map": {
+    "AutoConfig": "configuration_arkasr.ArkasrConfig",
+    "AutoModelForCausalLM": "modeling_arkasr.ArkasrForConditionalGeneration"
+  },
+  "bos_token_id": 151643,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 512,
+  "initializer_range": 0.02,
+  "intermediate_size": 1408,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_whisper_length": 1500,
+  "max_window_layers": 28,
+  "merge_factor": 4,
+  "mlp_adapter_act": "gelu",
+  "model_type": "arkasr",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 8,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "spec_aug": false,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.3",
+  "use_cache": true,
+  "use_rope": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936,
+  "whisper_config": {
+    "_name_or_path": "openai/whisper-small",
+    "activation_dropout": 0.0,
+    "activation_function": "gelu",
+    "apply_spec_augment": false,
+    "architectures": [
+      "WhisperForConditionalGeneration"
+    ],
+    "attention_dropout": 0.0,
+    "begin_suppress_tokens": [
+      220,
+      50257
+    ],
+    "bos_token_id": 50257,
+    "classifier_proj_size": 256,
+    "d_model": 768,
+    "decoder_attention_heads": 12,
+    "decoder_ffn_dim": 3072,
+    "decoder_layerdrop": 0.0,
+    "decoder_layers": 12,
+    "decoder_start_token_id": 50258,
+    "dropout": 0.0,
+    "dtype": "float32",
+    "encoder_attention_heads": 12,
+    "encoder_ffn_dim": 3072,
+    "encoder_layerdrop": 0.0,
+    "encoder_layers": 12,
+    "eos_token_id": 50257,
+    "forced_decoder_ids": [
+      [
+        1,
+        50259
+      ],
+      [
+        2,
+        50359
+      ],
+      [
+        3,
+        50363
+      ]
+    ],
+    "init_std": 0.02,
+    "mask_feature_length": 10,
+    "mask_feature_min_masks": 0,
+    "mask_feature_prob": 0.0,
+    "mask_time_length": 10,
+    "mask_time_min_masks": 2,
+    "mask_time_prob": 0.05,
+    "max_length": 448,
+    "max_source_positions": 1500,
+    "max_target_positions": 448,
+    "median_filter_width": 7,
+    "model_type": "whisper",
+    "num_hidden_layers": 12,
+    "num_mel_bins": 80,
+    "pad_token_id": 50257,
+    "scale_embedding": false,
+    "suppress_tokens": [
+      1,
+      2,
+      7,
+      8,
+      9,
+      10,
+      14,
+      25,
+      26,
+      27,
+      28,
+      29,
+      31,
+      58,
+      59,
+      60,
+      61,
+      62,
+      63,
+      90,
+      91,
+      92,
+      93,
+      359,
+      503,
+      522,
+      542,
+      873,
+      893,
+      902,
+      918,
+      922,
+      931,
+      1350,
+      1853,
+      1982,
+      2460,
+      2627,
+      3246,
+      3253,
+      3268,
+      3536,
+      3846,
+      3961,
+      4183,
+      4667,
+      6585,
+      6647,
+      7273,
+      9061,
+      9383,
+      10428,
+      10929,
+      11938,
+      12033,
+      12331,
+      12562,
+      13793,
+      14157,
+      14635,
+      15265,
+      15618,
+      16553,
+      16604,
+      18362,
+      18956,
+      20075,
+      21675,
+      22520,
+      26130,
+      26161,
+      26435,
+      28279,
+      29464,
+      31650,
+      32302,
+      32470,
+      36865,
+      42863,
+      47425,
+      49870,
+      50254,
+      50258,
+      50360,
+      50361,
+      50362
+    ],
+    "use_cache": true,
+    "use_weighted_layer_sum": false,
+    "vocab_size": 51865
+  },
+  "qwen3_asr_audio_config": {
+    "_name_or_path": "",
+    "activation_dropout": 0,
+    "activation_function": "gelu",
+    "add_cross_attention": false,
+    "architectures": null,
+    "attention_dropout": 0,
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "conv_chunksize": 500,
+    "cross_attention_hidden_size": null,
+    "d_model": 896,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "downsample_hidden_size": 480,
+    "dropout": 0,
+    "dtype": null,
+    "early_stopping": false,
+    "encoder_attention_heads": 14,
+    "encoder_ffn_dim": 3584,
+    "encoder_layers": 18,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "exponential_decay_length_penalty": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_source_positions": 1500,
+    "min_length": 0,
+    "model_type": "qwen3_asr_audio_encoder",
+    "n_window": 50,
+    "n_window_infer": 800,
+    "no_repeat_ngram_size": 0,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 18,
+    "num_mel_bins": 128,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_dim": 1024,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": null,
+    "prefix": null,
+    "problem_type": null,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "scale_embedding": false,
+    "sep_token_id": null,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torchscript": false,
+    "typical_p": 1.0,
+    "use_bfloat16": false
+  },
+  "qwen3_asr_mlp_tower_layers": 4,
+  "qwen3_asr_mlp_tower_hidden_size": 0,
+  "qwen3_asr_mlp_tower_dropout": 0.0,
+  "audio_backend": "qwen3_asr_mlp_tower",
+  "name_or_path": "audio8-asr-0.1B"
+}

audio8-asr-coreml/conversion.json

@@ -0,0 +1,13 @@
+{
+  "audio_tower": "Core ML",
+  "decoder": "PyTorch MPS",
+  "format": "audio8-asr-coreml-hybrid",
+  "functions": [
+    "tower_5s",
+    "tower_10s",
+    "tower_30s"
+  ],
+  "minimum_deployment_target": "macOS15",
+  "quantize_weights": true,
+  "schema_version": 1
+}

audio8-asr-coreml/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 151643,
+  "eos_token_id": 151645,
+  "pad_token_id": 151643,
+  "transformers_version": "4.57.3"
+}

audio8-asr-coreml/language_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01aae1649efb92ff5c9a2412a5a07cdbeae78b1891e109d94ccaca482c35e6b8
+size 207015496

audio8-asr-coreml/preprocessor_config.json

@@ -0,0 +1,18 @@
+{
+  "auto_map": {
+    "AutoProcessor": "processing_arkasr.ArkasrProcessor"
+  },
+  "chunk_length": 30,
+  "dither": 0.0,
+  "feature_extractor_type": "WhisperFeatureExtractor",
+  "feature_size": 128,
+  "hop_length": 160,
+  "n_fft": 400,
+  "n_samples": 480000,
+  "nb_max_frames": 3000,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "processor_class": "ArkasrProcessor",
+  "return_attention_mask": false,
+  "sampling_rate": 16000
+}

audio8-asr-coreml/processor_config.json

@@ -0,0 +1,9 @@
+{
+  "audio_dtype": "bfloat16",
+  "audio_token": "<|audio|>",
+  "auto_map": {
+    "AutoProcessor": "processing_arkasr.ArkasrProcessor"
+  },
+  "merge_factor": 4,
+  "processor_class": "ArkasrProcessor"
+}

audio8-asr-coreml/projector.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:036bffce0156dc6dfe49d858a753c021f60418c6b2f75c6e140d7d7bf7eef77d
+size 1053984