Instructions to use basyauma-tech/Haas-ja-v1.0-mlx-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use basyauma-tech/Haas-ja-v1.0-mlx-8bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download basyauma-tech/Haas-ja-v1.0-mlx-8bit --local-dir Haas-ja-v1.0-mlx-8bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Haas-ja-v1.0-mlx-8bit
้ๅ ฌๅผใฎๅคๆ็ฉใงใใ third-intelligence/Haas-ja-v1.0ใฎ็ทๅฝขๅฑคใ8bit้ๅญๅใใMLX็ใงใใThird Intelligence, Inc.ใจใฏๆๆบใปๆฟ่ชใปๅพๆดใฎ้ขไฟใซใใใพใใใ
้ใฟใฏ็ด0.73GBใงใbf16็ใฎ็ด0.59ๅใใใผใฏใกใขใชใฏๆธใใพใใใๆค่จผๆฉใงใฏRTFxไธญๅคฎๅคใงbf16ใใ็ด14%ไฝใ๏ผ้ใใฏใชใใชใ๏ผ็ตๆใงใใใ
ๆฆ่ฆใป็ฐๅข
- ๅ
จ็ทๅฝขๅฑคใ
mlx.nn.quantizeใง8bitใขใใฃใณ้ๅญๅ๏ผใฐใซใผใใตใคใบ64๏ผใ็ณใฟ่พผใฟใปๆญฃ่ฆๅๅฑคใปๅใ่พผใฟใฏbf16ใ็ถญๆใ่ชญใฟ่พผใฟๆใฎscalesใจbiasesใฏใจใณใณใผใใผใงbf16ใไบๆธฌใป็ตๅใใใใฏใผใฏใงfloat32ใ - Apple SiliconใฎMacๅฐ็จใNeMoใปCUDAใปTritonใปtorchไธ่ฆใPython 3.12ใจhaas-ja-localใไฝฟ็จใ
- ๆค่จผ็ฐๅขใฏbf16็ใจๅใApple M5 Max๏ผ128GB๏ผใmacOS 27.2ใmlx 0.32.4ใไปๆฉ็จฎใฏๆชๆธฌๅฎใ
ไฝฟใๆน
bf16็ใจๅใใณใผใใงๅใใพใใhfใใชใใใฐpip install huggingface_hubใงๅฐๅ
ฅใใฆใใ ใใใ
pip install "haas-ja-local[mlx] @ git+https://github.com/basyauma-tech/haas-ja-local"
hf download basyauma-tech/Haas-ja-v1.0-mlx-8bit --local-dir Haas-ja-v1.0-mlx-8bit
from haas_ja_local import load_audio, load_model, transcribe
backend, tokenizer = load_model("Haas-ja-v1.0-mlx-8bit")
result = transcribe(backend, tokenizer, load_audio("audio.wav"))
print(result.text)
haas-ja-local transcribe Haas-ja-v1.0-mlx-8bit audio.wav
haas-ja-local transcribe Haas-ja-v1.0-mlx-8bit audio.wav --json --timestamps
็ฒพๅบฆ
asr-ja-bench v1.1ใbf16็ใจๅใๆ้ ใง่ฉไพกใCER strictใฎๅไฝใฏ%ใๅทฎใฏใใคใณใใงใใbf16ใจใฎๅทฎใฏ่กจใฎไธธใๅคใใ็ฎๅบใใฆใใพใใ
| ใตใใปใใ | ็บ่ฉฑๆฐ | bf16 | 8bit | bf16ใจใฎๅทฎ | ๅ ฌ่กจๅค | ๅ ฌ่กจๅคใจใฎๅทฎ |
|---|---|---|---|---|---|---|
| FLEURS | 650 | 3.65 | 3.68 | +0.03 | 3.65ใ3.67 | +0.01 |
| JVNV | 1,375 | 4.51 | 4.49 | -0.02 | 4.50ใ4.53 | -0.01 |
ๅ ฌ่กจๅคใจใฎๅทฎใฏ็ฏๅฒใฎ่ฟใ็ซฏใใ่จ็ฎใใ็ฏๅฒๅ ใชใ0ใ8bitใฏๅๆ ผๅบๆบใฎๅทฎ0.3ใใคใณใไปฅๅ ใงใใ็ฒพๅบฆ่ชฟๆดใฏใใฆใใพใใใ
4bit็ใฏ็ฒพๅบฆๅบๆบใๅคใใใใๅ ฌ้ใใฆใใพใใใ CER strictใฏFLEURS 3.70%๏ผๅ ฌ่กจๅคใจใฎๅทฎ+0.03ใใคใณใ๏ผใJVNV 5.00%๏ผ+0.47ใใคใณใ๏ผใงใใใ
่ฉณ็ดฐใปๅ็พๆ้ : ้ๅญๅ็ใฎๆค่จผ่จ้ฒใ
ๅฎ่ฃ ใฎ็ขบ่ช
ๅฐใใชไนฑๆฐใขใใซใงๅคๆใป่ชญใฟ่พผใฟใฎๅพๅพฉใใญใผใจๅฝข็ถใ้ๅญๅใใชใๅคใฎใใใไธ่ดใๆค่จผๆธใฟใ8bit็ใฎๆฌๆใฏbf16็ใจFLEURS 88.8%ใปJVNV 84.9%ใฎ็บ่ฉฑใงไธ่ดใใพใใใ้้ๅญๅใใ้ใฟใfloat32ใงๆใคใขใใซใจใฎ็ธๅฏพ่ชคๅทฎใฏ1e-5ๆชๆบใงใใใๅฎใขใใซใจPyTorch๏ผfloat32๏ผใฎใใผใฏใณๅไธ่ดใฏๆชๆค่จผใงใใ
้ๅบฆใจใกใขใช
ไธ่จM5 Maxใใใใ1ใ้ๆฌก่ช่ญใFLEURSใฎๅ ้ ญ3็บ่ฉฑใใฆใฉใผใ ใขใใใจใใฆ้คใใ647็บ่ฉฑใง้่จใใญใฃใใทใฅไธ้ใฏๆขๅฎใฎ1GiBใbf16ใฏๅใๆ้ๅธฏใซๅๆธฌๅฎใใๅคใงใใGB๏ผ10^9ใใคใใ
| ้ ็ฎ | bf16 | 8bit |
|---|---|---|
| RTFx๏ผ1็บ่ฉฑใใจใฎไธญๅคฎๅค๏ผ | 260.1 | 223.3 |
| RTFx๏ผๅ่จ้ณๅฃฐรทๅ่จๅฆ็ๆ้๏ผ | 263.4 | 226.6 |
| 1็บ่ฉฑใฎๅฆ็ๆ้๏ผไธญๅคฎๅค๏ผ | 0.048็ง | 0.057็ง |
| ใใผใฏRSS | 1.57GB | 1.09GB |
| mlxใฎใใผใฏๅฒใๅฝใฆ | 1.79GB | 1.23GB |
| macOSใฎใใผใฏใกใขใชใใใใใชใณใ | 3.20GB | 2.63GB |
ใใกใคใซไธ่ฆง
ใตใคใบใฏใใคใใ
| ใใกใคใซ | ใตใคใบ | SHA-256 |
|---|---|---|
model.safetensors |
727,881,324 | 17ec1d12a67b35303c1bbc4defa3077b03f14665550e45d2749ada12ef73368e |
config.json |
726 | 0b7ad196af39a5120570d31bdcb0c9194e89d216892c9ea8e4d552170e8df6b3 |
tokenizer.model |
279,711 | 9bd7bcdff62e43200001cd4c258d22e475b7b3b1e0f64f8be8e6e046f402f38c |
LICENSE |
11,358 | cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 |
NOTICE |
741 | cb6145131e3cb48e0bb432aa5b949ae72330ef0dddaf1621de9b0cc65ab8b51b |
ๅถ้ไบ้
bf16็ใจๅใๅถ้ไบ้ ใ้ฉ็จใใใพใใ้ๅญๅ็ใงใฏ้ทๆ้้ณๅฃฐใฎๅๅฒ่ช่ญใ็ก้ณใป้้ณๅ ฅๅใJSUTใปTEDxJP-10KใปSpeechBSDใใฐใซใผใใตใคใบ64ไปฅๅคใใกใขใชใฎๅฐใชใๆฉ็จฎใใใใ2ไปฅไธใฏๆชๆธฌๅฎใงใใ
ใฉใคใปใณในใจๅธฐๅฑ
- bf16็ใจๅใApache-2.0๏ผ้ใฟใปๆจ่ซใณใผใ๏ผใๅ
้ใฟใฏthird-intelligence/Haas-ja-v1.0๏ผCopyright Third Intelligence, Inc.๏ผใๅธฐๅฑใฏ
NOTICEใใHaasใใฏๅคๆๅ ใ็คบใๅ็งฐใ - ๅ่ๅฎ่ฃ : NVIDIA NeMoใthird-intelligence/Haas-ASR๏ผๅ ฌๅผๆจ่ซใใใฑใผใธ๏ผใmatsuurajun/haas-mlxใSanami/haas-ja-onnxใฏApache-2.0ใflash-linear-attentionใฏMITใ
- Sanamiใฎfloat32 ONNXใฏๆฐๅคๆฏ่ผใฎใฟใงใ้ใฟใฏๅ้ ๅธใใพใใใ
- ่ฉไพกใใผใฟ: FLEURSใฏCC BY 4.0ใJVNVใฏCC BY-SA 4.0ใๅๆขฑใปๅ้ ๅธใใพใใใ
English
This is an unofficial conversion of third-intelligence/Haas-ja-v1.0 to MLX with 8-bit linear layers. It is not affiliated with, endorsed by, or sponsored by Third Intelligence, Inc.
Weights occupy about 0.73 GB, approximately 0.59 times the bf16 version. Peak memory decreases, but median RTFx is about 14% lower than bf16; quantization does not make inference faster on the tested machine.
Overview and requirements
- All linear layers use
mlx.nn.quantize8-bit affine quantization with group size 64; convolutions, normalization layers and embeddings stay bf16. At load time,scalesandbiasesare bf16 in the encoder and float32 in the prediction and joint networks. - Apple Silicon Macs only; no NeMo, CUDA, Triton or torch. Requires Python 3.12 and haas-ja-local.
- Same test environment as bf16: Apple M5 Max (128 GB), macOS 27.2, mlx 0.32.4. Other machines are unmeasured.
Usage
Uses the same code as bf16. Install hf with pip install huggingface_hub if needed.
pip install "haas-ja-local[mlx] @ git+https://github.com/basyauma-tech/haas-ja-local"
hf download basyauma-tech/Haas-ja-v1.0-mlx-8bit --local-dir Haas-ja-v1.0-mlx-8bit
from haas_ja_local import load_audio, load_model, transcribe
backend, tokenizer = load_model("Haas-ja-v1.0-mlx-8bit")
result = transcribe(backend, tokenizer, load_audio("audio.wav"))
print(result.text)
haas-ja-local transcribe Haas-ja-v1.0-mlx-8bit audio.wav
haas-ja-local transcribe Haas-ja-v1.0-mlx-8bit audio.wav --json --timestamps
Accuracy
asr-ja-bench v1.1, evaluated with the same procedure as bf16. CER strict values are percentages; differences are percentage points. Differences from bf16 use the rounded values shown.
| Subset | Utterances | bf16 | 8-bit | Difference from bf16 | Published | Difference from published |
|---|---|---|---|---|---|---|
| FLEURS | 650 | 3.65 | 3.68 | +0.03 | 3.65โ3.67 | +0.01 |
| JVNV | 1,375 | 4.51 | 4.49 | -0.02 | 4.50โ4.53 | -0.01 |
Differences from published values use the nearer endpoint, or 0 within the range. The 8-bit version passes the 0.3-point threshold. No accuracy tuning was applied.
The 4-bit version is not published because it failed the accuracy threshold. CER strict was 3.70% on FLEURS (+0.03 points from published) and 5.00% on JVNV (+0.47 points).
Details/reproduction (Japanese): quantization verification.
Implementation verification
Small random-model tests cover conversion/loading round trips, keys, shapes and bit-identical unquantized values. The 8-bit version produced identical text to bf16 for 88.8% of FLEURS utterances and 84.9% of JVNV utterances. Relative error against a model holding dequantized weights in float32 was below 1e-5. Token agreement between the full quantized model and PyTorch (float32) is unverified.
Speed and memory
M5 Max setup above, batch 1, sequential recognition. 647 FLEURS utterances after excluding the first 3 warm-ups; default 1 GiB cache cap. bf16 was remeasured in the same time window. GB = 10^9 bytes.
| Item | bf16 | 8-bit |
|---|---|---|
| RTFx (median per utterance) | 260.1 | 223.3 |
| RTFx (total audio / total processing time) | 263.4 | 226.6 |
| Median processing time per utterance | 0.048 s | 0.057 s |
| Peak RSS | 1.57 GB | 1.09 GB |
| mlx peak allocation | 1.79 GB | 1.23 GB |
| macOS peak memory footprint | 3.20 GB | 2.63 GB |
Files
Sizes in bytes.
| File | Size | SHA-256 |
|---|---|---|
model.safetensors |
727,881,324 | 17ec1d12a67b35303c1bbc4defa3077b03f14665550e45d2749ada12ef73368e |
config.json |
726 | 0b7ad196af39a5120570d31bdcb0c9194e89d216892c9ea8e4d552170e8df6b3 |
tokenizer.model |
279,711 | 9bd7bcdff62e43200001cd4c258d22e475b7b3b1e0f64f8be8e6e046f402f38c |
LICENSE |
11,358 | cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 |
NOTICE |
741 | cb6145131e3cb48e0bb432aa5b949ae72330ef0dddaf1621de9b0cc65ab8b51b |
Limitations
The bf16 limitations also apply. Quantized inference is unmeasured on long audio with splitting, silence/noise, JSUT, TEDxJP-10K, SpeechBSD, group sizes other than 64, low-memory machines and batch sizes โฅ2.
License and attribution
- Same as bf16: Apache-2.0 for weights/code. Source weights: third-intelligence/Haas-ja-v1.0, Copyright Third Intelligence, Inc. Attribution:
NOTICE. "Haas" identifies the source. - References: NVIDIA NeMo, third-intelligence/Haas-ASR (official inference package), matsuurajun/haas-mlx, Sanami/haas-ja-onnx: Apache-2.0. flash-linear-attention: MIT.
- Sanami's float32 ONNX is used only for numerical comparison; its weights are not redistributed.
- Evaluation data: FLEURS CC BY 4.0; JVNV CC BY-SA 4.0. Neither included nor redistributed.
- Downloads last month
- 107
8-bit
Model tree for basyauma-tech/Haas-ja-v1.0-mlx-8bit
Base model
third-intelligence/Haas-ja-v1.0