cohere-asr-ja-GGUF

GGUF quantization of efwkjn/cohere-asr-ja, for use with CrispASR's cohere backend.

Files

File	Type	Size
cohere-asr-ja-f16.gguf	F16 GGUF	4.14 GB
cohere-asr-ja-q8_0.gguf	Q8_0 GGUF	2.42 GB
cohere-asr-ja-q6_k.gguf	Q6_K GGUF	1.98 GB
cohere-asr-ja-q5_0.gguf	Q5_0 GGUF	1.74 GB
cohere-asr-ja-q4_k.gguf	Q4_K GGUF	1.51 GB

Quantization Rating

Informal local test results on Japanese dialogue, cover song, and ASMR-style audio. Speed is the observed average without VAD on an RTX 3080 system.

Quant	Size	Quality	Observed speed	Suggested role
F16	4.14 GB	5/5	20.91x	Reference quality, fastest in this local test
Q8_0	2.42 GB	5/5	15.41x	Near-F16 quality, smaller
Q6_K	1.98 GB	4.5/5	14.53x	Best default balance
Q5_0	1.74 GB	4/5	13.79x	Smaller practical option
Q4_K	1.51 GB	3/5	16.70x	Smallest option, not always most accurate

Conversion

Source model:

• efwkjn/cohere-asr-ja

Tokenizer check:

• tokenizer.model verified against tokenizer.json
• vocab size: 16384
• max token id: 16383
• no token/id mismatches found

Converted with CrispASR:

python convert-cohere-asr-to-gguf.py \
  --model-dir efwkjn-cohere-asr-ja \
  --output cohere-asr-ja-f16.gguf

crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q8_0.gguf q8_0
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q6_k.gguf q6_k
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q5_0.gguf q5_0
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q4_k.gguf q4_k

Usage

crispasr --backend cohere -m cohere-asr-ja-q6_k.gguf -f audio.wav -l ja

Notes

This GGUF is intended for CrispASR. Generic llama.cpp runtimes do not support the cohere_asr architecture.