Automatic Speech Recognition
GGUF
ggml
ASR
quantization
cpu-inference
bitnet
multilingual
thox
conversational
Instructions to use Thox-ai/ThoxSpeech-ASR-BitNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Thox-ai/ThoxSpeech-ASR-BitNet with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K # Run inference directly in the terminal: llama cli -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K # Run inference directly in the terminal: llama cli -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K # Run inference directly in the terminal: ./llama-cli -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
Use Docker
docker model run hf.co/Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
- LM Studio
- Jan
- Ollama
How to use Thox-ai/ThoxSpeech-ASR-BitNet with Ollama:
ollama run hf.co/Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
- Unsloth Studio
How to use Thox-ai/ThoxSpeech-ASR-BitNet with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Thox-ai/ThoxSpeech-ASR-BitNet to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Thox-ai/ThoxSpeech-ASR-BitNet to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Thox-ai/ThoxSpeech-ASR-BitNet to start chatting
- Atomic Chat new
- Docker Model Runner
How to use Thox-ai/ThoxSpeech-ASR-BitNet with Docker Model Runner:
docker model run hf.co/Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
- Lemonade
How to use Thox-ai/ThoxSpeech-ASR-BitNet with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Thox-ai/ThoxSpeech-ASR-BitNet:Q6_K
Run and chat with the model
lemonade run user.ThoxSpeech-ASR-BitNet-Q6_K
List all available models
lemonade list
| ThoxSpeech | |
| Copyright (c) 2026 THOX.ai LLC. | |
| This product is a rebranded derivative work. It bundles, links against, or | |
| derives from the third-party components listed below. All upstream copyright | |
| notices and license texts are preserved in this repository and MUST be retained | |
| in any redistribution, in source or binary form. | |
| ================================================================================ | |
| 1. VibeASR.cpp -- PRIMARY UPSTREAM (this repository is a fork of it) | |
| ================================================================================ | |
| Upstream: https://github.com/microsoft/VibeASR.cpp | |
| Copyright: (c) Microsoft Corporation. | |
| License: MIT -- full text in LICENSE, verbatim copy in THIRD_PARTY_NOTICES.md | |
| ThoxSpeech is a fork of VibeASR.cpp. The heterogeneous-quantization inference | |
| core -- the I8_S VAE tokenizer path, the I2_S BitNet LM path, the custom SIMD | |
| multiply-add-dot kernels, and the ggml operator fusion -- is Microsoft's work, | |
| used here unmodified in substance. | |
| Significant changes made by THOX.ai LLC are enumerated in CHANGES.md. | |
| ================================================================================ | |
| 2. VibeVoice-ASR-BitNet -- MODEL WEIGHTS | |
| ================================================================================ | |
| Upstream: https://huggingface.co/microsoft/VibeVoice-ASR-BitNet | |
| Copyright: (c) Microsoft Corporation. | |
| License: MIT (not gated) | |
| The quantized GGUF artifacts distributed as "ThoxSpeech ASR" weights are | |
| byte-identical mirrors of the upstream VibeVoice-ASR-BitNet GGUF files. THOX.ai | |
| did not retrain, fine-tune, or re-quantize these weights. See MODEL_CARD.md. | |
| Cite the original authors: | |
| @article{xu2025vibeasrbitnet, | |
| title={VibeVoice-ASR-BitNet Technical Report}, | |
| author={Xu, Songchen and Song, Ting and Huang, Shaohan and Peng, Zhiliang | |
| and Xia, Yan and Tu, Yujie and Huang, Xin and Yu, Jianwei and | |
| Dong, Li and Wei, Furu}, | |
| journal={arXiv preprint arXiv:2607.21075}, | |
| year={2025} | |
| } | |
| ================================================================================ | |
| 3. Qwen2.5-1.5B -- BASE LANGUAGE MODEL OF THE ASR DECODER | |
| ================================================================================ | |
| Upstream: https://huggingface.co/Qwen/Qwen2.5-1.5B | |
| Copyright: (c) Alibaba Cloud / Qwen Team. | |
| License: Apache License 2.0 -- verbatim copy in THIRD_PARTY_NOTICES.md | |
| The ASR decoder is derived from Qwen2.5-1.5B, which replaces the Qwen2.5-7B | |
| decoder used by the full-size VibeVoice-ASR model. Apache-2.0 section 4(b) | |
| requires derivative works to carry prominent notice of modification: the model | |
| has been quantized to I2_S (ternary BitNet) weights with Q6_K token embeddings | |
| and converted to GGUF. Apache-2.0 sections 4(c) and 4(d) require retention of | |
| this NOTICE file and the license text in redistributions. | |
| ================================================================================ | |
| 4. llama.cpp / ggml -- INFERENCE FRAMEWORK (git submodule, 3rdparty/llama.cpp) | |
| ================================================================================ | |
| Upstream: https://github.com/XsquirrelC/llama.cpp (branch vibeasr-dev) | |
| Ancestry: ggml-org/llama.cpp -> Eddie-Wang1120/llama.cpp -> XsquirrelC/llama.cpp | |
| Copyright: (c) 2023-2026 The ggml authors. | |
| License: MIT | |
| Vendored as a git submodule at the pinned vibeasr-dev branch. Its own LICENSE | |
| file travels with it and is not modified by this repository. | |
| ================================================================================ | |
| 5. dr_wav.h -- WAV DECODER (utils/dr_wav.h) | |
| ================================================================================ | |
| Upstream: https://github.com/mackron/dr_libs | |
| Copyright: (c) David Reid. | |
| License: public domain (Unlicense) OR MIT-0, at your option -- header retains | |
| its own embedded license block, unmodified. | |
| ================================================================================ | |
| TRADEMARK NOTICE | |
| ================================================================================ | |
| "ThoxSpeech", "ThoxRoute", "ThoxOS" and "THOX" are marks of THOX.ai LLC. | |
| Neither the MIT License nor the Apache License 2.0 grants trademark rights. | |
| This repository does NOT use the "Microsoft", "VibeVoice", "VibeASR", "Qwen", | |
| or "Alibaba" names to endorse or promote ThoxSpeech, and THOX.ai LLC claims no | |
| affiliation with, sponsorship by, or endorsement from Microsoft Corporation or | |
| Alibaba Cloud. Upstream names appear here solely as factual attribution of | |
| provenance, which is what the licenses require. | |