---
license: other
library_name: nemo-speech.cpp
base_model: nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
base_model_relation: quantized
tags:
- asr
- speech-recognition
- speech-to-text
- transcription
- gguf
- ggml
- llama.cpp
- nemo
- nvidia
- nemotron
- nemotron-3.5
- cuda
- cuda-12.8
- cuda-binary
- binary
- prebuilt-binary
- inference-server
- turing
- turing-gpu
- t4
- tesla-t4
- kaggle
- colab
---
# nemo-speech (CUDA build for Tesla T4)

Prebuilt `nemo-speech` server from NeMo-Speech.cpp, so Kaggle/Colab notebooks can skip the ~40 min compile. It runs [NVIDIA Nemotron-3.5-Transcribe-30B-A4B](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B) (speech recognition) from the GGUF conversion [AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF](https://huggingface.co/AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF). This repo contains the binary only, not the model weights.

## Parent model
| | |
|---|---|
| Original model | [nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B) |
| GGUF weights used | [AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF](https://huggingface.co/AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF) |
| Inference engine | NeMo-Speech.cpp (ggml / llama.cpp based) |

## Build
| | |
|---|---|
| Source | NeMo-Speech.cpp, commit `20999cc` (with this notebook's two-GPU split patch) |
| Preset | `cuda-server`, `-DCMAKE_CUDA_ARCHITECTURES=75-real` (Turing: T4 only) |
| CUDA | Build cuda_12.8.r12.8/compiler.35583870_0 |
| Built on | Linux-6.18.55+-x86_64-with-glibc2.39, Tesla T4, 7.5, 580.178.04 |

## Use
```python
from huggingface_hub import hf_hub_download
import tarfile, os
p = hf_hub_download("masterofaudio2077/nemo-speech-cpp-cuda-t4", "nemo-speech-cuda-t4.tar.gz")
tarfile.open(p).extractall("/tmp/nemo-bin")
os.chmod("/tmp/nemo-bin/bin/nemo-speech", 0o755)
# libraries sit next to the binary
os.environ["LD_LIBRARY_PATH"] = "/tmp/nemo-bin/bin:" + os.environ.get("LD_LIBRARY_PATH", "")
```
Then run `/tmp/nemo-bin/bin/nemo-speech serve --asr-model <model.gguf> ...`, with the GGUF files downloaded from the repo above.

## Limits
- Compiled for compute capability 7.5 only. It will not run on other GPUs (A100, L4, P100).
- Needs an NVIDIA driver with `libcuda.so.1` and a glibc/libstdc++ at least as new as the build machine's (see `ldd.txt`).
- Licence: the model is under NVIDIA's own licence (see the parent model page); the binary follows the NeMo-Speech.cpp, ggml and CUDA licences. Check them before redistributing.