asr-onnx-binary / linux /README.md
tyanfarm's picture
Upload folder using huggingface_hub
bf231d5 verified
## Linux
### 1. Requirements
- Already installed `CuDNN 9.x` & `CUDA 12.x` for _ONNX Runtime 1.23.2_
### 2. Inference
- `LD_LIBRARY_PATH=. ./asr --help`
- `LD_LIBRARY_PATH=. ./asr openai`
- `libonnxruntime_providers_cuda.so` & `libonnxruntime_providers_shared.so` - only need for CUDA inference
### 3. Build on Glibc>=2.28
- ``` sh
mkdir -p ~/lib/onnxruntime
cd ~/lib/onnxruntime
wget https://github.com/microsoft/onnxruntime/releases/download/v1.23.2/onnxruntime-linux-x64-gpu-1.23.2.tgz
tar -xzf onnxruntime-linux-x64-gpu-1.23.2.tgz
export ORT_STRATEGY=manual
export ORT_LIB_LOCATION=$HOME/lib/onnxruntime/onnxruntime-linux-x64-gpu-1.23.2/lib
export LD_LIBRARY_PATH=$ORT_LIB_LOCATION:$LD_LIBRARY_PATH
cargo clean
cargo build --release -p asr --features cuda
```