File size: 805 Bytes
8da87ef 5a7d329 8da87ef 6fabeec 8da87ef 5a7d329 a346010 bf231d5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | ## Linux
### 1. Requirements
- Already installed `CuDNN 9.x` & `CUDA 12.x` for _ONNX Runtime 1.23.2_
### 2. Inference
- `LD_LIBRARY_PATH=. ./asr --help`
- `LD_LIBRARY_PATH=. ./asr openai`
- `libonnxruntime_providers_cuda.so` & `libonnxruntime_providers_shared.so` - only need for CUDA inference
### 3. Build on Glibc>=2.28
- ``` sh
mkdir -p ~/lib/onnxruntime
cd ~/lib/onnxruntime
wget https://github.com/microsoft/onnxruntime/releases/download/v1.23.2/onnxruntime-linux-x64-gpu-1.23.2.tgz
tar -xzf onnxruntime-linux-x64-gpu-1.23.2.tgz
export ORT_STRATEGY=manual
export ORT_LIB_LOCATION=$HOME/lib/onnxruntime/onnxruntime-linux-x64-gpu-1.23.2/lib
export LD_LIBRARY_PATH=$ORT_LIB_LOCATION:$LD_LIBRARY_PATH
cargo clean
cargo build --release -p asr --features cuda
``` |