examples/inference-client/README.md

inference-client

👋 hello

This repository contains examples of image and video inference via Roboflow Inference (HTTP) and stream inference via Roboflow Inference (UDP).

The HTTP examples take in an image or video and run inference, whereas the UDP example listens for predictions from a UDP stream and processes them.

💻 install client environment

# clone repository and navigate to root directory
git clone https://github.com/roboflow/inference-client.git
cd inference-client

# setup python environment and activate it
python3 -m venv venv
source venv/bin/activate

# headless install
pip install -r requirements.txt

🐋 docker

You can learn more about Roboflow Inference Docker Image build, pull and run in our documentation.

HTTP

• Run on x86 CPU:

  docker run --net=host roboflow/roboflow-inference-server-cpu:latest
  

• Run on Nvidia GPU:

  docker run --network=host --gpus=all roboflow/roboflow-inference-server-gpu:latest
  

UDP

• Run on Nvidia GPU:

docker run --gpus=all --net=host -e STREAM_ID=0 -e MODEL_ID=<> -e API_KEY=<> roboflow/roboflow-inference-server-udp-gpu:latest

👉 more docker run options

HTTP

• Run on arm64 CPU:

  docker run -p 9001:9001 roboflow/roboflow-inference-server-arm-cpu:latest
  

• Run on Nvidia GPU with TensorRT Runtime:

  docker run --network=host --gpus=all roboflow/roboflow-inference-server-trt:latest
  

• Run on Nvidia Jetson with JetPack 4.x:

  docker run --privileged --net=host --runtime=nvidia roboflow/roboflow-inference-server-jetson:latest
  

• Run on Nvidia Jetson with JetPack 5.x:

  docker run --privileged --net=host --runtime=nvidia roboflow/roboflow-inference-server-jetson-5.1.1:latest
  

UDP

We only support one UDP container at the moment. Refer to the UDP command from earlier to set up UDP.

🔑 keys

Before running the inference script, ensure that the API_KEY is set as an environment variable. This key provides access to the inference API.

• For Unix/Linux:

  export API_KEY=your_api_key_here
  

• For Windows:

  set API_KEY=your_api_key_here
  

Replace your_api_key_here with your actual API key.

📷 image inference example (HTTP)

To run the image inference script:

python image.py \
--image_path data/a9f16c_8_9.png \
--class_list "ball" "goalkeeper" "player" "referee" \
--dataset_id "football-players-detection-3zvbc" \
--version_id 2 \
--confidence 0.5

🎬 video inference example (HTTP)

To run the video inference script:

python video.py \
--video_path "data/40cd38_5.mp4" \
--class_list "ball" "goalkeeper" "player" "referee" \
--dataset_id "football-players-detection-3zvbc" \
--version_id 2 \
--confidence 0.5

📺 stream inference example (UDP)

To run the UDP receiver, run:

python udp.py --port=12345