File size: 1,734 Bytes
661b502 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | # minimal-RITS-query
A tiny dependency-free Python client for calling a RITS-hosted chat model.
The script reads `.env` from the repository root and uses:
- `LLM_API_KEY`
- `LLM_BASE_URL`, for example `https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/`
- `LLM_MODEL`
## Usage
Call the default model from `.env`:
```bash
python3 rits_query.py "Explain retrieval augmented generation in one sentence."
```
Call a specific model. The script uses `LLM_BASE_URL` from `.env`, derives the
RITS route from the model name, and appends `/v1`:
```bash
python3 rits_query.py --model ibm/granite-4.0-micro "Write a short haiku about logs."
```
Read the prompt from a file. `--gpus` is accepted for job compatibility and
defaults to `1` if you do not pass it:
```bash
python3 rits_query.py \
--model meta-llama/llama-3-3-70b-instruct \
--gpus 1 \
--input-file /tmp/hf_fury_job_50f6mo8l/prompt.txt
```
You can also override the endpoint and key:
```bash
python3 rits_query.py \
--base-url "https://your-rits-host/" \
--api-key "$LLM_API_KEY" \
--model ibm/granite-4.0-micro \
"Hello"
```
If you do not pass `--model`, the script uses `LLM_MODEL` from `.env`.
If `LLM_BASE_URL` is the RITS host root, the script builds a route from the
final model name by replacing dots with hyphens. For example, model
`ibm/granite-4.0-micro` becomes:
`https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/granite-4-0-micro/v1`.
If `LLM_BASE_URL` already ends in `/some-model/v1`, the script replaces that
model route with the requested model.
If that model route is not deployed on the configured RITS host, RITS will
return `404`; in that case pass the exact model base URL with `--base-url`.
|