query_llm/README.md

# minimal-RITS-query

A tiny dependency-free Python client for calling a RITS-hosted chat model.

The script reads `.env` from the repository root and uses:

- `LLM_API_KEY`
- `LLM_BASE_URL`, for example `https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/`
- `LLM_MODEL`

## Usage

Call the default model from `.env`:

```bash
python3 rits_query.py "Explain retrieval augmented generation in one sentence."
```

Call a specific model. The script uses `LLM_BASE_URL` from `.env`, derives the
RITS route from the model name, and appends `/v1`:

```bash
python3 rits_query.py --model ibm/granite-4.0-micro "Write a short haiku about logs."
```

Read the prompt from a file. `--gpus` is accepted for job compatibility and
defaults to `1` if you do not pass it:

```bash
python3 rits_query.py \
  --model meta-llama/llama-3-3-70b-instruct \
  --gpus 1 \
  --input-file /tmp/hf_fury_job_50f6mo8l/prompt.txt
```

You can also override the endpoint and key:

```bash
python3 rits_query.py \
  --base-url "https://your-rits-host/" \
  --api-key "$LLM_API_KEY" \
  --model ibm/granite-4.0-micro \
  "Hello"
```

If you do not pass `--model`, the script uses `LLM_MODEL` from `.env`.
If `LLM_BASE_URL` is the RITS host root, the script builds a route from the
final model name by replacing dots with hyphens. For example, model
`ibm/granite-4.0-micro` becomes:
`https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/granite-4-0-micro/v1`.
If `LLM_BASE_URL` already ends in `/some-model/v1`, the script replaces that
model route with the requested model.
If that model route is not deployed on the configured RITS host, RITS will
return `404`; in that case pass the exact model base URL with `--base-url`.