Text Ranking
sentence-transformers
Safetensors
Transformers
multilingual
t5gemma2
text2text-generation
reranker
encoder-decoder
FBNL
Retrieval
RAG
cosyy commited on
Commit
1d3dcd7
·
verified ·
1 Parent(s): 343e2f6

Update vllm_support/README.md

Browse files
Files changed (1) hide show
  1. vllm_support/README.md +27 -2
vllm_support/README.md CHANGED
@@ -126,7 +126,7 @@ CUDA_VISIBLE_DEVICES=0 PORT=8000 \
126
  ./KaLM-Reranker-V1-Nano/vllm_support/examples/start_online_server.sh
127
  ```
128
 
129
- Check health and send demo requests:
130
 
131
  ```bash
132
  kalm-vllm-client --health
@@ -134,6 +134,30 @@ kalm-vllm-client --endpoint rerank --return-margin
134
  kalm-vllm-client --endpoint score --return-margin
135
  ```
136
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
137
  ### `POST /rerank`
138
 
139
  ```json
@@ -177,7 +201,8 @@ Results are returned in descending score order:
177
  }
178
  ```
179
 
180
- `/score` preserves pair order and includes an input `id` when provided.
 
181
 
182
  ### `GET /health`
183
 
 
126
  ./KaLM-Reranker-V1-Nano/vllm_support/examples/start_online_server.sh
127
  ```
128
 
129
+ In a second terminal, check health and send built-in demo requests:
130
 
131
  ```bash
132
  kalm-vllm-client --health
 
134
  kalm-vllm-client --endpoint score --return-margin
135
  ```
136
 
137
+ For custom input, pass one JSON object with `--json-file`. Use `/rerank` for
138
+ one query against multiple documents:
139
+
140
+ ```bash
141
+ kalm-vllm-client \
142
+ --endpoint rerank \
143
+ --json-file ./KaLM-Reranker-V1-Nano/vllm_support/examples/rerank_request.json \
144
+ --return-margin \
145
+ --top-k 10
146
+ ```
147
+
148
+ Use `/score` for a batch of independent query-document pairs:
149
+
150
+ ```bash
151
+ kalm-vllm-client \
152
+ --endpoint score \
153
+ --json-file ./KaLM-Reranker-V1-Nano/vllm_support/examples/score_request.json \
154
+ --return-margin
155
+ ```
156
+
157
+ When `--json-file` is used, `--return-margin` sets
158
+ `"return_margin": true` in the outgoing request, and `--top-k` overrides the
159
+ JSON value for `/rerank`.
160
+
161
  ### `POST /rerank`
162
 
163
  ```json
 
201
  }
202
  ```
203
 
204
+ `/score` accepts multiple entries in `pairs`, preserves their input order and
205
+ includes an input `id` when provided.
206
 
207
  ### `GET /health`
208