Instructions to use Qwen/Qwen3-Reranker-0.6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Qwen/Qwen3-Reranker-0.6B with Transformers:
# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-Reranker-0.6B") model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-Reranker-0.6B") - sentence-transformers
How to use Qwen/Qwen3-Reranker-0.6B with sentence-transformers:
from sentence_transformers import CrossEncoder model = CrossEncoder("Qwen/Qwen3-Reranker-0.6B") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) - Notebooks
- Google Colab
- Kaggle
vllm online serving
#7
by aperez900907 - opened
When running the model with VLLM, it logs this warning:
WARNING 06-13 09:05:21 [api_server.py:848] To indicate that the rerank API is not part of the standard OpenAI API, we have located it at /rerank. Please update your client accordingly. (Note: Conforms to JinaAI rerank API)
I query /rerank /v1/rerank /v2/rerank and always get this output error:
{
"object": "error",
"message": "The model does not support Rerank (Score) API",
"type": "BadRequestError",
"param": null,
"code": 400
}
The requests are 200 to all endpoints, but they do not work.
how are u running this
try to run with --task score