jina-reranker-v3

Multi-format version of jinaai/jina-reranker-v3 - optimized for deployment.

Model Information

Property	Value
Base Model	jinaai/jina-reranker-v3
Task	reranker-llm
Type	Text Model
Trust Remote Code	True

Available Versions

Folder	Format	Description	Size
safetensors-fp32/	PyTorch FP32	Baseline, highest accuracy	2289 MB
safetensors-fp16/	PyTorch FP16	GPU inference, ~50% smaller	1152 MB

Usage

PyTorch (GPU)

from transformers import AutoModel, AutoTokenizer
import torch

# GPU inference with FP16
model = AutoModel.from_pretrained(
    "n24q02m/jina-reranker-v3",
    subfolder="safetensors-fp16",
    torch_dtype=torch.float16,
    trust_remote_code=True
).cuda()
model.eval()

# Rerank documents
query = "What is machine learning?"
documents = [
    "Machine learning is a subset of AI.",
    "The weather is nice today."
]

with torch.no_grad():
    results = model.rerank(query, documents, top_n=2)
    for r in results:
        print(f"Score: {r['score']:.4f} - {r['text'][:50]}...")

Notes

1. SafeTensors FP16 is the primary format for GPU inference
2. Requires trust_remote_code=True for this model

License

Apache 2.0 (following the base model's license)

Credits

• Base Model: jinaai/jina-reranker-v3
• Conversion: PyTorch + SafeTensors