license: apache-2.0
base_model:
- tomaarsen/Qwen3-Reranker-0.6B-seq-cls
pipeline_tag: text-ranking
language:
- en
library_name: transformers
tags:
- text-generation-inference
Qwen3-Reranker-0.6B-seq-cls-GGUF
Qwen3-Reranker-0.6B-seq-cls is a sequence classification adaptation of the Qwen3-Reranker-0.6B model, designed for advanced text reranking and classification tasks across over 100 languages, including code and multilingual retrieval scenarios. Built on the Qwen3 series, this 0.6B parameter model offers a 32k context window and supports instruction-aware customization for specific tasks, typically improving performance by 1% to 5% when using tailored English instructions. It inherits the robust reasoning, long-text understanding, and versatility of its parent models, excelling in text retrieval, code retrieval, clustering, classification, and bitext mining, and ranks among the top models in benchmarks such as the MTEB multilingual leaderboard.
Model Files
| File name | Size | Quant Type |
|---|---|---|
| Qwen3-4B-abliterated.F32.gguf | 16.1 GB | F32 |
| Qwen3-4B-abliterated.BF16.gguf | 8.05 GB | BF16 |
| Qwen3-4B-abliterated.F16.gguf | 8.05 GB | F16 |
| Qwen3-4B-abliterated.Q8_0.gguf | 4.28 GB | Q8_0 |
| Qwen3-4B-abliterated.Q6_K.gguf | 3.31 GB | Q6_K |
| Qwen3-4B-abliterated.Q5_K_M.gguf | 2.89 GB | Q5_K_M |
| Qwen3-4B-abliterated.Q5_K_S.gguf | 2.82 GB | Q5_K_S |
| Qwen3-4B-abliterated.Q4_K_M.gguf | 2.5 GB | Q4_K_M |
| Qwen3-4B-abliterated.Q4_K_S.gguf | 2.38 GB | Q4_K_S |
| Qwen3-4B-abliterated.Q3_K_L.gguf | 2.24 GB | Q3_K_L |
| Qwen3-4B-abliterated.Q3_K_M.gguf | 2.08 GB | Q3_K_M |
| Qwen3-4B-abliterated.Q3_K_S.gguf | 1.89 GB | Q3_K_S |
| Qwen3-4B-abliterated.Q2_K.gguf | 1.67 GB | Q2_K |
Quants Usage
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
