File size: 2,817 Bytes
cc417b5 cf764ed c06bbac cc417b5 cf764ed cc417b5 c06bbac cc417b5 c06bbac cc417b5 cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac cf764ed c06bbac | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-0.6B
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-ranking
language:
- en
- vi
pretty_name: sec-rerank (fine-tuned Qwen3-Reranker)
tags:
- reranker
- cross-encoder
- text-ranking
- listwise-reranking
- generative-reranker
- cve
- cybersecurity
- qdrant
- secAI
---
# sec-rerank
**This is a fine-tuned version of [Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B)** — a **listwise generative reranker** (second-stage ranker), not an embedding model.
It was trained on grouped query–document examples whose **hard negatives were mined from a local Qdrant** collection (`cve_kb`) built from CVE investigation trajectories. It does not emit dense vectors; it reorders first-stage candidates (e.g. from [`DuyTa/sec-embedding`](https://huggingface.co/DuyTa/sec-embedding)).
## Training
From `notebooks/Qwen3_Reranker_Colab.ipynb` (ms-swift):
| | |
|---|---|
| Base | `Qwen/Qwen3-Reranker-0.6B` |
| Task | `generative_reranker` (causal-LM reranker / cross-encoder scoring) |
| Loss | **listwise reranking** (`--loss_type listwise_reranker`) |
| Tuner | full-parameter SFT (`--tuner_type full`) |
| Engine | [ms-swift](https://github.com/modelscope/ms-swift) `swift sft` |
| Max length | 2048 |
| Learning rate | 6e-6 |
`(query, positive, negative)` rows are converted to SWIFT **grouped ranking** schema:
- `messages` — system instruction + user query
- `positive_messages` — gold CVE passage
- `negative_messages` — Qdrant-mined hard-negative CVE passage(s)
Train-time instruction:
> Given a Vietnamese cybersecurity search query, retrieve passages from the CVE knowledge base that directly answer it.
SWIFT fills the native Qwen3-Reranker `{Instruction}` slot from that system message.
### Hard-negative mining
Positives and negatives are real CVE core-chunk text from local Qdrant `cve_kb` (NVD/MITRE). The negative is a **near-miss CVE** from the same collection — typically a different CWE (`hard_negative_type: different_cwe`): high lexical overlap, wrong document. The listwise objective ranks the gold passage above those mined hard negatives.
Split: 33.6k train / 4.2k validation grouped examples.
## Inference
Score each `(query, document)` with the native Qwen3-Reranker generative yes/no head. Use only to **rerank** a short candidate list from dense / hybrid retrieval.
```python
# vLLM / OpenAI-compatible rerank endpoint
# POST /v1/rerank
{
"model": "DuyTa/sec-rerank",
"query": "CVE-2021-44228 JNDI lookup on log4j",
"documents": ["...", "..."]
}
```
## Attribution & license
Derived from [Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) (**Apache-2.0**). Credit for the base reranker belongs to the Qwen team.
|