README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-0.6B
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-ranking
language:
  - en
  - vi
pretty_name: sec-rerank (fine-tuned Qwen3-Reranker)
tags:
  - reranker
  - cross-encoder
  - text-ranking
  - listwise-reranking
  - generative-reranker
  - cve
  - cybersecurity
  - qdrant
  - secAI
---

# sec-rerank

**This is a fine-tuned version of [Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B)** — a **listwise generative reranker** (second-stage ranker), not an embedding model.

It was trained on grouped query–document examples whose **hard negatives were mined from a local Qdrant** collection (`cve_kb`) built from CVE investigation trajectories. It does not emit dense vectors; it reorders first-stage candidates (e.g. from [`DuyTa/sec-embedding`](https://huggingface.co/DuyTa/sec-embedding)).

## Training

From `notebooks/Qwen3_Reranker_Colab.ipynb` (ms-swift):

| | |
|---|---|
| Base | `Qwen/Qwen3-Reranker-0.6B` |
| Task | `generative_reranker` (causal-LM reranker / cross-encoder scoring) |
| Loss | **listwise reranking** (`--loss_type listwise_reranker`) |
| Tuner | full-parameter SFT (`--tuner_type full`) |
| Engine | [ms-swift](https://github.com/modelscope/ms-swift) `swift sft` |
| Max length | 2048 |
| Learning rate | 6e-6 |

`(query, positive, negative)` rows are converted to SWIFT **grouped ranking** schema:

- `messages` — system instruction + user query
- `positive_messages` — gold CVE passage
- `negative_messages` — Qdrant-mined hard-negative CVE passage(s)

Train-time instruction:

> Given a Vietnamese cybersecurity search query, retrieve passages from the CVE knowledge base that directly answer it.

SWIFT fills the native Qwen3-Reranker `{Instruction}` slot from that system message.

### Hard-negative mining

Positives and negatives are real CVE core-chunk text from local Qdrant `cve_kb` (NVD/MITRE). The negative is a **near-miss CVE** from the same collection — typically a different CWE (`hard_negative_type: different_cwe`): high lexical overlap, wrong document. The listwise objective ranks the gold passage above those mined hard negatives.

Split: 33.6k train / 4.2k validation grouped examples.

## Inference

Score each `(query, document)` with the native Qwen3-Reranker generative yes/no head. Use only to **rerank** a short candidate list from dense / hybrid retrieval.

```python
# vLLM / OpenAI-compatible rerank endpoint
# POST /v1/rerank
{
  "model": "DuyTa/sec-rerank",
  "query": "CVE-2021-44228 JNDI lookup on log4j",
  "documents": ["...", "..."]
}
```

## Attribution & license

Derived from [Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) (**Apache-2.0**). Credit for the base reranker belongs to the Qwen team.