README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-0.6B
library_name: transformers
pipeline_tag: text-ranking
tags:
  - reranker
  - cross-encoder
  - retrieval
  - agent-skills
  - skill-routing
  - skillcorpus
language:
  - en
---

# skillcorpus-reranker-0.6b

A cross-encoder for **agent-skill retrieval**: given a task and a candidate skill
document, judge whether the skill helps. Fine-tuned from
[Qwen/Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B).

Reranks the candidates recalled by
[skillcorpus-embedding-0.6b](https://huggingface.co/EverMind-AI/skillcorpus-embedding-0.6b).
Because scoring is one forward pass per (task, skill) pair, run it on a shortlist
— typically the encoder's top 20–50 — not the whole corpus. The skill documents
it was built to score have the schema of
[skillcorpus-demo-1k](https://huggingface.co/datasets/EverMind-AI/skillcorpus-demo-1k).

| Property | Value |
|---|---|
| Parameters | 596M |
| Layers | 28 |
| Precision | bfloat16 |
| Output | `P("yes")` in `[0, 1]` |

Requires `transformers>=4.56` (the `dtype=` argument was named `torch_dtype=`
before that).

## Usage

The model answers a yes/no question; the relevance score is the softmax over the
`yes` and `no` logits at the final position.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "EverMind-AI/skillcorpus-reranker-0.6b"
tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval()

PREFIX = ('<|im_start|>system\nJudge whether the Document meets the requirements '
          'based on the Query and the Instruct provided. Note that the answer can '
          'only be "yes" or "no".<|im_end|>\n<|im_start|>user\n')
SUFFIX = '<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n'

YES, NO = tok.convert_tokens_to_ids("yes"), tok.convert_tokens_to_ids("no")


def score(pairs, max_length=4096):
    prompts = [PREFIX + p + SUFFIX for p in pairs]
    enc = tok(prompts, padding=True, truncation=True,
              max_length=max_length, return_tensors="pt").to(model.device)
    with torch.no_grad():
        logits = model(**enc).logits[:, -1, :]
    pair = torch.stack([logits[:, NO], logits[:, YES]], dim=-1)
    return torch.softmax(pair, dim=-1)[:, 1].float().tolist()


INSTRUCT = ("Given a task description, judge whether the skill document "
            "is relevant and useful for completing the task")


def pair(task, name, description, body):
    return (f"<Instruct>: {INSTRUCT}\n\n"
            f"<Query>: {task}\n\n"
            f"<Document>: {name} | {description} | {body}")


task = "resolve conflicts after a git merge"
print(score([
    pair(task, "resolve-conflicts", "Resolve git merge conflicts.", "..."),
    pair(task, "sourdough", "Bake sourdough bread.", "..."),
]))
# -> [0.97, 0.01]
```

Exact scores shift in the last decimal with dtype and hardware; the ordering is
what matters.

Two things to keep intact: the `PREFIX` / `SUFFIX` template, since the score is
read off the final-position logits, and the `<Instruct>` / `<Query>` /
`<Document>` layout with blank lines between the parts. Truncate the document
body, not the template.

### Via the chat template

`chat_template.jinja` reproduces exactly the string built above, so you can let
the tokenizer assemble it instead. It reads three roles — `system` carries the
instruction (omit it to get the default shown above), `query` the task, and
`document` the skill:

```python
prompt = tok.apply_chat_template([
    {"role": "system",   "content": INSTRUCT},
    {"role": "query",    "content": task},
    {"role": "document", "content": "resolve-conflicts | Resolve git merge conflicts. | ..."},
], tokenize=False)
```

### Truncation used in training

Beyond the token-level `max_length`, each field was cut to a fixed number of
**characters** before the prompt was assembled. Matching this keeps inference
inputs on the same distribution as training:

| field | limit |
|---|---|
| skill `description` | 500 chars |
| skill `body` | 2,000 chars |

## Intended use

Second-stage reranking over a shortlist recalled by
[skillcorpus-embedding-0.6b](https://huggingface.co/EverMind-AI/skillcorpus-embedding-0.6b)
— typically its top 20–50. Scoring is one forward pass per (task, skill) pair,
so it does not scale to a whole registry. Treat the output as a ranking signal
rather than a calibrated probability: compare scores within one candidate list,
not across different tasks.

## Citation

```bibtex
@article{wang2026skillcorpus,
  title         = {SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents},
  author        = {Wang, Yanze and Yao, Pengfei and Sun, Tianyi and Hu, Chuanrui and Xiao, Yan and Luo, Xiaotian and Han, Yunyun and Chen, Yifan and Sun, Jun and Deng, Yafeng},
  year          = {2026},
  eprint        = {2607.15557},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2607.15557}
}
```

## License

Apache-2.0, inherited from the base model. Skills in the corpus keep their own
upstream licenses.