README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-Embedding-0.6B
library_name: sentence-transformers
pipeline_tag: feature-extraction
tags:
  - transformers
  - sentence-similarity
  - retrieval
  - agent-skills
  - skill-routing
  - skillcorpus
  - contrastive-learning
  - text-embeddings-inference
language:
  - en
---

# skillcorpus-embedding-0.6b

A bi-encoder for **agent-skill retrieval**: given a task description, embed it and
the skill documents into one space so the relevant skills come back by nearest
neighbour. Fine-tuned from
[Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B).

Paired with [skillcorpus-reranker-0.6b](https://huggingface.co/EverMind-AI/skillcorpus-reranker-0.6b),
which reranks this model's top candidates. The skill documents it was built to
index have the schema of
[skillcorpus-demo-1k](https://huggingface.co/datasets/EverMind-AI/skillcorpus-demo-1k).

| Property | Value |
|---|---|
| Parameters | 596M |
| Hidden size | 1024 (embedding dimension) |
| Layers | 28 |
| Precision | bfloat16 |
| Pooling | last token |
| Normalization | L2 |

Requires `transformers>=4.56` (the `dtype=` argument was named `torch_dtype=`
before that) or `sentence-transformers>=3.0`.

## Usage

The two sides are encoded asymmetrically — a task description carries an
instruction prefix, a skill is the bare `name | description | body`
concatenation. Encode them the way the model was trained or retrieval quality
drops. Embeddings come back L2-normalized, so cosine similarity is a plain dot
product.

```python
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

MODEL = "EverMind-AI/skillcorpus-embedding-0.6b"
tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left")
model = AutoModel.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval()

QUERY_INSTRUCTION = (
    "Instruct: Given a task description, retrieve the most relevant "
    "skill document that would help an agent complete the task\nQuery:"
)


def doc(name, description, body):
    return f"{name} | {description} | {body}"


def last_token_pool(hidden, attention_mask):
    if attention_mask[:, -1].sum() == attention_mask.shape[0]:   # left padding
        return hidden[:, -1]
    idx = attention_mask.sum(dim=1) - 1
    return hidden[torch.arange(hidden.shape[0], device=hidden.device), idx]


def embed(texts, max_length=2048):
    enc = tok(texts, padding=True, truncation=True,
              max_length=max_length, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model(**enc).last_hidden_state
    return F.normalize(last_token_pool(out, enc["attention_mask"]), p=2, dim=1)


query = embed([QUERY_INSTRUCTION + "resolve conflicts after a git merge"])
docs = embed([doc("resolve-conflicts", "Resolve git merge conflicts.", "..."),
              doc("sourdough", "Bake sourdough bread.", "...")])
print((query @ docs.T).tolist())
# -> [[0.75, 0.07]]
```

Exact scores shift in the last decimal with dtype and hardware; the ordering is
what matters.

### With sentence-transformers

The repo ships a sentence-transformers configuration (last-token pooling + L2
normalization, and the query instruction registered as the `query` prompt), so
this path is equivalent to the code above, up to bf16 noise:

```python
from sentence_transformers import SentenceTransformer

st = SentenceTransformer("EverMind-AI/skillcorpus-embedding-0.6b")
q = st.encode(["resolve conflicts after a git merge"], prompt_name="query")
d = st.encode(["resolve-conflicts | Resolve git merge conflicts. | ...",
               "sourdough | Bake sourdough bread. | ..."])
print(st.similarity(q, d))
```

Pass `prompt_name="query"` for tasks and nothing for skill documents — that is
the asymmetry above, applied for you.

### Truncation used in training

Beyond the token-level `max_length`, each field was cut to a fixed number of
**characters** before the strings were assembled. Matching this keeps inference
inputs on the same distribution as training:

| field | limit |
|---|---|
| task description (after the instruction prefix) | 1,500 chars |
| skill `description` | 500 chars |
| skill `body` | 8,000 chars |

## Intended use

First-stage retrieval over a large skill registry: encode the registry offline,
encode each incoming task online, take the top K by cosine similarity (K in the
20–50 range is typical), then rerank that shortlist with
[skillcorpus-reranker-0.6b](https://huggingface.co/EverMind-AI/skillcorpus-reranker-0.6b).
Not a generative model — it produces embeddings, not answers.

## Citation

```bibtex
@article{wang2026skillcorpus,
  title         = {SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents},
  author        = {Wang, Yanze and Yao, Pengfei and Sun, Tianyi and Hu, Chuanrui and Xiao, Yan and Luo, Xiaotian and Han, Yunyun and Chen, Yifan and Sun, Jun and Deng, Yafeng},
  year          = {2026},
  eprint        = {2607.15557},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2607.15557}
}
```

## License

Apache-2.0, inherited from the base model. Skills in the corpus keep their own
upstream licenses.