---
license: apache-2.0
language:
  - en
tags:
  - zephyr
  - sentence-transformers
  - feature-extraction
  - sentence-similarity
  - retrieval
  - rag
base_model: sentence-transformers/all-MiniLM-L6-v2
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---

# Zephyr RTOS documentation embeddings

`all-MiniLM-L6-v2` fine-tuned on the Zephyr RTOS documentation, for retrieval over
that corpus.

- **Comparison Space:** https://huggingface.co/spaces/eoinedge/zephyrproject-embeddings-compare
- **Index and pipeline:** https://huggingface.co/eoinedge/zephyrproject
- **Source:** https://github.com/eoinjordan/zephyrproject-rag

## What it is for

The base model is trained on general web text, where "binding" means a contract and "west" is a direction. This copy is
adapted to Zephyr RTOS's vocabulary so those senses separate.

Use it as a drop-in replacement for the base model when embedding Zephyr RTOS
documentation:

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("eoinedge/zephyrproject-docs-embeddings")
vectors = model.encode(passages, normalize_embeddings=True)
```

## Training

| | |
|---|---|
| Base | `sentence-transformers/all-MiniLM-L6-v2` |
| Pairs | 6,079 mined from the docs' own heading/body structure |
| Loss | `MultipleNegativesRankingLoss` (in-batch negatives) |
| Epochs | 1 |
| Batch size | 32 |
| Held-out pairs | 400 |

No hand-labelled data and no synthetic questions from a generator — a section
heading is a natural query for the body beneath it, and it is already written by
the documentation authors.

## Results, and how to read them

| Metric | Base | Tuned |
|---|---|---|
| recall@5 on held-out pairs | 0.63 | **0.86** |

**That number flatters the model, and you should treat it with suspicion.** The
evaluation measures the same heading→body relationship the model was trained on,
so it partly measures whether training converged rather than whether retrieval
improved.

Spot-checking real questions gives a more mixed picture: some clearly improve,
several are unchanged, and at least one regressed. Real questions are not phrased
like section headings, which is exactly the gap this evaluation does not cover.

The comparison Space runs both models on the same query so you can judge for
yourself rather than trusting the headline.

## Limitations

- One epoch, one seed, no hyperparameter search.
- Evaluation is aligned with the training objective, as above.
- Trained on Zephyr RTOS documentation only — it will not help on other corpora and
  may be worse than the base model there.
- This adapts the **retriever**, not a generator. It does not teach a coding
  model anything about Zephyr RTOS; it changes which passages that model is handed.

## Licence

Apache-2.0, matching the base model.