README.md

---
license: apache-2.0
library_name: peft
pipeline_tag: text-generation
language:
  - en
tags:
  - lora
  - fine-tuned-model
  - instruction-tuning
  - adaptation
  - parameter-efficient-finetuning
metrics:
  - perplexity
  - bleu
  - rouge
datasets:
  - iohadrubin/wikitext-103-raw-v1
base_model:
  - Qwen/Qwen3-Embedding-0.6B
---

# dirlora-v3

`dirlora-v3` is a LoRA-adapted model checkpoint built for concise, instruction-style text embedding with
aware of previous text and following text.

## Model details
- **Model type:** Causal Language Model (LoRA adapter)
- **Base model:** `Qwen/Qwen3-Embedding-0.6B`
- **Fine-tuning method:** LoRA / PEFT
- **LoRA config:**
  - Rank (`r`): `16`
  - Alpha (`alpha`): `{{alpha}}`
  - Dropout: `{{lora_dropout}}`
- **Training data:** subset of `wikitext 103` for PoC
- **Framework:** PyTorch + Hugging Face `transformers` + `peft`

## Intended use
Use this model for:
- text embedding
- experimenting direction/sequence aware embedding

## Not intended for
- any engineering solution

## Usage

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("npc0/Qwen3-Embedding-0.6B-OED")

sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
```
## Limitations

- This is early stage PoC not converged but made progress comparing to previous versions

| 配置 | R@1 | R@5 | R@20 | MRR | med |
|---|---|---|---|---|---|
| **dirlora-v3 + next** | **0.110** | **0.297** | **0.505** | **0.200** | **20** |
| dirlora-v3 + prev（對照） | 0.043 | 0.167 | 0.368 | 0.114 | 41 |
| dirlora-v3 無 prompt | 0.058 | 0.210 | 0.418 | 0.137 | 31 |
| dirlora-v1（前最佳） | 0.075 | 0.242 | 0.465 | 0.162 | 27 |
| base | 0.046 | 0.150 | 0.318 | 0.108 | 53 |

## Citation

If you use this model, please cite:
```
@misc{npc0directionalembedderv0.3,
      title={Directional Embedder v0.3}, 
      author={NPC0},
      year={2026},
      url={https://huggingface.co/npc0/directional-embedder-v0.3}, 
}
```