File size: 2,152 Bytes
f188641 47e9630 f188641 47e9630 26eaba5 47e9630 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: apache-2.0
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- lora
- fine-tuned-model
- instruction-tuning
- adaptation
- parameter-efficient-finetuning
metrics:
- perplexity
- bleu
- rouge
datasets:
- iohadrubin/wikitext-103-raw-v1
base_model:
- Qwen/Qwen3-Embedding-0.6B
---
# dirlora-v3
`dirlora-v3` is a LoRA-adapted model checkpoint built for concise, instruction-style text embedding with
aware of previous text and following text.
## Model details
- **Model type:** Causal Language Model (LoRA adapter)
- **Base model:** `Qwen/Qwen3-Embedding-0.6B`
- **Fine-tuning method:** LoRA / PEFT
- **LoRA config:**
- Rank (`r`): `16`
- Alpha (`alpha`): `{{alpha}}`
- Dropout: `{{lora_dropout}}`
- **Training data:** subset of `wikitext 103` for PoC
- **Framework:** PyTorch + Hugging Face `transformers` + `peft`
## Intended use
Use this model for:
- text embedding
- experimenting direction/sequence aware embedding
## Not intended for
- any engineering solution
## Usage
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("npc0/Qwen3-Embedding-0.6B-OED")
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium."
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
```
## Limitations
- This is early stage PoC not converged but made progress comparing to previous versions
| 配置 | R@1 | R@5 | R@20 | MRR | med |
|---|---|---|---|---|---|
| **dirlora-v3 + next** | **0.110** | **0.297** | **0.505** | **0.200** | **20** |
| dirlora-v3 + prev（對照） | 0.043 | 0.167 | 0.368 | 0.114 | 41 |
| dirlora-v3 無 prompt | 0.058 | 0.210 | 0.418 | 0.137 | 31 |
| dirlora-v1（前最佳） | 0.075 | 0.242 | 0.465 | 0.162 | 27 |
| base | 0.046 | 0.150 | 0.318 | 0.108 | 53 |
## Citation
If you use this model, please cite:
```
@misc{npc0directionalembedderv0.3,
title={Directional Embedder v0.3},
author={NPC0},
year={2026},
url={https://huggingface.co/npc0/directional-embedder-v0.3},
}
``` |