README.md

metadata

license: apache-2.0
library_name: peft
pipeline_tag: text-generation
language:
  - en
tags:
  - lora
  - fine-tuned-model
  - instruction-tuning
  - adaptation
  - parameter-efficient-finetuning
metrics:
  - perplexity
  - bleu
  - rouge
datasets:
  - iohadrubin/wikitext-103-raw-v1
base_model:
  - Qwen/Qwen3-Embedding-0.6B

dirlora-v3

dirlora-v3 is a LoRA-adapted model checkpoint built for concise, instruction-style text embedding with aware of previous text and following text.

Model details

• Model type: Causal Language Model (LoRA adapter)
• Base model: Qwen/Qwen3-Embedding-0.6B
• Fine-tuning method: LoRA / PEFT
• LoRA config:
  • Rank (r): 16
  • Alpha (alpha): {{alpha}}
  • Dropout: {{lora_dropout}}
• Training data: subset of wikitext 103 for PoC
• Framework: PyTorch + Hugging Face transformers + peft

Intended use

Use this model for:

• text embedding
• experimenting direction/sequence aware embedding

Not intended for

• any engineering solution

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("npc0/Qwen3-Embedding-0.6B-OED")

sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)

Limitations

• This is early stage PoC not converged but made progress comparing to previous versions

配置	R@1	R@5	R@20	MRR	med
dirlora-v3 + next	0.110	0.297	0.505	0.200	20
dirlora-v3 + prev（對照）	0.043	0.167	0.368	0.114	41
dirlora-v3 無 prompt	0.058	0.210	0.418	0.137	31
dirlora-v1（前最佳）	0.075	0.242	0.465	0.162	27
base	0.046	0.150	0.318	0.108	53

Citation

If you use this model, please cite:

@misc{npc0directionalembedderv0.3,
      title={Directional Embedder v0.3}, 
      author={NPC0},
      year={2026},
      url={https://huggingface.co/npc0/directional-embedder-v0.3}, 
}