npc0 commited on
Commit
47e9630
·
verified ·
1 Parent(s): 1abad69

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +83 -0
README.md CHANGED
@@ -1,3 +1,86 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ language:
6
+ - en
7
+ tags:
8
+ - lora
9
+ - fine-tuned-model
10
+ - instruction-tuning
11
+ - adaptation
12
+ - parameter-efficient-finetuning
13
+ metrics:
14
+ - perplexity
15
+ - bleu
16
+ - rouge
17
+ datasets:
18
+ - iohadrubin/wikitext-103-raw-v1
19
+ base_model:
20
+ - Qwen/Qwen3-Embedding-0.6B
21
  ---
22
+
23
+ # dirlora-v3
24
+
25
+ `dirlora-v3` is a LoRA-adapted model checkpoint built for concise, instruction-style text generation with
26
+ improved in-context behavior on targeted tasks.
27
+
28
+ ## Model details
29
+ - **Model type:** Causal Language Model (LoRA adapter)
30
+ - **Base model:** `Qwen/Qwen3-Embedding-0.6B`
31
+ - **Fine-tuning method:** LoRA / PEFT
32
+ - **LoRA config:**
33
+ - Rank (`r`): `16`
34
+ - Alpha (`alpha`): `{{alpha}}`
35
+ - Dropout: `{{lora_dropout}}`
36
+ - **Training data:** subset of `wikitext 103` for PoC
37
+ - **Framework:** PyTorch + Hugging Face `transformers` + `peft`
38
+
39
+ ## Intended use
40
+ Use this model for:
41
+ - text embedding
42
+ - experimenting direction/sequence aware embedding
43
+
44
+ ## Not intended for
45
+ - any engineering solution
46
+
47
+ ## Usage
48
+
49
+ ```python
50
+ from sentence_transformers import SentenceTransformer
51
+
52
+ model = SentenceTransformer("npc0/Qwen3-Embedding-0.6B-OED")
53
+
54
+ sentences = [
55
+ "The weather is lovely today.",
56
+ "It's so sunny outside!",
57
+ "He drove to the stadium."
58
+ ]
59
+ embeddings = model.encode(sentences)
60
+
61
+ similarities = model.similarity(embeddings, embeddings)
62
+ print(similarities.shape)
63
+ ```
64
+ ## Limitations
65
+
66
+ - This is early stage PoC not converged but made progress comparing to previous versions
67
+
68
+ | 配置 | R@1 | R@5 | R@20 | MRR | med |
69
+ |---|---|---|---|---|---|
70
+ | **dirlora-v3 + next** | **0.110** | **0.297** | **0.505** | **0.200** | **20** |
71
+ | dirlora-v3 + prev(對照) | 0.043 | 0.167 | 0.368 | 0.114 | 41 |
72
+ | dirlora-v3 無 prompt | 0.058 | 0.210 | 0.418 | 0.137 | 31 |
73
+ | dirlora-v1(前最佳) | 0.075 | 0.242 | 0.465 | 0.162 | 27 |
74
+ | base | 0.046 | 0.150 | 0.318 | 0.108 | 53 |
75
+
76
+ ## Citation
77
+
78
+ If you use this model, please cite:
79
+ ```
80
+ @misc{npc0directionalembedderv0.3,
81
+ title={Directional Embedder v0.3},
82
+ author={NPC0},
83
+ year={2026},
84
+ url={https://huggingface.co/npc0/directional-embedder-v0.3},
85
+ }
86
+ ```