alfotech commited on
Commit
f36d007
·
verified ·
1 Parent(s): 8f337b3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +92 -331
README.md CHANGED
@@ -1,369 +1,130 @@
1
  ---
2
  license: apache-2.0
3
  base_model:
4
- - Qwen/Qwen3-Embedding-0.6B
5
- pipeline_tag: sentence-similarity
6
  library_name: sentence-transformers
 
7
  language:
8
- - multilingual
 
9
  tags:
10
- - transformers
11
- - sentence-transformers
12
- - text-embeddings
13
- - feature-extraction
14
- - retrieval
15
- - rag
16
- - multilingual
17
- - semantic-search
18
- - qwen3
19
- - alfotech
20
  model-index:
21
- - name: alfo-embedding-0.6b
22
- results:
23
- - task:
24
- type: sentence-similarity
25
- name: Retrieval (Easy Benchmark)
26
- metrics:
27
- - type: recall_at_1
28
- value: 0.967
29
- name: Recall@1
30
- - type: recall_at_5
31
- value: 0.999
32
- name: Recall@5
33
- - type: recall_at_10
34
- value: 1.000
35
- name: Recall@10
36
- - type: mrr_at_10
37
- value: 0.9815
38
- name: MRR@10
39
- - task:
40
- type: sentence-similarity
41
- name: Retrieval (Hard Benchmark, 72,635-doc corpus / 1,000 queries)
42
- metrics:
43
- - type: recall_at_1
44
- value: 0.614
45
- name: Recall@1
46
- - type: recall_at_5
47
- value: 0.883
48
- name: Recall@5
49
- - type: recall_at_10
50
- value: 0.929
51
- name: Recall@10
52
- - type: mrr_at_10
53
- value: 0.7268
54
- name: MRR@10
 
55
  ---
56
 
57
  <p align="center">
58
- <img src="image.png" width="200"/>
59
  </p>
60
 
61
- <h1 align="center">Alfo Embedding 0.6B</h1>
62
 
63
  <p align="center">
64
- <b>Production-oriented multilingual text embedding model by Alfo Tech Industries</b><br/>
65
- Built on Qwen3-Embedding-0.6B · Evaluation-first retrieval pipeline
66
  </p>
67
 
68
  <p align="center">
69
- <a href="https://huggingface.co/alfotech/alfo-embedding-0.6b"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow"></a>
70
- <a href="#license"><img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-blue"></a>
71
- <img alt="Params" src="https://img.shields.io/badge/params-0.6B-lightgrey">
72
- <img alt="Context" src="https://img.shields.io/badge/context-32K-lightgrey">
73
- <img alt="Languages" src="https://img.shields.io/badge/languages-100%2B-lightgrey">
 
 
74
  </p>
75
 
76
  ---
77
 
78
- ## Table of Contents
79
-
80
- - [Highlights](#highlights)
81
- - [Model Overview](#model-overview)
82
- - [Quickstart](#quickstart)
83
- - [Development Philosophy](#development-philosophy)
84
- - [Training Pipeline](#training-pipeline)
85
- - [Evaluation](#evaluation)
86
- - [Failure Analysis](#failure-analysis)
87
- - [Intended Use](#intended-use)
88
- - [Limitations & Bias](#limitations--bias)
89
- - [Production Recommendations](#production-recommendations)
90
- - [Roadmap](#roadmap)
91
- - [Transparency](#transparency)
92
- - [Citation](#citation)
93
- - [License](#license)
94
-
95
- ---
96
-
97
- ## Highlights
98
 
99
- Alfo Embedding 0.6B is a production-focused embedding model built on **Qwen3-Embedding-0.6B**, optimized through an evaluation-first retrieval pipeline.
100
 
101
- Rather than training from scratch, this project focuses on improving real-world retrieval quality through:
102
 
103
- - Hard retrieval evaluation
104
- - Semantic hard-negative analysis
105
- - Contrastive embedding fine-tuning
106
- - Retrieval-oriented optimization
107
- - Production deployment validation
108
 
109
- > **Goal:** build practical AI infrastructure for semantic search and Retrieval-Augmented Generation.
110
 
111
  ---
112
 
113
- ## Model Overview
114
 
115
- | Property | Value |
116
  |---|---|
117
- | Model Type | Text Embedding |
118
- | Base Model | [Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) |
119
- | Parameters | 0.6B |
120
- | Context Length | 32K tokens |
121
- | Default Embedding Dimension | 1024 |
122
- | Supported Dimensions | 32 – 1024 (Matryoshka-style truncation) |
123
- | Languages | 100+ |
 
124
  | Instruction Aware | Yes |
 
125
  | License | Apache-2.0 |
126
 
127
- The upstream Qwen3 embedding architecture provides multilingual understanding, long-context retrieval, configurable embedding dimensions, and instruction-aware query encoding. This project extends that foundation with a retrieval-focused fine-tuning and evaluation workflow.
128
-
129
- ---
130
-
131
- ## Quickstart
132
-
133
- ### Install
134
-
135
- ```bash
136
- pip install -U sentence-transformers
137
- ```
138
-
139
- ### Basic encoding
140
-
141
- ```python
142
- from sentence_transformers import SentenceTransformer
143
-
144
- model = SentenceTransformer("alfotech/alfo-embedding-0.6b")
145
-
146
- embeddings = model.encode(
147
- [
148
- "Artificial intelligence improves search.",
149
- "Vector embeddings represent semantic meaning.",
150
- ],
151
- normalize_embeddings=True,
152
- )
153
- ```
154
-
155
- ### Instruction-aware retrieval (recommended for queries)
156
-
157
- Qwen3-derived embedding models perform best when queries are prefixed with a retrieval instruction; documents are encoded without one.
158
-
159
- ```python
160
- from sentence_transformers import SentenceTransformer
161
-
162
- model = SentenceTransformer("alfotech/alfo-embedding-0.6b")
163
-
164
- queries = ["How can I optimize transformer inference?"]
165
- documents = [
166
- "Techniques such as quantization, KV-cache reuse, and batching reduce inference latency.",
167
- "The stock market closed higher today on strong earnings reports.",
168
- ]
169
-
170
- query_embeddings = model.encode(queries, prompt_name="query", normalize_embeddings=True)
171
- doc_embeddings = model.encode(documents, normalize_embeddings=True)
172
-
173
- scores = model.similarity(query_embeddings, doc_embeddings)
174
- print(scores)
175
- ```
176
-
177
- ### Reducing embedding dimension
178
-
179
- ```python
180
- embeddings = model.encode(
181
- ["Shorter vectors for cheaper storage."],
182
- normalize_embeddings=True,
183
- truncate_dim=256, # any value from 32–1024
184
- )
185
- ```
186
-
187
  ---
188
 
189
- ## Development Philosophy
190
 
191
- Rather than optimizing only for easy benchmarks, the training pipeline intentionally targets difficult retrieval scenarios where semantically similar distractors are present.
192
 
193
  ```text
194
- Baseline → Hard Benchmark → Failure Analysis → Hard Negative Mining
195
- → Fine-Tuning → Validation → Production Testing
196
- ```
197
-
198
- Every optimization decision is driven by measured evaluation rather than intuition.
199
-
200
- ---
201
-
202
- ## Training Pipeline
203
-
204
- Development was performed in **Google Colab** using dual Tesla T4 GPUs.
205
-
206
- **Pipeline components:**
207
-
208
- - Retrieval dataset preparation
209
- - Duplicate removal
210
- - Quality filtering
211
- - Validation leakage checks
212
- - Hard-negative mining
213
- - Multiple Negatives Ranking Loss with in-batch negatives
214
- - Continuous retrieval evaluation
215
- - Production validation
216
-
217
- ---
218
-
219
- ## Evaluation
220
-
221
- The model is evaluated under both easy and hard retrieval conditions.
222
-
223
- ### Easy Benchmark
224
-
225
- | Metric | Score |
226
- |---|---|
227
- | Recall@1 | **0.967** |
228
- | Recall@5 | **0.999** |
229
- | Recall@10 | **1.000** |
230
- | MRR@10 | **0.9815** |
231
-
232
- Validates the retrieval pipeline under standard, low-ambiguity conditions.
233
-
234
- ### Hard Retrieval Benchmark
235
-
236
- Corpus: **72,635 documents** · Queries: **1,000**
237
-
238
- | Metric | Score |
239
- |---|---|
240
- | Recall@1 | **0.614** |
241
- | Recall@5 | **0.883** |
242
- | Recall@10 | **0.929** |
243
- | MRR@10 | **0.7268** |
244
-
245
- Introduces semantically similar distractors, making it more representative of production retrieval systems than the easy benchmark alone.
246
-
247
- ---
248
-
249
- ## Failure Analysis
250
-
251
- | Metric | Value |
252
- |---|---|
253
- | Semantic hard negatives identified | **386** |
254
- | Avg. positive similarity | **0.689** |
255
- | Avg. top-negative similarity | **0.653** |
256
- | Avg. margin (positive − negative) | **0.036** |
257
- | Duplicate contamination detected | **0** |
258
- | Noisy positives detected | **0** |
259
-
260
- The narrow average margin (0.036) between true positives and top hard negatives is the primary remaining bottleneck and is the main target of the next fine-tuning iteration (see [Roadmap](#roadmap)).
261
-
262
- ---
263
-
264
- ## Intended Use
265
-
266
- **Recommended use cases:**
267
-
268
- - Semantic search
269
- - Retrieval-Augmented Generation (RAG)
270
- - Enterprise / knowledge-base search
271
- - FAQ matching
272
- - Code search
273
- - Similarity search
274
- - Vector database indexing
275
- - Multilingual retrieval
276
-
277
- **Out of scope:**
278
-
279
- - Text generation, classification, or summarization — this is an embedding-only model, not a generative LLM.
280
- - High-stakes decisions (legal, medical, financial, safety) made solely from similarity scores without human review.
281
- - Any use case requiring guarantees on protected or demographic attributes; no such evaluation has been performed (see below).
282
-
283
- ---
284
-
285
- ## Limitations & Bias
286
-
287
- - Evaluation to date is retrieval-accuracy focused (Recall@k, MRR@10). No dedicated fairness, bias, or demographic-parity audit has been conducted.
288
- - Hard-benchmark performance (Recall@1 of 0.614) is meaningfully lower than easy-benchmark performance (0.967), so retrieval quality should be validated on your own domain data before production rollout.
289
- - Long-context (32K) behavior is inherited from the base model and has not been independently re-validated by this project.
290
- - As with any embedding model, outputs reflect patterns in the underlying pretraining and fine-tuning data and may not generalize evenly across all 100+ supported languages.
291
-
292
- ---
293
-
294
- ## Production Recommendations
295
-
296
- | Setting | Value |
297
- |---|---|
298
- | Sequence Length | 512 |
299
- | Batch Size | 32 |
300
- | Normalize Embeddings | Enabled |
301
- | Similarity Metric | Cosine |
302
-
303
- These provide a practical starting balance between retrieval quality and inference throughput — tune per workload.
304
-
305
- ---
306
-
307
- ## Roadmap
308
-
309
- - Larger, curated retrieval datasets
310
- - Better hard-negative refinement (targeting the 0.036 avg. margin)
311
- - Domain-specific adaptation
312
- - Full blind evaluation
313
- - Extended benchmark coverage
314
- - Additional deployment optimizations
315
-
316
- ---
317
-
318
- ## Transparency
319
-
320
- This repository prioritizes reproducibility. Current documentation includes training methodology, retrieval evaluation, hard-benchmark results, production inference examples, and deployment recommendations.
321
-
322
- External benchmark leaderboard claims are intentionally omitted unless independently measured.
323
-
324
- ---
325
-
326
- ## Citation
327
-
328
- If this project helps your work, please cite both the upstream Qwen3 Embedding paper and this repository.
329
-
330
- **Qwen3 Embedding**
331
-
332
- ```bibtex
333
- @article{qwen3embedding,
334
- title={Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models},
335
- author={Zhang, Yanzhao and Li, Mingxin and Long, Dingkun and others},
336
- journal={arXiv preprint arXiv:2506.05176},
337
- year={2025}
338
- }
339
- ```
340
-
341
- **Alfo Embedding**
342
-
343
- ```bibtex
344
- @misc{alfotech_alfo_embedding_2026,
345
- title={Alfo Embedding 0.6B},
346
- author={Alfo Tech Industries},
347
- year={2026},
348
- publisher={Hugging Face},
349
- url={https://huggingface.co/alfotech/alfo-embedding-0.6b}
350
- }
351
- ```
352
-
353
- ---
354
-
355
- ## Acknowledgements
356
-
357
- This project builds upon the open-source **Qwen3-Embedding-0.6B** foundation model, whose multilingual and long-context embedding architecture made this work possible.
358
-
359
- ## License
360
-
361
- Apache-2.0, inherited from the upstream Qwen3-Embedding-0.6B model.
362
-
363
- ---
364
-
365
- <p align="center">
366
- <b>Built by Alfo Tech Industries</b><br/>
367
- <i>Building practical AI infrastructure for developers, startups, and enterprises.</i><br/>
368
- <a href="https://github.com/Alfo-Tech-Lab">GitHub</a>
369
- </p>
 
1
  ---
2
  license: apache-2.0
3
  base_model:
4
+ - Qwen/Qwen3-Embedding-0.6B
5
+ pipeline_tag: feature-extraction
6
  library_name: sentence-transformers
7
+
8
  language:
9
+ - multilingual
10
+
11
  tags:
12
+ - text-embeddings
13
+ - feature-extraction
14
+ - sentence-transformers
15
+ - transformers
16
+ - multilingual
17
+ - qwen3
18
+ - silas
19
+ - alfotech
20
+
 
21
  model-index:
22
+ - name: silas-embedding-0.6b
23
+ results:
24
+ - task:
25
+ type: feature-extraction
26
+ name: Embedding Model — Easy Benchmark
27
+ metrics:
28
+ - type: recall_at_1
29
+ name: Recall@1
30
+ value: 0.967
31
+ - type: recall_at_5
32
+ name: Recall@5
33
+ value: 0.999
34
+ - type: recall_at_10
35
+ name: Recall@10
36
+ value: 1.000
37
+ - type: mrr_at_10
38
+ name: MRR@10
39
+ value: 0.9815
40
+
41
+ - task:
42
+ type: feature-extraction
43
+ name: Embedding Model — Hard Benchmark
44
+ metrics:
45
+ - type: recall_at_1
46
+ name: Recall@1
47
+ value: 0.614
48
+ - type: recall_at_5
49
+ name: Recall@5
50
+ value: 0.883
51
+ - type: recall_at_10
52
+ name: Recall@10
53
+ value: 0.929
54
+ - type: mrr_at_10
55
+ name: MRR@10
56
+ value: 0.7268
57
  ---
58
 
59
  <p align="center">
60
+ <img src="image.png" width="220"/>
61
  </p>
62
 
63
+ <h1 align="center">Silas Embedding 0.6B</h1>
64
 
65
  <p align="center">
66
+ <b>Production-Grade Embedding Model by Alfo Tech Industries</b><br/>
67
+ Multilingual · Long Context · Semantic Understanding · RAG Ready
68
  </p>
69
 
70
  <p align="center">
71
+ <a href="https://huggingface.co/alfotech/silas-embedding-0.6b">
72
+ <img src="https://img.shields.io/badge/🤗-Hugging%20Face-yellow" alt="Hugging Face">
73
+ </a>
74
+ <img src="https://img.shields.io/badge/Parameters-0.6B-6D5BFF" alt="Parameters">
75
+ <img src="https://img.shields.io/badge/Context-32K-00E5FF" alt="Context">
76
+ <img src="https://img.shields.io/badge/Languages-100%2B-success" alt="Languages">
77
+ <img src="https://img.shields.io/badge/License-Apache--2.0-blue" alt="License">
78
  </p>
79
 
80
  ---
81
 
82
+ # Silas
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
 
84
+ ## An Embedding Model Built for Real-World AI Systems
85
 
86
+ **Silas Embedding 0.6B** is a production-oriented multilingual embedding model developed by **Alfo Tech Industries**.
87
 
88
+ Silas is designed to convert text into dense vector representations suitable for semantic understanding, similarity search, retrieval systems, knowledge bases, and Retrieval-Augmented Generation.
 
 
 
 
89
 
90
+ The model is built upon **Qwen3-Embedding-0.6B** and follows an evaluation-first development methodology focused on difficult semantic matching scenarios.
91
 
92
  ---
93
 
94
+ # At a Glance
95
 
96
+ | Specification | Details |
97
  |---|---|
98
+ | Model | Silas Embedding 0.6B |
99
+ | Organization | Alfo Tech Industries |
100
+ | Base Model | Qwen3-Embedding-0.6B |
101
+ | Parameters | **0.6B** |
102
+ | Context Window | **32K** |
103
+ | Default Embedding Dimension | **1024** |
104
+ | Supported Dimensions | **32–1024** |
105
+ | Languages | **100+** |
106
  | Instruction Aware | Yes |
107
+ | Framework | Sentence Transformers |
108
  | License | Apache-2.0 |
109
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
110
  ---
111
 
112
+ # What Silas Does
113
 
114
+ Silas transforms text into numerical representations that preserve semantic relationships.
115
 
116
  ```text
117
+ Text
118
+
119
+
120
+ Silas Embedding Model
121
+
122
+
123
+ Dense Vector
124
+
125
+ ├── Semantic Search
126
+ ├── RAG
127
+ ├── Similarity Search
128
+ ├── Knowledge Retrieval
129
+ ├── FAQ Matching
130
+ └── Vector Databases