Transformers
Safetensors
Japanese
rwkv7
text-embed
WariHima's picture
Update README.md
53d8090 verified
|
Raw History Blame Contribute Delete
2.49 kB
---
license: apache-2.0
datasets:
- hotchpotch/sentence_transformer_japanese
language:
- ja
library_name: transformers
tags:
- rwkv7
- text-embed
---
## 断松 kilo-size-japanese-text-embedding-models
![icon](assets/DanMatsu.png)
枯松や
庭にはいらぬ
御粗松の
我枝をかり
断松を聞く
![EcoAI](assets/ecoAI.png)
(多分国内で極小の)20k~60kパラメータの日本語テキスト埋め込みモデルです。
ベンチマークも含まれています。
気になったら測ったり監査してみてね!!
### 20k model
#### architecture
rwkv7
hidden size 16dim
2 layers
chr + bpe hybrid tokenizer (vocab size 512)
#### scores
##### JSTS
```
"pearson": 0.589115,
"spearman": 0.579923,
"cosine_pearson": 0.589115,
"cosine_spearman": 0.579923,
"manhattan_pearson": 0.601691,
"manhattan_spearman": 0.571704,
"euclidean_pearson": 0.60794,
"euclidean_spearman": 0.578109,
"main_score": 0.579923,
```
##### JaQuADRetrieval
```
"hit_rate_at_1": 0.07422,
"hit_rate_at_3": 0.14893,
"hit_rate_at_5": 0.2002,
"hit_rate_at_10": 0.27441,
"hit_rate_at_20": 0.3877,
"hit_rate_at_100": 0.67139,
"hit_rate_at_1000": 0.979,
"main_score": 0.10268,
```
---
### 40k model
#### architecture
rwkv7
hidden size 16dim
2 layers
chr + bpe hybrid tokenizer (vocab size 512)
#### scores
##### JSTS
```
"pearson": 0.622386,
"spearman": 0.601339,
"cosine_pearson": 0.622386,
"cosine_spearman": 0.601339,
"manhattan_pearson": 0.626993,
"manhattan_spearman": 0.594981,
"euclidean_pearson": 0.634518,
"euclidean_spearman": 0.601892,
"main_score": 0.601339,
```
##### JaQuADRetrieval
```
"hit_rate_at_1": 0.12988,
"hit_rate_at_3": 0.23926,
"hit_rate_at_5"20k~60kパラメータの: 0.30713,
"hit_rate_at_10": 0.41113,
"hit_rate_at_20": 0.51318,
"hit_rate_at_100": 0.74707,
"hit_rate_at_1000": 0.98096,
"main_score": 0.16319,
```
---
### 60k model
#### architecture
rwkv7
hidden size 32dim
2 layers
chr + bpe hybrid tokenizer (vocab size 512)
#### scores
##### JSTS
```
"pearson": 0.685757,
"spearman": 0.656014,
"cosine_pearson": 0.685757,
"cosine_spearman": 0.656014,
"manhattan_pearson": 0.679063,
"manhattan_spearman": 0.652057,
"euclidean_pearson": 0.682095,
"euclidean_spearman": 0.655796,
"main_score": 0.656014,
```
##### JaQuADRetrieval
```
"hit_rate_at_1": 0.1792,
"hit_rate_at_3": 0.31494,
"hit_rate_at_5": 0.39404,
"hit_rate_at_10": 0.49805,
"hit_rate_at_20": 0.60254,
"hit_rate_at_100": 0.79932,
"hit_rate_at_1000": 0.98682,
"main_score": 0.20629,
```