Sentence Similarity
sentence-transformers
ONNX
Safetensors
Chinese
modernbert
embeddings
clinical
healthcare
traditional-chinese
taiwan
medical
fhir
on-premise
text-embeddings-inference
Instructions to use weemed/IlhaEmbed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use weemed/IlhaEmbed with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("weemed/IlhaEmbed") sentences = [ "那是 個快樂的人", "那是 條快樂的狗", "那是 個非常幸福的人", "今天是晴天" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
rewrite: natural Taiwan wording, understated tone, correct spacing
Browse files
README.md
CHANGED
|
@@ -19,35 +19,35 @@ base_model: ibm-granite/granite-embedding-97m-multilingual-r2
|
|
| 19 |
|
| 20 |
# IlhaEmbed
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
A small embedding model
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
-
The name
|
| 29 |
|
| 30 |
-
##
|
| 31 |
|
| 32 |
-
- **Reads
|
| 33 |
-
- **Runs
|
| 34 |
-
- **
|
| 35 |
-
- **Open and commercial-
|
| 36 |
|
| 37 |
## What it is for
|
| 38 |
|
| 39 |
-
|
| 40 |
|
| 41 |
-
-
|
| 42 |
-
-
|
| 43 |
-
-
|
| 44 |
-
-
|
| 45 |
|
| 46 |
-
|
| 47 |
|
| 48 |
-
##
|
| 49 |
|
| 50 |
-
Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standard
|
| 51 |
|
| 52 |
| Model | On-prem | slang | abbrev | apposition | jargon top-1 (macro) |
|
| 53 |
|---|---|---|---|---|---|
|
|
@@ -56,9 +56,9 @@ Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standa
|
|
| 56 |
| ckip-base | yes | 0.00 | 0.01 | 0.36 | 0.12 |
|
| 57 |
| bge-small-zh | yes | 0.00 | 0.00 | 0.33 | 0.11 |
|
| 58 |
|
| 59 |
-
|
| 60 |
|
| 61 |
-
The
|
| 62 |
|
| 63 |
## Usage
|
| 64 |
|
|
@@ -79,21 +79,21 @@ model = SentenceTransformer("weemed/IlhaEmbed")
|
|
| 79 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 80 |
```
|
| 81 |
|
| 82 |
-
|
| 83 |
|
| 84 |
## How it was built
|
| 85 |
|
| 86 |
-
The base model is IBM Granite ModernBERT (Apache-2.0),
|
| 87 |
|
| 88 |
-
The base model ships a 180k multilingual BPE vocabulary, most of which Traditional Chinese clinical text never uses.
|
| 89 |
|
| 90 |
## Intended use and limits
|
| 91 |
|
| 92 |
-
|
| 93 |
|
| 94 |
-
## License
|
| 95 |
|
| 96 |
-
Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are
|
| 97 |
|
| 98 |
---
|
| 99 |
|
|
@@ -101,57 +101,57 @@ Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are docu
|
|
| 101 |
|
| 102 |
# IlhaEmbed(繁體中文)
|
| 103 |
|
| 104 |
-
[
|
| 105 |
|
| 106 |
一個讀得懂台灣臨床文字的小型語意模型。
|
| 107 |
|
| 108 |
-
|
| 109 |
|
| 110 |
-
名字取自Ilha Formosa
|
| 111 |
|
| 112 |
-
##
|
| 113 |
|
| 114 |
-
- **讀得懂
|
| 115 |
-
- **完全地端
|
| 116 |
-
- **
|
| 117 |
-
- **開源
|
| 118 |
|
| 119 |
-
## 用
|
| 120 |
|
| 121 |
-
|
| 122 |
|
| 123 |
-
- 臨床文字正規化:把
|
| 124 |
-
-
|
| 125 |
-
- 術語
|
| 126 |
-
- 跨
|
| 127 |
|
| 128 |
-
|
| 129 |
|
| 130 |
-
## 效
|
| 131 |
|
| 132 |
-
|
| 133 |
|
| 134 |
-
| 模型 |
|
| 135 |
|---|---|---|---|---|---|
|
| 136 |
| **IlhaEmbed** | 是,37 MB CPU | **0.84** | **0.82** | **0.90** | **0.85** |
|
| 137 |
| jina-embeddings-v2-base-zh | 否 | 0.06 | 0.14 | 0.45 | 0.22 |
|
| 138 |
| ckip-base | 是 | 0.00 | 0.01 | 0.36 | 0.12 |
|
| 139 |
| bge-small-zh | 是 | 0.00 | 0.00 | 0.33 | 0.11 |
|
| 140 |
|
| 141 |
-
|
| 142 |
|
| 143 |
-
|
| 144 |
|
| 145 |
-
##
|
| 146 |
|
| 147 |
```python
|
| 148 |
-
# ONNX(部署路徑,
|
| 149 |
from tokenizers import Tokenizer
|
| 150 |
import onnxruntime as ort, numpy as np
|
| 151 |
|
| 152 |
tok = Tokenizer.from_file("tokenizer.json")
|
| 153 |
sess = ort.InferenceSession("model_int8.onnx")
|
| 154 |
-
# 編碼(max_len 32)、推論、mean-pool、L2 normalize,得到384維向量
|
| 155 |
```
|
| 156 |
|
| 157 |
```python
|
|
@@ -161,18 +161,18 @@ model = SentenceTransformer("weemed/IlhaEmbed")
|
|
| 161 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 162 |
```
|
| 163 |
|
| 164 |
-
維度
|
| 165 |
|
| 166 |
-
## 怎麼做
|
| 167 |
|
| 168 |
-
|
| 169 |
|
| 170 |
-
|
| 171 |
|
| 172 |
-
## 適用範圍
|
| 173 |
|
| 174 |
-
|
| 175 |
|
| 176 |
-
## 授權
|
| 177 |
|
| 178 |
-
Apache-2.0。
|
|
|
|
| 19 |
|
| 20 |
# IlhaEmbed
|
| 21 |
|
| 22 |
+
English | [繁體中文](#繁體中文)
|
| 23 |
|
| 24 |
+
A small embedding model for Taiwanese clinical text.
|
| 25 |
|
| 26 |
+
Clinical records in Taiwan are full of local shorthand. 斷腦筋 means stroke, 不辣咖 means a blood culture, H/T is hypertension, and V(H/T) means the patient is followed for hypertension at an outside clinic. General-purpose and general Chinese models mostly miss these, because they never saw how doctors, nurses, and care workers in Taiwan actually write. IlhaEmbed was trained for it.
|
| 27 |
|
| 28 |
+
The name is from Ilha Formosa. It reads this island's clinical language.
|
| 29 |
|
| 30 |
+
## Highlights
|
| 31 |
|
| 32 |
+
- **Reads local clinical writing.** Slang, abbreviations, Taigi, handwritten shorthand. Where a general model returns noise, this one usually returns the right concept.
|
| 33 |
+
- **Runs on-premise.** 37 MB, INT8, CPU only. No GPU, no cloud, no API key. Patient data stays in the hospital.
|
| 34 |
+
- **Clear provenance.** Apache-2.0 weights, a public and documented base model (IBM Granite ModernBERT), and a pipeline you can trace. Ready for a procurement or security review.
|
| 35 |
+
- **Open and commercial-friendly.** Apache-2.0, no copyleft.
|
| 36 |
|
| 37 |
## What it is for
|
| 38 |
|
| 39 |
+
Turning messy clinical text into structured meaning. Common uses:
|
| 40 |
|
| 41 |
+
- Normalizing clinical text: mapping abbreviations and slang to standard concepts (H/T to 高血壓).
|
| 42 |
+
- Placing dirty data: on import, reading one cell and sending each fact to the field it belongs in (medication, condition, care source, and so on).
|
| 43 |
+
- Matching terms and codes: finding the right ICD, LOINC, or drug concept for a surface term.
|
| 44 |
+
- Searching across fields: finding a record from the clue at hand, without knowing the exact field name.
|
| 45 |
|
| 46 |
+
In a clinical pipeline built in Taiwan, it handles the "understanding" step: Breeze-ASR-26 for speech, IlhaEmbed for meaning, FHIR for structure.
|
| 47 |
|
| 48 |
+
## Results
|
| 49 |
|
| 50 |
+
Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standard concept from a combined pool. Slang, abbreviation, and apposition are each held out separately, self-matches excluded, scores macro-averaged. All four models use the same method and the same pool.
|
| 51 |
|
| 52 |
| Model | On-prem | slang | abbrev | apposition | jargon top-1 (macro) |
|
| 53 |
|---|---|---|---|---|---|
|
|
|
|
| 56 |
| ckip-base | yes | 0.00 | 0.01 | 0.36 | 0.12 |
|
| 57 |
| bge-small-zh | yes | 0.00 | 0.00 | 0.33 | 0.11 |
|
| 58 |
|
| 59 |
+
On these local terms the general and general-Chinese models mostly return nothing; this model finds the right concept. It is not meant to top general leaderboards such as MTEB. It does one thing, Taiwanese clinical vocabulary, and it is small enough to run on the ward.
|
| 60 |
|
| 61 |
+
The method is reproducible. The raw evaluation pairs carry third-party copyright and are not redistributed. A separate Han-Taigi to standard-concept benchmark is still in progress, and will be added once its concept mapping is clean.
|
| 62 |
|
| 63 |
## Usage
|
| 64 |
|
|
|
|
| 79 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 80 |
```
|
| 81 |
|
| 82 |
+
384 dimensions, up to 32 tokens. Vocab 25.5k (pruned to Traditional Chinese and clinical). Size 37 MB (INT8 ONNX) or 152 MB (fp32 safetensors).
|
| 83 |
|
| 84 |
## How it was built
|
| 85 |
|
| 86 |
+
The base model is IBM Granite ModernBERT (Apache-2.0), permissively licensed. A Taiwanese clinical teacher signal supplies the domain knowledge through knowledge distillation, so the student learns to place Taiwanese slang, abbreviations, and Taigi next to their standard concepts.
|
| 87 |
|
| 88 |
+
The base model ships a 180k multilingual BPE vocabulary, most of which Traditional Chinese clinical text never uses. Pruning it to the 25.5k tokens that actually appear cut the size by 68 percent (117 MB to 37 MB) with no loss of accuracy: jargon top-1 held, and the real import routing came out bit-identical.
|
| 89 |
|
| 90 |
## Intended use and limits
|
| 91 |
|
| 92 |
+
This is a tool for semantic representation, not a diagnostic system. It offers likely meanings and matches for a person to confirm, and does not decide clinical facts on its own. It is tuned for Traditional Chinese clinical text in Taiwan. General prose and Simplified Chinese are out of scope.
|
| 93 |
|
| 94 |
+
## License
|
| 95 |
|
| 96 |
+
Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are listed in `SOURCES.md`. What we release is the trained weights, a distributable derivative; the raw third-party pairs are not included.
|
| 97 |
|
| 98 |
---
|
| 99 |
|
|
|
|
| 101 |
|
| 102 |
# IlhaEmbed(繁體中文)
|
| 103 |
|
| 104 |
+
[English](#ilhaembed) | 繁體中文
|
| 105 |
|
| 106 |
一個讀得懂台灣臨床文字的小型語意模型。
|
| 107 |
|
| 108 |
+
台灣的病歷、備註、社區照護表上,有很多在地的寫法。斷腦筋是中風,不辣咖是血液培養,H/T 是高血壓,V(H/T) 是這位病人在外院追蹤高血壓。這些寫法,通用模型和一般中文模型多半讀不出來,因為它們沒看過台灣的醫師、護理師、照服員實際怎麼寫。IlhaEmbed 是針對這件事訓練的。
|
| 109 |
|
| 110 |
+
名字取自 Ilha Formosa。它專門讀這座島的臨床語言。
|
| 111 |
|
| 112 |
+
## 特點
|
| 113 |
|
| 114 |
+
- **讀得懂在地的臨床寫法。** 行話、縮寫、台語、手寫簡寫。通用模型讀成雜訊的地方,它多半能對到正確的概念。
|
| 115 |
+
- **完全在地端。** 37 MB、INT8、純 CPU。不用 GPU、雲端或 API key,病人資料不離開院內。
|
| 116 |
+
- **來源清楚。** Apache-2.0 權重,基礎模型是公開、有文件的 IBM Granite ModernBERT,訓練流程可追溯。採購或資安要看,都拿得出來。
|
| 117 |
+
- **開源、可商用。** Apache-2.0,沒有 copyleft 的限制。
|
| 118 |
|
| 119 |
+
## 用途
|
| 120 |
|
| 121 |
+
把凌亂的臨床文字整理成有結構的意義。常見的用法:
|
| 122 |
|
| 123 |
+
- 臨床文字正規化:把縮寫、行話對到標準概念(H/T 對到高血壓)。
|
| 124 |
+
- 髒資料歸位:匯入時讀一格內容,把裡面每個事實送到該去的欄位(用藥、疾病、就醫來源等)。
|
| 125 |
+
- 術語、代碼比對:幫一個詞找到對應的 ICD、LOINC 或藥品概念。
|
| 126 |
+
- 跨欄位搜尋:用手上僅有的線索找到紀錄,不必先知道正確的欄位名稱。
|
| 127 |
|
| 128 |
+
在一條台灣自己的臨床流程裡,它負責「理解」這一段:Breeze-ASR-26 聽打,IlhaEmbed 理解,FHIR 存成結構。
|
| 129 |
|
| 130 |
+
## 成效
|
| 131 |
|
| 132 |
+
測的是 jargon top-1:給一個台灣臨床的表面詞,從概念池裡找出對應的標準概念。行話、縮寫、同位語三類各自留作測試,排除自我匹配,取各類平均。四個模型用同一套方法、同一個池。
|
| 133 |
|
| 134 |
+
| 模型 | 在地端 | slang | abbrev | apposition | jargon top-1(macro) |
|
| 135 |
|---|---|---|---|---|---|
|
| 136 |
| **IlhaEmbed** | 是,37 MB CPU | **0.84** | **0.82** | **0.90** | **0.85** |
|
| 137 |
| jina-embeddings-v2-base-zh | 否 | 0.06 | 0.14 | 0.45 | 0.22 |
|
| 138 |
| ckip-base | 是 | 0.00 | 0.01 | 0.36 | 0.12 |
|
| 139 |
| bge-small-zh | 是 | 0.00 | 0.00 | 0.33 | 0.11 |
|
| 140 |
|
| 141 |
+
在這些在地詞彙上,通用模型和一般中文模型多半讀不出來,這個模型能對到正確概念。它不是要在通用排行榜(MTEB)上比高下,只把台灣臨床詞彙這一件事做好,而且小到能在病房的機器上跑。
|
| 142 |
|
| 143 |
+
這套方法可以重現。原始的評估配對有第三方著作權,不隨模型散佈。另外一個「漢字台語對到標準概念」的測試還在做,等對應整理乾淨再補上。
|
| 144 |
|
| 145 |
+
## 用法
|
| 146 |
|
| 147 |
```python
|
| 148 |
+
# ONNX(部署路徑,不需 torch、CPU)
|
| 149 |
from tokenizers import Tokenizer
|
| 150 |
import onnxruntime as ort, numpy as np
|
| 151 |
|
| 152 |
tok = Tokenizer.from_file("tokenizer.json")
|
| 153 |
sess = ort.InferenceSession("model_int8.onnx")
|
| 154 |
+
# 編碼(max_len 32)、推論、mean-pool、L2 normalize,得到 384 維向量
|
| 155 |
```
|
| 156 |
|
| 157 |
```python
|
|
|
|
| 161 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 162 |
```
|
| 163 |
|
| 164 |
+
維度 384,最長 32 個 token。詞表 25.5k(剪過,只留繁體中文和臨床會用到的)。大小 37 MB(INT8 ONNX)或 152 MB(fp32 safetensors)。
|
| 165 |
|
| 166 |
+
## 怎麼做的
|
| 167 |
|
| 168 |
+
基礎模型用 IBM Granite ModernBERT(Apache-2.0),授權寬鬆。用台灣臨床的 teacher 訊號做知識蒸餾,讓學生模型學會把台灣的行話、縮寫、台語,擺到對應的標準概念旁邊。
|
| 169 |
|
| 170 |
+
基礎模型原本帶 18 萬詞的多語 BPE 詞表,繁中臨床大多用不到。把它剪到實際會用到的 25.5k,體積少了 68%(117 MB 到 37 MB),準確度沒掉:jargon top-1 一樣,實際跑匯入的路由結果逐位相同。
|
| 171 |
|
| 172 |
+
## 適用範圍
|
| 173 |
|
| 174 |
+
這是一個做語意表示的工具,不是診斷系統。它給的是可能的意義和比對,最後由人確認,不會自己下臨床判斷。它針對台灣的繁體中文臨床文字調過,一般文章或簡體中文不在範圍內。
|
| 175 |
|
| 176 |
+
## 授權
|
| 177 |
|
| 178 |
+
Apache-2.0。基礎模型 IBM Granite(Apache-2.0)。訓練配對的來源記在 `SOURCES.md`。釋出的是訓練後的權重(可散佈的衍生成果),原始的第三方配對不隨附。
|