Sentence Similarity
sentence-transformers
ONNX
Safetensors
Chinese
modernbert
embeddings
clinical
healthcare
traditional-chinese
taiwan
medical
fhir
on-premise
text-embeddings-inference
Instructions to use weemed/IlhaEmbed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use weemed/IlhaEmbed with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("weemed/IlhaEmbed") sentences = [ "那是 個快樂的人", "那是 條快樂的狗", "那是 個非常幸福的人", "今天是晴天" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
model card: natural voice, no em-dashes, real numbers
Browse files
README.md
CHANGED
|
@@ -19,70 +19,55 @@ base_model: ibm-granite/granite-embedding-97m-multilingual-r2
|
|
| 19 |
|
| 20 |
# IlhaEmbed
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
|
| 27 |
-
attends an outside clinic *for* hypertension. Generic and general-Chinese embedders read none of these —
|
| 28 |
-
they were never trained on how Taiwanese clinicians, nurses, and care workers really write in charts,
|
| 29 |
-
remarks, and community-care sheets. **IlhaEmbed was.**
|
| 30 |
-
|
| 31 |
-
*Ilha* — from *Ilha Formosa*, "beautiful island." An embedder that understands this island's clinical
|
| 32 |
-
mother tongue.
|
| 33 |
|
| 34 |
## Why IlhaEmbed
|
| 35 |
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
| 🧾 **Transparent, auditable lineage** | Apache-2.0 weights, a documented base model (IBM Granite ModernBERT), and a fully traceable training pipeline. Provenance you can hand to a procurement or infosec review, not a black box. |
|
| 41 |
-
| 🪶 **Open and commercial-ready** | Apache-2.0 — use it, ship it, embed it in a product, no copyleft strings. |
|
| 42 |
|
| 43 |
-
## What it
|
| 44 |
|
| 45 |
-
IlhaEmbed turns messy Taiwanese clinical text into structured meaning.
|
| 46 |
|
| 47 |
-
-
|
| 48 |
-
-
|
| 49 |
-
-
|
| 50 |
-
-
|
| 51 |
|
| 52 |
-
It is the normalization layer of a
|
| 53 |
-
**`Breeze-ASR-26` (speech) → `IlhaEmbed` (meaning) → `FHIR` (structure).**
|
| 54 |
|
| 55 |
## Benchmarks
|
| 56 |
|
| 57 |
-
|
| 58 |
-
from a combined concept pool. Each register (slang, abbreviation, apposition) is held out separately;
|
| 59 |
-
self-matches excluded; macro-averaged over registers. Same methodology and pool for every model.
|
| 60 |
|
| 61 |
-
| Model |
|
| 62 |
|---|---|---|---|---|---|
|
| 63 |
-
| **IlhaEmbed** |
|
| 64 |
-
| jina-embeddings-v2-base-zh |
|
| 65 |
-
| ckip-base |
|
| 66 |
-
| bge-small-zh |
|
| 67 |
|
| 68 |
-
|
| 69 |
-
were never trained to read. We do **not** claim to top general leaderboards (MTEB): IlhaEmbed is a
|
| 70 |
-
*specialist* that wins where generic models fail, while staying small enough to run on the ward.
|
| 71 |
|
| 72 |
-
|
| 73 |
-
evaluation pairs carry third-party copyright and are not redistributed. A dedicated Han-Taigi →
|
| 74 |
-
standard-concept benchmark is in progress and reported separately once its concept mapping is clean.)*
|
| 75 |
|
| 76 |
## Usage
|
| 77 |
|
| 78 |
```python
|
| 79 |
-
# ONNX (deployment path
|
| 80 |
from tokenizers import Tokenizer
|
| 81 |
import onnxruntime as ort, numpy as np
|
| 82 |
|
| 83 |
tok = Tokenizer.from_file("tokenizer.json")
|
| 84 |
sess = ort.InferenceSession("model_int8.onnx")
|
| 85 |
-
#
|
| 86 |
```
|
| 87 |
|
| 88 |
```python
|
|
@@ -92,23 +77,21 @@ model = SentenceTransformer("weemed/IlhaEmbed")
|
|
| 92 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 93 |
```
|
| 94 |
|
| 95 |
-
|
| 96 |
|
| 97 |
## How it was built
|
| 98 |
|
| 99 |
-
|
| 100 |
-
- **Distillation:** a Taiwan-clinical teacher signal supplies the domain knowledge; the student learns to place Taiwanese slang, abbreviations, and Taigi near their canonical concepts.
|
| 101 |
-
- **Vocabulary pruning:** the base 180k multilingual BPE vocab was pruned to the 25.5k tokens Traditional-Chinese clinical text actually uses — a **68% size cut (117 MB → 37 MB) with zero accuracy loss** (jargon top-1 held; real-pipeline routing bit-identical).
|
| 102 |
|
| 103 |
-
|
| 104 |
|
| 105 |
-
|
| 106 |
|
| 107 |
-
|
| 108 |
|
| 109 |
-
|
| 110 |
|
| 111 |
-
|
| 112 |
|
| 113 |
---
|
| 114 |
|
|
@@ -118,59 +101,55 @@ Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources document
|
|
| 118 |
|
| 119 |
[**English**](#ilhaembed) | **繁體中文**
|
| 120 |
|
| 121 |
-
|
| 122 |
|
| 123 |
-
|
| 124 |
-
模型全都讀不懂——它們從沒學過台灣的醫師、護理師、照服員真正怎麼在病歷、備註、社區照護表上書寫。**IlhaEmbed 學過。**
|
| 125 |
|
| 126 |
-
|
| 127 |
|
| 128 |
-
## 為什麼選
|
| 129 |
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
| 🧾 **透明可稽核的血統** | Apache-2.0 權重、有文件的底模(IBM Granite ModernBERT)、可完整追溯的訓練管線。是可以交給採購與資安審查的來源證明,不是黑盒子。 |
|
| 135 |
-
| 🪶 **開源且可商用** | Apache-2.0——自己用、包進產品、隨貨出貨都行,沒有 copyleft 綁約。 |
|
| 136 |
|
| 137 |
## 用來做什麼
|
| 138 |
|
| 139 |
-
IlhaEmbed
|
| 140 |
|
| 141 |
-
-
|
| 142 |
-
-
|
| 143 |
-
-
|
| 144 |
-
-
|
| 145 |
|
| 146 |
-
它是一條
|
| 147 |
-
**`Breeze-ASR-26`(聽音)→ `IlhaEmbed`(理解)→ `FHIR`(結構)。**
|
| 148 |
|
| 149 |
## 效能指標
|
| 150 |
|
| 151 |
-
|
| 152 |
|
| 153 |
-
| 模型 | 可地端
|
| 154 |
|---|---|---|---|---|---|
|
| 155 |
-
| **IlhaEmbed** |
|
| 156 |
-
| jina-embeddings-v2-base-zh |
|
| 157 |
-
| ckip-base |
|
| 158 |
-
| bge-small-zh |
|
| 159 |
|
| 160 |
-
|
| 161 |
|
| 162 |
-
|
| 163 |
|
| 164 |
## 使用方式
|
| 165 |
|
| 166 |
```python
|
| 167 |
-
# ONNX(部署路徑
|
| 168 |
from tokenizers import Tokenizer
|
| 169 |
import onnxruntime as ort, numpy as np
|
| 170 |
|
| 171 |
tok = Tokenizer.from_file("tokenizer.json")
|
| 172 |
sess = ort.InferenceSession("model_int8.onnx")
|
| 173 |
-
#
|
| 174 |
```
|
| 175 |
|
| 176 |
```python
|
|
@@ -180,20 +159,18 @@ model = SentenceTransformer("weemed/IlhaEmbed")
|
|
| 180 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 181 |
```
|
| 182 |
|
| 183 |
-
|
| 184 |
|
| 185 |
## 怎麼做出來的
|
| 186 |
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
|
| 191 |
## 適用範圍與限制
|
| 192 |
|
| 193 |
-
IlhaEmbed
|
| 194 |
|
| 195 |
## 授權與引用
|
| 196 |
|
| 197 |
-
Apache-2.0。底模:IBM Granite(Apache-2.0)。訓練配對來源記於
|
| 198 |
-
|
| 199 |
-
*IlhaEmbed——讀得懂這座島的臨床母語。*
|
|
|
|
| 19 |
|
| 20 |
# IlhaEmbed
|
| 21 |
|
| 22 |
+
A small embedding model that reads Taiwanese clinical text.
|
| 23 |
|
| 24 |
+
These are words that clinical staff in Taiwan actually write in charts, remarks, and community-care sheets. 斷腦筋 means stroke. 不辣咖 means a blood culture. H/T is hypertension, and V(H/T) means the patient is seen at an outside clinic for hypertension. General-purpose models and general Chinese models read none of these, because they never learned how doctors, nurses, and care workers in Taiwan really write. IlhaEmbed learned it.
|
| 25 |
|
| 26 |
+
The name comes from Ilha Formosa. It is an embedder for this island's clinical language.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## Why IlhaEmbed
|
| 29 |
|
| 30 |
+
- **Reads Taiwan's clinical language.** Slang, abbreviations, Taigi, the handwritten shorthand in real records. Where a general model returns noise, this one returns the right concept.
|
| 31 |
+
- **Runs fully on-premise.** 37 MB, INT8, CPU only. No GPU, no cloud, no API key. Patient data never leaves the hospital.
|
| 32 |
+
- **Transparent lineage.** Apache-2.0 weights, a documented base model (IBM Granite ModernBERT), and a training pipeline you can trace. Provenance you can hand to a procurement or security review, not a black box.
|
| 33 |
+
- **Open and commercial-ready.** Apache-2.0, with no copyleft obligations.
|
|
|
|
|
|
|
| 34 |
|
| 35 |
+
## What it is for
|
| 36 |
|
| 37 |
+
IlhaEmbed turns messy Taiwanese clinical text into structured meaning. Common uses:
|
| 38 |
|
| 39 |
+
- Clinical text normalization: mapping free-text abbreviations and slang to standard concepts (H/T to 高血壓).
|
| 40 |
+
- Routing dirty data into structured fields: the semantic layer of an importer that reads a spreadsheet cell and sends each fact to its correct home (medication, condition, care source, and so on).
|
| 41 |
+
- Terminology and code matching: retrieving the right ICD, LOINC, or drug concept for a surface term.
|
| 42 |
+
- Cross-boundary search: finding a record from the clue a nurse actually has, not the exact field name.
|
| 43 |
|
| 44 |
+
It is the normalization layer of a clinical pipeline built entirely in Taiwan: Breeze-ASR-26 for speech, IlhaEmbed for meaning, FHIR for structure.
|
|
|
|
| 45 |
|
| 46 |
## Benchmarks
|
| 47 |
|
| 48 |
+
Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standard clinical concept from a combined concept pool. Each register (slang, abbreviation, apposition) is held out separately, self-matches are excluded, and the score is macro-averaged. Every model uses the same method and the same pool.
|
|
|
|
|
|
|
| 49 |
|
| 50 |
+
| Model | On-prem | slang | abbrev | apposition | jargon top-1 (macro) |
|
| 51 |
|---|---|---|---|---|---|
|
| 52 |
+
| **IlhaEmbed** | yes, 37 MB CPU | **0.84** | **0.82** | **0.90** | **0.85** |
|
| 53 |
+
| jina-embeddings-v2-base-zh | no | 0.06 | 0.14 | 0.45 | 0.22 |
|
| 54 |
+
| ckip-base | yes | 0.00 | 0.01 | 0.36 | 0.12 |
|
| 55 |
+
| bge-small-zh | yes | 0.00 | 0.00 | 0.33 | 0.11 |
|
| 56 |
|
| 57 |
+
That is about four times the best general model and about eight times bge, on exactly the vocabulary those models were never trained to read. IlhaEmbed does not aim to top general leaderboards such as MTEB. It is a specialist: it wins where general models fail, and it is small enough to run on the ward.
|
|
|
|
|
|
|
| 58 |
|
| 59 |
+
The register-held-out method above is reproducible. The raw evaluation pairs carry third-party copyright and are not redistributed. A separate Han-Taigi to standard-concept benchmark is in progress and will be reported once its concept mapping is clean.
|
|
|
|
|
|
|
| 60 |
|
| 61 |
## Usage
|
| 62 |
|
| 63 |
```python
|
| 64 |
+
# ONNX (deployment path, no torch, CPU)
|
| 65 |
from tokenizers import Tokenizer
|
| 66 |
import onnxruntime as ort, numpy as np
|
| 67 |
|
| 68 |
tok = Tokenizer.from_file("tokenizer.json")
|
| 69 |
sess = ort.InferenceSession("model_int8.onnx")
|
| 70 |
+
# encode (max_len 32), run, mean-pool, L2-normalize, giving a 384-d vector
|
| 71 |
```
|
| 72 |
|
| 73 |
```python
|
|
|
|
| 77 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 78 |
```
|
| 79 |
|
| 80 |
+
Dimensions: 384. Max sequence: 32 tokens. Vocab: 25.5k (pruned to Traditional Chinese and clinical). Size: 37 MB (INT8 ONNX) or 152 MB (fp32 safetensors).
|
| 81 |
|
| 82 |
## How it was built
|
| 83 |
|
| 84 |
+
The base model is IBM Granite ModernBERT (Apache-2.0), a permissively licensed foundation. A Taiwanese clinical teacher signal supplies the domain knowledge through knowledge distillation, so the student learns to place Taiwanese slang, abbreviations, and Taigi near their standard concepts.
|
|
|
|
|
|
|
| 85 |
|
| 86 |
+
The base model ships a 180k multilingual BPE vocabulary, most of which Traditional Chinese clinical text never uses. We pruned it to the 25.5k tokens that actually appear, which cut the size by 68 percent (117 MB to 37 MB) with no loss of accuracy: jargon top-1 held, and real-pipeline routing stayed bit-identical.
|
| 87 |
|
| 88 |
+
## Intended use and limits
|
| 89 |
|
| 90 |
+
IlhaEmbed is a tool for semantic representation, not a diagnostic system. It surfaces likely meanings and matches for a person to confirm; it must never decide clinical facts on its own without review. It is tuned for Traditional Chinese clinical text in Taiwan. General prose or Simplified Chinese is out of scope.
|
| 91 |
|
| 92 |
+
## License and citation
|
| 93 |
|
| 94 |
+
Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are documented in `SOURCES.md`. The released weights are a distributable derivative; the raw third-party pairs are not redistributed.
|
| 95 |
|
| 96 |
---
|
| 97 |
|
|
|
|
| 101 |
|
| 102 |
[**English**](#ilhaembed) | **繁體中文**
|
| 103 |
|
| 104 |
+
一個讀得懂台灣臨床文字的小型語意模型。
|
| 105 |
|
| 106 |
+
以下這些是台灣臨床人員實際會寫在病歷、備註、社區照護表上的詞。斷腦筋是中風。不辣咖是血液培養。H/T是高血壓,V(H/T)是這位病人在外院看高血壓。通用模型與一般中文模型全都讀不懂,因為它們從沒學過台灣的醫師、護理師、照服員真正怎麼書寫。IlhaEmbed學過。
|
|
|
|
| 107 |
|
| 108 |
+
名字取自Ilha Formosa(美麗島)。它是一個讀這座島臨床語言的語意模型。
|
| 109 |
|
| 110 |
+
## 為什麼選IlhaEmbed
|
| 111 |
|
| 112 |
+
- **讀得懂台灣臨床語言。** 行話、縮寫、台語、手寫備註簡寫,都是真實病歷裡的文字。通用模型讀成雜訊的地方,它讀出正確概念。
|
| 113 |
+
- **完全地端執行。** 37 MB、INT8、純CPU。免GPU、免雲端、免API key。病人資料不離開醫院。
|
| 114 |
+
- **透明的血統。** Apache-2.0權重、有文件的底模(IBM Granite ModernBERT)、可追溯的訓練管線。是可以交給採購與資安審查的來源證明,不是黑盒子。
|
| 115 |
+
- **開源且可商用。** Apache-2.0,沒有copyleft綁約。
|
|
|
|
|
|
|
| 116 |
|
| 117 |
## 用來做什麼
|
| 118 |
|
| 119 |
+
IlhaEmbed把凌亂的台灣臨床文字轉成結構化的意義。常見用途:
|
| 120 |
|
| 121 |
+
- 臨床文字正規化:把free-text縮寫與行話對應到標準概念(H/T對到高血壓)。
|
| 122 |
+
- 把髒資料路由到結構化欄位:匯入器的語意層,讀一格試算表、把每個事實送到正確的歸宿(用藥、疾病、就醫來源等)。
|
| 123 |
+
- 術語與代碼比對:為一個表面詞找到正確的ICD、LOINC或藥品概念。
|
| 124 |
+
- 跨界搜尋:用護理師手上真正有的線索找到紀錄,而不是要求精確欄位名。
|
| 125 |
|
| 126 |
+
它是一條完全在台灣打造的臨床管線的正規化層:Breeze-ASR-26聽音,IlhaEmbed理解,FHIR結構化。
|
|
|
|
| 127 |
|
| 128 |
## 效能指標
|
| 129 |
|
| 130 |
+
任務:jargon top-1。給一個台灣臨床表面詞,於合併概念池中檢索其標準臨床概念。各語域(行話、縮寫、同位語)分別留出、排除自我匹配、macro平均。所有模型使用同一方法與同一概念池。
|
| 131 |
|
| 132 |
+
| 模型 | 可地端 | slang | abbrev | apposition | jargon top-1(macro) |
|
| 133 |
|---|---|---|---|---|---|
|
| 134 |
+
| **IlhaEmbed** | 是,37 MB CPU | **0.84** | **0.82** | **0.90** | **0.85** |
|
| 135 |
+
| jina-embeddings-v2-base-zh | 否 | 0.06 | 0.14 | 0.45 | 0.22 |
|
| 136 |
+
| ckip-base | 是 | 0.00 | 0.01 | 0.36 | 0.12 |
|
| 137 |
+
| bge-small-zh | 是 | 0.00 | 0.00 | 0.33 | 0.11 |
|
| 138 |
|
| 139 |
+
約為最佳通用模型的四倍、bge的近八倍,就贏在那些模型從沒學過的在地詞彙。IlhaEmbed不以在通用排行榜(MTEB)霸榜為目標。它是專科模型:只贏在通用模型會爆掉的地方,同時小到能在病房裡跑。
|
| 140 |
|
| 141 |
+
上述register-held-out方法可重現。原始評估配對含第三方著作權,不隨附散佈。專門的「漢字台語對到標準概念」基準進行中,待概念對應乾淨後另行報告。
|
| 142 |
|
| 143 |
## 使用方式
|
| 144 |
|
| 145 |
```python
|
| 146 |
+
# ONNX(部署路徑,免torch、CPU)
|
| 147 |
from tokenizers import Tokenizer
|
| 148 |
import onnxruntime as ort, numpy as np
|
| 149 |
|
| 150 |
tok = Tokenizer.from_file("tokenizer.json")
|
| 151 |
sess = ort.InferenceSession("model_int8.onnx")
|
| 152 |
+
# 編碼(max_len 32)、推論、mean-pool、L2 normalize,得到384維向量
|
| 153 |
```
|
| 154 |
|
| 155 |
```python
|
|
|
|
| 159 |
model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
|
| 160 |
```
|
| 161 |
|
| 162 |
+
維度:384。最大序列:32 tokens。詞表:25.5k(剪枝至繁體中文與臨床)。大小:37 MB(INT8 ONNX)或152 MB(fp32 safetensors)。
|
| 163 |
|
| 164 |
## 怎麼做出來的
|
| 165 |
|
| 166 |
+
底模是IBM Granite ModernBERT(Apache-2.0),一個寬鬆授權的基礎。由台灣臨床teacher訊號透過知識蒸餾提供領域知識,讓學生學會把台灣行話、縮寫、台語擺到它們的標準概念旁邊。
|
| 167 |
+
|
| 168 |
+
底模原帶18萬詞的多語BPE詞表,其中大多數繁中臨床文字用不到。我們把它剪到實際會用到的25.5k token,體積因此減少68%(117 MB到37 MB),且準確度零損:jargon top-1不掉,真實管線路由逐位一致。
|
| 169 |
|
| 170 |
## 適用範圍與限制
|
| 171 |
|
| 172 |
+
IlhaEmbed是語意表示工具,不是診斷系統。它浮現可能的意義與比對供人確認,絕不可在未經審核下自行判定臨床事實。它為台灣的繁體中文臨床文字調校。一般散文或簡體中文不在範圍內。
|
| 173 |
|
| 174 |
## 授權與引用
|
| 175 |
|
| 176 |
+
Apache-2.0。底模:IBM Granite(Apache-2.0)。訓練配對來源記於`SOURCES.md`。釋出的權重是可散佈的衍生作,原始第三方配對不隨附散佈。
|
|
|
|
|
|