gloomcheng commited on
Commit
d5c01d8
·
verified ·
1 Parent(s): 8987818

rewrite: natural Taiwan wording, understated tone, correct spacing

Browse files
Files changed (1) hide show
  1. README.md +56 -56
README.md CHANGED
@@ -19,35 +19,35 @@ base_model: ibm-granite/granite-embedding-97m-multilingual-r2
19
 
20
  # IlhaEmbed
21
 
22
- **English** | [**繁體中文**](#繁體中文)
23
 
24
- A small embedding model that reads Taiwanese clinical text.
25
 
26
- These are words that clinical staff in Taiwan actually write in charts, remarks, and community-care sheets. 斷腦筋 means stroke. 不辣咖 means a blood culture. H/T is hypertension, and V(H/T) means the patient is seen at an outside clinic for hypertension. General-purpose models and general Chinese models read none of these, because they never learned how doctors, nurses, and care workers in Taiwan really write. IlhaEmbed learned it.
27
 
28
- The name comes from Ilha Formosa. It is an embedder for this island's clinical language.
29
 
30
- ## Why IlhaEmbed
31
 
32
- - **Reads Taiwan's clinical language.** Slang, abbreviations, Taigi, the handwritten shorthand in real records. Where a general model returns noise, this one returns the right concept.
33
- - **Runs fully on-premise.** 37 MB, INT8, CPU only. No GPU, no cloud, no API key. Patient data never leaves the hospital.
34
- - **Transparent lineage.** Apache-2.0 weights, a documented base model (IBM Granite ModernBERT), and a training pipeline you can trace. Provenance you can hand to a procurement or security review, not a black box.
35
- - **Open and commercial-ready.** Apache-2.0, with no copyleft obligations.
36
 
37
  ## What it is for
38
 
39
- IlhaEmbed turns messy Taiwanese clinical text into structured meaning. Common uses:
40
 
41
- - Clinical text normalization: mapping free-text abbreviations and slang to standard concepts (H/T to 高血壓).
42
- - Routing dirty data into structured fields: the semantic layer of an importer that reads a spreadsheet cell and sends each fact to its correct home (medication, condition, care source, and so on).
43
- - Terminology and code matching: retrieving the right ICD, LOINC, or drug concept for a surface term.
44
- - Cross-boundary search: finding a record from the clue a nurse actually has, not the exact field name.
45
 
46
- It is the normalization layer of a clinical pipeline built entirely in Taiwan: Breeze-ASR-26 for speech, IlhaEmbed for meaning, FHIR for structure.
47
 
48
- ## Benchmarks
49
 
50
- Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standard clinical concept from a combined concept pool. Each register (slang, abbreviation, apposition) is held out separately, self-matches are excluded, and the score is macro-averaged. Every model uses the same method and the same pool.
51
 
52
  | Model | On-prem | slang | abbrev | apposition | jargon top-1 (macro) |
53
  |---|---|---|---|---|---|
@@ -56,9 +56,9 @@ Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standa
56
  | ckip-base | yes | 0.00 | 0.01 | 0.36 | 0.12 |
57
  | bge-small-zh | yes | 0.00 | 0.00 | 0.33 | 0.11 |
58
 
59
- That is about four times the best general model and about eight times bge, on exactly the vocabulary those models were never trained to read. IlhaEmbed does not aim to top general leaderboards such as MTEB. It is a specialist: it wins where general models fail, and it is small enough to run on the ward.
60
 
61
- The register-held-out method above is reproducible. The raw evaluation pairs carry third-party copyright and are not redistributed. A separate Han-Taigi to standard-concept benchmark is in progress and will be reported once its concept mapping is clean.
62
 
63
  ## Usage
64
 
@@ -79,21 +79,21 @@ model = SentenceTransformer("weemed/IlhaEmbed")
79
  model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
80
  ```
81
 
82
- Dimensions: 384. Max sequence: 32 tokens. Vocab: 25.5k (pruned to Traditional Chinese and clinical). Size: 37 MB (INT8 ONNX) or 152 MB (fp32 safetensors).
83
 
84
  ## How it was built
85
 
86
- The base model is IBM Granite ModernBERT (Apache-2.0), a permissively licensed foundation. A Taiwanese clinical teacher signal supplies the domain knowledge through knowledge distillation, so the student learns to place Taiwanese slang, abbreviations, and Taigi near their standard concepts.
87
 
88
- The base model ships a 180k multilingual BPE vocabulary, most of which Traditional Chinese clinical text never uses. We pruned it to the 25.5k tokens that actually appear, which cut the size by 68 percent (117 MB to 37 MB) with no loss of accuracy: jargon top-1 held, and real-pipeline routing stayed bit-identical.
89
 
90
  ## Intended use and limits
91
 
92
- IlhaEmbed is a tool for semantic representation, not a diagnostic system. It surfaces likely meanings and matches for a person to confirm; it must never decide clinical facts on its own without review. It is tuned for Traditional Chinese clinical text in Taiwan. General prose or Simplified Chinese is out of scope.
93
 
94
- ## License and citation
95
 
96
- Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are documented in `SOURCES.md`. The released weights are a distributable derivative; the raw third-party pairs are not redistributed.
97
 
98
  ---
99
 
@@ -101,57 +101,57 @@ Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are docu
101
 
102
  # IlhaEmbed(繁體中文)
103
 
104
- [**English**](#ilhaembed) | **繁體中文**
105
 
106
  一個讀得懂台灣臨床文字的小型語意模型。
107
 
108
- 以下這些是台灣臨床人員實際會寫在病歷、備註、社區照護表上的。斷腦筋是中風不辣咖是血液培養H/T是高血壓,V(H/T)是這位病人在外院高血壓。通用模型一般中文模型全都讀不,因為它們過台灣的醫師、護理師、照服員真正怎麼寫。IlhaEmbed學過
109
 
110
- 名字取自Ilha Formosa(美麗島)。它是一個讀這座島臨床語言的語意模型
111
 
112
- ## 為什麼選IlhaEmbed
113
 
114
- - **讀得懂台灣臨床語言。** 行話、縮寫、台語、手寫備註簡寫,都是真實病歷裡的文字。通用模型讀成雜訊的地方,它讀出正確概念。
115
- - **完全地端執行。** 37 MB、INT8、純CPU。GPU、雲端、免API key病人資料不離開院。
116
- - **透明的血統。** Apache-2.0權重、有文件的底模(IBM Granite ModernBERT)、可追溯的訓練管線是可以交給採購資安審查的來源證明不是黑盒子
117
- - **開源可商用。** Apache-2.0,沒有copyleft綁約
118
 
119
- ## 用來做什麼
120
 
121
- IlhaEmbed把凌亂的台灣臨床文字成結構的意義。常見用
122
 
123
- - 臨床文字正規化:把free-text縮寫行話對到標準概念(H/T對到高血壓)。
124
- - 髒資料路由到結構化欄位:匯入器的語意層,讀一格試算表、把每個事實送到正確歸宿(用藥、疾病、就醫來源等)。
125
- - 術語代碼比對:一個表面詞找到正確的ICD、LOINC或藥品概念。
126
- - 跨搜尋:用護理師手上真正有的線索找到紀錄,是要求精確欄位名。
127
 
128
- 它是一條完全在台灣打造的臨床管線的正規化層:Breeze-ASR-26聽,IlhaEmbed理解,FHIR結構
129
 
130
- ## 效能指標
131
 
132
- 任務:jargon top-1給一個台灣臨床表面詞,於合併概念池中檢索其標準臨床概念。各語域(行話、縮寫、同位語)分別出、排除自我匹配、macro平均。所有模型使用同一方法同一概念池。
133
 
134
- | 模型 | 地端 | slang | abbrev | apposition | jargon top-1(macro) |
135
  |---|---|---|---|---|---|
136
  | **IlhaEmbed** | 是,37 MB CPU | **0.84** | **0.82** | **0.90** | **0.85** |
137
  | jina-embeddings-v2-base-zh | 否 | 0.06 | 0.14 | 0.45 | 0.22 |
138
  | ckip-base | 是 | 0.00 | 0.01 | 0.36 | 0.12 |
139
  | bge-small-zh | 是 | 0.00 | 0.00 | 0.33 | 0.11 |
140
 
141
- 約為最佳通用模型的四倍、bge的近八倍就贏在那些模型從沒學過的在地詞彙IlhaEmbed在通用排行榜(MTEB)霸榜為目標。它是專科模型:贏在通用模型會爆掉的地方同時小到能在病房跑。
142
 
143
- 上述register-held-out方法可重現。原始評估配對第三方著作權,不隨散佈。專門的「漢字台語對到標準概念」基準進行中待概念對應乾淨後另行報告
144
 
145
- ## 使方式
146
 
147
  ```python
148
- # ONNX(部署路徑,torch、CPU)
149
  from tokenizers import Tokenizer
150
  import onnxruntime as ort, numpy as np
151
 
152
  tok = Tokenizer.from_file("tokenizer.json")
153
  sess = ort.InferenceSession("model_int8.onnx")
154
- # 編碼(max_len 32)、推論、mean-pool、L2 normalize,得到384維向量
155
  ```
156
 
157
  ```python
@@ -161,18 +161,18 @@ model = SentenceTransformer("weemed/IlhaEmbed")
161
  model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
162
  ```
163
 
164
- 維度384大序列:32 tokens。詞表25.5k(剪枝至繁體中文臨床)。大小37 MB(INT8 ONNX)或152 MB(fp32 safetensors)。
165
 
166
- ## 怎麼做出來
167
 
168
- IBM Granite ModernBERT(Apache-2.0),一個寬鬆授權的基礎台灣���床teacher訊號透過知識蒸餾提供領域知識,讓學生學會把台灣行話、縮寫、台語擺到它們的標準概念旁邊。
169
 
170
- 模原帶18萬詞的多語BPE詞表,其中大多數繁中臨床文字用不到。我們把它剪到實際會用到的25.5k token,體積因此減少68%(117 MB到37 MB),準確度零損:jargon top-1不掉管線路由逐位一致
171
 
172
- ## 適用範圍與限制
173
 
174
- IlhaEmbed是語意表示工具,不是診斷系統。它浮現可能的意義比對人確認,可在未經審核下行判定臨床事實。它台灣的繁體中文臨床文字調校。一般文或簡體中文不在範圍內。
175
 
176
- ## 授權與引用
177
 
178
- Apache-2.0。IBM Granite(Apache-2.0)。訓練配對來源記`SOURCES.md`。釋出的權重可散佈的衍生,原始第三方配對不隨附散佈
 
19
 
20
  # IlhaEmbed
21
 
22
+ English | [繁體中文](#繁體中文)
23
 
24
+ A small embedding model for Taiwanese clinical text.
25
 
26
+ Clinical records in Taiwan are full of local shorthand. 斷腦筋 means stroke, 不辣咖 means a blood culture, H/T is hypertension, and V(H/T) means the patient is followed for hypertension at an outside clinic. General-purpose and general Chinese models mostly miss these, because they never saw how doctors, nurses, and care workers in Taiwan actually write. IlhaEmbed was trained for it.
27
 
28
+ The name is from Ilha Formosa. It reads this island's clinical language.
29
 
30
+ ## Highlights
31
 
32
+ - **Reads local clinical writing.** Slang, abbreviations, Taigi, handwritten shorthand. Where a general model returns noise, this one usually returns the right concept.
33
+ - **Runs on-premise.** 37 MB, INT8, CPU only. No GPU, no cloud, no API key. Patient data stays in the hospital.
34
+ - **Clear provenance.** Apache-2.0 weights, a public and documented base model (IBM Granite ModernBERT), and a pipeline you can trace. Ready for a procurement or security review.
35
+ - **Open and commercial-friendly.** Apache-2.0, no copyleft.
36
 
37
  ## What it is for
38
 
39
+ Turning messy clinical text into structured meaning. Common uses:
40
 
41
+ - Normalizing clinical text: mapping abbreviations and slang to standard concepts (H/T to 高血壓).
42
+ - Placing dirty data: on import, reading one cell and sending each fact to the field it belongs in (medication, condition, care source, and so on).
43
+ - Matching terms and codes: finding the right ICD, LOINC, or drug concept for a surface term.
44
+ - Searching across fields: finding a record from the clue at hand, without knowing the exact field name.
45
 
46
+ In a clinical pipeline built in Taiwan, it handles the "understanding" step: Breeze-ASR-26 for speech, IlhaEmbed for meaning, FHIR for structure.
47
 
48
+ ## Results
49
 
50
+ Task: jargon top-1. Given a Taiwanese clinical surface term, retrieve its standard concept from a combined pool. Slang, abbreviation, and apposition are each held out separately, self-matches excluded, scores macro-averaged. All four models use the same method and the same pool.
51
 
52
  | Model | On-prem | slang | abbrev | apposition | jargon top-1 (macro) |
53
  |---|---|---|---|---|---|
 
56
  | ckip-base | yes | 0.00 | 0.01 | 0.36 | 0.12 |
57
  | bge-small-zh | yes | 0.00 | 0.00 | 0.33 | 0.11 |
58
 
59
+ On these local terms the general and general-Chinese models mostly return nothing; this model finds the right concept. It is not meant to top general leaderboards such as MTEB. It does one thing, Taiwanese clinical vocabulary, and it is small enough to run on the ward.
60
 
61
+ The method is reproducible. The raw evaluation pairs carry third-party copyright and are not redistributed. A separate Han-Taigi to standard-concept benchmark is still in progress, and will be added once its concept mapping is clean.
62
 
63
  ## Usage
64
 
 
79
  model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
80
  ```
81
 
82
+ 384 dimensions, up to 32 tokens. Vocab 25.5k (pruned to Traditional Chinese and clinical). Size 37 MB (INT8 ONNX) or 152 MB (fp32 safetensors).
83
 
84
  ## How it was built
85
 
86
+ The base model is IBM Granite ModernBERT (Apache-2.0), permissively licensed. A Taiwanese clinical teacher signal supplies the domain knowledge through knowledge distillation, so the student learns to place Taiwanese slang, abbreviations, and Taigi next to their standard concepts.
87
 
88
+ The base model ships a 180k multilingual BPE vocabulary, most of which Traditional Chinese clinical text never uses. Pruning it to the 25.5k tokens that actually appear cut the size by 68 percent (117 MB to 37 MB) with no loss of accuracy: jargon top-1 held, and the real import routing came out bit-identical.
89
 
90
  ## Intended use and limits
91
 
92
+ This is a tool for semantic representation, not a diagnostic system. It offers likely meanings and matches for a person to confirm, and does not decide clinical facts on its own. It is tuned for Traditional Chinese clinical text in Taiwan. General prose and Simplified Chinese are out of scope.
93
 
94
+ ## License
95
 
96
+ Apache-2.0. Base model: IBM Granite (Apache-2.0). Training-pair sources are listed in `SOURCES.md`. What we release is the trained weights, a distributable derivative; the raw third-party pairs are not included.
97
 
98
  ---
99
 
 
101
 
102
  # IlhaEmbed(繁體中文)
103
 
104
+ [English](#ilhaembed) | 繁體中文
105
 
106
  一個讀得懂台灣臨床文字的小型語意模型。
107
 
108
+ 台灣病歷、備註、社區照護表上,有很多在地寫法。斷腦筋是中風不辣咖是血液培養H/T 是高血壓,V(H/T) 是這位病人在外院追蹤高血壓。這些寫法,通用模型一般中文模型多半讀不出來,因為它們沒過台灣的醫師、護理師、照服員實際怎麼寫。IlhaEmbed 是針對這件事訓練的
109
 
110
+ 名字取自 Ilha Formosa。它專門讀這座島臨床語言。
111
 
112
+ ## 特點
113
 
114
+ - **讀得懂在地的臨床寫法。** 行話、縮寫、台語、手寫簡寫。通用模型讀成雜訊的地方,它多半能對到正確概念。
115
+ - **完全地端。** 37 MB、INT8、純 CPU。不用 GPU、雲端API key病人資料不離開院
116
+ - **來源清楚。** Apache-2.0 權重,基礎模型是公開、有文件的 IBM Granite ModernBERT,訓練流程可追溯。採購資安要看都拿得出來
117
+ - **開源可商用。** Apache-2.0,沒有 copyleft 的限制
118
 
119
+ ## 用
120
 
121
+ 把凌亂的臨床文字整理結構的意義。常見
122
 
123
+ - 臨床文字正規化:把縮寫行話對到標準概念(H/T 對到高血壓)。
124
+ - 髒資料位:匯入讀一格內容,裡面每個事實送到該去欄位(用藥、疾病、就醫來源等)。
125
+ - 術語代碼比對:一個詞找到對應 ICD、LOINC 或藥品概念。
126
+ - 跨欄位搜尋:用手上有的線索找到紀錄,不必先知道正欄位名
127
 
128
+ 一條台灣自己的臨床流程裡,它負責「理解」這一段:Breeze-ASR-26 ,IlhaEmbed 理解,FHIR 存成結構。
129
 
130
+ ##
131
 
132
+ 測的是 jargon top-1給一個台灣臨床表面詞,概念池裡找出對應的標準概念。行話、縮寫、同位語三類各自作測試,排除自我匹配,取各類平均。四個模型用同一方法同一池。
133
 
134
+ | 模型 | 地端 | slang | abbrev | apposition | jargon top-1(macro) |
135
  |---|---|---|---|---|---|
136
  | **IlhaEmbed** | 是,37 MB CPU | **0.84** | **0.82** | **0.90** | **0.85** |
137
  | jina-embeddings-v2-base-zh | 否 | 0.06 | 0.14 | 0.45 | 0.22 |
138
  | ckip-base | 是 | 0.00 | 0.01 | 0.36 | 0.12 |
139
  | bge-small-zh | 是 | 0.00 | 0.00 | 0.33 | 0.11 |
140
 
141
+ 在這些在地詞彙上,通用模型和一般中文模型多半讀不出來這個模型能對到正確概念是要在通用排行榜(MTEB)上比高下,把台灣臨床詞彙這一件事做好而且小到能在病房的機器上跑。
142
 
143
+ 這套方法可重現。原始評估配對第三方著作權,不隨模型散佈。另外一個「漢字台語對到標準概念」的測試還在做對應整理乾淨再補上
144
 
145
+ ## 用
146
 
147
  ```python
148
+ # ONNX(部署路徑,不需 torch、CPU)
149
  from tokenizers import Tokenizer
150
  import onnxruntime as ort, numpy as np
151
 
152
  tok = Tokenizer.from_file("tokenizer.json")
153
  sess = ort.InferenceSession("model_int8.onnx")
154
+ # 編碼(max_len 32)、推論、mean-pool、L2 normalize,得到 384 維向量
155
  ```
156
 
157
  ```python
 
161
  model.encode(["斷腦筋", "H/T", "定期心內門診-戒菸"])
162
  ```
163
 
164
+ 維度 38432 個 token。詞表 25.5k(剪過,只留繁體中文臨床會用到的)。大小 37 MB(INT8 ONNX)或 152 MB(fp32 safetensors)。
165
 
166
+ ## 怎麼做的
167
 
168
+ 基礎型用 IBM Granite ModernBERT(Apache-2.0),授權寬鬆台灣teacher 訊號知識蒸餾,讓學生模型學會把台灣行話、縮寫、台語擺到對應的標準概念旁邊。
169
 
170
+ 基礎 18 萬詞的多語 BPE 詞表,繁中臨床大多用不到。把它剪到實際會用到的 25.5k,體積少68%(117 MB 37 MB),準確度沒掉:jargon top-1 一樣,實際跑匯入的路由結果逐位相同
171
 
172
+ ## 適用範圍
173
 
174
+ 一個做語意表示工具,不是診斷系統。它給的是可能的意義比對,最後由人確認,不己下臨床判斷。它針對台灣的繁體中文臨床文字調過,一般文或簡體中文不在範圍內。
175
 
176
+ ## 授權
177
 
178
+ Apache-2.0。基礎IBM Granite(Apache-2.0)。訓練配對來源記`SOURCES.md`。釋出的是訓練後的權重可散佈的衍生成果),原始第三方配對不隨附。