Robin Claude Sonnet 4.6 commited on
Commit
f90826c
ยท
1 Parent(s): 2288fd7

fix(zh): slice entity text from original input to avoid BERT tokenizer spaces

Browse files

The transformers token-classification pipeline with aggregation_strategy="simple"
emits Chinese entity text with whitespace between characters (e.g. "้ฉฌ ไบ‘"
instead of "้ฉฌไบ‘"). This breaks downstream string matching and recall metrics.

Fix: use the start/end character offsets returned by the pipeline to slice
the entity text directly from the original input string. The offsets are
correct; only the joined token text is wrong.

Also adds scripts/test_remote_api.py โ€” end-to-end API test harness that hits
the deployed HF Space, exercises every routing branch (EN/ZH/AR/Mixed/auto/
min_entities/threshold/edge), and writes a Markdown report with timings
and per-case recall.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

app/ner.py CHANGED
@@ -216,8 +216,11 @@ class ChineseBERTBackend(_Backend):
216
 
217
  std_label = BERT_TYPE_TO_LABEL.get(bert_type, bert_type)
218
  labels_seen.add(std_label)
 
 
 
219
  entities.append(Entity(
220
- text=r["word"],
221
  label=std_label,
222
  score=round(score, 4),
223
  start=r["start"],
 
216
 
217
  std_label = BERT_TYPE_TO_LABEL.get(bert_type, bert_type)
218
  labels_seen.add(std_label)
219
+ # Chinese BERT tokenizer ไผšๅœจๅญ่ฏ้—ดๆ’ๅ…ฅ็ฉบๆ ผ๏ผˆ"้ฉฌ ไบ‘"๏ผ‰๏ผŒ
220
+ # ็›ดๆŽฅ็”จ start/end ไปŽๅŽŸๆ–‡ๅˆ‡็‰‡๏ผŒ้ฟๅ…็ฉบๆ ผๆฑกๆŸ“
221
+ entity_text = text[r["start"]:r["end"]]
222
  entities.append(Entity(
223
+ text=entity_text,
224
  label=std_label,
225
  score=round(score, 4),
226
  start=r["start"],
reports/remote_api_test_report.md ADDED
@@ -0,0 +1,401 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ่ฟœ็ซฏ API ๆต‹่ฏ•ๆŠฅๅ‘Š
2
+
3
+ - ๆœๅŠกๅœฐๅ€๏ผš`https://robinwu-nerserver.hf.space`
4
+ - ๆต‹่ฏ•ๆ—ถ้—ด๏ผš2026-04-30 09:43:24
5
+ - ๅฅๅบทๆฃ€ๆŸฅ๏ผšโœ“ OK (1259ms) โ€” {"status":"ok"}
6
+ - ็”จไพ‹ๆ€ปๆ•ฐ๏ผš15
7
+
8
+ ## ไธ€ใ€ๆฑ‡ๆ€ป
9
+
10
+ | ็”จไพ‹ | ๆ่ฟฐ | HTTP | ๅฎžไฝ“ๆ•ฐ | ๅฌๅ›ž | ่€—ๆ—ถ |
11
+ |---|---|---|---|---|---|
12
+ | **EN-01** | ่‹ฑๆ–‡็Ÿญๅฅ๏ผŒๆ˜พๅผ language=en๏ผŒ่‡ชๅฎšไน‰ๆ ‡็ญพ | โœ“ 200 | 5 | 5/5 | 1632ms |
13
+ | **EN-02** | ่‹ฑๆ–‡้•ฟๆฎต๏ผŒlabels ็•™็ฉบ่งฆๅ‘้ป˜่ฎคๅŒ่ฏญๆ ‡็ญพ้›† | โœ“ 200 | 5 | 1/4 | 1542ms |
14
+ | **ZH-01** | ไธญๆ–‡็Žฐไปฃๅ•†ไธšๆ–‡ๆœฌ๏ผŒๆ˜พๅผ language=zh | โœ“ 200 | 6 | 0/4 | 1305ms |
15
+ | **ZH-02** | ไธญๆ–‡ๅŒป็–—ๅœบๆ™ฏ๏ผŒ่‡ชๅฎšไน‰ๅŒ่ฏญๆ ‡็ญพ | โœ“ 200 | 5 | 0/3 | 1282ms |
16
+ | **ZH-03** | ๅคๅ…ธๆ–‡ๅญฆ่พน็•Œๆต‹่ฏ• โ€” ใ€Œๅฐคๆฐๆฅ่ฏทใ€ๅบ”ๅชๅ–ใ€Œๅฐคๆฐใ€ | โœ“ 200 | 6 | 0/6 | 1330ms |
17
+ | **AR-01** | ้˜ฟๆ‹‰ไผฏ่ฏญๆ–ฐ้—ป | โœ“ 200 | 4 | 2/2 | 1307ms |
18
+ | **MIX-01** | ไธญ่‹ฑๆททๅˆ ยท ่Œๅœบๅœบๆ™ฏ๏ผŒlanguage=mixed ๅผบๅˆถๅŒ่ท‘ | โœ“ 200 | 7 | 3/7 | 1652ms |
19
+ | **MIX-02** | ๅญฆๆœฏๅœบๆ™ฏ๏ผŒlabels ็•™็ฉบ | โœ“ 200 | 4 | 2/5 | 1635ms |
20
+ | **AUTO-01** | ็บฏไธญๆ–‡ๆ–‡ๆœฌ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ zh | โœ“ 200 | 3 | 0/3 | 1267ms |
21
+ | **AUTO-02** | ็บฏ่‹ฑๆ–‡ๆ–‡ๆœฌ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ en | โœ“ 200 | 4 | 3/3 | 1940ms |
22
+ | **AUTO-03** | ไธญ่‹ฑๆททๅˆ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ mixed ๅนถๅŒ่ท‘ๅˆๅนถ | โœ“ 200 | 4 | 2/3 | 1428ms |
23
+ | **MIN-01** | min_entities=10 ๅผบๅˆถๅ…œๅบ•๏ผˆ็Ÿญๆ–‡ๆœฌๅฏๅ‘ๅผๅชๆœŸๆœ› 1 ไธช๏ผ‰ | โœ“ 200 | 1 | 0/1 | 1501ms |
24
+ | **MIN-02** | min_entities=0 ๅ…ณ้—ญๅ…œๅบ• | โœ“ 200 | 1 | 0/1 | 1183ms |
25
+ | **THR-01** | ้ซ˜้˜ˆๅ€ผ 0.8 - ๆœŸๆœ›่ฟ”ๅ›žๆ›ดๅฐ‘ไฝ†ๆ›ด้ซ˜็ฝฎไฟกๅบฆ็š„ๅฎžไฝ“ | โœ“ 200 | 1 | 1/3 | 1525ms |
26
+ | **EDGE-01** | ็ฉบๆ–‡ๆœฌ | โœ“ 200 | 0 | โ€” | 1160ms |
27
+
28
+ - ้€š่ฟ‡็އ๏ผš**15/15**
29
+ - ็ดฏ่ฎก่€—ๆ—ถ๏ผš**21688ms**๏ผˆๅนณๅ‡ 1446ms/่ฏทๆฑ‚๏ผ‰
30
+
31
+ ## ไบŒใ€ๅˆ†็ป„่ฏฆ็ป†็ป“ๆžœ
32
+
33
+ ### EN โ€” GLiNER ไธป่ทฏๅพ„
34
+
35
+ #### EN-01 ยท ่‹ฑๆ–‡็Ÿญๅฅ๏ผŒๆ˜พๅผ language=en๏ผŒ่‡ชๅฎšไน‰ๆ ‡็ญพ
36
+
37
+ **่ฏทๆฑ‚**
38
+ ```json
39
+ {
40
+ "text": "Elon Musk founded SpaceX in Hawthorne, California in 2002.",
41
+ "labels": [
42
+ "full name of a person",
43
+ "company or organization name",
44
+ "geographical location",
45
+ "date or year"
46
+ ],
47
+ "language": "en"
48
+ }
49
+ ```
50
+
51
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1632ms ยท 5 ไธชๅฎžไฝ“
52
+
53
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
54
+ |---|---|---|---|
55
+ | `Elon Musk` | ไบบๅๆˆ–ๅง“ๅ | 0.85 | 0โ€“9 |
56
+ | `SpaceX` | company or organization name | 0.85 | 18โ€“24 |
57
+ | `Hawthorne` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 0.70 | 28โ€“37 |
58
+ | `California` | geographical location | 0.57 | 39โ€“49 |
59
+ | `2002` | date or year | 0.89 | 53โ€“57 |
60
+
61
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 5/5๏ผš`Elon Musk`, `2002`, `SpaceX`, `Hawthorne`, `California`
62
+
63
+ #### EN-02 ยท ่‹ฑๆ–‡้•ฟๆฎต๏ผŒlabels ็•™็ฉบ่งฆๅ‘้ป˜่ฎคๅŒ่ฏญๆ ‡็ญพ้›†
64
+
65
+ **่ฏทๆฑ‚**
66
+ ```json
67
+ {
68
+ "text": "President Biden signed the Inflation Reduction Act in Washington D.C. on August 16, 2022. The legislation was championed by Senator Chuck Schumer and was seen as a major win for the Democratic Party.",
69
+ "language": "en"
70
+ }
71
+ ```
72
+
73
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1542ms ยท 5 ไธชๅฎžไฝ“
74
+
75
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
76
+ |---|---|---|---|
77
+ | `President Biden` | ไบบๅๆˆ–ๅง“ๅ | 0.66 | 0โ€“15 |
78
+ | `Inflation Reduction Act` | legislation or policy name | 0.78 | 27โ€“50 |
79
+ | `Washington D.C.` | geographical location | 0.64 | 54โ€“69 |
80
+ | `August 16, 2022` | date or year | 0.92 | 73โ€“88 |
81
+ | `Senator Chuck Schumer` | ไบบๅๆˆ–ๅง“ๅ | 0.66 | 124โ€“145 |
82
+
83
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 1/4๏ผš`Chuck Schumer`, `Democratic Party`, `Biden`, `Washington D.C.`
84
+ **ๆœชๅ‘ฝไธญ**๏ผš`Chuck Schumer`, `Democratic Party`, `Biden`
85
+
86
+ ### ZH โ€” BERT ไธป่ทฏๅพ„
87
+
88
+ #### ZH-01 ยท ไธญๆ–‡็Žฐไปฃๅ•†ไธšๆ–‡ๆœฌ๏ผŒๆ˜พๅผ language=zh
89
+
90
+ **่ฏทๆฑ‚**
91
+ ```json
92
+ {
93
+ "text": "้˜ฟ้‡Œๅทดๅทด้›†ๅ›ขๅˆ›ๅง‹ไบบ้ฉฌไบ‘ไบŽ2019ๅนดๅธไปป่‘ฃไบ‹ๅฑ€ไธปๅธญ๏ผŒ็”ฑๅผ ๅ‹‡ๆŽฅไปปใ€‚ๆ€ป้ƒจไฝไบŽๆญๅทž็š„้˜ฟ้‡Œๅทดๅทดๆ——ไธ‹ๆ‹ฅๆœ‰ๆท˜ๅฎใ€ๅคฉ็Œซใ€ๆ”ฏไป˜ๅฎ็ญ‰ไธšๅŠกๆฟๅ—ใ€‚",
94
+ "language": "zh"
95
+ }
96
+ ```
97
+
98
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1305ms ยท 6 ไธชๅฎžไฝ“
99
+
100
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
101
+ |---|---|---|---|
102
+ | `้˜ฟ ้‡Œ ๅทด ๅทด ้›† ๅ›ข` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 1.00 | 0โ€“6 |
103
+ | `้ฉฌ ไบ‘` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 9โ€“11 |
104
+ | `2019 ๅนด` | ๆ—ฅๆœŸๆˆ–ๅนดไปฝ | 1.00 | 12โ€“17 |
105
+ | `ๅผ  ๅ‹‡` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 26โ€“28 |
106
+ | `ๆญ ๅทž` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 1.00 | 35โ€“37 |
107
+ | `้˜ฟ ้‡Œ ๅทด ๅทด` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 0.99 | 38โ€“42 |
108
+
109
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 0/4๏ผš`ๅผ ๅ‹‡`, `ๆญๅทž`, `้˜ฟ้‡Œๅทดๅทด`, `้ฉฌไบ‘`
110
+ **ๆœชๅ‘ฝไธญ**๏ผš`ๆญๅทž`, `้˜ฟ้‡Œๅทดๅทด`, `ๅผ ๅ‹‡`, `้ฉฌไบ‘`
111
+
112
+ #### ZH-02 ยท ไธญๆ–‡ๅŒป็–—ๅœบๆ™ฏ๏ผŒ่‡ชๅฎšไน‰ๅŒ่ฏญๆ ‡็ญพ
113
+
114
+ **่ฏทๆฑ‚**
115
+ ```json
116
+ {
117
+ "text": "ๅŒ—ไบฌๅๅ’ŒๅŒป้™ขๅฟƒๅ†…็ง‘ไธปไปป็Ž‹ๅปบๅ›ฝๆ•™ๆŽˆๅ›ข้˜Ÿ๏ผŒไบŽ2023ๅนดๆˆๅŠŸๅฎŒๆˆ้ฆ–ไพ‹ๆœบๅ™จไบบ่พ…ๅŠฉๅ† ็ŠถๅŠจ่„‰ๆญๆกฅๆ‰‹ๆœฏ๏ผŒๆ‚ฃ่€…ๆฅ่‡ชๅฑฑไธœ็œๆตŽๅ—ๅธ‚ใ€‚",
118
+ "labels": [
119
+ "ไบบๅๆˆ–ๅง“ๅ",
120
+ "ๅŒป้™ขๆˆ–ๅŒป็–—ๆœบๆž„ๅ็งฐ",
121
+ "ๅœฐๅๆˆ–ๅŸŽๅธ‚",
122
+ "ๆ—ฅๆœŸๆˆ–ๅนดไปฝ"
123
+ ],
124
+ "language": "zh"
125
+ }
126
+ ```
127
+
128
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1282ms ยท 5 ไธชๅฎžไฝ“
129
+
130
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
131
+ |---|---|---|---|
132
+ | `ๅŒ— ไบฌ ๅ ๅ’Œ ๅŒป ้™ข ๅฟƒ ๅ†… ็ง‘` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 0.98 | 0โ€“9 |
133
+ | `๏ฟฝ๏ฟฝ๏ฟฝ ๅปบ ๅ›ฝ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 11โ€“14 |
134
+ | `2023 ๅนด` | ๆ—ฅๆœŸๆˆ–ๅนดไปฝ | 0.96 | 20โ€“25 |
135
+ | `ๅฑฑ ไธœ ็œ` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 1.00 | 49โ€“52 |
136
+ | `ๆตŽ ๅ— ๅธ‚` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 1.00 | 52โ€“55 |
137
+
138
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 0/3๏ผš`ๆตŽๅ—`, `็Ž‹ๅปบๅ›ฝ`, `ๅŒ—ไบฌๅๅ’ŒๅŒป้™ข`
139
+ **ๆœชๅ‘ฝไธญ**๏ผš`ๆตŽๅ—`, `็Ž‹ๅปบๅ›ฝ`, `ๅŒ—ไบฌๅๅ’ŒๅŒป้™ข`
140
+
141
+ ### ZH โ€” BERT ่พน็•Œ่ฏ†ๅˆซ
142
+
143
+ #### ZH-03 ยท ๅคๅ…ธๆ–‡ๅญฆ่พน็•Œๆต‹่ฏ• โ€” ใ€Œๅฐคๆฐๆฅ่ฏทใ€ๅบ”ๅชๅ–ใ€Œๅฐคๆฐใ€
144
+
145
+ **่ฏทๆฑ‚**
146
+ ```json
147
+ {
148
+ "text": "ๅฐคๆฐๆฅ่ฏท๏ผŒ็Ž‹็†™ๅ‡ค็ฌ‘้“๏ผšไฝ ๆฅไบ†ใ€‚่ดพๆฏๅ‘ฝไบบๆ‘†้…’๏ผŒๅฎ็މๅ’Œ้ป›็މๅœจๅคง่ง‚ๅ›ญๆ•ฃๆญฅใ€‚",
149
+ "language": "zh"
150
+ }
151
+ ```
152
+
153
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1330ms ยท 6 ไธชๅฎžไฝ“
154
+
155
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
156
+ |---|---|---|---|
157
+ | `ๅฐค ๆฐ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 0โ€“2 |
158
+ | `็Ž‹ ็†™ ๅ‡ค` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 5โ€“8 |
159
+ | `่ดพ ๆฏ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 15โ€“17 |
160
+ | `ๅฎ ็މ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 22โ€“24 |
161
+ | `้ป› ็މ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 25โ€“27 |
162
+ | `ๅคง ่ง‚ ๅ›ญ` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 0.93 | 28โ€“31 |
163
+
164
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 0/6๏ผš`่ดพๆฏ`, `ๅคง่ง‚ๅ›ญ`, `ๅฎ็މ`, `ๅฐคๆฐ`, `้ป›็މ`, `็Ž‹็†™ๅ‡ค`
165
+ **ๆœชๅ‘ฝไธญ**๏ผš`ๅฐคๆฐ`, `้ป›็މ`, `็Ž‹็†™ๅ‡ค`, `่ดพๆฏ`, `ๅคง่ง‚ๅ›ญ`, `ๅฎ็މ`
166
+
167
+ > โœ“ ่พน็•Œๆญฃ็กฎ๏ผˆๆœชๅ‡บ็Žฐ {'็Ž‹็†™ๅ‡ค็ฌ‘้“', 'ๅฐคๆฐๆฅ่ฏท'}๏ผ‰
168
+
169
+ ### AR โ€” GLiNER ไธป่ทฏๅพ„
170
+
171
+ #### AR-01 ยท ้˜ฟๆ‹‰ไผฏ่ฏญๆ–ฐ้—ป
172
+
173
+ **่ฏทๆฑ‚**
174
+ ```json
175
+ {
176
+ "text": "ุฃุนู„ู† ุงู„ุฑุฆูŠุณ ู…ุญู…ุฏ ุจู† ุณู„ู…ุงู† ุนู† ุฅุทู„ุงู‚ ู…ุดุฑูˆุน ู†ูŠูˆู… ููŠ ุงู„ู…ู…ู„ูƒุฉ ุงู„ุนุฑุจูŠุฉ ุงู„ุณุนูˆุฏูŠุฉ ุนุงู… 2017ุŒ ูˆุชุจู„ุบ ุชูƒู„ูุชู‡ 500 ู…ู„ูŠุงุฑ ุฏูˆู„ุงุฑ.",
177
+ "labels": [
178
+ "full name of a person",
179
+ "geographical location",
180
+ "project or initiative name",
181
+ "date or year"
182
+ ],
183
+ "language": "ar"
184
+ }
185
+ ```
186
+
187
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1307ms ยท 4 ไธชๅฎžไฝ“
188
+
189
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
190
+ |---|---|---|---|
191
+ | `ู…ุญู…ุฏ ุจู† ุณู„ู…ุงู†` | ไบบๅๆˆ–ๅง“ๅ | 0.71 | 12โ€“25 |
192
+ | `ู†ูŠูˆู…` | project or initiative name | 0.55 | 41โ€“45 |
193
+ | `ุงู„ู…ู…ู„ูƒุฉ ุงู„ุนุฑุจูŠุฉ ุงู„ุณุนูˆุฏูŠุฉ` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 0.65 | 49โ€“73 |
194
+ | `2017` | date or year | 0.82 | 78โ€“82 |
195
+
196
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 2/2๏ผš`ู…ุญู…ุฏ ุจู† ุณู„ู…ุงู†`, `ุงู„ู…ู…ู„ูƒุฉ ุงู„ุนุฑุจูŠุฉ ุงู„ุณุนูˆุฏูŠุฉ`
197
+
198
+ ### Mixed โ€” ๅŒๆจกๅž‹ๅˆๅนถ
199
+
200
+ #### MIX-01 ยท ไธญ่‹ฑๆททๅˆ ยท ่Œๅœบๅœบๆ™ฏ๏ผŒlanguage=mixed ๅผบๅˆถๅŒ่ท‘
201
+
202
+ **่ฏทๆฑ‚**
203
+ ```json
204
+ {
205
+ "text": "ๅผ ไผŸๅŠ ๅ…ฅไบ† Google ๅŒ—ไบฌ็ ”ๅ‘ไธญๅฟƒ๏ผŒ่ดŸ่ดฃ Android ็ณป็ปŸไผ˜ๅŒ–ใ€‚ไป–็š„ๅŒไบ‹ Sarah Chen ๆฅ่‡ช Meta๏ผŒไธคไบบๅ…ฑๅŒๅ‚ไธŽไบ† 2024 ๅนด็š„ AI Summitใ€‚",
206
+ "language": "mixed"
207
+ }
208
+ ```
209
+
210
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1652ms ยท 7 ไธชๅฎžไฝ“
211
+
212
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
213
+ |---|---|---|---|
214
+ | `ๅผ  ไผŸ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 0โ€“2 |
215
+ | `Google` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 0.82 | 6โ€“12 |
216
+ | `ๅŒ— ไบฌ ็ ” ๅ‘ ไธญ ๅฟƒ` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 0.86 | 13โ€“19 |
217
+ | `Sarah Chen` | ไบบๅๆˆ–ๅง“ๅ | 0.89 | 41โ€“51 |
218
+ | `Meta` | company or organization name | 0.85 | 55โ€“59 |
219
+ | `2024 ๅนด` | ๆ—ฅๆœŸๆˆ–ๅนดไปฝ | 0.66 | 68โ€“74 |
220
+ | `AI Summit` | event name | 0.49 | 76โ€“85 |
221
+
222
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 3/7๏ผš`2024`, `Meta`, `Google`, `Android`, `ๅŒ—ไบฌ`, `Sarah Chen`, `ๅผ ไผŸ`
223
+ **ๆœชๅ‘ฝไธญ**๏ผš`2024`, `Android`, `ๅŒ—ไบฌ`, `ๅผ ไผŸ`
224
+
225
+ #### MIX-02 ยท ๅญฆๆœฏๅœบๆ™ฏ๏ผŒlabels ็•™็ฉบ
226
+
227
+ **่ฏทๆฑ‚**
228
+ ```json
229
+ {
230
+ "text": "ๆธ…ๅŽๅคงๅญฆ่ฎก็ฎ—ๆœบ็ณปๆ•™ๆŽˆๆŽๆ˜Žๅœจ NeurIPS 2023 ๅ‘่กจไบ†ๅ…ณไบŽ Transformer ๆžถๆž„็š„่ฎบๆ–‡๏ผŒๅˆไฝœ่€…ๆฅ่‡ช MIT ๅ’Œ Stanford Universityใ€‚",
231
+ "language": "mixed"
232
+ }
233
+ ```
234
+
235
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1635ms ยท 4 ไธชๅฎžไฝ“
236
+
237
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
238
+ |---|---|---|---|
239
+ | `ๆธ… ๅŽ ๅคง ๅญฆ ่ฎก ็ฎ— ๆœบ ็ณป` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 1.00 | 0โ€“8 |
240
+ | `ๆŽ ๆ˜Ž` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 10โ€“12 |
241
+ | `MIT` | university or research institution | 0.76 | 57โ€“60 |
242
+ | `Stanford University` | university or research institution | 0.85 | 63โ€“82 |
243
+
244
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 2/5๏ผš`Stanford University`, `ๆŽๆ˜Ž`, `ๆธ…ๅŽๅคงๅญฆ`, `Transformer`, `MIT`
245
+ **ๆœชๅ‘ฝไธญ**๏ผš`ๆŽๆ˜Ž`, `ๆธ…ๅŽๅคงๅญฆ`, `Transformer`
246
+
247
+ ### auto โ€” ่‡ชๅŠจ่ฏญ่จ€ๆฃ€ๆต‹
248
+
249
+ #### AUTO-01 ยท ็บฏไธญๆ–‡ๆ–‡ๆœฌ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ zh
250
+
251
+ **่ฏทๆฑ‚**
252
+ ```json
253
+ {
254
+ "text": "้ฉฌไบ‘ๅˆ›็ซ‹ไบ†้˜ฟ้‡Œๅทดๅทด๏ผŒๆ€ป้ƒจๅœจๆญๅทžใ€‚"
255
+ }
256
+ ```
257
+
258
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1267ms ยท 3 ไธชๅฎžไฝ“
259
+
260
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
261
+ |---|---|---|---|
262
+ | `้ฉฌ ไบ‘` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 0โ€“2 |
263
+ | `้˜ฟ ้‡Œ ๅทด ๅทด` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 1.00 | 5โ€“9 |
264
+ | `ๆญ ๅทž` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 1.00 | 13โ€“15 |
265
+
266
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 0/3๏ผš`ๆญๅทž`, `้˜ฟ้‡Œๅทดๅทด`, `้ฉฌไบ‘`
267
+ **ๆœชๅ‘ฝไธญ**๏ผš`ๆญๅทž`, `้˜ฟ้‡Œๅทดๅทด`, `้ฉฌไบ‘`
268
+
269
+ #### AUTO-02 ยท ็บฏ่‹ฑๆ–‡ๆ–‡ๆœฌ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ en
270
+
271
+ **่ฏทๆฑ‚**
272
+ ```json
273
+ {
274
+ "text": "Tim Cook is the CEO of Apple in Cupertino."
275
+ }
276
+ ```
277
+
278
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1940ms ยท 4 ไธชๅฎžไฝ“
279
+
280
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
281
+ |---|---|---|---|
282
+ | `Tim Cook` | ไบบๅๆˆ–ๅง“ๅ | 0.86 | 0โ€“8 |
283
+ | `CEO` | ไบบๅๆˆ–ๅง“ๅ | 0.61 | 16โ€“19 |
284
+ | `Apple` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 0.90 | 23โ€“28 |
285
+ | `Cupertino` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 0.88 | 32โ€“41 |
286
+
287
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 3/3๏ผš`Cupertino`, `Tim Cook`, `Apple`
288
+
289
+ #### AUTO-03 ยท ไธญ่‹ฑๆททๅˆ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ mixed ๅนถๅŒ่ท‘ๅˆๅนถ
290
+
291
+ **่ฏทๆฑ‚**
292
+ ```json
293
+ {
294
+ "text": "ๆŽๅŽๅœจ Microsoft ๆ‹…ไปปๅทฅ็จ‹ๅธˆ๏ผŒๅธธ้ฉป Seattle ๅŠžๅ…ฌๅฎคใ€‚"
295
+ }
296
+ ```
297
+
298
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1428ms ยท 4 ไธชๅฎžไฝ“
299
+
300
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
301
+ |---|---|---|---|
302
+ | `ๆŽๅŽๅœจ` | ไบบๅๆˆ–ๅง“ๅ | 0.41 | 0โ€“3 |
303
+ | `ๆŽ ๅŽ` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 0โ€“2 |
304
+ | `Microsoft` | ๅ…ฌๅธๆˆ–็ป„็ป‡ๆœบๆž„ๅ็งฐ | 0.75 | 4โ€“13 |
305
+ | `Seattle` | ๅœฐๅๆˆ–ๅŸŽๅธ‚ | 0.79 | 23โ€“30 |
306
+
307
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 2/3๏ผš`ๆŽๅŽ`, `Microsoft`, `Seattle`
308
+ **ๆœชๅ‘ฝไธญ**๏ผš`ๆŽๅŽ`
309
+
310
+ ### min_entities ่ฆ†็›–ๅฏๅ‘ๅผ
311
+
312
+ #### MIN-01 ยท min_entities=10 ๅผบๅˆถๅ…œๅบ•๏ผˆ็Ÿญๆ–‡ๆœฌๅฏๅ‘ๅผๅชๆœŸๆœ› 1 ไธช๏ผ‰
313
+
314
+ **่ฏทๆฑ‚**
315
+ ```json
316
+ {
317
+ "text": "้ฉฌไบ‘",
318
+ "language": "zh",
319
+ "min_entities": 10
320
+ }
321
+ ```
322
+
323
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1501ms ยท 1 ไธชๅฎžไฝ“
324
+
325
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
326
+ |---|---|---|---|
327
+ | `้ฉฌ ไบ‘` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 0โ€“2 |
328
+
329
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 0/1๏ผš`้ฉฌไบ‘`
330
+ **ๆœชๅ‘ฝไธญ**๏ผš`้ฉฌไบ‘`
331
+
332
+ #### MIN-02 ยท min_entities=0 ๅ…ณ้—ญๅ…œๅบ•
333
+
334
+ **่ฏทๆฑ‚**
335
+ ```json
336
+ {
337
+ "text": "้ฉฌไบ‘",
338
+ "language": "zh",
339
+ "min_entities": 0
340
+ }
341
+ ```
342
+
343
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1183ms ยท 1 ไธชๅฎžไฝ“
344
+
345
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
346
+ |---|---|---|---|
347
+ | `้ฉฌ ไบ‘` | ไบบๅๆˆ–ๅง“ๅ | 1.00 | 0โ€“2 |
348
+
349
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 0/1๏ผš`้ฉฌไบ‘`
350
+ **ๆœชๅ‘ฝไธญ**๏ผš`้ฉฌไบ‘`
351
+
352
+ ### Threshold ๅ˜ๅŒ–
353
+
354
+ #### THR-01 ยท ้ซ˜้˜ˆๅ€ผ 0.8 - ๆœŸๆœ›่ฟ”ๅ›žๆ›ดๅฐ‘ไฝ†ๆ›ด้ซ˜็ฝฎไฟกๅบฆ็š„ๅฎžไฝ“
355
+
356
+ **่ฏทๆฑ‚**
357
+ ```json
358
+ {
359
+ "text": "Tesla and SpaceX are companies founded by Elon Musk.",
360
+ "language": "en",
361
+ "threshold": 0.8
362
+ }
363
+ ```
364
+
365
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1525ms ยท 1 ไธชๅฎžไฝ“
366
+
367
+ | ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |
368
+ |---|---|---|---|
369
+ | `Elon Musk` | ไบบๅๆˆ–ๅง“ๅ | 0.86 | 42โ€“51 |
370
+
371
+ **ๆœŸๆœ›ๅ‘ฝไธญ** 1/3๏ผš`Tesla`, `SpaceX`, `Elon Musk`
372
+ **ๆœชๅ‘ฝไธญ**๏ผš`Tesla`, `SpaceX`
373
+
374
+ ### Edge cases
375
+
376
+ #### EDGE-01 ยท ็ฉบๆ–‡ๆœฌ
377
+
378
+ **่ฏทๆฑ‚**
379
+ ```json
380
+ {
381
+ "text": ""
382
+ }
383
+ ```
384
+
385
+ **ๅ“ๅบ”**๏ผšHTTP 200 ยท 1160ms ยท 0 ไธชๅฎžไฝ“
386
+
387
+ _ๆœช่ฏ†ๅˆซๅˆฐๅฎžไฝ“_
388
+
389
+ ## ไธ‰ใ€ๆŒ‰่ทฏ็”ฑๅˆ†็ป„ๆ€ง่ƒฝ
390
+
391
+ | ๅˆ†็ป„ | ็”จไพ‹ๆ•ฐ | ๆœ€ๅฟซ | ๆœ€ๆ…ข | ๅนณๅ‡ |
392
+ |---|---|---|---|---|
393
+ | EN โ€” GLiNER ไธป่ทฏๅพ„ | 2 | 1542ms | 1632ms | 1587ms |
394
+ | ZH โ€” BERT ไธป่ทฏๅพ„ | 2 | 1282ms | 1305ms | 1294ms |
395
+ | ZH โ€” BERT ่พน็•Œ่ฏ†ๅˆซ | 1 | 1330ms | 1330ms | 1330ms |
396
+ | AR โ€” GLiNER ไธป่ทฏๅพ„ | 1 | 1307ms | 1307ms | 1307ms |
397
+ | Mixed โ€” ๅŒๆจกๅž‹ๅˆๅนถ | 2 | 1635ms | 1652ms | 1643ms |
398
+ | auto โ€” ่‡ชๅŠจ่ฏญ่จ€ๆฃ€ๆต‹ | 3 | 1267ms | 1940ms | 1545ms |
399
+ | min_entities ่ฆ†็›–ๅฏๅ‘ๅผ | 2 | 1183ms | 1501ms | 1342ms |
400
+ | Threshold ๅ˜ๅŒ– | 1 | 1525ms | 1525ms | 1525ms |
401
+ | Edge cases | 1 | 1160ms | 1160ms | 1160ms |
scripts/test_remote_api.py ADDED
@@ -0,0 +1,360 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ ๅฏน่ฟœ็ซฏ HF Spaces ไธŠ้ƒจ็ฝฒ็š„ NER API ๅš็ซฏๅˆฐ็ซฏๆต‹่ฏ•๏ผŒ่ฆ†็›–ๆ‰€ๆœ‰่ทฏ็”ฑๅˆ†ๆ”ฏไธŽ่พน็•Œๆƒ…ๅ†ตใ€‚
3
+ ไธบๆฏไธช็”จไพ‹่ฎฐๅฝ•๏ผšHTTP ็Šถๆ€ใ€่ฏ†ๅˆซๅˆฐ็š„ๅฎžไฝ“ใ€่ฐƒ็”จ่€—ๆ—ถใ€่‡ชๅŠจๆฃ€ๆต‹็š„่ฏญ่จ€๏ผˆๅฆ‚ๆœ‰๏ผ‰ใ€‚
4
+ ๆœ€็ปˆ่พ“ๅ‡บ Markdown ๆŠฅๅ‘Š๏ผšreports/remote_api_test_report.md
5
+ """
6
+ import io
7
+ import json
8
+ import time
9
+ import urllib.request
10
+ import urllib.error
11
+ from dataclasses import dataclass, field
12
+ from pathlib import Path
13
+
14
+ BASE_URL = "https://robinwu-nerserver.hf.space"
15
+ EXTRACT = f"{BASE_URL}/api/v1/extract"
16
+ HEALTH = f"{BASE_URL}/api/v1/health"
17
+ REPORT = Path("reports/remote_api_test_report.md")
18
+
19
+
20
+ # โ”€โ”€ ็”จไพ‹ๅฎšไน‰ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
21
+ #
22
+ # ๆฏไธช็”จไพ‹ๅญ—ๆฎต๏ผš
23
+ # id ็Ÿญ็ผ–ๅท
24
+ # group ๅˆ†็ป„๏ผˆ็”จไบŽๆŠฅๅ‘Šๅˆ†็ฑป๏ผ‰
25
+ # description ไธญๆ–‡ๆ่ฟฐ
26
+ # payload ไผ ็ป™ /api/v1/extract ็š„ JSON
27
+ # expected ๆœŸๆœ›ๅ‘ฝไธญ็š„ๅฎžไฝ“ๆ–‡ๆœฌ๏ผˆ็”จไบŽๅฌๅ›ž็އ็ปŸ่ฎก๏ผ›ๅฏไธบ็ฉบ้›†ๅˆ่กจ็คบไธๆ ก้ชŒ๏ผ‰
28
+
29
+ CASES: list[dict] = [
30
+ # โ”€โ”€ EN ่ทฏ็”ฑ โ”€โ”€
31
+ {
32
+ "id": "EN-01", "group": "EN โ€” GLiNER ไธป่ทฏๅพ„",
33
+ "description": "่‹ฑๆ–‡็Ÿญๅฅ๏ผŒๆ˜พๅผ language=en๏ผŒ่‡ชๅฎšไน‰ๆ ‡็ญพ",
34
+ "payload": {
35
+ "text": "Elon Musk founded SpaceX in Hawthorne, California in 2002.",
36
+ "labels": ["full name of a person", "company or organization name",
37
+ "geographical location", "date or year"],
38
+ "language": "en",
39
+ },
40
+ "expected": {"Elon Musk", "SpaceX", "Hawthorne", "California", "2002"},
41
+ },
42
+ {
43
+ "id": "EN-02", "group": "EN โ€” GLiNER ไธป่ทฏๅพ„",
44
+ "description": "่‹ฑๆ–‡้•ฟๆฎต๏ผŒlabels ็•™็ฉบ่งฆๅ‘้ป˜่ฎคๅŒ่ฏญๆ ‡็ญพ้›†",
45
+ "payload": {
46
+ "text": ("President Biden signed the Inflation Reduction Act in "
47
+ "Washington D.C. on August 16, 2022. The legislation was "
48
+ "championed by Senator Chuck Schumer and was seen as a major "
49
+ "win for the Democratic Party."),
50
+ "language": "en",
51
+ },
52
+ "expected": {"Biden", "Chuck Schumer", "Washington D.C.", "Democratic Party"},
53
+ },
54
+ # โ”€โ”€ ZH ่ทฏ็”ฑ โ”€โ”€
55
+ {
56
+ "id": "ZH-01", "group": "ZH โ€” BERT ไธป่ทฏๅพ„",
57
+ "description": "ไธญๆ–‡็Žฐไปฃๅ•†ไธšๆ–‡ๆœฌ๏ผŒๆ˜พๅผ language=zh",
58
+ "payload": {
59
+ "text": "้˜ฟ้‡Œๅทดๅทด้›†ๅ›ขๅˆ›ๅง‹ไบบ้ฉฌไบ‘ไบŽ2019ๅนดๅธไปป่‘ฃไบ‹ๅฑ€ไธปๅธญ๏ผŒ็”ฑๅผ ๅ‹‡ๆŽฅไปปใ€‚"
60
+ "ๆ€ป้ƒจไฝไบŽๆญๅทž็š„้˜ฟ้‡Œๅทดๅทดๆ——ไธ‹ๆ‹ฅๆœ‰ๆท˜ๅฎใ€ๅคฉ็Œซใ€ๆ”ฏไป˜ๅฎ็ญ‰ไธšๅŠกๆฟๅ—ใ€‚",
61
+ "language": "zh",
62
+ },
63
+ "expected": {"้ฉฌไบ‘", "ๅผ ๅ‹‡", "้˜ฟ้‡Œๅทดๅทด", "ๆญๅทž"},
64
+ },
65
+ {
66
+ "id": "ZH-02", "group": "ZH โ€” BERT ไธป่ทฏๅพ„",
67
+ "description": "ไธญๆ–‡ๅŒป็–—ๅœบๆ™ฏ๏ผŒ่‡ชๅฎšไน‰ๅŒ่ฏญๆ ‡็ญพ",
68
+ "payload": {
69
+ "text": "ๅŒ—ไบฌๅๅ’ŒๅŒป้™ขๅฟƒๅ†…็ง‘ไธปไปป็Ž‹ๅปบๅ›ฝๆ•™ๆŽˆๅ›ข้˜Ÿ๏ผŒไบŽ2023ๅนดๆˆๅŠŸๅฎŒๆˆ้ฆ–ไพ‹"
70
+ "ๆœบๅ™จไบบ่พ…ๅŠฉๅ† ็ŠถๅŠจ่„‰ๆญๆกฅๆ‰‹ๆœฏ๏ผŒๆ‚ฃ่€…ๆฅ่‡ชๅฑฑไธœ็œๆตŽๅ—ๅธ‚ใ€‚",
71
+ "labels": ["ไบบๅๆˆ–ๅง“ๅ", "ๅŒป้™ขๆˆ–ๅŒป็–—ๆœบๆž„ๅ็งฐ", "ๅœฐๅๆˆ–ๅŸŽๅธ‚", "ๆ—ฅๆœŸๆˆ–ๅนดไปฝ"],
72
+ "language": "zh",
73
+ },
74
+ "expected": {"็Ž‹ๅปบๅ›ฝ", "ๅŒ—ไบฌๅๅ’ŒๅŒป้™ข", "ๆตŽๅ—"},
75
+ },
76
+ {
77
+ "id": "ZH-03", "group": "ZH โ€” BERT ่พน็•Œ่ฏ†ๅˆซ",
78
+ "description": "ๅคๅ…ธๆ–‡ๅญฆ่พน็•Œๆต‹่ฏ• โ€” ใ€Œๅฐคๆฐๆฅ่ฏทใ€ๅบ”ๅชๅ–ใ€Œๅฐคๆฐใ€",
79
+ "payload": {
80
+ "text": "ๅฐคๆฐๆฅ่ฏท๏ผŒ็Ž‹็†™ๅ‡ค็ฌ‘้“๏ผšไฝ ๆฅไบ†ใ€‚่ดพๆฏๅ‘ฝไบบๆ‘†้…’๏ผŒๅฎ็މๅ’Œ้ป›็މๅœจๅคง่ง‚ๅ›ญๆ•ฃๆญฅใ€‚",
81
+ "language": "zh",
82
+ },
83
+ "expected": {"ๅฐคๆฐ", "็Ž‹็†™ๅ‡ค", "่ดพๆฏ", "ๅฎ็މ", "้ป›็މ", "ๅคง่ง‚ๅ›ญ"},
84
+ "must_not_contain": {"ๅฐคๆฐๆฅ่ฏท", "็Ž‹็†™ๅ‡ค็ฌ‘้“"},
85
+ },
86
+ # โ”€โ”€ AR ่ทฏ็”ฑ โ”€โ”€
87
+ {
88
+ "id": "AR-01", "group": "AR โ€” GLiNER ไธป่ทฏๅพ„",
89
+ "description": "้˜ฟๆ‹‰ไผฏ่ฏญๆ–ฐ้—ป",
90
+ "payload": {
91
+ "text": ("ุฃุนู„ู† ุงู„ุฑุฆูŠุณ ู…ุญู…ุฏ ุจู† ุณู„ู…ุงู† ุนู† ุฅุทู„ุงู‚ ู…ุดุฑูˆุน ู†ูŠูˆู… ููŠ ุงู„ู…ู…ู„ูƒุฉ "
92
+ "ุงู„ุนุฑุจูŠุฉ ุงู„ุณุนูˆุฏูŠุฉ ุนุงู… 2017ุŒ ูˆุชุจู„ุบ ุชูƒู„ูุชู‡ 500 ู…ู„ูŠุงุฑ ุฏูˆู„ุงุฑ."),
93
+ "labels": ["full name of a person", "geographical location",
94
+ "project or initiative name", "date or year"],
95
+ "language": "ar",
96
+ },
97
+ "expected": {"ู…ุญู…ุฏ ุจู† ุณู„ู…ุงู†", "ุงู„ู…ู…ู„ูƒุฉ ุงู„ุนุฑุจูŠุฉ ุงู„ุณุนูˆุฏูŠุฉ"},
98
+ },
99
+ # โ”€โ”€ Mixed ่ทฏ็”ฑ๏ผˆๅŒ่ท‘ๅˆๅนถ๏ผ‰ โ”€โ”€
100
+ {
101
+ "id": "MIX-01", "group": "Mixed โ€” ๅŒๆจกๅž‹ๅˆๅนถ",
102
+ "description": "ไธญ่‹ฑๆททๅˆ ยท ่Œๅœบๅœบๆ™ฏ๏ผŒlanguage=mixed ๅผบๅˆถๅŒ่ท‘",
103
+ "payload": {
104
+ "text": "ๅผ ไผŸๅŠ ๅ…ฅไบ† Google ๅŒ—ไบฌ็ ”ๅ‘ไธญๅฟƒ๏ผŒ่ดŸ่ดฃ Android ็ณป็ปŸไผ˜ๅŒ–ใ€‚"
105
+ "ไป–็š„ๅŒไบ‹ Sarah Chen ๆฅ่‡ช Meta๏ผŒไธคไบบ๏ฟฝ๏ฟฝๅŒๅ‚ไธŽไบ† 2024 ๅนด็š„ AI Summitใ€‚",
106
+ "language": "mixed",
107
+ },
108
+ "expected": {"ๅผ ไผŸ", "Google", "Sarah Chen", "Meta", "Android", "ๅŒ—ไบฌ", "2024"},
109
+ },
110
+ {
111
+ "id": "MIX-02", "group": "Mixed โ€” ๅŒๆจกๅž‹ๅˆๅนถ",
112
+ "description": "ๅญฆๆœฏๅœบๆ™ฏ๏ผŒlabels ็•™็ฉบ",
113
+ "payload": {
114
+ "text": "ๆธ…ๅŽๅคงๅญฆ่ฎก็ฎ—ๆœบ็ณปๆ•™ๆŽˆๆŽๆ˜Žๅœจ NeurIPS 2023 ๅ‘่กจไบ†ๅ…ณไบŽ "
115
+ "Transformer ๆžถๆž„็š„่ฎบๆ–‡๏ผŒๅˆไฝœ่€…ๆฅ่‡ช MIT ๅ’Œ Stanford Universityใ€‚",
116
+ "language": "mixed",
117
+ },
118
+ "expected": {"ๆŽๆ˜Ž", "ๆธ…ๅŽๅคงๅญฆ", "MIT", "Stanford University", "Transformer"},
119
+ },
120
+ # โ”€โ”€ auto ่‡ชๅŠจๆฃ€ๆต‹ โ”€โ”€
121
+ {
122
+ "id": "AUTO-01", "group": "auto โ€” ่‡ชๅŠจ่ฏญ่จ€ๆฃ€ๆต‹",
123
+ "description": "็บฏไธญๆ–‡ๆ–‡ๆœฌ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ zh",
124
+ "payload": {
125
+ "text": "้ฉฌไบ‘ๅˆ›็ซ‹ไบ†้˜ฟ้‡Œๅทดๅทด๏ผŒๆ€ป้ƒจๅœจๆญๅทžใ€‚",
126
+ },
127
+ "expected": {"้ฉฌไบ‘", "้˜ฟ้‡Œๅทดๅทด", "ๆญๅทž"},
128
+ },
129
+ {
130
+ "id": "AUTO-02", "group": "auto โ€” ่‡ชๅŠจ่ฏญ่จ€ๆฃ€ๆต‹",
131
+ "description": "็บฏ่‹ฑๆ–‡ๆ–‡ๆœฌ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ en",
132
+ "payload": {
133
+ "text": "Tim Cook is the CEO of Apple in Cupertino.",
134
+ },
135
+ "expected": {"Tim Cook", "Apple", "Cupertino"},
136
+ },
137
+ {
138
+ "id": "AUTO-03", "group": "auto โ€” ่‡ชๅŠจ่ฏญ่จ€ๆฃ€ๆต‹",
139
+ "description": "ไธญ่‹ฑๆททๅˆ๏ผŒๅบ”่ขซๆฃ€ๆต‹ไธบ mixed ๅนถๅŒ่ท‘ๅˆๅนถ",
140
+ "payload": {
141
+ "text": "ๆŽๅŽๅœจ Microsoft ๆ‹…ไปปๅทฅ็จ‹ๅธˆ๏ผŒๅธธ้ฉป Seattle ๅŠžๅ…ฌๅฎคใ€‚",
142
+ },
143
+ "expected": {"ๆŽๅŽ", "Microsoft", "Seattle"},
144
+ },
145
+ # โ”€โ”€ min_entities ่ฆ†็›– โ”€โ”€
146
+ {
147
+ "id": "MIN-01", "group": "min_entities ่ฆ†็›–ๅฏๅ‘ๅผ",
148
+ "description": "min_entities=10 ๅผบๅˆถๅ…œๅบ•๏ผˆ็Ÿญๆ–‡ๆœฌๅฏๅ‘ๅผๅชๆœŸๆœ› 1 ไธช๏ผ‰",
149
+ "payload": {
150
+ "text": "้ฉฌไบ‘",
151
+ "language": "zh",
152
+ "min_entities": 10,
153
+ },
154
+ "expected": {"้ฉฌไบ‘"},
155
+ },
156
+ {
157
+ "id": "MIN-02", "group": "min_entities ่ฆ†็›–ๅฏๅ‘ๅผ",
158
+ "description": "min_entities=0 ๅ…ณ้—ญๅ…œๅบ•",
159
+ "payload": {
160
+ "text": "้ฉฌไบ‘",
161
+ "language": "zh",
162
+ "min_entities": 0,
163
+ },
164
+ "expected": {"้ฉฌไบ‘"},
165
+ },
166
+ # โ”€โ”€ ้˜ˆๅ€ผๅ˜ๅŒ– โ”€โ”€
167
+ {
168
+ "id": "THR-01", "group": "Threshold ๅ˜ๅŒ–",
169
+ "description": "้ซ˜้˜ˆๅ€ผ 0.8 - ๆœŸๆœ›่ฟ”ๅ›žๆ›ดๅฐ‘ไฝ†ๆ›ด้ซ˜็ฝฎไฟกๅบฆ็š„ๅฎžไฝ“",
170
+ "payload": {
171
+ "text": "Tesla and SpaceX are companies founded by Elon Musk.",
172
+ "language": "en",
173
+ "threshold": 0.8,
174
+ },
175
+ "expected": {"Tesla", "SpaceX", "Elon Musk"},
176
+ },
177
+ # โ”€โ”€ ่พน็•Œ่ฏทๆฑ‚ โ”€โ”€
178
+ {
179
+ "id": "EDGE-01", "group": "Edge cases",
180
+ "description": "็ฉบๆ–‡ๆœฌ",
181
+ "payload": {"text": ""},
182
+ "expected": set(),
183
+ },
184
+ ]
185
+
186
+
187
+ # โ”€โ”€ HTTP ่ฐƒ็”จ + ่ฎกๆ—ถ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
188
+
189
+ @dataclass
190
+ class CallResult:
191
+ case_id: str
192
+ status: int
193
+ elapsed_ms: float
194
+ entities: list[dict] = field(default_factory=list)
195
+ labels_used: list[str] = field(default_factory=list)
196
+ error: str | None = None
197
+
198
+
199
+ def post_extract(payload: dict, timeout: int = 60) -> CallResult:
200
+ body = json.dumps(payload).encode("utf-8")
201
+ req = urllib.request.Request(
202
+ EXTRACT,
203
+ data=body,
204
+ headers={"Content-Type": "application/json"},
205
+ method="POST",
206
+ )
207
+ t0 = time.perf_counter()
208
+ try:
209
+ with urllib.request.urlopen(req, timeout=timeout) as resp:
210
+ elapsed = (time.perf_counter() - t0) * 1000
211
+ data = json.loads(resp.read().decode())
212
+ return CallResult(
213
+ case_id="",
214
+ status=resp.status,
215
+ elapsed_ms=elapsed,
216
+ entities=data.get("entities", []),
217
+ labels_used=data.get("labels_used", []),
218
+ )
219
+ except urllib.error.HTTPError as e:
220
+ elapsed = (time.perf_counter() - t0) * 1000
221
+ return CallResult(case_id="", status=e.code, elapsed_ms=elapsed,
222
+ error=e.read().decode("utf-8", errors="replace"))
223
+ except Exception as e:
224
+ elapsed = (time.perf_counter() - t0) * 1000
225
+ return CallResult(case_id="", status=0, elapsed_ms=elapsed, error=str(e))
226
+
227
+
228
+ # โ”€โ”€ ๅฅๅบทๆฃ€ๆŸฅ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
229
+
230
+ def check_health() -> tuple[bool, float, str]:
231
+ t0 = time.perf_counter()
232
+ try:
233
+ with urllib.request.urlopen(HEALTH, timeout=30) as resp:
234
+ elapsed = (time.perf_counter() - t0) * 1000
235
+ return resp.status == 200, elapsed, resp.read().decode()
236
+ except Exception as e:
237
+ return False, (time.perf_counter() - t0) * 1000, str(e)
238
+
239
+
240
+ # โ”€โ”€ ๆŠฅๅ‘Š็”Ÿๆˆ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
241
+
242
+ def write_report(results: list[tuple[dict, CallResult]], health: tuple[bool, float, str]):
243
+ buf = io.StringIO()
244
+ w = buf.write
245
+
246
+ w("# ่ฟœ็ซฏ API ๆต‹่ฏ•ๆŠฅๅ‘Š\n\n")
247
+ w(f"- ๆœๅŠกๅœฐๅ€๏ผš`{BASE_URL}`\n")
248
+ w(f"- ๆต‹่ฏ•ๆ—ถ้—ด๏ผš{time.strftime('%Y-%m-%d %H:%M:%S')}\n")
249
+ ok, hms, hbody = health
250
+ w(f"- ๅฅๅบทๆฃ€ๆŸฅ๏ผš{'โœ“ OK' if ok else 'โœ— FAIL'} ({hms:.0f}ms) โ€” {hbody}\n")
251
+ w(f"- ็”จไพ‹ๆ€ปๆ•ฐ๏ผš{len(results)}\n\n")
252
+
253
+ # โ”€โ”€ ๆฑ‡ๆ€ป่กจ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
254
+ w("## ไธ€ใ€ๆฑ‡ๆ€ป\n\n")
255
+ w("| ็”จไพ‹ | ๆ่ฟฐ | HTTP | ๅฎžไฝ“ๆ•ฐ | ๅฌๅ›ž | ่€—ๆ—ถ |\n")
256
+ w("|---|---|---|---|---|---|\n")
257
+ total_ms = 0.0
258
+ pass_n = 0
259
+ for case, res in results:
260
+ expected = case.get("expected", set())
261
+ found = {e["text"] for e in res.entities}
262
+ hit = len(expected & found)
263
+ recall = f"{hit}/{len(expected)}" if expected else "โ€”"
264
+ ok_mark = "โœ“" if res.status == 200 else "โœ—"
265
+ w(f"| **{case['id']}** | {case['description']} | {ok_mark} {res.status} | "
266
+ f"{len(res.entities)} | {recall} | {res.elapsed_ms:.0f}ms |\n")
267
+ if res.status == 200:
268
+ pass_n += 1
269
+ total_ms += res.elapsed_ms
270
+ w(f"\n- ้€š่ฟ‡็އ๏ผš**{pass_n}/{len(results)}**\n")
271
+ w(f"- ็ดฏ่ฎก่€—ๆ—ถ๏ผš**{total_ms:.0f}ms**๏ผˆๅนณๅ‡ {total_ms/len(results):.0f}ms/่ฏทๆฑ‚๏ผ‰\n\n")
272
+
273
+ # โ”€โ”€ ๅˆ†็ป„่ฏฆๆƒ… โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
274
+ groups: dict[str, list] = {}
275
+ for case, res in results:
276
+ groups.setdefault(case["group"], []).append((case, res))
277
+
278
+ w("## ไบŒใ€ๅˆ†็ป„่ฏฆ็ป†็ป“ๆžœ\n\n")
279
+ for group_name, items in groups.items():
280
+ w(f"### {group_name}\n\n")
281
+ for case, res in items:
282
+ w(f"#### {case['id']} ยท {case['description']}\n\n")
283
+ w("**่ฏทๆฑ‚**\n```json\n")
284
+ w(json.dumps(case["payload"], ensure_ascii=False, indent=2))
285
+ w("\n```\n\n")
286
+
287
+ w(f"**ๅ“ๅบ”**๏ผšHTTP {res.status} ยท {res.elapsed_ms:.0f}ms ยท "
288
+ f"{len(res.entities)} ไธชๅฎžไฝ“\n\n")
289
+
290
+ if res.error:
291
+ w(f"```\nERROR: {res.error}\n```\n\n")
292
+ continue
293
+
294
+ if res.entities:
295
+ w("| ๆ–‡ๆœฌ | ๆ ‡็ญพ | ็ฝฎไฟกๅบฆ | ่ตทๆญข |\n|---|---|---|---|\n")
296
+ for e in res.entities:
297
+ w(f"| `{e['text']}` | {e['label']} | {e['score']:.2f} | "
298
+ f"{e['start']}โ€“{e['end']} |\n")
299
+ else:
300
+ w("_ๆœช่ฏ†ๅˆซๅˆฐๅฎžไฝ“_\n")
301
+
302
+ expected = case.get("expected", set())
303
+ if expected:
304
+ found = {e["text"] for e in res.entities}
305
+ hits = expected & found
306
+ misses = expected - found
307
+ w(f"\n**ๆœŸๆœ›ๅ‘ฝไธญ** {len(hits)}/{len(expected)}๏ผš")
308
+ w(", ".join(f"`{x}`" for x in expected) + " \n")
309
+ if misses:
310
+ w(f"**ๆœชๅ‘ฝไธญ**๏ผš{', '.join(f'`{x}`' for x in misses)} \n")
311
+
312
+ mnc = case.get("must_not_contain", set())
313
+ if mnc:
314
+ bad = {e["text"] for e in res.entities} & mnc
315
+ if bad:
316
+ w(f"\n> โš ๏ธ **่พน็•Œ้”™่ฏฏ**๏ผš{bad}\n")
317
+ else:
318
+ w(f"\n> โœ“ ่พน็•Œๆญฃ็กฎ๏ผˆๆœชๅ‡บ็Žฐ {mnc}๏ผ‰\n")
319
+ w("\n")
320
+
321
+ # โ”€โ”€ ๆ€ง่ƒฝ่šๅˆ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
322
+ w("## ไธ‰ใ€ๆŒ‰่ทฏ็”ฑๅˆ†็ป„ๆ€ง่ƒฝ\n\n")
323
+ by_group: dict[str, list[float]] = {}
324
+ for case, res in results:
325
+ if res.status == 200:
326
+ by_group.setdefault(case["group"], []).append(res.elapsed_ms)
327
+ w("| ๅˆ†็ป„ | ็”จไพ‹ๆ•ฐ | ๆœ€ๅฟซ | ๆœ€ๆ…ข | ๅนณๅ‡ |\n|---|---|---|---|---|\n")
328
+ for g, times in by_group.items():
329
+ w(f"| {g} | {len(times)} | {min(times):.0f}ms | "
330
+ f"{max(times):.0f}ms | {sum(times)/len(times):.0f}ms |\n")
331
+
332
+ REPORT.parent.mkdir(parents=True, exist_ok=True)
333
+ REPORT.write_text(buf.getvalue(), encoding="utf-8")
334
+ print(f"\nReport: {REPORT.resolve()}")
335
+
336
+
337
+ # โ”€โ”€ ไธป็จ‹ๅบ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
338
+
339
+ def main():
340
+ print(f"Target: {BASE_URL}")
341
+ health = check_health()
342
+ print(f"Health: {'OK' if health[0] else 'FAIL'} ({health[1]:.0f}ms)")
343
+ if not health[0]:
344
+ print(f" -> {health[2]}")
345
+ return
346
+
347
+ results: list[tuple[dict, CallResult]] = []
348
+ for case in CASES:
349
+ print(f" {case['id']:8s} ", end="", flush=True)
350
+ res = post_extract(case["payload"])
351
+ res.case_id = case["id"]
352
+ results.append((case, res))
353
+ status = "OK" if res.status == 200 else f"FAIL({res.status})"
354
+ print(f"{status:8s} {res.elapsed_ms:6.0f}ms {len(res.entities)} entities")
355
+
356
+ write_report(results, health)
357
+
358
+
359
+ if __name__ == "__main__":
360
+ main()