cp500 commited on
Commit
073ec8d
Β·
verified Β·
1 Parent(s): d2c0775

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +138 -0
README.md ADDED
@@ -0,0 +1,138 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers.js
4
+ language:
5
+ - en
6
+ - ja
7
+ - zh
8
+ - ko
9
+ - th
10
+ tags:
11
+ - coreference
12
+ - multilingual
13
+ - onnx
14
+ - onnxruntime-web
15
+ - transformers.js
16
+ pipeline_tag: token-classification
17
+ ---
18
+
19
+ # Infon multilingual coreference pointer
20
+
21
+ Multilingual coreference resolution: detects mentions and links them
22
+ into clusters across **English, Japanese, Korean, Thai, and Chinese**.
23
+ Designed for browser inference via ONNX, replacing the English-only
24
+ fastcoref baseline for multilingual workloads.
25
+
26
+ ## Quick start (JavaScript)
27
+
28
+ ```bash
29
+ npm install @cp500/infon-coref onnxruntime-web
30
+ ```
31
+
32
+ ```ts
33
+ import { InfonCorefModel } from '@cp500/infon-coref';
34
+
35
+ const model = await InfonCorefModel.fromHub('cp500/infon-coref-pointer', {
36
+ precision: 'fp16', // 235 MB (default) β€” vs 470 MB for fp32
37
+ device: 'auto', // tries WebGPU, falls back to WASM
38
+ });
39
+
40
+ const result = await model.resolve(
41
+ 'Toyota announced a partnership with Panasonic. ' +
42
+ 'The Japanese automaker said the deal is worth $250M.'
43
+ );
44
+
45
+ for (const cluster of result.clusters) {
46
+ console.log(cluster.map(i => result.mentions[i].text).join(' = '));
47
+ // Toyota = The Japanese automaker
48
+ }
49
+ ```
50
+
51
+ The JS client source is mirrored under [`js/`](./tree/main/js) in this
52
+ repo for self-contained installs:
53
+
54
+ ```bash
55
+ npm install ./js
56
+ ```
57
+
58
+ ## Quick start (Python / PyTorch)
59
+
60
+ ```python
61
+ import torch
62
+ from transformers import AutoModel, AutoTokenizer
63
+ # Architecture lives in scripts/train_coref_pointer.py / coref_onnx_experiment.py
64
+ # (the training repo). Loading the heads is a 4-line check:
65
+ heads = torch.load("heads.pt", map_location="cpu", weights_only=True)
66
+ backbone = AutoModel.from_pretrained("./backbone/")
67
+ tokenizer = AutoTokenizer.from_pretrained("./backbone/")
68
+ ```
69
+
70
+ ## Architecture
71
+
72
+ ```
73
+ text ─▢ tokenize ─▢ MiniLM-L12 backbone ─▢ ┬─▢ last_hidden_state ─┐
74
+ └─▢ bio_logits (T,3) β”‚
75
+ β”‚ β”‚
76
+ β–Ό β”‚
77
+ decode BIO spans β”‚
78
+ β”‚ β”‚
79
+ β–Ό β”‚
80
+ mention_scorer β—€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
81
+ β”‚
82
+ β–Ό
83
+ pair_scores (P,)
84
+ β”‚
85
+ β–Ό
86
+ per-mention argmax
87
+ β”‚
88
+ β–Ό
89
+ coreference clusters
90
+ ```
91
+
92
+ Two ONNX graphs:
93
+
94
+ - `onnx/coref_backbone_bio.onnx` β€” XLM-R-distilled MiniLM-L12 (H=384,
95
+ 12 layers, 117M params) plus a 3-class BIO mention-detection head.
96
+ - `onnx/coref_mention_scorer.onnx` β€” vectorised mention pooling
97
+ (boundary tokens + segment-mean) and a pairwise antecedent scorer.
98
+ DUMMY antecedent is concatenated at index 0 so `pair_j == 0` means
99
+ "no antecedent."
100
+
101
+ ## Evaluation
102
+
103
+ Best checkpoint (selected on combined `(ptr_acc + bio_f1) / 2`):
104
+
105
+ | Language | Pointer acc | BIO F1 | Val mentions |
106
+ |----------|-------------|--------|--------------|
107
+ | en | 0.805 | 0.809 | 1827 |
108
+ | ja | 0.823 | 0.794 | 1601 |
109
+ | ko | 0.824 | 0.814 | 1702 |
110
+ | th | 0.820 | 0.906 | 1495 |
111
+ | zh | 0.829 | 0.872 | 1589 |
112
+
113
+ **Aggregate**: pointer accuracy 0.820, BIO F1 0.815,
114
+ combined score 0.817.
115
+
116
+ Trained on
117
+ [cp500/infon-coref-multilingual](https://huggingface.co/datasets/cp500/infon-coref-multilingual).
118
+
119
+ ### Known limits
120
+
121
+ - BIO precision degrades after epoch 0 if training continues with the
122
+ default joint-loss schedule (pointer head saturates and the
123
+ optimizer pushes BIO toward recall). The deployed checkpoint is
124
+ from epoch 0 to keep BIO precision and pointer accuracy balanced.
125
+ A fix using separate optimizers per head is on the roadmap.
126
+ - Trained only on the 5 listed languages. Other XLM-R-supported
127
+ languages may work via zero-shot transfer; verify on your domain.
128
+ - Synthetic training data follows news-article register; out-of-domain
129
+ text (chat, code comments, formal contracts) may underperform.
130
+
131
+ ## Backbone
132
+
133
+ `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` β€” public Apache-2.0 distillation of XLM-R-base.
134
+ Tokenizer copied here for offline-installable parity.
135
+
136
+ ## License
137
+
138
+ Apache 2.0 for both weights and code.