Token Classification
Transformers.js
ONNX
bert
feature-extraction
coreference
multilingual
onnxruntime-web
Instructions to use cp500/infon-coref-pointer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use cp500/infon-coref-pointer with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('token-classification', 'cp500/infon-coref-pointer');
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,138 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: transformers.js
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
- ja
|
| 7 |
+
- zh
|
| 8 |
+
- ko
|
| 9 |
+
- th
|
| 10 |
+
tags:
|
| 11 |
+
- coreference
|
| 12 |
+
- multilingual
|
| 13 |
+
- onnx
|
| 14 |
+
- onnxruntime-web
|
| 15 |
+
- transformers.js
|
| 16 |
+
pipeline_tag: token-classification
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# Infon multilingual coreference pointer
|
| 20 |
+
|
| 21 |
+
Multilingual coreference resolution: detects mentions and links them
|
| 22 |
+
into clusters across **English, Japanese, Korean, Thai, and Chinese**.
|
| 23 |
+
Designed for browser inference via ONNX, replacing the English-only
|
| 24 |
+
fastcoref baseline for multilingual workloads.
|
| 25 |
+
|
| 26 |
+
## Quick start (JavaScript)
|
| 27 |
+
|
| 28 |
+
```bash
|
| 29 |
+
npm install @cp500/infon-coref onnxruntime-web
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
```ts
|
| 33 |
+
import { InfonCorefModel } from '@cp500/infon-coref';
|
| 34 |
+
|
| 35 |
+
const model = await InfonCorefModel.fromHub('cp500/infon-coref-pointer', {
|
| 36 |
+
precision: 'fp16', // 235 MB (default) β vs 470 MB for fp32
|
| 37 |
+
device: 'auto', // tries WebGPU, falls back to WASM
|
| 38 |
+
});
|
| 39 |
+
|
| 40 |
+
const result = await model.resolve(
|
| 41 |
+
'Toyota announced a partnership with Panasonic. ' +
|
| 42 |
+
'The Japanese automaker said the deal is worth $250M.'
|
| 43 |
+
);
|
| 44 |
+
|
| 45 |
+
for (const cluster of result.clusters) {
|
| 46 |
+
console.log(cluster.map(i => result.mentions[i].text).join(' = '));
|
| 47 |
+
// Toyota = The Japanese automaker
|
| 48 |
+
}
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
The JS client source is mirrored under [`js/`](./tree/main/js) in this
|
| 52 |
+
repo for self-contained installs:
|
| 53 |
+
|
| 54 |
+
```bash
|
| 55 |
+
npm install ./js
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
## Quick start (Python / PyTorch)
|
| 59 |
+
|
| 60 |
+
```python
|
| 61 |
+
import torch
|
| 62 |
+
from transformers import AutoModel, AutoTokenizer
|
| 63 |
+
# Architecture lives in scripts/train_coref_pointer.py / coref_onnx_experiment.py
|
| 64 |
+
# (the training repo). Loading the heads is a 4-line check:
|
| 65 |
+
heads = torch.load("heads.pt", map_location="cpu", weights_only=True)
|
| 66 |
+
backbone = AutoModel.from_pretrained("./backbone/")
|
| 67 |
+
tokenizer = AutoTokenizer.from_pretrained("./backbone/")
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
## Architecture
|
| 71 |
+
|
| 72 |
+
```
|
| 73 |
+
text ββΆ tokenize ββΆ MiniLM-L12 backbone ββΆ β¬ββΆ last_hidden_state ββ
|
| 74 |
+
βββΆ bio_logits (T,3) β
|
| 75 |
+
β β
|
| 76 |
+
βΌ β
|
| 77 |
+
decode BIO spans β
|
| 78 |
+
β β
|
| 79 |
+
βΌ β
|
| 80 |
+
mention_scorer βββββββββββββ
|
| 81 |
+
β
|
| 82 |
+
βΌ
|
| 83 |
+
pair_scores (P,)
|
| 84 |
+
β
|
| 85 |
+
βΌ
|
| 86 |
+
per-mention argmax
|
| 87 |
+
β
|
| 88 |
+
βΌ
|
| 89 |
+
coreference clusters
|
| 90 |
+
```
|
| 91 |
+
|
| 92 |
+
Two ONNX graphs:
|
| 93 |
+
|
| 94 |
+
- `onnx/coref_backbone_bio.onnx` β XLM-R-distilled MiniLM-L12 (H=384,
|
| 95 |
+
12 layers, 117M params) plus a 3-class BIO mention-detection head.
|
| 96 |
+
- `onnx/coref_mention_scorer.onnx` β vectorised mention pooling
|
| 97 |
+
(boundary tokens + segment-mean) and a pairwise antecedent scorer.
|
| 98 |
+
DUMMY antecedent is concatenated at index 0 so `pair_j == 0` means
|
| 99 |
+
"no antecedent."
|
| 100 |
+
|
| 101 |
+
## Evaluation
|
| 102 |
+
|
| 103 |
+
Best checkpoint (selected on combined `(ptr_acc + bio_f1) / 2`):
|
| 104 |
+
|
| 105 |
+
| Language | Pointer acc | BIO F1 | Val mentions |
|
| 106 |
+
|----------|-------------|--------|--------------|
|
| 107 |
+
| en | 0.805 | 0.809 | 1827 |
|
| 108 |
+
| ja | 0.823 | 0.794 | 1601 |
|
| 109 |
+
| ko | 0.824 | 0.814 | 1702 |
|
| 110 |
+
| th | 0.820 | 0.906 | 1495 |
|
| 111 |
+
| zh | 0.829 | 0.872 | 1589 |
|
| 112 |
+
|
| 113 |
+
**Aggregate**: pointer accuracy 0.820, BIO F1 0.815,
|
| 114 |
+
combined score 0.817.
|
| 115 |
+
|
| 116 |
+
Trained on
|
| 117 |
+
[cp500/infon-coref-multilingual](https://huggingface.co/datasets/cp500/infon-coref-multilingual).
|
| 118 |
+
|
| 119 |
+
### Known limits
|
| 120 |
+
|
| 121 |
+
- BIO precision degrades after epoch 0 if training continues with the
|
| 122 |
+
default joint-loss schedule (pointer head saturates and the
|
| 123 |
+
optimizer pushes BIO toward recall). The deployed checkpoint is
|
| 124 |
+
from epoch 0 to keep BIO precision and pointer accuracy balanced.
|
| 125 |
+
A fix using separate optimizers per head is on the roadmap.
|
| 126 |
+
- Trained only on the 5 listed languages. Other XLM-R-supported
|
| 127 |
+
languages may work via zero-shot transfer; verify on your domain.
|
| 128 |
+
- Synthetic training data follows news-article register; out-of-domain
|
| 129 |
+
text (chat, code comments, formal contracts) may underperform.
|
| 130 |
+
|
| 131 |
+
## Backbone
|
| 132 |
+
|
| 133 |
+
`sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` β public Apache-2.0 distillation of XLM-R-base.
|
| 134 |
+
Tokenizer copied here for offline-installable parity.
|
| 135 |
+
|
| 136 |
+
## License
|
| 137 |
+
|
| 138 |
+
Apache 2.0 for both weights and code.
|