Instructions to use hedwigai/citeparse-32m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hedwigai/citeparse-32m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="hedwigai/citeparse-32m")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("hedwigai/citeparse-32m") model = AutoModelForTokenClassification.from_pretrained("hedwigai/citeparse-32m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
citeparse-32m
citeparse-32m labels the parts of a reference string: authors, title, container, date, volume, issue, pages, DOI and 8 more fields. It is the default model of the citeparse package, which turns the labels into CSL-JSON or BibTeX.
pip install citeparse
citeparse refs.txt --bibtex
Results
700 real references from Crossref, each scored against the cited work's own Crossref record. None of these DOIs were in training. GROBID was given the same DOI rule as citeparse.
| Parser | All fields | References fully right | Title | Container |
|---|---|---|---|---|
| GROBID 0.8.1 (CRF) | 0.911 | 71.1% | 0.882 | 0.805 |
| citeparse-32m | 0.939 | 80.3% | 0.921 | 0.858 |
| citeparse-150m | 0.952 | 83.4% | 0.934 | 0.878 |
GROBID wins on its own held-out citations: 0.943 against 0.882 for this model, on GROBID's own scorer. Those come from the same sources as GROBID's training data.
Speed on CPU (8 threads): about 118 references a second.
Training
From jhu-clsp/ettin-encoder-32m, two epochs of BIO token classification over 16 labels. Data: GROBID's citation corpus (Apache-2.0) with its evaluation files and every reference that also appears in them removed, plus Crossref records (CC0) written out in 11 citation styles with typing noise. Code: github.com/un-seen/citeparse.
Limits
Trained on English-language styles. It labels one reference at a time, so split a bibliography into lines first. "et al." comes out as a literal author.
In the browser
onnx/model.onnx runs in transformers.js. Its logits match PyTorch to within 1.5e-5. A demo is at spaces/hedwigai/citeparse.
- Downloads last month
- 52
Model tree for hedwigai/citeparse-32m
Base model
jhu-clsp/ettin-encoder-32mDataset used to train hedwigai/citeparse-32m
Space using hedwigai/citeparse-32m 1
Evaluation results
- Fields right (share) on citeparse-wild-700self-reported0.939
- References fully right on citeparse-wild-700self-reported0.803