Instructions to use hedwigai/citeparse-150m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hedwigai/citeparse-150m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="hedwigai/citeparse-150m")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("hedwigai/citeparse-150m") model = AutoModelForTokenClassification.from_pretrained("hedwigai/citeparse-150m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
citeparse-150m
citeparse-150m labels the parts of a reference string: authors, title, container, date, volume, issue, pages, DOI and 8 more fields. It is the larger model of the citeparse package, which turns the labels into CSL-JSON or BibTeX.
pip install citeparse
citeparse refs.txt --bibtex --model hedwigai/citeparse-150m
Results
700 real references from Crossref, each scored against the cited work's own Crossref record. None of these DOIs were in training. GROBID was given the same DOI rule as citeparse.
| Parser | All fields | References fully right | Title | Container |
|---|---|---|---|---|
| GROBID 0.8.1 (CRF) | 0.911 | 71.1% | 0.882 | 0.805 |
| citeparse-32m | 0.939 | 80.3% | 0.921 | 0.858 |
| citeparse-150m | 0.952 | 83.4% | 0.934 | 0.878 |
GROBID wins on its own held-out citations: 0.943 against 0.900 for this model, on GROBID's own scorer. Those come from the same sources as GROBID's training data.
Speed on CPU (8 threads): about 20 references a second.
Training
From jhu-clsp/ettin-encoder-150m, two epochs of BIO token classification over 16 labels. Data: GROBID's citation corpus (Apache-2.0) with its evaluation files and every reference that also appears in them removed, plus Crossref records (CC0) written out in 11 citation styles with typing noise. Code: github.com/un-seen/citeparse.
Limits
Trained on English-language styles. It labels one reference at a time, so split a bibliography into lines first. "et al." comes out as a literal author.
- Downloads last month
- 30
Model tree for hedwigai/citeparse-150m
Base model
jhu-clsp/ettin-encoder-150mDataset used to train hedwigai/citeparse-150m
Evaluation results
- Fields right (share) on citeparse-wild-700self-reported0.952
- References fully right on citeparse-wild-700self-reported0.834