hongccccccc commited on
Commit
2c507de
·
verified ·
1 Parent(s): 99a86c5

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +5 -44
README.md CHANGED
@@ -7,7 +7,6 @@ library_name: transformers
7
  tags:
8
  - scientific-text
9
  - citation-intent
10
- - citation-analysis
11
  datasets:
12
  - allenai/multicite
13
  widget:
@@ -19,9 +18,7 @@ widget:
19
 
20
  # SciBERT Citation-Background Classifier
21
 
22
- A [`allenai/scibert_scivocab_uncased`](https://huggingface.co/allenai/scibert_scivocab_uncased) model fine-tuned as a **binary classifier of citation intent**: does a citation sentence cite prior work as **background** (`BACKGROUND`) or for any other reason — method use, comparison, extension, motivation, etc. (`NOT_BACKGROUND`)?
23
-
24
- It was built during a large-scale **citation-fidelity** study as a filter for background citations. Note: in the paper's final pipeline this filtering was ultimately performed with Semantic Scholar's (S2ORC) built-in citation-intent labels rather than this model; it is released for transparency and reuse.
25
 
26
  ## Labels
27
 
@@ -42,46 +39,10 @@ clf("Deep learning has achieved remarkable success in many NLP tasks [1, 2].")
42
 
43
  ## Training
44
 
45
- - **Base model:** SciBERT (uncased, scivocab), fine-tuned with `BertForSequenceClassification` (single-label, 2 classes).
46
- - **Data:** binary labels derived from [MultiCite](https://github.com/allenai/multicite) (Lauscher et al., 2022): citation contexts with intent `background` → `BACKGROUND` (1), all other intents → `NOT_BACKGROUND` (0).
47
- - **Trained:** August 2023, `transformers` 4.32.0. The training script was not preserved, so exact hyperparameters are unknown.
48
 
49
  ## Evaluation
50
 
51
- Evaluated zero-shot on the **SciCite** test set (n = 1,861; label `background` vs. the rest) as a release sanity check — note SciCite is a *different* dataset/annotation scheme than the MultiCite-derived training data:
52
-
53
- | metric | value |
54
- |---------------------------------|-------|
55
- | Accuracy | 0.897 |
56
- | Majority-class baseline | 0.536 |
57
- | BACKGROUND precision | 0.888 |
58
- | BACKGROUND recall | 0.925 |
59
- | BACKGROUND F1 | 0.906 |
60
-
61
- No evaluation from the original training run survives; evaluate on your own data before critical use.
62
-
63
- ## Limitations
64
-
65
- - Trained on citation **context sentences** from scientific papers; not suited to other text types.
66
- - The binary label collapses many distinct intents into `NOT_BACKGROUND`.
67
- - The label mapping was reconstructed from the project's dataset-construction notebook and verified empirically on SciCite (above).
68
- - This model was **not** used to produce the paper's final results (S2ORC intents were used instead).
69
-
70
- ## Citation
71
-
72
- If you use this model, please cite the citation-fidelity paper (to appear — citation forthcoming). Related resources:
73
-
74
- ```bibtex
75
- @inproceedings{lauscher2022multicite,
76
- title={MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting},
77
- author={Lauscher, Anne and Ko, Brandon and Kuehl, Bailey and Johnson, Sophie and Cohan, Arman and Jurgens, David and Lo, Kyle},
78
- booktitle={NAACL},
79
- year={2022}
80
- }
81
- @inproceedings{beltagy2019scibert,
82
- title={SciBERT: A Pretrained Language Model for Scientific Text},
83
- author={Beltagy, Iz and Lo, Kyle and Cohan, Arman},
84
- booktitle={EMNLP},
85
- year={2019}
86
- }
87
- ```
 
7
  tags:
8
  - scientific-text
9
  - citation-intent
 
10
  datasets:
11
  - allenai/multicite
12
  widget:
 
18
 
19
  # SciBERT Citation-Background Classifier
20
 
21
+ [`allenai/scibert_scivocab_uncased`](https://huggingface.co/allenai/scibert_scivocab_uncased) fine-tuned as a binary classifier of citation intent: does a citation sentence cite prior work as background (`BACKGROUND`) or for any other reason (`NOT_BACKGROUND`)?
 
 
22
 
23
  ## Labels
24
 
 
39
 
40
  ## Training
41
 
42
+ - **Base model:** SciBERT (uncased, scivocab; `BertForSequenceClassification`, single-label, 2 classes)
43
+ - **Data:** citation contexts from [MultiCite](https://github.com/allenai/multicite) intent `background` → `BACKGROUND`, all other intents → `NOT_BACKGROUND`
44
+ - **Fine-tuned:** August 2023, `transformers` 4.32.0
45
 
46
  ## Evaluation
47
 
48
+ On the SciCite test set (n = 1,861; `background` vs. rest): accuracy **0.897**, `BACKGROUND` F1 **0.906**.