Sihao Chen commited on
Commit ·
34da9a3
1
Parent(s): affb7bf
Update README.md
Browse files
README.md
CHANGED
|
@@ -21,8 +21,25 @@ A BART (base) model trained to classify whether a summary is *faithful* to the o
|
|
| 21 |
## Usage
|
| 22 |
Concatenate a summary and a source document as input (note that the summary needs to be the **first** sentence).
|
| 23 |
|
|
|
|
| 24 |
```python
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
```
|
| 27 |
|
| 28 |
|
|
|
|
| 21 |
## Usage
|
| 22 |
Concatenate a summary and a source document as input (note that the summary needs to be the **first** sentence).
|
| 23 |
|
| 24 |
+
Here's an example usage (with PyTorch)
|
| 25 |
```python
|
| 26 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 27 |
+
|
| 28 |
+
tokenizer = AutoTokenizer.from_pretrained("CogComp/bart-faithful-summary-detector")
|
| 29 |
+
model = AutoModelForSequenceClassification.from_pretrained("CogComp/bart-faithful-summary-detector")
|
| 30 |
+
|
| 31 |
+
article = "Ban-Ki Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011"
|
| 32 |
+
|
| 33 |
+
bad_summary = "Ban Ki-moon was elected for a second term in 2007"
|
| 34 |
+
good_summary = "Ban Ki-moon was elected for a second term in 2011"
|
| 35 |
+
|
| 36 |
+
bad_pair = tokenizer(text=bad_summary, text_pair=article, return_tensors='pt')
|
| 37 |
+
good_pair = tokenizer(text=good_summary, text_pair=article, return_tensors='pt')
|
| 38 |
+
|
| 39 |
+
bad_score = model(**bad_pair)
|
| 40 |
+
good_score = model(**good_pair)
|
| 41 |
+
|
| 42 |
+
print(good_score[0][:, 1] > bad_score[0][:, 1]) # True, label mapping: "0" -> "Hallucinated" "1" -> "Faithful"
|
| 43 |
```
|
| 44 |
|
| 45 |
|