harsharajkumar273 commited on
Commit
ded4a27
·
verified ·
1 Parent(s): 5e84513

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +71 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: apache-2.0
4
+ base_model: facebook/bart-base
5
+ tags:
6
+ - summarization
7
+ - research-paper
8
+ - seq2seq
9
+ - bart
10
+ datasets:
11
+ - custom
12
+ metrics:
13
+ - rouge
14
+ - bertscore
15
+ ---
16
+
17
+ # Bart-Base-Summarization
18
+
19
+ A fine-tuned version of [facebook/bart-base](https://huggingface.co/facebook/bart-base) for summarizing research papers into concise summaries. This is the first stage of a two-step **Research Paper Simplifier** pipeline.
20
+
21
+ ## Model Description
22
+
23
+ This model takes a section of a research paper as input and generates a plain-language summary approximately 1/10th the length of the original text. It was fine-tuned end-to-end (no LoRA) on a custom dataset of research papers.
24
+
25
+ ## Pipeline
26
+
27
+ ```
28
+ Research Paper ──► [Bart-Base-Summarization] ──► Summary ──► [Bart-Base-Story-Generation] ──► Story
29
+ ```
30
+
31
+ ## Training Details
32
+
33
+ | Parameter | Value |
34
+ |-----------|-------|
35
+ | Base model | facebook/bart-base |
36
+ | Task | Summarization |
37
+ | Max input length | 1024 tokens |
38
+ | Max target length | 128 tokens |
39
+ | Learning rate | 5e-5 |
40
+ | Batch size | 8 |
41
+ | Warmup steps | 1000 |
42
+ | Weight decay | 0.01 |
43
+ | Fine-tuning method | Full fine-tuning |
44
+
45
+ ## Usage
46
+
47
+ ```python
48
+ from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
49
+
50
+ tokenizer = AutoTokenizer.from_pretrained("harsharajkumar273/Bart-Base-Summarization")
51
+ model = AutoModelForSeq2SeqLM.from_pretrained("harsharajkumar273/Bart-Base-Summarization")
52
+
53
+ text = "Your research paper section here..."
54
+ word_count = len(text.split())
55
+ prompt = f"Summarize this part of the research paper to less than {word_count // 10} words:\n{text}"
56
+
57
+ inputs = tokenizer(prompt, return_tensors="pt", max_length=1024, truncation=True)
58
+ outputs = model.generate(**inputs, max_length=128, num_beams=4, length_penalty=1.0)
59
+ summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
60
+ print(summary)
61
+ ```
62
+
63
+ ## Evaluation Metrics
64
+
65
+ Evaluated using ROUGE and BERTScore on a held-out 10% test split.
66
+
67
+ ## Related Models
68
+
69
+ - [harsharajkumar273/T5-Base-Summarization](https://huggingface.co/harsharajkumar273/T5-Base-Summarization)
70
+ - [harsharajkumar273/ProphetNet-Large-Summarization](https://huggingface.co/harsharajkumar273/ProphetNet-Large-Summarization)
71
+ - [harsharajkumar273/Bart-Base-Story-Generation](https://huggingface.co/harsharajkumar273/Bart-Base-Story-Generation) — next stage