Add library_name and pipeline_tag
#3
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,13 +1,16 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
| 4 |
# Model Card for SciLitLLM-7B
|
| 5 |
|
| 6 |
SciLitLLM-7B adapts a general large language model for effective scientific literature understanding. Starting from the Qwen2-7B model, SciLitLLM-7B goes through a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT), to simultaneously infuse scientific domain knowledge and enhance instruction-following capabilities for domain-specific tasks.
|
| 7 |
|
| 8 |
In this process, we identify two key challenges: (1) constructing high-quality CPT corpora, and (2) generating diverse SFT instructions. We address these challenges through a meticulous pipeline, including PDF text extraction, parsing content error correction, quality filtering, and synthetic instruction creation.
|
| 9 |
|
| 10 |
-
Applying this strategy, we present SciLitLLM-7B, specialized in scientific literature understanding, which demonstrates promising performance on scientific literature understanding benchmarks. Specifically, it shows an average performance improvement of 3.6
|
| 11 |
|
| 12 |
See the [paper](https://arxiv.org/abs/2408.15545) for more details and [github](https://github.com/dptech-corp/Uni-SMART) for data processing codes.
|
| 13 |
|
|
@@ -32,7 +35,8 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
| 32 |
)
|
| 33 |
tokenizer = AutoTokenizer.from_pretrained("Uni-SMART/SciLitLLM")
|
| 34 |
|
| 35 |
-
prompt = "Can you summarize this article for me
|
|
|
|
| 36 |
messages = [
|
| 37 |
{"role": "system", "content": "You are a helpful assistant."},
|
| 38 |
{"role": "user", "content": prompt}
|
|
@@ -68,5 +72,4 @@ If you find our work helpful, feel free to give us a cite.
|
|
| 68 |
archivePrefix={arXiv},
|
| 69 |
primaryClass={cs.LG},
|
| 70 |
url={https://arxiv.org/abs/2408.15545},
|
| 71 |
-
}
|
| 72 |
-
```
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: transformers
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
---
|
| 6 |
+
|
| 7 |
# Model Card for SciLitLLM-7B
|
| 8 |
|
| 9 |
SciLitLLM-7B adapts a general large language model for effective scientific literature understanding. Starting from the Qwen2-7B model, SciLitLLM-7B goes through a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT), to simultaneously infuse scientific domain knowledge and enhance instruction-following capabilities for domain-specific tasks.
|
| 10 |
|
| 11 |
In this process, we identify two key challenges: (1) constructing high-quality CPT corpora, and (2) generating diverse SFT instructions. We address these challenges through a meticulous pipeline, including PDF text extraction, parsing content error correction, quality filtering, and synthetic instruction creation.
|
| 12 |
|
| 13 |
+
Applying this strategy, we present SciLitLLM-7B, specialized in scientific literature understanding, which demonstrates promising performance on scientific literature understanding benchmarks. Specifically, it shows an average performance improvement of 3.6% on SciAssess and 10.1% on SciRIFF compared to leading LLMs with fewer than 15B parameters.
|
| 14 |
|
| 15 |
See the [paper](https://arxiv.org/abs/2408.15545) for more details and [github](https://github.com/dptech-corp/Uni-SMART) for data processing codes.
|
| 16 |
|
|
|
|
| 35 |
)
|
| 36 |
tokenizer = AutoTokenizer.from_pretrained("Uni-SMART/SciLitLLM")
|
| 37 |
|
| 38 |
+
prompt = "Can you summarize this article for me?
|
| 39 |
+
<ARTICLE>"
|
| 40 |
messages = [
|
| 41 |
{"role": "system", "content": "You are a helpful assistant."},
|
| 42 |
{"role": "user", "content": prompt}
|
|
|
|
| 72 |
archivePrefix={arXiv},
|
| 73 |
primaryClass={cs.LG},
|
| 74 |
url={https://arxiv.org/abs/2408.15545},
|
| 75 |
+
}
|
|
|