Upload folder using huggingface_hub

README.md

@@ -0,0 +1,45 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- facebook/contriever
+---
+
+OpenScholar_Retriever is a continued pre-trained version of [facebook/contriever](https://huggingface.co/facebook/contriever) for scientific literature synthesis. 
+
+
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+- **Developed by:** University of Washigton, Allen Institute for AI (AI2)
+- **Model type:** a masked language model. 
+- **Language(s) (NLP):** English
+- **License:** The code and model are released under apache-2.0. 
+- **Date cutoff:** The pre-training data is mixture of [peS2o](https://huggingface.co/datasets/allenai/peS2o), [CCNews](https://huggingface.co/datasets/vblagoje/cc_news) and [Proofpile2](https://huggingface.co/datasets/EleutherAI/proof-pile-2).
+
+### Model Sources
+
+<!-- Provide the basic links for the model. -->
+
+- **Project Page:** https://open-scholar.allen.ai/
+- **Repositories:** 
+    - Core repo (training, inference, fine-tuning etc.): https://github.com/AkariAsai/OpenScholar
+    - Evaluation code: https://github.com/AkariAsai/ScholarQABench
+- **Paper:** [Link](https://openscholar.allen.ai/paper)
+- **Technical blog post:** https://allenai.org/blog/openscholar
+<!-- - **Press release:** TODO -->
+
+### Citation
+If you find it useful in this work, cite our paper.  
+
+```
+@article{openscholar,
+  title={{OpenScholar}: Synthesizing Scientific Literature with Retrieval-Augmented Language Models},
+  author={ Asai, Akari and He*, Jacqueline and Shao*, Rulin and Shi, Weijia and Singh, Amanpreet and Chang, Joseph Chee  and Lo,  Kyle and Soldaini, Luca and Feldman, Tian, Sergey and Mike, D’arcy and Wadden, David and Latzke, Matt and Minyang and Ji, Pan and Liu, Shengyan and Tong, Hao and Wu, Bohao and Xiong, Yanyu and Zettlemoyer, Luke and Weld, Dan and Neubig, Graham and Downey, Doug and Yih, Wen-tau and Koh, Pang Wei and Hajishirzi, Hannaneh},
+  journal={Arxiv},
+  year={2024},
+}
+```

config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "Contriever"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.15.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb5226976e362e070ad5b3f6c5bd38ed03a770b7d349b9334e4db8d61cc5f8fa
+size 435633182

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fd1c882abbd30517dced455a2c9768945ec726b96727927e4959348d9de550b
+size 466081

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "bert-base-uncased", "tokenizer_class": "BertTokenizer"}