Update README.md

README.md

@@ -1,144 +1,91 @@
 ---
-base_model: WhereIsAI/pubmed-angle-base-en
-datasets: []
-language: []
-library_name: sentence-transformers
-pipeline_tag: sentence-similarity
-tags:
-- sentence-transformers
-- sentence-similarity
-- feature-extraction
-widget: []
+license: mit
+base_model: microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext
+model-index:
+- name: pre-pubmedbert-base-embedding
+  results: []
+datasets:
+- WhereIsAI/medical-triples
+- WhereIsAI/pubmedqa-test-angle-format-a
+language:
+- en
 ---
 
-# SentenceTransformer based on WhereIsAI/pubmed-angle-base-en
+# WhereIsAI/pubmed-angle-base-en
 
-This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [WhereIsAI/pubmed-angle-base-en](https://huggingface.co/WhereIsAI/pubmed-angle-base-en). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+This model is an example model for the Chinese blog post [title](#) and [angle tutorial](https://angle.readthedocs.io/en/latest/notes/tutorial.html#tutorial). 
 
-## Model Details
+It was fine-tuned with [AnglE Loss](https://arxiv.org/abs/2309.12871) using the official [angle-emb](https://github.com/SeanLee97/AnglE).
 
-### Model Description
-- **Model Type:** Sentence Transformer
-- **Base model:** [WhereIsAI/pubmed-angle-base-en](https://huggingface.co/WhereIsAI/pubmed-angle-base-en) <!-- at revision 3f914ca29238808434100c8db529391cad5c115b -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 768 tokens
-- **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+**1. Training Setup:**
 
-### Model Sources
+- Base model: [microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext)
+- Training Data: [WhereIsAI/medical-triples](https://huggingface.co/datasets/WhereIsAI/medical-triples), processed from [PubMedQA](https://huggingface.co/datasets/qiaojin/PubMedQA).
+- Test Data: [WhereIsAI/pubmedqa-test-angle-format-a](https://huggingface.co/datasets/WhereIsAI/pubmedqa-test-angle-format-a)
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+**2. Performance:**
 
-### Full Model Architecture
+| Model                                  | Pooling Strategy | Spearman's Correlation |
+|----------------------------------------|------------------|:----------------------:|
+| tavakolih/all-MiniLM-L6-v2-pubmed-full | avg              | 84.56                  |
+| NeuML/pubmedbert-base-embeddings       | avg              | 84.88                  |
+| **WhereIsAI/pubmed-angle-base-en**     | cls              | 86.01                  |
+| WhereIsAI/pubmed-angle-large-en        | cls              | 86.21                  |
 
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
-  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
-)
-```
-
-## Usage
-
-### Direct Usage (Sentence Transformers)
-
-First install the Sentence Transformers library:
+**3. Citation**
 
-```bash
-pip install -U sentence-transformers
-```
+If you use this model for academic papers, please cite angle's paper, as follows:
 
-Then you can load this model and run inference.
-```python
-from sentence_transformers import SentenceTransformer
-
-# Download from the 🤗 Hub
-model = SentenceTransformer("WhereIsAI/pubmed-angle-base-en")
-# Run inference
-sentences = [
-    'The weather is lovely today.',
-    "It's so sunny outside!",
-    'He drove to the stadium.',
-]
-embeddings = model.encode(sentences)
-print(embeddings.shape)
-# [3, 768]
-
-# Get the similarity scores for the embeddings
-similarities = model.similarity(embeddings, embeddings)
-print(similarities.shape)
-# [3, 3]
+```bibtext
+@article{li2023angle,
+  title={AnglE-optimized Text Embeddings},
+  author={Li, Xianming and Li, Jing},
+  journal={arXiv preprint arXiv:2309.12871},
+  year={2023}
+}
 ```
 
-<!--
-### Direct Usage (Transformers)
-
-<details><summary>Click to see the direct usage in Transformers</summary>
-
-</details>
--->
 
-<!--
-### Downstream Usage (Sentence Transformers)
-
-You can finetune this model on your own dataset.
-
-<details><summary>Click to expand</summary>
-
-</details>
--->
-
-<!--
-### Out-of-Scope Use
+## Usage
 
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
+### via angle-emb
 
-<!--
-## Bias, Risks and Limitations
+```bash
+python -m pip install -U angle-emb
+```
 
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
+Example:
 
-<!--
-### Recommendations
+```python
+from angle_emb import AnglE
+from angle_emb.utils import cosine_similarity
 
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
+angle = AnglE.from_pretrained('WhereIsAI/pubmed-angle-base-en', pooling_strategy='cls').cuda()
 
-## Training Details
+query = 'How to treat childhood obesity and overweight?'
+docs = [
+    query,
+    'The child is overweight. Parents should relieve their children\'s symptoms through physical activity and healthy eating. First, they can let them do some aerobic exercise, such as jogging, climbing, swimming, etc. In terms of diet, children should eat more cucumbers, carrots, spinach, etc. Parents should also discourage their children from eating fried foods and dried fruits, which are high in calories and fat. Parents should not let their children lie in bed without moving after eating. If their children\'s condition is serious during the treatment of childhood obesity, parents should go to the hospital for treatment under the guidance of a doctor in a timely manner.',
+    'If you want to treat tonsillitis better, you can choose some anti-inflammatory drugs under the guidance of a doctor, or use local drugs, such as washing the tonsil crypts, injecting drugs into the tonsils, etc. If your child has a sore throat, you can also give him or her some pain relievers. If your child has a fever, you can give him or her antipyretics. If the condition is serious, seek medical attention as soon as possible. If the medication does not have a good effect and the symptoms recur, the author suggests surgical treatment. Parents should also make sure to keep their children warm to prevent them from catching a cold and getting tonsillitis again.',
+]
 
-### Framework Versions
-- Python: 3.10.12
-- Sentence Transformers: 3.0.1
-- Transformers: 4.42.3
-- PyTorch: 2.3.0+cu121
-- Accelerate: 0.30.1
-- Datasets: 2.19.1
-- Tokenizers: 0.19.1
+embeddings = angle.encode(docs)
+query_emb = embeddings[0]
 
-## Citation
+for doc, emb in zip(docs[1:], embeddings[1:]):
+    print(cosine_similarity(query_emb, emb))
 
-### BibTeX
+# 0.8029839020052982
+# 0.4260630076818197
+```
 
-<!--
-## Glossary
 
-*Clearly define terms in order to be accessible across audiences.*
--->
+### via sentence-transformers
 
-<!--
-## Model Card Authors
+Install sentence-transformers
 
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
+```bash
+python -m pip install -U sentence-transformers
+```
 
-<!--
-## Model Card Contact
 
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->