Update README.md

README.md

@@ -1,143 +1,116 @@
 ---
 tags:
+- transformers
 - sentence-transformers
-- sentence-similarity
-- feature-extraction
-- dense
-pipeline_tag: sentence-similarity
-library_name: sentence-transformers
+language:
+- en
+license: apache-2.0
+library_name: transformers
 ---
+## Introduction
 
-# SentenceTransformer
+A key limitation of large language models (LLMs) is their inability to capture less-frequent or up-to-date entity knowledge, often leading to factual inaccuracies and hallucinations. Retrieval-augmented generation (RAG), which incorporates external knowledge through retrieval, is a common approach to mitigate this issue.
 
-This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+Although RAG typically relies on embedding-based retrieval, the embedding models themselves are also based on language models and therefore struggle with queries involving less-frequent entities, often failing to retrieve the crucial knowledge needed to overcome this limitation.
 
-## Model Details
+**Knowledgeable Passage Retriever** enhances the performance with such queries by injecting real-world entity knowledge into embeddings, making them more *knowledgeable*.
+**The entity knowledge is pluggable and can be dynamically updated with ease.**
 
-### Model Description
-- **Model Type:** Sentence Transformer
-<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 768 dimensions
-- **Similarity Function:** Dot Product
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+For more details, refer to [our GitHub repository](https://github.com/knowledgeable-embedding/knowledgeable-embedding).
 
-### Model Sources
+## Model List
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+| Model | Model Size | Base Model |
+| --- | --- | --- |
+| [knowledgeable-ai/kpr-bert-base-uncased](https://huggingface.co/knowledgeable-ai/kpr-bert-base-uncased) | 112M | [bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) |
+| [knowledgeable-ai/kpr-retromae](https://huggingface.co/knowledgeable-ai/kpr-retromae) | 112M | [RetroMAE](https://huggingface.co/Shitao/RetroMAE) |
+| [knowledgeable-ai/kpr-bge-base-en](https://huggingface.co/knowledgeable-ai/kpr-bge-base-en) | 112M | [bge-base-en](https://huggingface.co/BAAI/bge-base-en) |
+| [knowledgeable-ai/kpr-bge-base-en-v1.5](https://huggingface.co/knowledgeable-ai/kpr-bge-base-en-v1.5) | 112M | [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) |
+| [knowledgeable-ai/kpr-bge-large-en-v1.5](https://huggingface.co/knowledgeable-ai/kpr-bge-large-en-v1.5) | 340M | [bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) |
 
-### Full Model Architecture
+For practical use, we recommend `knowledgeable-ai/kpr-bge-*`, which significantly outperforms state-of-the-art models on queries involving less-frequent entities while performing comparably on other queries, as reported in [our paper](https://arxiv.org/abs/2507.03922).
 
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'KPRModelForBert'})
-  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
-)
-```
+Regarding the model size, we do not count the entity embeddings since they are stored in CPU memory and have a negligible impact on runtime performance. See [this page](https://github.com/knowledgeable-embedding/knowledgeable-embedding/wiki/Internals-of-Knowledgeable-Embedding) for details.
 
-## Usage
+## Model Details
 
-### Direct Usage (Sentence Transformers)
+- Base Model: [bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased)
+- Maximum Sequence Length: 512
+- Embedding Dimension: 768
 
-First install the Sentence Transformers library:
+## How to use
 
-```bash
-pip install -U sentence-transformers
-```
+This model can be used via [Hugging Face Transformers](https://github.com/huggingface/transformers) or [Sentence Transformers](https://github.com/UKPLab/sentence-transformers):
+
+### Hugging Face Transformers
 
-Then you can load this model and run inference.
 ```python
-from sentence_transformers import SentenceTransformer
+from transformers import AutoTokenizer, AutoModel
+import torch
+
+MODEL_NAME_OR_PATH = "knowledgeable-ai/kpr-bge-base-en"
 
-# Download from the 🤗 Hub
-model = SentenceTransformer("knowledgeable-ai/kpr-bert-base-uncased")
-# Run inference
-sentences = [
-    'The weather is lovely today.',
-    "It's so sunny outside!",
-    'He drove to the stadium.',
+input_texts = [
+  "Who founded Dominican Liberation Party?",
+  "Who owns Mompesson House?"
 ]
-embeddings = model.encode(sentences)
-print(embeddings.shape)
-# [3, 768]
-
-# Get the similarity scores for the embeddings
-similarities = model.similarity(embeddings, embeddings)
-print(similarities)
-# tensor([[743.6603, 712.7500, 674.8392],
-#         [712.7500, 743.7998, 678.3881],
-#         [674.8391, 678.3880, 743.6827]])
-```
 
-<!--
-### Direct Usage (Transformers)
+# Load model and tokenizer from the Hugging Face Hub
+tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME_OR_PATH, trust_remote_code=True)
+model = AutoModel.from_pretrained(MODEL_NAME_OR_PATH, trust_remote_code=True)
 
-<details><summary>Click to see the direct usage in Transformers</summary>
+# Preprocess the text
+preprocessed_inputs = tokenizer(input_texts, return_tensors="pt", padding=True)
 
-</details>
--->
+# Compute embeddings
+with torch.no_grad():
+    embeddings = model.encode(**preprocessed_inputs)
 
-<!--
-### Downstream Usage (Sentence Transformers)
+print("Embeddings:", embeddings)
+```
 
-You can finetune this model on your own dataset.
+### Sentence Transformers
 
-<details><summary>Click to expand</summary>
+```python
+from sentence_transformers import SentenceTransformer
 
-</details>
--->
+MODEL_NAME_OR_PATH = "knowledgeable-ai/kpr-bge-base-en"
 
-<!--
-### Out-of-Scope Use
+input_texts = [
+  "Who founded Dominican Liberation Party?",
+  "Who owns Mompesson House?"
+]
 
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
+# Load model from the Hugging Face Hub
+model = SentenceTransformer(MODEL_NAME_OR_PATH, trust_remote_code=True)
 
-<!--
-## Bias, Risks and Limitations
+# Compute embeddings
+embeddings = model.encode(input_texts)
 
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
+print("Embeddings:", embeddings)
+```
 
-<!--
-### Recommendations
+**IMPORTANT:** This code will be supported in versions of Sentence Transformers later than v5.1.0,  
+which have not yet been released at the time of writing. Until then, please install the library directly from GitHub:
 
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
+```bash
+pip install git+https://github.com/UKPLab/sentence-transformers.git
+```
 
-## Training Details
+## License
 
-### Framework Versions
-- Python: 3.10.14
-- Sentence Transformers: 5.2.0.dev0
-- Transformers: 4.55.4
-- PyTorch: 2.4.0+cu121
-- Accelerate: 0.34.2
-- Datasets: 2.16.1
-- Tokenizers: 0.21.4
+This model is licensed under the Apache License, Version 2.0.
 
 ## Citation
 
-### BibTeX
-
-<!--
-## Glossary
+If you use this model in your research, please cite the following paper:
+[Dynamic Injection of Entity Knowledge into Dense Retrievers](https://arxiv.org/abs/2507.03922)
 
-*Clearly define terms in order to be accessible across audiences.*
--->
-
-<!--
-## Model Card Authors
-
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
-
-<!--
-## Model Card Contact
-
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->
+```bibtex
+@article{yamada2025kpr,
+  title={Dynamic Injection of Entity Knowledge into Dense Retrievers},
+  author={Ikuya Yamada and Ryokan Ri and Takeshi Kojima and Yusuke Iwasawa and Yutaka Matsuo},
+  journal={arXiv preprint arXiv:2507.03922},
+  year={2025}
+}
+```