Update README.md

Add Langchain RAG example

README.md

@@ -5,11 +5,11 @@ metrics:
 - spearmanr
 ---
 
-### Overview
+## Overview
 This model is primarily designed for language understanding between Chinese texts.<br>
 It utilizes the **CoSENT** training framework for the purpose of the Retrieval-Augmented Generation (RAG) task.
 
-### Download the model
+## Download the model
 
 ```python
 from transformers import AutoTokenizer, AutoModel
@@ -17,7 +17,7 @@ tokenizer = AutoTokenizer.from_pretrained("Mike0307/text2vec-base-chinese-rag")
 model = AutoModel.from_pretrained("Mike0307/text2vec-base-chinese-rag")
 ```
 
-### Example of similarity comparison
+## Example of similarity comparison
 ```python
 import torch
 def mean_pooling(model_output, attention_mask):
@@ -43,9 +43,9 @@ torch.cosine_similarity(embeddings[0], embeddings[1], dim=0)
 
 ```
 
-### Example of Langchain Retriever
+## Example of Langchain RAG
 
-RAG with langchain: https://python.langchain.com/v0.1/docs/use_cases/question_answering/
+RAG with Langchain: https://python.langchain.com/v0.1/docs/use_cases/question_answering/
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6414866f1cbd604c9217c7d0/RrBoHJINfrSWtCNkePs7g.png)
 
 Install the langchain packages
@@ -54,6 +54,8 @@ Install the langchain packages
 pip install --upgrade --quiet  langchain langchain-community
 ```
 
+### Use this embedding model to build a retiever
+
 Download HuggingFace model through langchain_community
 
 ```python
@@ -77,6 +79,89 @@ documents = [
     Document(page_content="滾石國際音樂股份有限公司 Rock Records Co., Ltd.  曾用名 滾石雜誌社 滾石有聲出版社 公司類型 股份有限公司 統一編號 22012304 成立 1976年，滾石雜誌社 1980年，滾石有聲出版社 1986年1月28日（公司登記日期）（38年113天） 創辦人 段鍾沂、段鍾潭 代表人物 段鍾沂、段鍾潭 "),
 ]
 db = FAISS.from_documents(documents, embeddings)
-db.similarity_search("福井舞所屬哪家唱片公司？", k=1)
+retriever = db.as_retriever(search_kwargs = {"k" : 1})
+retriever.invoke("福井舞所屬哪家唱片公司？")
 # [Document(page_content='23歲時出道、血型A型的福井舞是出身於京都的日本女創作歌手，所屬唱片公司為J-more。2004年，與WADAGAKI、SHINO組合地下音樂隊Poplar，發表了兩張專輯，天照和夢死物語。在2006年時退出，2007年10月加入了Avex獨立發展。')]
 ```
+
+### Use HuggingFace LLM as the langchain LLM
+
+First, download the HuggingFace LLM via the bellow. Check this [repo](https://huggingface.co/Mike0307/Phi-3-mini-4k-instruct-chinese-lora) if you encounter any problems.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+llm_id = "Mike0307/Phi-3-mini-4k-instruct-chinese-lora"
+model = AutoModelForCausalLM.from_pretrained(
+    llm_id, 
+    device_map="mps", # Change mps if not MacOS
+    torch_dtype=torch.float32,  # try float16 for M1 chip
+    trust_remote_code=True,
+    attn_implementation="eager", # without flash_attn
+)
+tokenizer = AutoTokenizer.from_pretrained(llm_id)
+```
+
+Second, construct a valid langchain LLM class using customized HuggingFace model.
+
+```python
+import re
+from pydantic import Field
+from typing import Any, List, Optional
+from langchain.prompts import PromptTemplate
+from langchain.schema.runnable import RunnablePassthrough
+from langchain_core.callbacks.manager import CallbackManagerForLLMRun
+from langchain_core.language_models.llms import LLM
+
+class CustomLLM(LLM):
+    model : Any = Field(..., description="The huggingface llm model")
+    tokenizer : Any = Field(..., description="The huggingface llm tokenizer.")
+    def __init__(self, model, tokenizer):
+        super().__init__(model = model, tokenizer = tokenizer)
+    
+    def _call(self, prompt: str, stop: Optional[List[str]] = None,
+        run_manager: Optional[CallbackManagerForLLMRun] = None,**kwargs: Any,) -> str:
+        if stop is not None:
+            raise ValueError("stop kwargs are not permitted.")
+        inputs = self.tokenizer(prompt, return_tensors="pt").to(self.model.device)
+        outputs = self.model.generate(**inputs, temperature = 0.0, max_length = 500, do_sample = False)
+        generated_text = tokenizer.decode(outputs[0], skip_special_tokens=False)
+        return self.output_parser(generated_text)
+    
+    @property
+    def _llm_type(self) -> str:
+        return "custom"
+    
+    def output_parser(output):
+        pattern = "<\|assistant\|>(.*?)<\|endoftext\|>"
+        match = re.search(pattern, output, re.DOTALL)
+        if match:
+            return match.group(1).strip()
+        return output.strip()
+
+```
+
+### Make a simple RAG chain
+
+Use `promt`, `llm`, `retriever` to build a simple RAG chain and try inference.
+
+```python
+import langchain
+langchain.debug = True # Check the chain process and validate the retrieved documents
+
+prompt = PromptTemplate.from_template(template="<|user|>{documents}\n{question} <|end|>\n<|assistant|>")
+llm = CustomLLM(model, tokenizer)
+rag = {
+    "question" : RunnablePassthrough(),
+    "documents" : retriever
+} | prompt | llm
+
+## example of inference
+query = "埃及聖䴉是什麼？"
+rag.invoke(query)
+## '埃及聖䴉是一種埃及的朱鷺，它在埃及備受尊敬，經常被製成木乃伊當做托特的象徵。它也被引入到法國、義大利、西班牙及美國。現在，在臺灣西部濱海地區也可看到埃及聖䴉。'
+
+```
+
+