Update README.md

README.md

@@ -1,15 +1,23 @@
 ---
 library_name: peft
 ---
-## Training procedure
+These are LoRA adaption weights for [mT5]<https://huggingface.co/google/mt5-xxl> encoder.
+
+## Multilingual Sentence T5
+This model is a multilingual extension of Sentence T5 and was created using the [mT5]<https://huggingface.co/google/mt5-xxl> encoder. It is proposed in this [paper]<hoge>.
+It is an encoder for sentence embedding, and its performance has been verified in cross-lingual STS and sentence retrieval.
 
 ### Framework versions
 
 
 - PEFT 0.4.0.dev0
 
-## 
-
+## Hot to use
+0. If you have not installed peft, please do so.
+```
+pip install -q git+https://github.com/huggingface/transformers.git@main git+https://github.com/huggingface/peft.git
+```
+1. Load the model.
 ```
 from transformers import MT5EncoderModel
 from peft import PeftModel
@@ -19,3 +27,21 @@ model.enable_input_require_grads()
 model.gradient_checkpointing_enable()
 model: PeftModel = PeftModel.from_pretrained(model, "pkshatech/m-ST5")
 ```
+2. To obtain sentence embedding, use the mean pooling.
+```
+tokenizer = AutoTokenizer.from_pretrained("google/mt5-xxl", use_fast=False)
+model.eval()
+
+texts = ["I am a dog.","You are a cat."]
+inputs = tokenizer(
+    texts,
+    padding=True,
+    truncation=True,
+    return_tensors="pt",
+)
+outputs = model(**inputs)
+last_hidden_state = outputs.last_hidden_state
+last_hidden_state[inputs.attention_mask == 0, :] = 0
+sent_len = inputs.attention_mask.sum(dim=1, keepdim=True)
+sent_emb = last_hidden_state.sum(dim=1) / sent_len
+```