Update README.md

README.md

@@ -9,13 +9,13 @@ On all three widely-used instruct model benchmarks: **Alpaca-Eval-V2**, **MT-Ben
 and strong proprietary models (e.g., GPT-3.5-turbo-0613). The model is trained with open-sourced datasets without any additional human- or GPT4-labeling.
 
 ## Model Releases
-- SFT model
-- Reward model
-- RLHF model
+- [SFT model](https://huggingface.co/Salesforce/SFR-SFT-LLaMA-3-8B-R)
+- [Reward model](https://huggingface.co/Salesforce)
+- [RLHF model](https://huggingface.co/Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R)
 
 ## Dataset Releases
-- Preference data mix
-- Prompt collection for RLHF training
+- [Preference data mix]()
+- [Prompt collection for RLHF training]()
 
 ## Training methods
 The key to our training is iterative RLHF.
@@ -53,20 +53,35 @@ The key to our training is iterative RLHF.
 
 ## Academic Benchmarks
 
-| **Model**              | **Size** | **Method**    | **GSM-8K** | **MMLU** | **HumanEval** | **TruthfulQA** | **ARC** | **MBPP** |
-|------------------------|----------|---------------|------------|----------|---------------|----------------|---------|----------|
-| LLaMA-3-8B-it          | 8B       | RS+DPO+PPO    | 79.6       | 66.0     | 61.6          | 43.9           | 59.5    | 61.1     |
-| Ours (SFT baseline)    | 8B       | SFT           | 76.7       |          | 61.0          |                |         | 63.5     |
-| Ours (Offline baseline)| 8B       | Vanilla DPO   | 79.8       |          | 63.4          |                |         | 60.3     |
-| Ours (Online RLHF)     | 8B       | Iterative DPO | 80.7       | 65.3     | 64.6          | 60.4           | 64.3    | 60.8     |
+| **Model**                  | **Size** | **Method**      | **GSM-8K** | **MMLU** | **HumanEval** | **TruthfulQA** | **ARC** | **MBPP** |
+|----------------------------|----------|-----------------|------------|----------|---------------|----------------|---------|----------|
+| LLaMA-3-8B-it              | 8B       | RS+DPO+PPO      | 79.6       | 66.0     | 61.6          | 43.9           | 59.5    | 61.1     |
+| Ours (SFT baseline)        | 8B       | SFT             | 74.2       | 64.7     | 65.2          | 53.4           | 61.4    | 62.3     |
+| Ours (DPO baseline)        | 8B       | Vanilla DPO     | 79.8       | 64.5     | 63.4          | 61.8           | 65.2    | 60.3     |
+| Ours (Iterative RLHF)      | 8B       | Iterative DPO   | 80.7       | 65.3     | 64.6          | 60.4           | 64.3    | 60.8     |
 
 
 ## Usage
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
+
+device = "cuda" 
+
 model = AutoModelForCausalLM.from_pretrained("Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R")
 tokenizer = AutoTokenizer.from_pretrained("Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R")
 
+messages = [
+    {"role": "user", "content": "I'm trying to teach myself to have nicer handwriting. Can you help?"},
+]
+
+model_inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
+
+model_inputs = model_inputs.to(device)
+model.to(device)
+
+output_tokens = model.generate(model_inputs, max_new_tokens=1024, do_sample=True)
+model_outputs = tokenizer.batch_decode(output_tokens)
+print(model_outputs[0])
 ```