Update README.md

README.md

@@ -29,8 +29,9 @@ The model is trained on a mixture of the following datasets. We also provide the
 - [HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer)
 - [Orca](argilla/distilabel-intel-orca-dpo-pairs)
 
-Difference between this mixture and that of
+Difference between this mixture and the original dataset
 
+- HH-RLHF: we only use the helpful subset and we delete the noisy samples where chosen_response == rejected_response;
 - SHP: we only use the samples with score ratio > 2, for each prompt, we take 5 comparison at most, leading to 109526;
 - Ultrafeedback: similar to UltraFeedback-Binarized, we use the fine-grained score instead of the overall one to rank samples. Meanwhile, for each prompt, we take all possible 6 pairs of comparisons. Finally, we delete the selected pairs with equal scores, leading to 267416.
 - HelpSteer: we use the mean of helpfulness and correctness to rank samples. Meanwhile, we take all possible 6 pairs of comparisons. Finally, we delete the selected pairs with equal scores, leading to 21576;