Update README.md

README.md

@@ -17,10 +17,21 @@ LoRAs and fusion gates for our paper
 
 We released all of our checkpoints used in [LoRA-Flow](https://aclanthology.org/2024.acl-long.695.pdf) which has been accepted to ACL 2024 main conference.
 # Summary
->  In this repo, we release LoRA and the gate of 7B models trained in our paper in HuggingFace format.
+>  In this repo, we release LoRA modules and the gate of 7B models trained in our paper in HuggingFace format.
 # Introduction
 LoRA-Flow provides an efficient way to fuse different LoRA modules which can outperform existing methods significantly. The following picture shows our proposed method, we use layer-wise fusion gates to facilitate dynamic LoRA fusion, which project input hidden states of each layer into fusion weights. For more details, please refer to our paper.
 ![1.jpg](https://cdn-uploads.huggingface.co/production/uploads/64d99f6cd7e30889c6c477b4/ifiu1FTHilrmUkD4FKkgV.jpeg)
+# Training Details
+## LoRA modules Training
+For language LoRA modules: we use the data 52K training examples respectively which from [Okapi](https://aclanthology.org/2023.emnlp-demo.28).
+
+For math LoRA module: the training data for English math LoRA is constructed by [Metamath](https://arxiv.org/abs/2309.12284), which is comprised of 395K mathematical problems in English.
+ 
+For code LoRA module: we train the English code LoRA with the Magicoder dataset [Magicoder](https://arxiv.org/abs/2312.02120), which consists of 186K code generation problems in English.
+
+## Gate Training
+We use gates to fuse different LoRA modules. We employ few-shot training and have released our training data for further details please refer to our GitHub.
+
 # Citation
 if you find our repo is helpful, please cite the following 
 ```bibtex