Update README.md

README.md

@@ -3,7 +3,9 @@ license: apache-2.0
 ---
 
 
-This model is continually pre-trained from [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) with the structure proposed in [MemoryLLM](https://arxiv.org/abs/2402.04624). 
+This model is continually pre-trained from [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) with the structure proposed in [MemoryLLM](https://arxiv.org/abs/2402.04624).  
+We equip Llama-3 with 12800 memory tokens in each layer, leading to a memory pool of 1.67B parameters. 
+
 
 To use the model, please use the following code: 
 ```