Update README.md

README.md

@@ -10,6 +10,14 @@ license: apache-2.0
 language:
 - nl
 ---
+
+## Why prune?
+
+Falcon-11B is still undertrained, as can be seen by this graph:
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/QeaL9bOrPskustzFpjMUP.png)
+This is why the choice is made by prune 50% of the layers.
+Note that ~1B of continued pre-training (~1M rows of 1k tokens) is still required to restore the perplexity of this model.
+
 # sliced
 
 This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).