Created README.md from the official repository

RADME.md

@@ -0,0 +1,51 @@
+---
+license: apache-2.0
+---
+# MatMul-Free LL
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+## Model Details
+
+[[Paper](https://arxiv.org/abs/2406.02528)] [[Code](https://github.com/ridgerchu/matmulfreellm/tree/master)]
+
+MatMul-Free LM is a language model architecture that eliminates the need for Matrix Multiplication (MatMul) operations. 
+This repository provides an implementation of MatMul-Free LM that is compatible with the 🤗 Transformers library.
+
+![MatMul-Free LM](https://raw.githubusercontent.com/ridgerchu/matmulfreellm/master/__assets__/main.png)
+
+## Scaling Law
+
+We evaluate how the scaling law fits to the 370M, 1.3B and 2.7B parameter models in both Transformer++ and our model. 
+For a fair comparison, each operation is treated identically, though our model uses more efficient ternary weights in some layers. 
+Interestingly, the scaling projection for our model exhibits a steeper descent compared to Transformer++, 
+suggesting our architecture is more efficient in leveraging additional compute to improve performance.
+
+![Scaling Law](https://raw.githubusercontent.com/ridgerchu/matmulfreellm/master/__assets__/scaling_law.png)
+
+## Usage
+
+We provide the implementations of models that are compatible with 🤗 Transformers library. 
+Here's an example of how to initialize a model from the default configs in ```matmulfreelm```: 
+This is a huggingface-compatible library that you can use such command to initialize the model with huggingface ```AutoModel```:
+
+```shell
+pip install transformers
+pip install -U git+https://github.com/ridgerchu/matmulfreellm
+```
+
+```python
+from mmfreelm.models import HGRNBitConfig
+from mmfreelm.layers import hgrn_bit
+
+from transformers import AutoModelForCausalLM
+model = AutoModelForCausalLM.from_pretrained("ridger/MMfreeLM-2.7B")
+```
+
+## Pre-trained Model Zoo
+
+| Model Size | Layer | Hidden dimension | Trained tokens |
+| [370M](https://huggingface.co/ridger/MMfreeLM-370M) | 24 | 1024 | 15B |
+| :---: | :---: | :---: | :---: |
+| [1.3B](https://huggingface.co/ridger/MMfreeLM-1.3B)     |  24   |   2048    |   100B    |
+| [2.7B](https://huggingface.co/ridger/MMfreeLM-2.7B)     |  32   |   2560    |   100B    |