Create README.md

README.md

@@ -0,0 +1,44 @@
+---
+extra_gated_heading: Access Llama 2 on Hugging Face
+extra_gated_description: >-
+  This is a form to enable access to Llama 2 on Hugging Face after you have been
+  granted access from Meta. Please visit the [Meta
+  website](https://ai.meta.com/resources/models-and-libraries/llama-downloads)
+  and accept our license terms and acceptable use policy before submitting this
+  form. Requests will be processed in 1-2 days.
+extra_gated_prompt: >-
+  **Your Hugging Face account email address MUST match the email you provide on
+  the Meta website, or your request will not be approved.**
+extra_gated_button_content: Submit
+extra_gated_fields:
+  I agree to share my name, email address and username with Meta and confirm that I have already been granted download access on the Meta website: checkbox
+language:
+  - en
+pipeline_tag: text-generation
+inference: false
+tags:
+  - facebook
+  - meta
+  - pytorch
+  - llama
+  - llama-2
+---
+This is 82M parameters llama model of random weights. This model can be use for proof of concept. <br/>
+Tokenizer is copy of meta-llama/Llama-2-7b
+
+```
+# Use a pipeline as a high-level helper
+from transformers import LlamaConfig, LlamaForCausalLM, LlamaTokenizer
+import numpy as np
+
+config = LlamaConfig(vocab_size=32000, hidden_size=768, intermediate_size=768*4, num_hidden_layers=4, num_attention_heads=8)
+tokenizer = LlamaTokenizer.from_pretrained("meta-llama/Llama-2-7b")
+model = LlamaForCausalLM(config).half()
+model_parameters = filter(lambda p: p.requires_grad, model.parameters())
+params = sum([np.prod(p.size()) for p in model_parameters])
+print(params / 1024 / 1024) # 82.881591796875
+
+hub_id = "heegyu/llama-small-randomweights"
+tokenizer.push_to_hub(hub_id)
+model.push_to_hub(hub_id)
+```