| | --- |
| | extra_gated_heading: Access Llama 2 on Hugging Face |
| | extra_gated_description: >- |
| | This is a form to enable access to Llama 2 on Hugging Face after you have been |
| | granted access from Meta. Please visit the [Meta |
| | website](https://ai.meta.com/resources/models-and-libraries/llama-downloads) |
| | and accept our license terms and acceptable use policy before submitting this |
| | form. Requests will be processed in 1-2 days. |
| | extra_gated_prompt: >- |
| | **Your Hugging Face account email address MUST match the email you provide on |
| | the Meta website, or your request will not be approved.** |
| | extra_gated_button_content: Submit |
| | extra_gated_fields: |
| | I agree to share my name, email address and username with Meta and confirm that I have already been granted download access on the Meta website: checkbox |
| | language: |
| | - en |
| | pipeline_tag: text-generation |
| | inference: false |
| | tags: |
| | - facebook |
| | - meta |
| | - pytorch |
| | - llama |
| | - llama-2 |
| | --- |
| | This is 82M parameters llama model of random weights. This model can be use for proof of concept. <br/> |
| | Tokenizer is copy of meta-llama/Llama-2-7b |
| |
|
| | ``` |
| | # Use a pipeline as a high-level helper |
| | from transformers import LlamaConfig, LlamaForCausalLM, LlamaTokenizer |
| | import numpy as np |
| | |
| | config = LlamaConfig(vocab_size=32000, hidden_size=768, intermediate_size=768*4, num_hidden_layers=4, num_attention_heads=8) |
| | tokenizer = LlamaTokenizer.from_pretrained("meta-llama/Llama-2-7b") |
| | model = LlamaForCausalLM(config).half() |
| | model_parameters = filter(lambda p: p.requires_grad, model.parameters()) |
| | params = sum([np.prod(p.size()) for p in model_parameters]) |
| | print(params / 1024 / 1024) # 82.881591796875 |
| | |
| | hub_id = "heegyu/llama-small-randomweights" |
| | tokenizer.push_to_hub(hub_id) |
| | model.push_to_hub(hub_id) |
| | ``` |
