| | --- |
| | language: |
| | - code |
| | license: llama2 |
| | pipeline_tag: text-generation |
| | tags: |
| | - facebook |
| | - meta |
| | - pytorch |
| | - llama |
| | - llama-2 |
| | - mlx |
| | extra_gated_heading: You need to share contact information with Meta to access this |
| | model |
| | extra_gated_prompt: "### LLAMA 2 COMMUNITY LICENSE AGREEMENT\n\"Agreement\" means\ |
| | \ the terms and conditions for use, reproduction, distribution and modification\ |
| | \ of the Llama Materials set forth herein. \"Documentation\" means the specifications,\ |
| | \ manuals and documentation accompanying Llama 2 distributed by Meta at https://ai.meta.com/resources/models-and-libraries/llama-downloads/.\ |
| | \ \"Licensee\" or \"you\" means you, or your employer or any other person or entity\ |
| | \ (if you are entering into this Agreement on such person or entity's behalf), of\ |
| | \ the age required under applicable laws, rules or regulations to provide legal\ |
| | \ consent and that has legal authority to bind your employer or such other person\ |
| | \ or entity if you are entering in this Agreement on their behalf. \"Llama 2\"\ |
| | \ means the foundational large language models and software and algorithms, including\ |
| | \ machine-learning model code, trained model weights, inference-enabling code, training-enabling\ |
| | \ code, fine-tuning enabling code and other elements of the foregoing distributed\ |
| | \ by Meta at ai.meta.com/resources/models-and-libraries/llama-downloads/. \"Llama\ |
| | \ Materials\" means, collectively, Meta's proprietary Llama 2 and documentation\ |
| | \ (and any portion thereof) made available under this Agreement. \"Meta\" or \"\ |
| | we\" means Meta Platforms Ireland Limited (if you are located in or, if you are\ |
| | \ an entity, your principal place of business is in the EEA or Switzerland) and\ |
| | \ Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland). \ |
| | \ By clicking \"I Accept\" below or by using or distributing any portion or element\ |
| | \ of the Llama Materials, you agree to be bound by this Agreement. 1. License Rights\ |
| | \ and Redistribution. a. Grant of Rights. You are granted a non-exclusive, worldwide,\ |
| | \ non- transferable and royalty-free limited license under Meta's intellectual property\ |
| | \ or other rights owned by Meta embodied in the Llama Materials to use, reproduce,\ |
| | \ distribute, copy, create derivative works of, and make modifications to the Llama\ |
| | \ Materials. \n \nb. Redistribution and Use. i. If you distribute or make\ |
| | \ the Llama Materials, or any derivative works thereof, available to a third party,\ |
| | \ you shall provide a copy of this Agreement to such third party. ii. If you\ |
| | \ receive Llama Materials, or any derivative works thereof, from a Licensee as\ |
| | \ part of an integrated end user product, then Section 2 of this Agreement will\ |
| | \ not apply to you. iii. You must retain in all copies of the Llama Materials that\ |
| | \ you distribute the following attribution notice within a \"Notice\" text file\ |
| | \ distributed as a part of such copies: \"Llama 2 is licensed under the LLAMA 2\ |
| | \ Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.\"\ |
| | \ iv. Your use of the Llama Materials must comply with applicable laws and regulations\ |
| | \ (including trade compliance laws and regulations) and adhere to the Acceptable\ |
| | \ Use Policy for the Llama Materials (available at https://ai.meta.com/llama/use-policy),\ |
| | \ which is hereby incorporated by reference into this Agreement. v. You will not\ |
| | \ use the Llama Materials or any output or results of the Llama Materials to improve\ |
| | \ any other large language model (excluding Llama 2 or derivative works thereof).\ |
| | \ 2. Additional Commercial Terms. If, on the Llama 2 version release date, the\ |
| | \ monthly active users of the products or services made available by or for Licensee,\ |
| | \ or Licensee's affiliates, is greater than 700 million monthly active users in\ |
| | \ the preceding calendar month, you must request a license from Meta, which Meta\ |
| | \ may grant to you in its sole discretion, and you are not authorized to exercise\ |
| | \ any of the rights under this Agreement unless or until Meta otherwise expressly\ |
| | \ grants you such rights. 3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE\ |
| | \ LAW, THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON\ |
| | \ AN \"AS IS\" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,\ |
| | \ INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY,\ |
| | \ OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING\ |
| | \ THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND ASSUME\ |
| | \ ANY RISKS ASSOCIATED WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND\ |
| | \ RESULTS. 4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE\ |
| | \ LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE,\ |
| | \ PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST\ |
| | \ PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE\ |
| | \ DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY\ |
| | \ OF ANY OF THE FOREGOING. 5. Intellectual Property. a. No trademark licenses are\ |
| | \ granted under this Agreement, and in connection with the Llama Materials, neither\ |
| | \ Meta nor Licensee may use any name or mark owned by or associated with the other\ |
| | \ or any of its affiliates, except as required for reasonable and customary use\ |
| | \ in describing and redistributing the Llama Materials. b. Subject to Meta's ownership\ |
| | \ of Llama Materials and derivatives made by or for Meta, with respect to any derivative\ |
| | \ works and modifications of the Llama Materials that are made by you, as between\ |
| | \ you and Meta, you are and will be the owner of such derivative works and modifications.\ |
| | \ c. If you institute litigation or other proceedings against Meta or any entity\ |
| | \ (including a cross-claim or counterclaim in a lawsuit) alleging that the Llama\ |
| | \ Materials or Llama 2 outputs or results, or any portion of any of the foregoing,\ |
| | \ constitutes infringement of intellectual property or other rights owned or licensable\ |
| | \ by you, then any licenses granted to you under this Agreement shall terminate\ |
| | \ as of the date such litigation or claim is filed or instituted. You will indemnify\ |
| | \ and hold harmless Meta from and against any claim by any third party arising\ |
| | \ out of or related to your use or distribution of the Llama Materials. 6. Term\ |
| | \ and Termination. The term of this Agreement will commence upon your acceptance\ |
| | \ of this Agreement or access to the Llama Materials and will continue in full\ |
| | \ force and effect until terminated in accordance with the terms and conditions\ |
| | \ herein. Meta may terminate this Agreement if you are in breach of any term or\ |
| | \ condition of this Agreement. Upon termination of this Agreement, you shall delete\ |
| | \ and cease use of the Llama Materials. Sections 3, 4 and 7 shall survive the \ |
| | \ termination of this Agreement. 7. Governing Law and Jurisdiction. This Agreement\ |
| | \ will be governed and construed under the laws of the State of California without\ |
| | \ regard to choice of law principles, and the UN Convention on Contracts for the\ |
| | \ International Sale of Goods does not apply to this Agreement. The courts of California\ |
| | \ shall have exclusive jurisdiction of any dispute arising out of this Agreement.\ |
| | \ USE POLICY ### Llama 2 Acceptable Use Policy Meta is committed to promoting safe\ |
| | \ and fair use of its tools and features, including Llama 2. If you access or use\ |
| | \ Llama 2, you agree to this Acceptable Use Policy (“Policy”). The most recent copy\ |
| | \ of this policy can be found at [ai.meta.com/llama/use-policy](http://ai.meta.com/llama/use-policy).\ |
| | \ #### Prohibited Uses We want everyone to use Llama 2 safely and responsibly. You\ |
| | \ agree you will not use, or allow others to use, Llama 2 to: 1. Violate the law\ |
| | \ or others’ rights, including to:\n 1. Engage in, promote, generate, contribute\ |
| | \ to, encourage, plan, incite, or further illegal or unlawful activity or content,\ |
| | \ such as: \n 1. Violence or terrorism \n 2. Exploitation or harm to children,\ |
| | \ including the solicitation, creation, acquisition, or dissemination of child exploitative\ |
| | \ content or failure to report Child Sexual Abuse Material\n 3. Human trafficking,\ |
| | \ exploitation, and sexual violence\n 4. The illegal distribution of information\ |
| | \ or materials to minors, including obscene materials, or failure to employ legally\ |
| | \ required age-gating in connection with such information or materials.\n 5.\ |
| | \ Sexual solicitation\n 6. Any other criminal activity\n 2. Engage in, promote,\ |
| | \ incite, or facilitate the harassment, abuse, threatening, or bullying of individuals\ |
| | \ or groups of individuals\n 3. Engage in, promote, incite, or facilitate discrimination\ |
| | \ or other unlawful or harmful conduct in the provision of employment, employment\ |
| | \ benefits, credit, housing, other economic benefits, or other essential goods and\ |
| | \ services\n 4. Engage in the unauthorized or unlicensed practice of any profession\ |
| | \ including, but not limited to, financial, legal, medical/health, or related professional\ |
| | \ practices \n 5. Collect, process, disclose, generate, or infer health, demographic,\ |
| | \ or other sensitive personal or private information about individuals without rights\ |
| | \ and consents required by applicable laws\n 6. Engage in or facilitate any action\ |
| | \ or generate any content that infringes, misappropriates, or otherwise violates\ |
| | \ any third-party rights, including the outputs or results of any products or services\ |
| | \ using the Llama 2 Materials\n 7. Create, generate, or facilitate the creation\ |
| | \ of malicious code, malware, computer viruses or do anything else that could disable,\ |
| | \ overburden, interfere with or impair the proper working, integrity, operation\ |
| | \ or appearance of a website or computer system \n2. Engage in, promote, incite,\ |
| | \ facilitate, or assist in the planning or development of activities that present\ |
| | \ a risk of death or bodily harm to individuals, including use of Llama 2 related\ |
| | \ to the following:\n 1. Military, warfare, nuclear industries or applications,\ |
| | \ espionage, use for materials or activities that are subject to the International\ |
| | \ Traffic Arms Regulations (ITAR) maintained by the United States Department of\ |
| | \ State\n 2. Guns and illegal weapons (including weapon development)\n 3. Illegal\ |
| | \ drugs and regulated/controlled substances\n 4. Operation of critical infrastructure,\ |
| | \ transportation technologies, or heavy machinery\n 5. Self-harm or harm to others,\ |
| | \ including suicide, cutting, and eating disorders\n 6. Any content intended to\ |
| | \ incite or promote violence, abuse, or any infliction of bodily harm to an individual\n\ |
| | 3. Intentionally deceive or mislead others, including use of Llama 2 related to\ |
| | \ the following:\n 1. Generating, promoting, or furthering fraud or the creation\ |
| | \ or promotion of disinformation\n 2. Generating, promoting, or furthering defamatory\ |
| | \ content, including the creation of defamatory statements, images, or other content\n\ |
| | \ 3. Generating, promoting, or further distributing spam\n 4. Impersonating another\ |
| | \ individual without consent, authorization, or legal right\n 5. Representing that\ |
| | \ the use of Llama 2 or outputs are human-generated\n 6. Generating or facilitating\ |
| | \ false online engagement, including fake reviews and other means of fake online\ |
| | \ engagement \n 4. Fail to appropriately disclose to end users any known dangers\ |
| | \ of your AI system \nPlease report any violation of this Policy, software “bug,”\ |
| | \ or other problems that could lead to a violation of this Policy through one of\ |
| | \ the following means: * Reporting issues with the model: [github.com/facebookresearch/llama](http://github.com/facebookresearch/llama)\ |
| | \ * Reporting risky content generated by the model: [developers.facebook.com/llama_output_feedback](http://developers.facebook.com/llama_output_feedback)\ |
| | \ * Reporting bugs and security concerns: [facebook.com/whitehat/info](http://facebook.com/whitehat/info)\ |
| | \ * Reporting violations of the Acceptable Use Policy or unlicensed uses of Llama:\ |
| | \ [LlamaUseReport@meta.com](mailto:LlamaUseReport@meta.com)" |
| | extra_gated_fields: |
| | First Name: text |
| | Last Name: text |
| | Date of birth: date_picker |
| | Country: country |
| | Affiliation: text |
| | geo: ip_location |
| | ? By clicking Submit below I accept the terms of the license and acknowledge that |
| | the information I provide will be collected stored processed and shared in accordance |
| | with the Meta Privacy Policy |
| | : checkbox |
| | extra_gated_description: The information you provide will be collected, stored, processed |
| | and shared in accordance with the [Meta Privacy Policy](https://www.facebook.com/privacy/policy/). |
| | extra_gated_button_content: Submit |
| | --- |
| | |
| | # ipetrukha/CodeLlama-7b-Instruct-hf-4bit |
| |
|
| | The Model [ipetrukha/CodeLlama-7b-Instruct-hf-4bit](https://huggingface.co/ipetrukha/CodeLlama-7b-Instruct-hf-4bit) was converted to MLX format from [meta-llama/CodeLlama-7b-Instruct-hf](https://huggingface.co/meta-llama/CodeLlama-7b-Instruct-hf) using mlx-lm version **0.16.1**. |
| |
|
| | ## Use with mlx |
| |
|
| | ```bash |
| | pip install mlx-lm |
| | ``` |
| |
|
| | ```python |
| | from mlx_lm import load, generate |
| | |
| | model, tokenizer = load("ipetrukha/CodeLlama-7b-Instruct-hf-4bit") |
| | response = generate(model, tokenizer, prompt="hello", verbose=True) |
| | ``` |
| |
|
