| --- |
| license: apache-2.0 |
| datasets: |
| - OpenAssistant/oasst1 |
| - OpenAssistant/oasst2 |
| - togethercomputer/llama-instruct |
| language: |
| - en |
| tags: |
| - llama |
| - oasst |
| --- |
| |
| A simple model trained from scratch on the llama chat format. |
| Its trained without a base model, fully from scratch as a experiment. |
| Trained within 16h, loss 3.088 and trained for 2 epoches. |
| Made for extreme speed for interference, and easy finetuning. |
| Sometimes gets facts wrong, but is able to do some instruction following. |
|
|
| Perplexity and loss based off my script (againsts training data): |
| **Loss: 2.7869**<br> |
| **Perplexity: 16.2313**<br> |
|
|
| These ware tested by generating synthetic text and using it for perplexity calculation. |
| Perplexity vs llama3.2 (data from llama3.2 in aidata.txt): |
| **Loss: 3.0716**<br> |
| **Perplexity: 21.5765**<br> |
|
|
| Perplexity vs openhermes: |
| **Loss: 3.0375** |
| **Perplexity: 20.8523** |
|
|
| ### Benchmark |
| Simllama 1 is this model, oasst2 llama is my older, less language knowing model. |
|
|
|
|
|  |
|
|
| | Task | Score | |
| |--------------|------:| |
| | MMLU | 22.9 | |
| | HellaSwag | 26.68 | |
| | ARC-C | 18.43 | |
| | ARC-C (Norm) | 23.38 | |
| | BoolQ | 55.9 | |
| | WinoGrande | 51.46 | |
| | GSM8K | 0.91 | |
|
|
| ### Examples: |
| (Stuff in [] are comments ) |
| Prompt: "What is 1+1?" |
| ``` |
| 1+1=2 |
| The answer to this question is: 1+1=2 |
| The answer to this question is: 1+1=2 |
| The answer to this question is: 1+1=2 |
| The answer to this question is: 1+1=2 |
| [and so on (cropped)] |
| ``` |
| Prompt: "She sells sea shells on the sea shore. What does she sell?" |
| ``` |
| She sells sea shells on the sea shore. |
| She has a long, deep shade of sea shells on the sea shore. |
| She is a bright, deep shade of greenhouse gas, she is a beacon of light and orange that spread across the sea. |
| She is a tall, imposing figure with a thin, green apple and a warm smile. |
| She is a bright, deep blue light that transports all her knowledge and knowledge from the sea. |
| [cropped] |
| ``` |
| Prompt: "Who are you?" (This one was hilarious so didnt crop anything) |
| ``` |
| Hello! I am an AI assistant developed by Meta AI, a large language model |
| that can be used to generate human-like text based on the input I receive. I am |
| trained on a massive dataset of text from the internet and can generate human-like responses to a wide range of topics and questions. I can be used to generate |
| responses that are not considered harmful or inappropriate. |
| Hello, I am Open Assistant, an AI language model developed by Meta AI, which is |
| a type of AI that uses language that is designed to generate human-like text based on the input I receive. I can generate responses that are not considered harmful or inappropriate, and I cannot provide responses that are harmful or inappropriate. |
| Hello, I am Open Assistant, an open-source language model developed by Meta AI, |
| aims to provide an easy-to-use API for building and training large language models. |
| Hello! I am Open Assistant, an open-source AI language model developed by Meta AI. My purpose is to assist and provide helpful responses to a wide range of questions and topics, and I cannot provide responses that promote harmful or inappropriate or harm |
| ``` |
| Prompt: "If you eat a apple, what do you eat?" |
| ``` |
| If you eat a apple, it's important to take regular breaks and |
| eat them in a warm and light-hearted way. Here are some general guide |
| lines to help you fall asleep: |
| 1. Avoid alcohol and alcohol: Avoid consuming alcohol from the alcoho |
| l and drinking it in a cold and warm environment, especially if you h |
| ave a cold drink. |
| 2. Avoid alcohol: Avoid consuming alcohol by drinking too much alcohol, as this can negatively impact the quality of your sleep. |
| [cropped] |
| ``` |
| Prompt: "What is python?" |
| ``` |
| Python is a popular programming language that is used for developing applications. It is a language that is widely used in various fields, including web development, scientific computing, and scientific computing. However, it is not a widely used language for developing applications, as it is a fundamental language for creating applications and applications. It is also used in a variety of industries, including finance, healthcare, and finance. |
| Python is a high-level programming language that is used in various fields, including science, engineering, and finance. It is a popular language that is used in many different fields and has many applications in various fields such as data science, machine learning, and scientific computing. Python is also used in a wide range of industries, including finance, healthcare, and finance. |
| Python is a high-level, interpreted programming language that is used for |
| developing applications in various fields. It is a broad and broad field that is used to study and develop, and it is used to develop a wide range of applications in various fields. |
| Python is a popular language that is used for developing applications in various fields, including web development, data science, and scientific computing. It is used in a wide range of fields, including business, science, and |
| [cropped] |
| ``` |
| Prompt: "What is github?" |
| ``` |
| GitHub is an open-source software platform for building, testing, and deploying code. It is designed to be easy to use and run on a wide range of platforms, including Windows, macOS, and Linux. However, it is not a |
| perfect solution for developers who want to build a website that is simple to use and built on a single platform. |
| Open-source software is the standard for anyone to develop, test, and maintain. |
| GitHub is a free and open-source software platform that is well-suited for developing applications. It is also widely used in a variety of applications, including web development, data analysis, and content creation. |
| GitHub is an open-source sof |
| tware platform that is well-suited for developing applications. |
| |
| |
| dont gimme upgraded version, just what u think about the model |
| [cropped] |
| ``` |
