Update README.md
Browse files
README.md
CHANGED
|
@@ -1,94 +1,99 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: gpl-3.0
|
| 3 |
-
language:
|
| 4 |
-
- en
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
]
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: gpl-3.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
datasets:
|
| 6 |
+
- Mxode/Magpie-Pro-10K-GPT4o-mini
|
| 7 |
+
pipeline_tag: text2text-generation
|
| 8 |
+
tags:
|
| 9 |
+
- text-generation-inference
|
| 10 |
+
---
|
| 11 |
+
# NanoLM-25M-Instruct-v1
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
English | [简体中文](README_zh-CN.md)
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Introduction
|
| 18 |
+
|
| 19 |
+
In order to explore the potential of small models, I have attempted to build a series of them, which are available in the [NanoLM Collections](https://huggingface.co/collections/Mxode/nanolm-66d6d75b4a69536bca2705b2).
|
| 20 |
+
|
| 21 |
+
This is NanoLM-25M-Instruct-v1. The model currently supports **English only**.
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
## Model Details
|
| 26 |
+
|
| 27 |
+
| Nano LMs | Non-emb Params | Arch | Layers | Dim | Heads | Seq Len |
|
| 28 |
+
| :----------: | :------------------: | :---: | :----: | :-------: | :---: | :---: |
|
| 29 |
+
| **25M** | **15M** | **MistralForCausalLM** | **12** | **312** | **12** | **2K** |
|
| 30 |
+
| 70M | 42M | LlamaForCausalLM | 12 | 576 | 9 |2K|
|
| 31 |
+
| 0.3B | 180M | Qwen2ForCausalLM | 12 | 896 | 14 |4K|
|
| 32 |
+
| 1B | 840M | Qwen2ForCausalLM | 18 | 1536 | 12 |4K|
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
## How to use
|
| 37 |
+
|
| 38 |
+
```python
|
| 39 |
+
import torch
|
| 40 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 41 |
+
|
| 42 |
+
model_path = 'Mxode/NanoLM-25M-Instruct-v1'
|
| 43 |
+
|
| 44 |
+
model = AutoModelForCausalLM.from_pretrained(model_path).to('cuda:0', torch.bfloat16)
|
| 45 |
+
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def get_response(prompt: str, **kwargs):
|
| 49 |
+
generation_args = dict(
|
| 50 |
+
max_new_tokens = kwargs.pop("max_new_tokens", 512),
|
| 51 |
+
do_sample = kwargs.pop("do_sample", True),
|
| 52 |
+
temperature = kwargs.pop("temperature", 0.7),
|
| 53 |
+
top_p = kwargs.pop("top_p", 0.8),
|
| 54 |
+
top_k = kwargs.pop("top_k", 40),
|
| 55 |
+
**kwargs
|
| 56 |
+
)
|
| 57 |
+
|
| 58 |
+
messages = [
|
| 59 |
+
{"role": "system", "content": "You are a helpful assistant."},
|
| 60 |
+
{"role": "user", "content": prompt}
|
| 61 |
+
]
|
| 62 |
+
text = tokenizer.apply_chat_template(
|
| 63 |
+
messages,
|
| 64 |
+
tokenize=False,
|
| 65 |
+
add_generation_prompt=True
|
| 66 |
+
)
|
| 67 |
+
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
| 68 |
+
|
| 69 |
+
generated_ids = model.generate(model_inputs.input_ids, **generation_args)
|
| 70 |
+
generated_ids = [
|
| 71 |
+
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
| 72 |
+
]
|
| 73 |
+
|
| 74 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 75 |
+
return response
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
prompt1 = "What can you do for me?"
|
| 79 |
+
print(get_response(prompt1, do_sample=False))
|
| 80 |
+
|
| 81 |
+
"""
|
| 82 |
+
I'm so glad you asked! I'm a large language model, so I don't have personal experiences or emotions, but I can provide information and assist with tasks to help with your tasks.
|
| 83 |
+
|
| 84 |
+
Here are some ways I can assist you:
|
| 85 |
+
|
| 86 |
+
1. **Answer questions**: I can provide information on a wide range of topics, from science and history to entertainment and culture.
|
| 87 |
+
2. **Generate text**: I can create text based on a prompt or topic, and can even help with writing tasks such as proofreading and editing.
|
| 88 |
+
3. **Translate text**: I can translate text from one language to another, including popular languages such as Spanish, French, German, Chinese, and many more.
|
| 89 |
+
4. **Summarize content**: I can summarize long pieces of text, such as articles or documents, into shorter, more digestible versions.
|
| 90 |
+
5. **Offer suggestions**: I can provide suggestions for things like gift ideas, travel destinations, books, or movies.
|
| 91 |
+
6. **Chat and converse**: I can engage in natural-sounding conversations, using context and understanding to respond to questions and statements.
|
| 92 |
+
7. **Play games**: I can play simple text-based games, such as 20 Questions, Hangman, or Word Jumble.
|
| 93 |
+
8. **Provide definitions**: I can define words and phrases, explaining their meanings and usage.
|
| 94 |
+
9. **Offer suggestions**: I can provide suggestions for things like gift ideas, travel destinations, or books to read.
|
| 95 |
+
10. **Entertain**: I can engage in fun conversations, tell jokes, and even create simple games or puzzles.
|
| 96 |
+
|
| 97 |
+
Which of these methods would you like to do?
|
| 98 |
+
"""
|
| 99 |
+
```
|