Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
base_model: unsloth/
|
| 3 |
tags:
|
| 4 |
- text-generation-inference
|
| 5 |
- transformers
|
|
@@ -64,14 +64,14 @@ Install the required packages:
|
|
| 64 |
|
| 65 |
## Model Initialization
|
| 66 |
|
| 67 |
-
Load the **
|
| 68 |
|
| 69 |
```python
|
| 70 |
from unsloth import FastLanguageModel
|
| 71 |
import torch
|
| 72 |
|
| 73 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 74 |
-
model_name="unsloth/
|
| 75 |
max_seq_length=2048,
|
| 76 |
dtype=torch.bfloat16,
|
| 77 |
load_in_4bit=True,
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model: unsloth/tinyllama-bnb-4bit
|
| 3 |
tags:
|
| 4 |
- text-generation-inference
|
| 5 |
- transformers
|
|
|
|
| 64 |
|
| 65 |
## Model Initialization
|
| 66 |
|
| 67 |
+
Load the **Tinyllama** model with 4-bit quantization for reduced resource usage:
|
| 68 |
|
| 69 |
```python
|
| 70 |
from unsloth import FastLanguageModel
|
| 71 |
import torch
|
| 72 |
|
| 73 |
model, tokenizer = FastLanguageModel.from_pretrained(
|
| 74 |
+
model_name="unsloth/tinyllama-bnb-4bit",
|
| 75 |
max_seq_length=2048,
|
| 76 |
dtype=torch.bfloat16,
|
| 77 |
load_in_4bit=True,
|