|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: mit |
|
|
tags: |
|
|
- text-generation |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
# Nanochat |
|
|
|
|
|
Nanochat is a small language model from Andrej Karpathy, converted to HuggingFace format. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Architecture**: GPT-style transformer with RoPE, QK normalization, ReLU², and logits softcap |
|
|
- **Parameters**: ~393M |
|
|
- **Hidden Size**: 1280 |
|
|
- **Layers**: 20 |
|
|
- **Attention Heads**: 10 |
|
|
- **Vocabulary**: 65536 tokens |
|
|
- **Context Length**: 2048 tokens |
|
|
|
|
|
## Usage |
|
|
|
|
|
### With Transformers (PyTorch) |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("<model-path>", trust_remote_code=True) |
|
|
tokenizer = AutoTokenizer.from_pretrained("<model-path>") |
|
|
|
|
|
prompt = "Once upon a time" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=50) |
|
|
print(tokenizer.decode(outputs[0])) |
|
|
``` |
|
|
|
|
|
### Converting to MLX |
|
|
|
|
|
To use with Apple's MLX framework: |
|
|
|
|
|
```bash |
|
|
mlx_lm.convert --hf-path <model-path> --mlx-path nanochat-mlx --trust-remote-code |
|
|
mlx_lm.generate --model nanochat-mlx --prompt "Once upon a time" |
|
|
``` |
|
|
|
|
|
## Citation |
|
|
|
|
|
Original model by Andrej Karpathy: https://github.com/karpathy/nanochat |
|
|
|
