I am still learning
Model Description
This is a toy language model trained from scratch for educational purposes.
Model Type: Causal Language Model Architecture: Transformer decoder-only Parameters: 30,044,544 Context Length: 256 tokens Vocabulary Size: 50257
Training Details
Architecture
- Layers: 6
- Hidden Size: 384
- Attention Heads: 6
- Dropout: 0.1
Training Configuration
- Batch Size: 8
- Learning Rate: 0.0003
- Optimizer: AdamW
- Final Loss: 8.232668980683012
Usage
Quick Start
import torch
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("Sak2004/I-am-still-learning", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("Sak2004/I-am-still-learning")
inputs = tokenizer("The benefits of renewable energy include", return_tensors="pt")
with torch.no_grad():
outputs = model.generate(inputs.input_ids, max_length=100, temperature=0.8)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Limitations
- This is an educational/toy model and is not intended for production use.
- Generated text may be incoherent or factually incorrect.
- Trained on a limited dataset; performance is expected to be modest.
License
This model is released under the MIT License.
Citation
If you use this model, please cite:
@misc{toyllm_2026,
title={I am still learning - A Toy Language Model},
author={Sak2004},
year={2026},
url={https://huggingface.co/Sak2004/I-am-still-learning}
}
- Downloads last month
- -