Sak2004/I-am-still-learning

I am still learning

Model Description

This is a toy language model trained from scratch for educational purposes.

Model Type: Causal Language Model Architecture: Transformer decoder-only Parameters: 30,044,544 Context Length: 256 tokens Vocabulary Size: 50257

Training Details

Architecture

• Layers: 6
• Hidden Size: 384
• Attention Heads: 6
• Dropout: 0.1

Training Configuration

• Batch Size: 8
• Learning Rate: 0.0003
• Optimizer: AdamW
• Final Loss: 8.232668980683012

Usage

Quick Start

import torch
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Sak2004/I-am-still-learning", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("Sak2004/I-am-still-learning")

inputs = tokenizer("The benefits of renewable energy include", return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(inputs.input_ids, max_length=100, temperature=0.8)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• This is an educational/toy model and is not intended for production use.
• Generated text may be incoherent or factually incorrect.
• Trained on a limited dataset; performance is expected to be modest.

License

This model is released under the MIT License.

Citation

If you use this model, please cite:

@misc{toyllm_2026,
  title={I am still learning - A Toy Language Model},
  author={Sak2004},
  year={2026},
  url={https://huggingface.co/Sak2004/I-am-still-learning}
}