Kite
🎉 You are looking at Kite 9, which uses different hyperparameters and a larger tokenizer!
Kite is a small, trained, 16 million parameter language model.
Training
It was trained on the first shard of Andrej Karpathy's shuffle of nvidia/Nemotron-ClimbMix, using 1 epoch, 8 batch size, 0.001 learning rate, and the Mistral 7B v0.3 tokenizer.
Limitations
Due to its size, the model is not suitable for production workloads.
- Downloads last month
- 10