-
Add: Vanilla model trained on the entire tinystories dataset. Delete: Previous vanilla models that were trained on 25%, 50% and 75% of the dataset.
-
Add: Vanilla model trained on 75% of the dataset
-
Add: Heliumlm vanilla trained on 50% of the dataset
-
Add: New Vanilla model trained on 25% of data with no ignore_index in loss function
-
Add: Vanilla SwiGLU
-
Delete: bad models
-
Added gitignore and new primer model
-
Removed bad models
-
Upload checkpoints/heliumlm-primer-iter-10000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-primer-iter-5000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-final-phase2-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-final-phase2-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-final-phase2-model-iter-2000.pt with huggingface_hub
-
Added first model of the final phase
-
Upload checkpoints/heliumlm-final-phase-25percent-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-final-phase-25percent-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-final-phase-25percent-model-iter-2000.pt with huggingface_hub
-
Deleted irrelevant models and added grammatically correct model trained in phases on entire tinystories dataset (using quartely batch technique)
-
Upload checkpoints/heliumlm-phases4-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phases3-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phases2-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phases-model-iter-2000.pt with huggingface_hub
-
Deleted phase models
-
Upload checkpoints/heliumlm-phase4-75percent-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phase2-50-75percent-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phase2-25-50percent-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phase2-25-50percent-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/heliumlm-phase1-25percent-model-iter-2000.pt with huggingface_hub
-
Removed uneccessary models and renamed models for better understanding
-
Upload checkpoints/helium-distill-4-08-model-iter-14000.pt with huggingface_hub
-
Upload checkpoints/helium-distill-4-08-model-iter-12000.pt with huggingface_hub
-
Upload checkpoints/helium-distill-4-08-model-iter-10000.pt with huggingface_hub
-
Upload checkpoints/helium-distill-4-08-model-iter-8000.pt with huggingface_hub
-
Upload checkpoints/helium-distill-4-08-model-iter-6000.pt with huggingface_hub
-
Upload checkpoints/helium-distill-4-08-model-iter-4000.pt with huggingface_hub
-
Upload checkpoints/helium-distill-4-08-model-iter-2000.pt with huggingface_hub
-
Upload checkpoints/mini_model_iter_8000.pt with huggingface_hub
-
Upload checkpoints/mini_model_iter_6000.pt with huggingface_hub
-
Upload checkpoints/mini_model_iter_4000.pt with huggingface_hub
-
Upload checkpoints/mini_model_iter_2000.pt with huggingface_hub
-
Upload checkpoints/model_iter_4000.pt with huggingface_hub
-
Upload checkpoints/model_iter_2000.pt with huggingface_hub