BitMar 100M tokens - Epoch 10 - 996,862,184 tokens processed

README.md

@@ -19,9 +19,9 @@ This model was trained on exactly 100 million tokens as part of the BabyLM chall
 
 ## Training Details
 - Total tokens: 100,000,000
-- Epochs completed: 9
-- Tokens processed: 897,175,985
-- Cross-modal similarity: 0.4368
+- Epochs completed: 10
+- Tokens processed: 996,862,184
+- Cross-modal similarity: 0.4552
 
 ## Model Architecture
 - Text encoder: 4 layers, 128 hidden size
@@ -38,6 +38,6 @@ tokenizer = AutoTokenizer.from_pretrained("euhidaman/bitmar-attention-multimodal
 
 
 ## Training Status
-- **Status**: In Progress (Epoch 9)
-- **Tokens Processed**: 897,175,985
-- **Best Cross-modal Similarity**: 0.4368
+- **Status**: In Progress (Epoch 10)
+- **Tokens Processed**: 996,862,184
+- **Best Cross-modal Similarity**: 0.4552

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b3ca5993fbf9f2896ae5f6f9d90fbbdc635f7161c756da042fe21d52786c3b2e
+oid sha256:d05b530305b4708c0680d672e825ad410d74cd906b96385d8d6e9ec67e4b15eb
 size 86128991

training_metadata.json

@@ -1,9 +1,9 @@
 {
-  "epoch": 8,
-  "global_step": 895482,
-  "tokens_processed": 897175985,
+  "epoch": 9,
+  "global_step": 994980,
+  "tokens_processed": 996862184,
   "target_tokens": 100000000,
-  "best_similarity": 0.4368094205856323,
+  "best_similarity": 0.4551965296268463,
   "training_config": {
     "model": {
       "vocab_size": 50257,