BitMar 100M tokens (no memory) - Epoch 7 - 697,775,898 tokens processed

README.md

@@ -20,9 +20,9 @@ This model was trained on exactly 100 million tokens as part of the BabyLM chall
 
 ## Training Details
 - Total tokens: 100,000,000
-- Epochs completed: 6
-- Tokens processed: 598,093,703
-- Cross-modal similarity: 0.3192
+- Epochs completed: 7
+- Tokens processed: 697,775,898
+- Cross-modal similarity: 0.3340
 - Episodic memory: Disabled
 
 ## Model Architecture
@@ -40,6 +40,6 @@ tokenizer = AutoTokenizer.from_pretrained("estebancarlin/bitmar-no-memory")
 
 
 ## Training Status
-- **Status**: In Progress (Epoch 6)
-- **Tokens Processed**: 598,093,703
-- **Best Cross-modal Similarity**: 0.3192
+- **Status**: In Progress (Epoch 7)
+- **Tokens Processed**: 697,775,898
+- **Best Cross-modal Similarity**: 0.3340

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:50a04ec2a46b421c2a94c06fb50eaf4828d2f5cfbd7989b27e817dc41a1a42a8
+oid sha256:f7c0255243f07e53cb5ae469ff79780ff268d73271e3456d411686e5933b0c40
 size 85226595

training_metadata.json

@@ -1,9 +1,9 @@
 {
-  "epoch": 5,
-  "global_step": 99486,
-  "tokens_processed": 598093703,
+  "epoch": 6,
+  "global_step": 116067,
+  "tokens_processed": 697775898,
   "target_tokens": 100000000,
-  "best_similarity": 0.3191567361354828,
+  "best_similarity": 0.33396559953689575,
   "training_config": {
     "model": {
       "vocab_size": 50257,