Create REPORT3_Modifications_1+2 _PiT_Training_Results_in_Colab

REPORT3_Modifications_1+2 _PiT_Training_Results_in_Colab

@@ -0,0 +1,80 @@
+I modified the vanilla PiT model in a first undisclosed manner plus a second dislosed manner, essentially doubling the parameter count, and the training results greatly improved, and the two-ways modified model was consistently ahead of the unmodified Vanilla PiT model.  
+My two-ways modified PiT model consistently exceeded the vanilla PiT Val Accuracy until epoch 11 (exceeding the final/highest 94.75% Val Accuracy of the vanilla PiT model at Epoch 10).
+By Epoch 17 and Epoch 19, the two-way modified PiT model had the highest Val Accuracy 95.90%, 95.95% (already surpassing both the Vanilla and the Modification1 PiT models)
+By Epoch 20, the two-ways modified PiT model exceeded 96% Val Accuracy, 
+and Val Accuracy continued to increase up to the highest Val Accuracy of 96.50% before the training was hardcoded terminated Epoch 25.
+
+--- Configuration V3.0 (Encoder-Decoder Architecture) ---
+train_file: /content/sample_data/mnist_train_small.csv
+test_file: /content/sample_data/mnist_test.csv
+image_size: 28
+num_classes: 10
+embed_dim: XXX
+num_layers: X
+num_heads: X
+mlp_dim: XXX
+dropout: 0.1
+batch_size: 128
+epochs: 25
+learning_rate: 0.0001
+XXXXXX
+device: cuda
+image_height: 28
+image_width: 28
+sequence_length: 784
+------------------------------------------------------
+
+Data loaded. Training on cuda.
+  Training samples:   17999
+  Validation samples: 2000
+  Test samples:       9999
+
+Model V3.0 initialized with 2,7XX,XXX trainable parameters.
+
+--- Starting Training (V3.0 Encoder-Decoder Model) ---
+Epoch 01/25 | Train Loss: 2.2017 | Val Loss: 2.0235 | Val Acc: 24.65%
+  -> New best validation accuracy! Saving model state.
+Epoch 02/25 | Train Loss: 1.7963 | Val Loss: 1.2618 | Val Acc: 58.45%
+  -> New best validation accuracy! Saving model state.
+Epoch 03/25 | Train Loss: 0.9849 | Val Loss: 0.7252 | Val Acc: 76.15%
+  -> New best validation accuracy! Saving model state.
+Epoch 04/25 | Train Loss: 0.6703 | Val Loss: 0.4835 | Val Acc: 85.45%
+  -> New best validation accuracy! Saving model state.
+Epoch 05/25 | Train Loss: 0.4813 | Val Loss: 0.3480 | Val Acc: 89.40%
+  -> New best validation accuracy! Saving model state.
+Epoch 06/25 | Train Loss: 0.3694 | Val Loss: 0.2687 | Val Acc: 91.75%
+  -> New best validation accuracy! Saving model state.
+Epoch 07/25 | Train Loss: 0.2967 | Val Loss: 0.2549 | Val Acc: 91.95%
+  -> New best validation accuracy! Saving model state.
+Epoch 08/25 | Train Loss: 0.2568 | Val Loss: 0.2090 | Val Acc: 94.40%
+  -> New best validation accuracy! Saving model state.
+Epoch 09/25 | Train Loss: 0.2338 | Val Loss: 0.1867 | Val Acc: 94.90%
+  -> New best validation accuracy! Saving model state.
+Epoch 10/25 | Train Loss: 0.2137 | Val Loss: 0.1783 | Val Acc: 94.85%
+Epoch 11/25 | Train Loss: 0.1885 | Val Loss: 0.1778 | Val Acc: 94.65%
+Epoch 12/25 | Train Loss: 0.1778 | Val Loss: 0.1747 | Val Acc: 94.65%
+Epoch 13/25 | Train Loss: 0.1635 | Val Loss: 0.1561 | Val Acc: 95.15%
+  -> New best validation accuracy! Saving model state.
+Epoch 14/25 | Train Loss: 0.1549 | Val Loss: 0.1535 | Val Acc: 95.45%
+  -> New best validation accuracy! Saving model state.
+Epoch 15/25 | Train Loss: 0.1498 | Val Loss: 0.1290 | Val Acc: 95.95%
+  -> New best validation accuracy! Saving model state.
+Epoch 16/25 | Train Loss: 0.1363 | Val Loss: 0.1323 | Val Acc: 96.15%
+  -> New best validation accuracy! Saving model state.
+Epoch 17/25 | Train Loss: 0.1260 | Val Loss: 0.1277 | Val Acc: 95.90%
+Epoch 18/25 | Train Loss: 0.1225 | Val Loss: 0.1370 | Val Acc: 95.65%
+Epoch 19/25 | Train Loss: 0.1249 | Val Loss: 0.1395 | Val Acc: 95.95%
+Epoch 20/25 | Train Loss: 0.1150 | Val Loss: 0.1267 | Val Acc: 96.10%
+Epoch 21/25 | Train Loss: 0.1075 | Val Loss: 0.1203 | Val Acc: 96.40%
+  -> New best validation accuracy! Saving model state.
+Epoch 22/25 | Train Loss: 0.1023 | Val Loss: 0.1232 | Val Acc: 96.10%
+Epoch 23/25 | Train Loss: 0.1013 | Val Loss: 0.1277 | Val Acc: 95.80%
+Epoch 24/25 | Train Loss: 0.0975 | Val Loss: 0.1141 | Val Acc: 96.50%
+  -> New best validation accuracy! Saving model state.
+Epoch 25/25 | Train Loss: 0.0853 | Val Loss: 0.1192 | Val Acc: 96.15%
+--- Training Finished ---
+
+--- Evaluating on Test Set (V3.0 Model) ---
+                                                                     Final Test Loss: 0.1096
+Final Test Accuracy: 96.83%
+-------------------------------------------