Update README.md

README.md

@@ -70,3 +70,67 @@ The following hyperparameters were used during training:
 - Pytorch 1.13.0+cu116
 - Datasets 2.8.0
 - Tokenizers 0.13.2
+# Day 1 
+
+1.  Tried to use the Neural Magic Model "neuralmagic/oBERT-12-upstream-pruned-unstructured-97".  The macro and micro f1 scores were much smaller at the
+beginning of the model; the initial step did not increase much. However, it did outperform in the same epoch by .159 difference in the f1 score.
+2. Modification of the code was more significant was able to add errors in my program  to move to the CPU if there was an error in my program 
+``` Python
+import gc
+'''
+Try and Catch the block when training the model using more memory than the GPU, it will produce an error.
+1. Check the Amount of GPU memory used
+2. Move the model to the CPU
+3. Call the garbage collector 
+4. Free the GPU memory in the cache
+5. Check the amount of GPU memory used to see if it is freed
+'''
+def check_gpu_memory():
+    print(torch.cuda.memory_allocated()/1e9)
+    return torch.cuda.memory_allocated()/1e9
+try:
+    trainer.train()
+except RuntimeError as e:
+    if "CUDA out of memory" in str(e):
+        print("CUDA out of memory")
+        print("Let's free some GPU memory and re-allocate")
+        check_gpu_memory()
+        ## Move the model to CPU
+        model.to("cpu")
+        gc.collect()
+        ## Free the GPU memory
+        torch.cuda.empty_cache()
+        check_gpu_memory()
+    else:
+        raise e
+```
+4. Able to check if there was a number of support my model can support in my model 
+``` Python
+from transformers import Trainer, TrainingArguments
+def is_on_colab():
+    if 'google.colab' in sys.modules:
+        return True
+    return False
+
+training_args_fine_tune = TrainingArguments(
+    output_dir  = "./multi-label-class-classification-on-github-issues" , 
+    num_train_epochs = 15,
+    learning_rate = 3e-5,
+    per_device_train_batch_size = 64 , 
+    evaluation_strategy = "epoch" , 
+    save_strategy="epoch"  , 
+    load_best_model_at_end=True, 
+    metric_for_best_model='micro f1',
+    save_total_limit=1, 
+    log_level='error', 
+    push_to_hub = True  if is_on_colab else False ,
+    )
+if torch.cuda.is_available():
+    ## check if the Cuda GPU can bfloat16
+    if torch.cuda.is_bf16_supported():
+        print("Cuda GPU can support bfloat16")
+        training_args_fine_tune.fp16 = True
+    else:
+        print("Cuda GPU cannot support bfloat16 so instead we will use float16 ")
+        training_args_fine_tune.fp16 = True
+```