marianeft commited on
Commit
230bb4c
·
verified ·
1 Parent(s): 7a7005b

Update fine_tune_gpt2_medquad.py

Browse files
Files changed (1) hide show
  1. fine_tune_gpt2_medquad.py +13 -3
fine_tune_gpt2_medquad.py CHANGED
@@ -1,3 +1,8 @@
 
 
 
 
 
1
  import pandas as pd
2
  from transformers import GPT2LMHeadModel, GPT2Tokenizer, Trainer, TrainingArguments
3
  from datasets import load_dataset
@@ -42,6 +47,11 @@ trainer = Trainer(
42
  # Fine-tune the model
43
  trainer.train()
44
 
45
- # Save the model
46
- model.save_pretrained("./fine_tuned_gpt2_medquad")
47
- tokenizer.save_pretrained("./fine_tuned_gpt2_medquad")
 
 
 
 
 
 
1
+ It looks like the `marianeft/MedQuAD` model you're trying to load does not have the required files (`pytorch_model.bin`, `model.safetensors`, `tf_model.h5`, `model.ckpt` or `flax_model.msgpack`). To address this, let's use a different approach.
2
+
3
+ Instead of saving the fine-tuned model to the `marianeft/MedQuAD` directory, let's save it to a new directory. Here’s the modified code to handle this issue:
4
+
5
+ ```python
6
  import pandas as pd
7
  from transformers import GPT2LMHeadModel, GPT2Tokenizer, Trainer, TrainingArguments
8
  from datasets import load_dataset
 
47
  # Fine-tune the model
48
  trainer.train()
49
 
50
+ # Save the model to a new directory
51
+ model.save_pretrained("fine_tuned_medquad")
52
+ tokenizer.save_pretrained("fine_tuned_medquad")
53
+ ```
54
+
55
+ This code will save your fine-tuned model and tokenizer to the "fine_tuned_medquad" directory instead of "marianeft/MedQuAD". This way, you avoid trying to save the model to the original model directory which caused the issue.
56
+
57
+ Give this a try and let me know if it works for you!