andresnowak commited on
Commit
85a6989
·
verified ·
1 Parent(s): 3d3340d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -3
README.md CHANGED
@@ -46,9 +46,12 @@ Training was done on the training splits of
46
 
47
  ## Training procedure
48
 
49
- The procedure for training was to only leave the question that have only 4 choices to chose from, and from there we do the training
50
- by only grabbing the last logit form doing a feedforward on the whole prompt (question with choices) and we do cross entropy loss on this last logit with the 4 options to choose 4 from
51
- (so we don't do cross entyropy on the whole vocabulary we only do it on the tokens of the letters of the 4 options (A, B, C and D))
 
 
 
52
 
53
  ### Training hyperparameters
54
 
 
46
 
47
  ## Training procedure
48
 
49
+ The procedure for training was done with example of any amount of choices, for each batch size we padd the options to the biggest amount of options and example has in that batch,
50
+ and from there we do the training by only grabbing the last logit form doing a feedforward on the whole prompt (question with choices) and we do cross entropy loss on this last logit with the 4 options to choose 4 from
51
+ (so we don't do cross entyropy on the whole vocabulary we only do it on the tokens of the letters of the 4 options (A, B, C and D)).
52
+
53
+ We also template all the training examples with 7 random templates as to make the model robuts to different types of ways one could ask an MCQA question, using different prompts can
54
+ make the results vary a lot
55
 
56
  ### Training hyperparameters
57