contrastive pretraining (pious-haze-45) 3987233
Andrew Green commited on
Commit History
Reset to last good pretrained version 84f48a4
Andrew Green commited on
re-finetuned on deduplicated similarity data 579981a
Andrew Green commited on
Prototype after contrastive learning first pass. Unlikely to be good due to data duplication 519ad17
Andrew Green commited on
Updated pretrained model on 4 epochs (58,500 steps) 7341c83
Andrew Green commited on
Updated model with better pretraining. 30% MLM rate and shuffled source data c65927d
Andrew Green commited on
Upload model 10e91d9 verified
Upload model cd68b5d verified
Upload model 101372f verified
Upload model 79da6c6 verified
Upload model 9472c5e verified
Upload model b49ae26 verified
Update README.md 53e89e4 verified
Move MLM readme into this repo. bc15b88 verified
Should be a base model with no MLM head cb63d3a
Andrew Green commited on