rayraycano
/

finetune-demo-lora

Generated from Trainer

Model card Files Files and versions

finetune-demo-lora / TODO.md

rayraycano's picture

Training in progress, step 20

fcca8c8 verified 4 months ago

|

history blame contribute delete

262 Bytes

todo list

[] Validation of parameters for combinations that won't work

things that are known not to work

FSDP offload and gradient_checkpointing - https://github.com/pytorch/pytorch/issues/82203
adamw_bnb_8bit doesn't play well with FSDP offload