fix FT score

update OOM fix

Fix OOM: prev LoRA on CPU + no_grad entire prev contribution (no saved tensors for backward)

Fix OOM: agg_lora_states accumulate weighted sum incrementally instead of O(N) cat

Fix deprecation warnings: torch.load weights_only, cupy fromDlpack, as_target_tokenizer

fix: re-initialize trans_input/prompt_key after from_pretrained, add epsilon to norm, use_reentrant=False

fix: pass attention_mask directly to model.generate(), not via GenerationConfig

fix: override _save to disable safetensors for T5 shared embedding weights

fix: denumpify_detensorize moved to trainer_utils in transformers 4.40+

fix: add explicit trainer_pt_utils imports (nested_truncate etc.) removed from trainer.* in 4.40+

fix: comprehensive transformers 4.40 compat across all trainer files

fix: replace removed _pad_across_processes with accelerator API

fix: synced_gpus must be passed to generate(), not GenerationConfig

fix: LoRALayer.forward reshape incompatible with T5-small

fix: restore _set_gradient_checkpointing + enable_input_require_grads for gradient checkpointing

fix: use default use_reentrant=True for gradient checkpointing (model expects it)

fix: add typing imports to all trainer files

fix: add Union to typing imports in cl_collator

fix: add Optional/Any import to cl_collator, ipdb try/except in score.py

fix: add FP16 safety check, dataset script check, LoRA sanity check

fix: LoRA reinit, gradient checkpointing use_reentrant=False, checkpoint existence check, collator padding alignment

add root_gainlora to repo for testing

reduce rubish

fix: _maybe_log_save_evaluate add grad_norm param for transformers 4.40.2

fix: do_grad_scaling compat for transformers 4.40.2

fix: gradient checkpointing compat for transformers 4.40.2 (OOM fix)

fix: sanitize NaN/Inf values before SVD computation

fix: add compat shim for is_torch_tpu_available

fix: SVD CPU fallback for ill-conditioned matrices

fix: use gesvd driver for robust SVD + larger regularization

fix: robust SVD fallback with direct regularization instead of pinv

fix: replace self.use_apex and self.fsdp with getattr for compat

fix: compat shim for ShardedDDPOption and sharded_ddp attribute

fix: replace deprecated torch.pinv with torch.linalg.pinv

fix: add explicit imports for DistributedSampler, DataLoader, IterableDatasetShard, ipdb

fix: trust_remote_code and abs paths for load_dataset

update

clean code

revert: discard all 2-GPU DataParallel/DDP changes, back to f51f791

DDP 2 GPU

fix: robust SVD with try-except+pinv fallback + trans_input.pt exists guard

fix: SVD jitter + trans_input.pt exists guard in both root and improve

setup root

fix: port all compat fixes from improve to root trainer (ShardedDDPOption, ipdb, nested_truncate, gradient_checkpointing use_reentrant)

test root

new change