new v2

v2

v2

fix FT score

fix FT score

update OOM fix

Fix OOM: prev LoRA on CPU + no_grad entire prev contribution (no saved tensors for backward)

Fix OOM: agg_lora_states accumulate weighted sum incrementally instead of O(N) cat

Fix deprecation warnings: torch.load weights_only, cupy fromDlpack, as_target_tokenizer

fix: re-initialize trans_input/prompt_key after from_pretrained, add epsilon to norm, use_reentrant=False

fix: pass attention_mask directly to model.generate(), not via GenerationConfig

fix: override _save to disable safetensors for T5 shared embedding weights

fix: denumpify_detensorize moved to trainer_utils in transformers 4.40+

fix: add explicit trainer_pt_utils imports (nested_truncate etc.) removed from trainer.* in 4.40+

fix: comprehensive transformers 4.40 compat across all trainer files

fix: replace removed _pad_across_processes with accelerator API

fix: synced_gpus must be passed to generate(), not GenerationConfig

fix: LoRALayer.forward reshape incompatible with T5-small

fix: preserve task_config_dir in T5_small scripts

feat: add T5_small benchmark scripts for 4 comparison scenarios

fix: restore _set_gradient_checkpointing + enable_input_require_grads for gradient checkpointing

fix: use default use_reentrant=True for gradient checkpointing (model expects it)

fix: enable --gradient_checkpointing to fit T4 GPU memory

fix: revert eval_strategy back to evaluation_strategy (4.40.2 uses old name)

fix: rename --evaluation_strategy to --eval_strategy (transformers 4.40+)

fix: add typing imports to all trainer files

fix: add Union to typing imports in cl_collator

fix: add Optional/Any import to cl_collator, ipdb try/except in score.py

fix: add FP16 safety check, dataset script check, LoRA sanity check

fix: LoRA reinit, gradient checkpointing use_reentrant=False, checkpoint existence check, collator padding alignment

fix: restore notebook cells

add root_gainlora to repo for testing

reduce rubish

resize

fix bug root

fix: _maybe_log_save_evaluate add grad_norm param for transformers 4.40.2

fix: do_grad_scaling compat for transformers 4.40.2

fix: gradient checkpointing compat for transformers 4.40.2 (OOM fix)

fix: sanitize NaN/Inf values before SVD computation

fix: add compat shim for is_torch_tpu_available

fix: SVD CPU fallback for ill-conditioned matrices

fix: use gesvd driver for robust SVD + larger regularization

fix: robust SVD fallback with direct regularization instead of pinv

fix: replace self.use_apex and self.fsdp with getattr for compat

fix: compat shim for ShardedDDPOption and sharded_ddp attribute

fix: replace deprecated torch.pinv with torch.linalg.pinv

fix: add explicit imports for DistributedSampler, DataLoader, IterableDatasetShard, ipdb

fix: trust_remote_code and abs paths for load_dataset

update

clean code