Instructions to use yoorhim/26f-CSE576 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use yoorhim/26f-CSE576 with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
CSE576 β LoRA adapters for LLaVA-1.5-7B (NoCaps contamination study)
LoRA adapters (+ fully trained multi_modal_projector) on llava-hf/llava-1.5-7b-hf.
Each run: M = 130,000 examples, 1 epoch, LoRA r=128 / alpha=256 on all LM linear layers
(q,k,v,o,gate,up,down), lr 2e-4 (projector 2e-5), global batch 128, bf16, split A.
Full hyper-parameters, data hashes and environment are in each run's manifest.json.
Layout
ckpt/<run_id>/:adapter_config.json,adapter_model.safetensors,manifest.json(one folder per run)outputs/: evaluation results (T=0 captions, metrics, short-prompt metrics, data hashes); verify withsha256sum -c outputs/SHA256SUMS_sol.txt
hf download yoorhim/26f-CSE576 --include "ckpt/G0_d0_s0/*" --local-dir $SHARED # -> $SHARED/ckpt/G0_d0_s0
hf download yoorhim/26f-CSE576 --include "outputs/*" --local-dir $SHARED
Runs
Naming: {group}_{condition}_d{dose}[_n{N_exp}]_s{seed}. d = how many times each exposed NoCaps
image is repeated; N_exp = number of exposed images (1,000 unless noted).
| Group | Runs | Purpose |
|---|---|---|
| G0 | G0_d0_s0..s4 |
d=0 baseline (no NoCaps exposure), 5 seeds |
| G1 | {Caption,Paired,SameDist}_d{4,64}_s{0,1} |
dose sweep, high doses |
| G2 | {Caption,Paired,SameDist}_d1_s* |
d=1 |
| EX | Caption/Paired_d2_n1000_s{0,1}, {Paired,SameDist}_d1_n{30,100,300}_s0 |
d=2 and exposed-set-size (K) sweep |
Conditions: Paired = exposed NoCaps image + its reference caption; Caption = the reference caption only (text-only); SameDist = COCO images / captions of the same style (no NoCaps content, control). Code: https://github.com/ofzlo/data-contamination
Usage
from transformers import LlavaForConditionalGeneration, AutoProcessor
from peft import PeftModel
import torch
base = LlavaForConditionalGeneration.from_pretrained("llava-hf/llava-1.5-7b-hf", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "yoorhim/26f-CSE576", subfolder="ckpt/G1_Paired_d4_s0")
processor = AutoProcessor.from_pretrained("llava-hf/llava-1.5-7b-hf")
Trained with torch 2.5.1, transformers 4.46.3, peft 0.13.2.
- Downloads last month
- -
Model tree for yoorhim/26f-CSE576
Base model
llava-hf/llava-1.5-7b-hf