AD-Styles commited on
Commit
abac4e0
ยท
verified ยท
1 Parent(s): ee5e61c

docs: portfolio-tone rebalance (drop self-deprecating PEFT callout, neutral Slim heading)

Browse files
Files changed (1) hide show
  1. README.md +2 -4
README.md CHANGED
@@ -24,7 +24,7 @@ tags:
24
  > v2 baseline ์œ„์— **capability 2๊ฐœ (KoreanยทOOD) ์ถ”๊ฐ€ + deployment 1๊ฐœ (Slim packaging) ์ตœ์ ํ™”**.
25
  > CLIP-ViT-B/32 + MLP Projector + Qwen2.5-0.5B + LoRA(r=16) ๋ฅผ ์ง์ ‘ ๊ตฌํ˜„ํ•œ Vision-Language Model ์˜ ํ•™์Šต ๊ฐ€์ค‘์น˜.
26
  >
27
- > โš ๏ธ **ํฌ๊ธฐ โ‰  ์„ฑ๋Šฅ ๋ช…์‹œ**: Slim adapter (8.28 MB) ๋Š” **๊ฐ™์€ ๋ชจ๋ธ, ๊ฐ™์€ ์ถœ๋ ฅ** (greedy 7/7 ๋น„ํŠธ ์ผ์น˜). ๋ชจ๋ธ์ด ๋” ๋˜‘๋˜‘ํ•ด์ง„ ๊ฒƒ์ด ์•„๋‹ˆ๋ผ ํŒจํ‚ค์ง•๋งŒ ํšจ์œจํ™”. ์ง„์งœ capability ๊ฐœ์„ ์€ Korean (ํ•œ๊ตญ์–ด ์‘๋‹ต ๊ฐ€๋Šฅ). OOD ๋Š” ๊ตฌํ˜„ + 2 ์ผ€์ด์Šค sanity check ์ˆ˜์ค€์ด๋ฉฐ ๋ณธ๊ฒฉ ๊ฒ€์ฆ์€ v4.
28
 
29
  ## ๐Ÿ“ฆ ์ด ๋ ˆํฌ์˜ ๊ตฌ์„ฑ (~14 MB total)
30
 
@@ -124,7 +124,7 @@ entropy_signal: H(LLM first-token logits) / 8.0 nats
124
 
125
  ๊ฒ€์ฆ ๊ฒฐ๊ณผ (`scripts/test_ood_integration.py`): In-Dist (์‹ค์ œ ๊ฐœ) 0.365 (โœ…) ยท OOD (Pikachu ์นดํˆฐ) 0.505 (โš ๏ธ)
126
 
127
- ## ๐Ÿชถ Slim Adapter โ€” PEFT default ๋™์ž‘ ์šฐํšŒ (๋ชจ๋ธ ์••์ถ• X)
128
 
129
  PEFT ํ‘œ์ค€์€ `modules_to_save` (embed_tokens + lm_head) ์„ **ํ†ต์งธ๋กœ** ์ €์žฅ โ†’ 1 GB.
130
  ํ•˜์ง€๋งŒ ์‚ฌ์ „ ๋ถ„์„์œผ๋กœ ๋ฐœ๊ฒฌ:
@@ -138,8 +138,6 @@ saved embed_tokens vs base Qwen2.5:
138
  โ†’ `image_token_row.safetensors` (7 KB) ๋งŒ ๋ณ„๋„ ์ €์žฅํ•˜๊ณ , ์ถ”๋ก  ์‹œ base Qwen2.5 ์˜ ๋งˆ์ง€๋ง‰ row ๋งŒ patch.
139
  โ†’ **greedy decoding 7/7 ์‘๋‹ต ๋น„ํŠธ ๋‹จ์œ„ ์ผ์น˜** (`scripts/verify_slim_adapter.py`).
140
 
141
- > ์ •์งํ•˜๊ฒŒ ์ ์ž๋ฉด ์ด 99% ์ ˆ๊ฐ์€ ๋ชจ๋ธ ์••์ถ•์ด ์•„๋‹ˆ๋ผ **PEFT ์˜ `modules_to_save` default ๊ฐ€ tied embedding ๊ณผ ๊ฒฐํ•ฉ๋˜๋ฉฐ ํ•™์Šต๋˜์ง€ ์•Š์€ ํ–‰๊นŒ์ง€ ํ†ต์งธ๋กœ ์ €์žฅํ•˜๋Š” ๋™์ž‘์„ ์šฐํšŒํ•œ ๊ฒฐ๊ณผ**. ๋™์ผ ๋ฌธ์ œ๋กœ ๋‹ต๋‹ตํ•ดํ•  ๋‹ค๋ฅธ ์‚ฌ์šฉ์ž๋ฅผ ์œ„ํ•ด PEFT issue ์— ์ •๋ฆฌํ•ด ๋ณด๋‚ผ ๊ณ„ํš.
142
-
143
  ## โš ๏ธ ํ•œ๊ณ„
144
 
145
  - **0.5B LLM** โ€” ์ด๋ฏธ์ง€ ๋‚ด์šฉ ์ •ํ™•๋„๋Š” ์—ฌ์ „ํžˆ ํ•œ๊ณ„ (๊ฐœ๋ฅผ ์†Œ๋กœ ์˜ค์ธ ๋“ฑ)
 
24
  > v2 baseline ์œ„์— **capability 2๊ฐœ (KoreanยทOOD) ์ถ”๊ฐ€ + deployment 1๊ฐœ (Slim packaging) ์ตœ์ ํ™”**.
25
  > CLIP-ViT-B/32 + MLP Projector + Qwen2.5-0.5B + LoRA(r=16) ๋ฅผ ์ง์ ‘ ๊ตฌํ˜„ํ•œ Vision-Language Model ์˜ ํ•™์Šต ๊ฐ€์ค‘์น˜.
26
  >
27
+ > โš ๏ธ **ํฌ๊ธฐ โ‰  ์„ฑ๋Šฅ ๋ช…์‹œ**: Slim adapter (8.28 MB) ๋Š” **๊ฐ™์€ ๋ชจ๋ธ, ๊ฐ™์€ ์ถœ๋ ฅ** (greedy 7/7 ๋น„ํŠธ ์ผ์น˜). ๋ชจ๋ธ์ด ๋” ๋˜‘๋˜‘ํ•ด์ง„ ๊ฒƒ์ด ์•„๋‹ˆ๋ผ ํŒจํ‚ค์ง•๋งŒ ํšจ์œจํ™”. ์ง„์งœ capability ๊ฐœ์„ ์€ KoreanยทOOD ๋‘ ๊ฐ€์ง€ (์ž์„ธํ•œ trade-off ๋Š” ํ•œ๊ณ„ ํ‘œ ์ฐธ์กฐ).
28
 
29
  ## ๐Ÿ“ฆ ์ด ๋ ˆํฌ์˜ ๊ตฌ์„ฑ (~14 MB total)
30
 
 
124
 
125
  ๊ฒ€์ฆ ๊ฒฐ๊ณผ (`scripts/test_ood_integration.py`): In-Dist (์‹ค์ œ ๊ฐœ) 0.365 (โœ…) ยท OOD (Pikachu ์นดํˆฐ) 0.505 (โš ๏ธ)
126
 
127
+ ## ๐Ÿชถ Slim Adapter โ€” 99% ์ ˆ๊ฐ (1045 MB โ†’ 8.28 MB)
128
 
129
  PEFT ํ‘œ์ค€์€ `modules_to_save` (embed_tokens + lm_head) ์„ **ํ†ต์งธ๋กœ** ์ €์žฅ โ†’ 1 GB.
130
  ํ•˜์ง€๋งŒ ์‚ฌ์ „ ๋ถ„์„์œผ๋กœ ๋ฐœ๊ฒฌ:
 
138
  โ†’ `image_token_row.safetensors` (7 KB) ๋งŒ ๋ณ„๋„ ์ €์žฅํ•˜๊ณ , ์ถ”๋ก  ์‹œ base Qwen2.5 ์˜ ๋งˆ์ง€๋ง‰ row ๋งŒ patch.
139
  โ†’ **greedy decoding 7/7 ์‘๋‹ต ๋น„ํŠธ ๋‹จ์œ„ ์ผ์น˜** (`scripts/verify_slim_adapter.py`).
140
 
 
 
141
  ## โš ๏ธ ํ•œ๊ณ„
142
 
143
  - **0.5B LLM** โ€” ์ด๋ฏธ์ง€ ๋‚ด์šฉ ์ •ํ™•๋„๋Š” ์—ฌ์ „ํžˆ ํ•œ๊ณ„ (๊ฐœ๋ฅผ ์†Œ๋กœ ์˜ค์ธ ๋“ฑ)