n8n-workflow-evaluator / evaluation_criteria.py
suwankim
Deploy n8n workflow evaluator
1c1e7bc
"""
n8n ์›Œํฌํ”Œ๋กœ์šฐ ์‹ฌ์‚ฌ ๊ธฐ์ค€ ์ •์˜
readme.md์˜ ์ž์„ธํ•œ ๊ธฐ์ค€์„ ๋ฐ”ํƒ•์œผ๋กœ ์ž‘์„ฑ๋จ
ํ‰๊ฐ€ ํ•ญ๋ชฉ ๊ตฌ์„ฑ:
1. ๊ธฐ์ˆ ์  ์™„์„ฑ๋„ (15์ )
2. ์—…์Šคํ…Œ์ด์ง€ ์ œํ’ˆ ํ™œ์šฉ๋„ (15์ )
3. ์‹ค์šฉ์„ฑ (30์ )
4. ์ฐฝ์˜์„ฑ / ๋ฌธ์ œ ํ•ด๊ฒฐ ์ ‘๊ทผ๋ฒ• (20์ )
์ด์ : 80์ 
"""
# =============================================================================
# ๋‹ค์ค‘ ์›Œํฌํ”Œ๋กœ์šฐ ์ข…ํ•ฉ ํ‰๊ฐ€ ๊ธฐ์ค€ (์—ฌ๋Ÿฌ JSON ํŒŒ์ผ์ด ํฌํ•จ๋œ ZIP ํŒŒ์ผ์šฉ)
# =============================================================================
MULTI_WORKFLOW_TECHNICAL_CRITERIA = """
# ๋‹ค์ค‘ ์›Œํฌํ”Œ๋กœ์šฐ ๊ธฐ์ˆ ์  ์™„์„ฑ๋„ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (15์ )
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ํ‰๊ฐ€ ์›์น™**
- ์ ์ˆ˜๋Š” ๋งค์šฐ ์—„๊ฒฉํ•˜๊ฒŒ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค. ์™„๋ฒฝํ•˜์ง€ ์•Š์œผ๋ฉด ๊ฐ์ ํ•˜์‹ญ์‹œ์˜ค.
- ์—ฌ๋Ÿฌ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ํ•˜๋‚˜์˜ ํ”„๋กœ์ ํŠธ๋กœ ๊ตฌ์„ฑ๋œ ๊ฒฝ์šฐ๋ฅผ ํ‰๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
## ํ‰๊ฐ€ ํ•ต์‹ฌ ์งˆ๋ฌธ
1. ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ๊ฐœ๋ณ„์ ์œผ๋กœ ์™„์„ฑ๋˜์–ด ์žˆ๋Š”๊ฐ€?
2. ์—ฌ๋Ÿฌ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ์„œ๋กœ ์ž˜ ์—ฐ๊ฒฐ๋˜์–ด ์žˆ๋Š”๊ฐ€? (๋ฐ์ดํ„ฐ ํ๋ฆ„, ๊ธฐ๋Šฅ์  ์—ฐ๊ณ„์„ฑ)
3. ์ „์ฒด ์‹œ์Šคํ…œ์œผ๋กœ์„œ์˜ ์ผ๊ด€์„ฑ๊ณผ ๋ชจ๋“ˆํ™”๊ฐ€ ์ž˜ ๋˜์–ด ์žˆ๋Š”๊ฐ€?
## ํ‰๊ฐ€ ํ•ญ๋ชฉ:
### 1-1. ๊ฐœ๋ณ„ ์›Œํฌํ”Œ๋กœ์šฐ ์™„์„ฑ๋„ (6์ )
- ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ Triggerโ†’Output๊นŒ์ง€ ์™„์„ฑ๋œ ๊ฒฝ๋กœ๋ฅผ ๊ฐ–๋Š”๊ฐ€?
- Dead node๊ฐ€ ์—†๋Š”๊ฐ€?
- ์—ญํ•  ๊ธฐ๋ฐ˜ ๋„ค์ด๋ฐ์ด ๋˜์–ด ์žˆ๋Š”๊ฐ€?
### 1-2. ์›Œํฌํ”Œ๋กœ์šฐ ๊ฐ„ ์—ฐ๊ฒฐ์„ฑ ๋ฐ ํ†ตํ•ฉ์„ฑ (6์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ์—ฌ๋Ÿฌ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ์„œ๋กœ ๋ณด์™„์ ์ธ ๊ธฐ๋Šฅ์„ ํ•˜๋Š”๊ฐ€?
- ์›Œํฌํ”Œ๋กœ์šฐ ๊ฐ„ ๋ฐ์ดํ„ฐ ํ๋ฆ„์ด ๋…ผ๋ฆฌ์ ์œผ๋กœ ์—ฐ๊ฒฐ๋˜์–ด ์žˆ๋Š”๊ฐ€?
- ์ „์ฒด์ ์ธ ์‹œ์Šคํ…œ ๊ตฌ์กฐ๊ฐ€ ์ผ๊ด€์„ฑ ์žˆ๊ฒŒ ์„ค๊ณ„๋˜์—ˆ๋Š”๊ฐ€?
- ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ๋ชจ๋“ˆํ™”๋˜์–ด ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๊ตฌ์กฐ์ธ๊ฐ€?
**ํ‰๊ฐ€ ์‹œ ๊ณ ๋ ค์‚ฌํ•ญ:**
- ์›Œํฌํ”Œ๋กœ์šฐ ๊ฐ„ ๋ช…์‹œ์  ์—ฐ๊ฒฐ(webhook, HTTP ํ˜ธ์ถœ ๋“ฑ)์ด ์žˆ์œผ๋ฉด ๊ฐ€์ 
- ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ๋…๋ฆฝ์ ์œผ๋กœ๋„ ๋™์ž‘ ๊ฐ€๋Šฅํ•˜๋ฉด์„œ ํ†ตํ•ฉ๋„ ๊ฐ€๋Šฅํ•˜๋ฉด ๊ฐ€์ 
- ์ „์ฒด ์‹œ์Šคํ…œ์˜ ๋ฐ์ดํ„ฐ ์Šคํ‚ค๋งˆ ์ผ๊ด€์„ฑ์ด ์žˆ์œผ๋ฉด ๊ฐ€์ 
### 1-3. ๋ถ„๊ธฐ/์˜ˆ์™ธ ์ฒ˜๋ฆฌ (3์ )
- ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ ์˜๋ฏธ ์žˆ๋Š” ์กฐ๊ฑด ๋ถ„๊ธฐ๊ฐ€ ์žˆ๋Š”๊ฐ€?
- Fallback/default ๊ฒฝ๋กœ๊ฐ€ ์กด์žฌํ•˜๋Š”๊ฐ€?
- ์›Œํฌํ”Œ๋กœ์šฐ ๊ฐ„ ์˜ค๋ฅ˜ ์ „ํŒŒ ๋ฐ ์ฒ˜๋ฆฌ ๋ฉ”์ปค๋‹ˆ์ฆ˜์ด ์žˆ๋Š”๊ฐ€?
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{
"๊ฐœ๋ณ„_์›Œํฌํ”Œ๋กœ์šฐ_์™„์„ฑ๋„": {"์†Œ๊ณ„": <0-6>, "ํ‰๊ฐ€": "<ํ‰๊ฐ€>"},
"์›Œํฌํ”Œ๋กœ์šฐ_๊ฐ„_์—ฐ๊ฒฐ์„ฑ_๋ฐ_ํ†ตํ•ฉ์„ฑ": {"์†Œ๊ณ„": <0-6>, "ํ‰๊ฐ€": "<ํ‰๊ฐ€>"},
"๋ถ„๊ธฐ_์˜ˆ์™ธ_์ฒ˜๋ฆฌ": {"์†Œ๊ณ„": <0-3>, "ํ‰๊ฐ€": "<ํ‰๊ฐ€>"},
"์ด์ ": <0-15>,
"์ข…ํ•ฉ_ํ‰๊ฐ€": "<์ „์ฒด์ ์ธ ํ‰๊ฐ€ ์š”์•ฝ>"
}
"""
MULTI_WORKFLOW_UPSTAGE_CRITERIA = """
# ๋‹ค์ค‘ ์›Œํฌํ”Œ๋กœ์šฐ ์—…์Šคํ…Œ์ด์ง€ ์ œํ’ˆ ํ™œ์šฉ๋„ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (15์ )
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ํ‰๊ฐ€ ์›์น™**
- ์ ์ˆ˜๋Š” ๋งค์šฐ ์—„๊ฒฉํ•˜๊ฒŒ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- **์ค‘์š”**: ๋ชจ๋“  ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ ์—…์Šคํ…Œ์ด์ง€ ๊ธฐ์ˆ ์„ ํ™œ์šฉํ•  ํ•„์š”๋Š” ์—†์Šต๋‹ˆ๋‹ค.
- ํ”„๋กœ์ ํŠธ ์ „์ฒด์—์„œ ์—…์Šคํ…Œ์ด์ง€ ๊ธฐ์ˆ ์ด ์ ์ ˆํžˆ ํ™œ์šฉ๋˜์—ˆ๋Š”์ง€ ํ‰๊ฐ€ํ•˜์„ธ์š”.
## ํ‰๊ฐ€ ํ•ต์‹ฌ ์งˆ๋ฌธ
1. ํ”„๋กœ์ ํŠธ ์ „์ฒด์—์„œ ์—…์Šคํ…Œ์ด์ง€ API๊ฐ€ ์ ์ ˆํžˆ ์‚ฌ์šฉ๋˜์—ˆ๋Š”๊ฐ€?
2. ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ์˜ ๋ชฉ์ ์— ๋งž๋Š” API๊ฐ€ ์„ ํƒ๋˜์—ˆ๋Š”๊ฐ€?
3. ์—…์Šคํ…Œ์ด์ง€ ๊ธฐ์ˆ ์ด ํ•„์š”ํ•œ ๋ถ€๋ถ„์—๋งŒ ์‚ฌ์šฉ๋˜์—ˆ๋Š”๊ฐ€? (๊ณผ๋„ํ•œ ์‚ฌ์šฉ์€ ๊ฐ์ )
## ํ‰๊ฐ€ ํ•ญ๋ชฉ:
### 2-1. API ์‚ฌ์šฉ ์ ํ•ฉ์„ฑ (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ํ”„๋กœ์ ํŠธ ์ „์ฒด์—์„œ ์—…์Šคํ…Œ์ด์ง€ API๊ฐ€ ๊ณต์‹ ์—”๋“œํฌ์ธํŠธ๋กœ ์‚ฌ์šฉ๋˜์—ˆ๋Š”๊ฐ€?
- ๊ฐ ์›Œํฌํ”Œ๋กœ์šฐ์˜ ๋ชฉ์ ์— ๋งž๋Š” API๊ฐ€ ์„ ํƒ๋˜์—ˆ๋Š”๊ฐ€?
- Credential์ด ์•ˆ์ „ํ•˜๊ฒŒ ๊ด€๋ฆฌ๋˜์—ˆ๋Š”๊ฐ€?
**์ค‘์š”**:
- ๋ชจ๋“  ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ ์—…์Šคํ…Œ์ด์ง€ API๋ฅผ ์‚ฌ์šฉํ•  ํ•„์š”๋Š” ์—†์Šต๋‹ˆ๋‹ค.
- ์—…์Šคํ…Œ์ด์ง€ ๊ธฐ์ˆ ์ด ํ•„์š”ํ•œ ๋ถ€๋ถ„์—๋งŒ ์ ์ ˆํžˆ ์‚ฌ์šฉ๋˜์—ˆ๋Š”์ง€ ํ‰๊ฐ€ํ•˜์„ธ์š”.
- ์˜ˆ: ๋ฌธ์„œ ํŒŒ์‹ฑ์ด ํ•„์š”ํ•œ ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ๋งŒ Document Parse API ์‚ฌ์šฉ
### 2-2. Prompt / System ์„ค๊ณ„ (5์ )
- ์—…์Šคํ…Œ์ด์ง€ API๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ ๋ช…ํ™•ํ•œ ์—ญํ• (system message)์ด ์ •์˜๋˜์—ˆ๋Š”๊ฐ€?
- JSON/schema ๊ฐ•์ œ๋ ฅ์ด ์žˆ๋Š”๊ฐ€?
- ํ”„๋กฌํ”„ํŠธ๊ฐ€ ๊น”๋”ํ•˜๊ณ  ๊ตฌ์กฐํ™”๋˜์–ด ์žˆ๋Š”๊ฐ€?
### 2-3. ๊ธฐ๋Šฅ ์กฐํ•ฉยท์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜ (5์ )
- ์—ฌ๋Ÿฌ ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ ์—…์Šคํ…Œ์ด์ง€ API๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ, ๋‹จ๊ณ„์  ํŒŒ์ดํ”„๋ผ์ธ์œผ๋กœ ๊ตฌ์„ฑ๋˜์—ˆ๋Š”๊ฐ€?
- ์ „/ํ›„์ฒ˜๋ฆฌ๊ฐ€ ์ ์ ˆํ•œ๊ฐ€?
- ์ค‘๋ณต ํ˜ธ์ถœ์ด ์—†๋Š”๊ฐ€?
**์ค‘์š”**:
- ์—…์Šคํ…Œ์ด์ง€ API๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š๋Š” ์›Œํฌํ”Œ๋กœ์šฐ๊ฐ€ ์žˆ์–ด๋„ ๊ฐ์ ํ•˜์ง€ ๋งˆ์„ธ์š”.
- ํ”„๋กœ์ ํŠธ ์ „์ฒด์—์„œ ์—…์Šคํ…Œ์ด์ง€ ๊ธฐ์ˆ ์ด ํ•„์š”ํ•œ ๋ถ€๋ถ„์— ์ ์ ˆํžˆ ํ™œ์šฉ๋˜์—ˆ๋Š”์ง€ ํ‰๊ฐ€ํ•˜์„ธ์š”.
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{
"API_์‚ฌ์šฉ_์ ํ•ฉ์„ฑ": {"์†Œ๊ณ„": <0-5>, "ํ‰๊ฐ€": "<ํ‰๊ฐ€>"},
"Prompt_System_์„ค๊ณ„": {"์†Œ๊ณ„": <0-5>, "ํ‰๊ฐ€": "<ํ‰๊ฐ€>"},
"๊ธฐ๋Šฅ_์กฐํ•ฉ_์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜": {"์†Œ๊ณ„": <0-5>, "ํ‰๊ฐ€": "<ํ‰๊ฐ€>"},
"์ด์ ": <0-15>,
"์ข…ํ•ฉ_ํ‰๊ฐ€": "<์ „์ฒด์ ์ธ ํ‰๊ฐ€ ์š”์•ฝ>"
}
"""
# =============================================================================
# 1. ๊ธฐ์ˆ ์  ์™„์„ฑ๋„ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (15์ )
# =============================================================================
TECHNICAL_CRITERIA = """
# ๊ธฐ์ˆ ์  ์™„์„ฑ๋„ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (15์ )
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ํ‰๊ฐ€ ์›์น™**
- ์ ์ˆ˜๋Š” ๋งค์šฐ ์—„๊ฒฉํ•˜๊ฒŒ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค. ์™„๋ฒฝํ•˜์ง€ ์•Š์œผ๋ฉด ๊ฐ์ ํ•˜์‹ญ์‹œ์˜ค.
- "๊ฑฐ์˜ ์™„์„ฑ" ๋˜๋Š” "๋Œ€๋ถ€๋ถ„ ์ข‹์Œ"์€ ์ถฉ๋ถ„ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๊ธฐ์ค€์„ ์ •ํ™•ํžˆ ์ถฉ์กฑํ•ด์•ผ๋งŒ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๋ถˆํ™•์‹คํ•˜๊ฑฐ๋‚˜ ๋ช…ํ™•ํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๋Š” ๋‚ฎ์€ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๊ธฐ์ค€์„ ์™„์ „ํžˆ ์ถฉ์กฑํ•˜์ง€ ์•Š์œผ๋ฉด ํ•ด๋‹น ํ•ญ๋ชฉ์— ๋Œ€ํ•œ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์ง€ ๋งˆ์‹ญ์‹œ์˜ค.
์›Œํฌํ”Œ๋กœ์šฐ JSON ํŒŒ์ผ์„ ๋ถ„์„ํ•˜์—ฌ ๋‹ค์Œ ๊ธฐ์ค€์œผ๋กœ ํ‰๊ฐ€ํ•˜์‹ญ์‹œ์˜ค.
## LLM์ด ๋ณผ ์ˆ˜ ์žˆ๋Š” ๊ฒƒ (JSON ๊ตฌ์กฐ๋งŒ์œผ๋กœ ํŒ๋‹จ ๊ฐ€๋Šฅ)
- ๋…ธ๋“œ ๊ตฌ์กฐ & ์—ฐ๊ฒฐ ๊ด€๊ณ„ (`nodes`, `connections`)
- ์–ด๋–ค ํŠธ๋ฆฌ๊ฑฐ์—์„œ ์‹œ์ž‘ํ•˜๋Š”์ง€ (์˜ˆ: ChatTrigger, Webhook ๋“ฑ)
- ๋ธŒ๋žœ์น˜/๋ถ„๊ธฐ ์„ค๊ณ„ (IF, Switch, Merge ๋…ธ๋“œ ๋“ฑ)
- ์˜ˆ์™ธ ์ฒ˜๋ฆฌ/์—๋Ÿฌ ์ฒ˜๋ฆฌ ํŒจํ„ด์ด ์žˆ๋Š”์ง€
- HTTP ์š”์ฒญ ๋…ธ๋“œ์—์„œ:
- ์–ด๋–ค API๋ฅผ ํ˜ธ์ถœํ•˜๋Š”์ง€ (URL, `api.upstage.ai`, `model: solar-pro2` ๋“ฑ)
- ์š”์ฒญ/์‘๋‹ต์„ ์–ด๋–ป๊ฒŒ ํ›„์ฒ˜๋ฆฌํ•˜๋Š”์ง€ (Code ๋…ธ๋“œ, Set ๋…ธ๋“œ)
- Code ๋…ธ๋“œ์˜ JS ๋กœ์ง ์ˆ˜์ค€ (๋ฐ์ดํ„ฐ ๋จธ์ง€, ํ‚ค ๊ด€๋ฆฌ, ์˜ˆ์™ธ ์ฒ˜๋ฆฌ ๋“ฑ)
- "TODO", "DEACTIVATED", "์ˆ˜์ •ํ•  ๊ฒƒ" ๊ฐ™์€ ๋ฏธ์™„์„ฑ ์‹œ๊ทธ๋„
- intent ๋ถ„๋ฅ˜ โ†’ Switch โ†’ ๊ฐ intent๋ณ„ ๋ถ„๊ธฐ ๊ตฌ์กฐ๊ฐ€ "๋‹ซํ˜€ ์žˆ๋Š”์ง€"
- ์ตœ์ข… ์•ก์…˜๊นŒ์ง€ ๋„๋‹ฌํ•˜๋Š”์ง€ (์˜ˆ: ์ด๋ฉ”์ผ ๋ฐœ์†ก, DB ์ €์žฅ ๋“ฑ)
## ํ‰๊ฐ€ ํ•ญ๋ชฉ:
### 1-1. ๊ตฌ์กฐ ์™„๊ฒฐ์„ฑ & ๋ชจ๋“ˆํ™” (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- Triggerโ†’Output๊นŒ์ง€ ๊ฒฝ๋กœ ์™„์„ฑ, dead node ์—†์Œ
- ์—ญํ•  ๊ธฐ๋ฐ˜ ๋„ค์ด๋ฐยท๋ชจ๋“ˆํ™”
#### ์ „์ฒด ๊ฒฝ๋กœ ์™„์„ฑ๋„ & Dead ๋…ธ๋“œ ์—†์Œ (3์ )
- ์ตœ์†Œ 1๊ฐœ์˜ **trigger node** ์กด์žฌ (`type`: `webhook`, `scheduleTrigger`, `chatTrigger` ๋“ฑ)
- trigger โ†’ intermediate nodes โ†’ output nodes๋กœ ์ด์–ด์ง€๋Š” **๋‹จ์ ˆ ์—†๋Š” ๊ฒฝ๋กœ** ์กด์žฌ
- ์–ด๋–ค ๋…ธ๋“œ๋„ **incoming/outgoing connection ๋‘˜ ๋‹ค ์—†์Œ**์ด๋ฉด ๊ฐ์ 
- output ๋…ธ๋“œ๋Š” ์ด๋ฉ”์ผ, ํŒŒ์ผ ์—…๋กœ๋“œ, DB, webhook ์‘๋‹ต ๋“ฑ **์ตœ์ข… ๋ชฉ์  ๊ธฐ๋Šฅ์„ ๊ฐ–๋Š” ๋…ธ๋“œ**
#### ์—ญํ•  ๊ธฐ๋ฐ˜ ๋„ค์ด๋ฐ & ๋ชจ๋“ˆํ™” (2์ )
- ๋…ธ๋“œ๋ช…์ด "HTTP Request", "Code" ๊ฐ™์€ ๊ธฐ๋ณธ๋ช…์ด ์•„๋‹Œ **๊ธฐ๋Šฅ ์ค‘์‹ฌ ๋ช…๋ช…** (์˜ˆ: "Upstage DP ํ˜ธ์ถœ", "์š”์•ฝ ํ›„์ฒ˜๋ฆฌ")
- ๋ฐ˜๋ณต ๋กœ์ง์€ loop / batch ๋…ธ๋“œ๋กœ ๋ชจ๋“ˆํ™”
- ๊ณตํ†ต ์ „์ฒ˜๋ฆฌ๋Š” ๋ณ„๋„ ๋…ธ๋“œํ™” (๋ถ„์‚ฐ X)
### 1-2. ๋ฐ์ดํ„ฐยท์Šคํ‚ค๋งˆ ์ผ๊ด€์„ฑ (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- key ์ผ๊ด€์„ฑ(`text/data/json`)
- ๋ถˆํ•„์š”ํ•œ overwrite ์—†์Œ
- merge ์ •ํ•ฉ์„ฑ, output ํ•„๋“œ ๋ณด์กด
#### key ์ผ๊ด€์„ฑ (2์ )
- HTTP ๋…ธ๋“œ ์‘๋‹ต ํ•„๋“œ: `response.data` ๋˜๋Š” `body.choices[0].message.content`
- Code/Set ๋…ธ๋“œ์—์„œ output key ์ผ๊ด€์„ฑ: ๋‹ค์Œ ๋…ธ๋“œ์—์„œ ์ฐธ์กฐํ•˜๋Š” key์™€ ์ผ์น˜
- `{{ $json.XXX }}` ํ‘œํ˜„์‹์—์„œ key ์˜คํƒ€/๋ถˆ์ผ์น˜ ์—†์Œ
#### ๋ถˆํ•„์š”ํ•œ overwrite ์—†์Œ & output ํ•„๋“œ ๋ณด์กด (2์ )
- Set ๋…ธ๋“œ๋กœ "๋™์ผํ•œ key๋ฅผ ์—ฌ๋Ÿฌ ๋ฒˆ" overwriteํ•˜์ง€ ์•Š์Œ
- "์ด์ „ ๋‹จ๊ณ„ ๊ฒฐ๊ณผ๋ฅผ ๋ณด์กด"ํ•˜๋ฉด์„œ ์ƒˆ ํ•„๋“œ๋ฅผ ์ถ”๊ฐ€ํ•˜๋Š” ํŒจํ„ด
- ์ตœ์ข… ๋…ธ๋“œ๊นŒ์ง€ "ํ•„์š”ํ•œ ํ•„๋“œ"๊ฐ€ ์œ ์ง€๋จ
#### Merge ๋…ธ๋“œ ์ •ํ•ฉ์„ฑ (1์ )
- Merge ๋…ธ๋“œ์—์„œ key collision ์ฒ˜๋ฆฌ ๋ช…ํ™• (`mode: "merge"`, `"append"`, `"chooseBranch"` ๋“ฑ)
- ๋ณ‘ํ•ฉ ํ›„ schema๊ฐ€ ๋‹ค์Œ ๋…ธ๋“œ ์ž…๋ ฅ๊ณผ ํ˜ธํ™˜
### 1-3. ๋ถ„๊ธฐ/์˜ˆ์™ธ ์ฒ˜๋ฆฌ (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ์˜๋ฏธ ์žˆ๋Š” ์กฐ๊ฑด ๋ถ„๊ธฐ
- fallback/default ๊ฒฝ๋กœ ์กด์žฌ
#### ์˜๋ฏธ ์žˆ๋Š” ์กฐ๊ฑด ๋ถ„๊ธฐ (3์ )
- IF / Switch ๋…ธ๋“œ์—์„œ ์กฐ๊ฑด ๋กœ์ง ๋ช…ํ™• (์˜ˆ: `intent === "์งˆ๋ฌธ"` / `response.status === 200`)
- ๋ถ„๊ธฐ ํ›„ ๊ฐ ๊ฒฝ๋กœ๊ฐ€ **์˜๋ฏธ ์žˆ๋Š” ์ฐจ์ด**๋ฅผ ๊ฐ–๋Š” ์ฒ˜๋ฆฌ๋กœ ์ด์–ด์ง
#### Fallback / Default ๊ฒฝ๋กœ ์กด์žฌ (2์ )
- Switch ๋…ธ๋“œ์— `default` ์ผ€์ด์Šค ์กด์žฌ
- IF ๋…ธ๋“œ์˜ false ๊ฒฝ๋กœ์—๋„ ์ตœ์†Œํ•œ์˜ ์ฒ˜๋ฆฌ ๋˜๋Š” ๋ช…์‹œ์  "๋ฌด์‘๋‹ต" ๋…ธ๋“œ
- ์˜ˆ์™ธ ๋ฐœ์ƒ ์‹œ **์ตœ์†Œํ•œ์˜ ์‚ฌ์šฉ์ž ์‘๋‹ต ๋ฐ˜ํ™˜**
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{
"๊ตฌ์กฐ_์™„๊ฒฐ์„ฑ_๋ฐ_๋ชจ๋“ˆํ™”": {
"์ „์ฒด_๊ฒฝ๋กœ_์™„์„ฑ๋„_๋ฐ_dead๋…ธ๋“œ_์—†์Œ": <0-3>,
"์—ญํ• ๊ธฐ๋ฐ˜_๋„ค์ด๋ฐ_๋ชจ๋“ˆํ™”": <0-2>,
"์†Œ๊ณ„": <0-5>,
"ํ‰๊ฐ€": "<๊ตฌ์กฐ ์™„๊ฒฐ์„ฑ ๋ฐ ๋ชจ๋“ˆํ™” ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"๋ฐ์ดํ„ฐ_์Šคํ‚ค๋งˆ_์ผ๊ด€์„ฑ": {
"key_์ผ๊ด€์„ฑ": <0-2>,
"๋ถˆํ•„์š”ํ•œ_overwrite_์—†์Œ_๋ฐ_outputํ•„๋“œ_๋ณด์กด": <0-2>,
"merge_๋…ธ๋“œ_์ •ํ•ฉ์„ฑ": <0-1>,
"์†Œ๊ณ„": <0-5>,
"ํ‰๊ฐ€": "<๋ฐ์ดํ„ฐ ์Šคํ‚ค๋งˆ ์ผ๊ด€์„ฑ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"๋ถ„๊ธฐ_์˜ˆ์™ธ_์ฒ˜๋ฆฌ": {
"์˜๋ฏธ์žˆ๋Š”_์กฐ๊ฑด_๋ถ„๊ธฐ": <0-3>,
"fallback_default_๊ฒฝ๋กœ": <0-2>,
"์†Œ๊ณ„": <0-5>,
"ํ‰๊ฐ€": "<๋ถ„๊ธฐ/์˜ˆ์™ธ ์ฒ˜๋ฆฌ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์ด์ ": <0-15>,
"์ข…ํ•ฉ_ํ‰๊ฐ€": "<์ „์ฒด์ ์ธ ํ‰๊ฐ€ ์š”์•ฝ>"
}
"""
# =============================================================================
# 2. ์—…์Šคํ…Œ์ด์ง€ ์ œํ’ˆ ํ™œ์šฉ๋„ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (15์ )
# =============================================================================
UPSTAGE_CRITERIA = """
# ์—…์Šคํ…Œ์ด์ง€ ์ œํ’ˆ ํ™œ์šฉ๋„ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (15์ )
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ํ‰๊ฐ€ ์›์น™**
- ์ ์ˆ˜๋Š” ๋งค์šฐ ์—„๊ฒฉํ•˜๊ฒŒ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค. ์™„๋ฒฝํ•˜์ง€ ์•Š์œผ๋ฉด ๊ฐ์ ํ•˜์‹ญ์‹œ์˜ค.
- "๊ฑฐ์˜ ์™„์„ฑ" ๋˜๋Š” "๋Œ€๋ถ€๋ถ„ ์ข‹์Œ"์€ ์ถฉ๋ถ„ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๊ธฐ์ค€์„ ์ •ํ™•ํžˆ ์ถฉ์กฑํ•ด์•ผ๋งŒ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๋ถˆํ™•์‹คํ•˜๊ฑฐ๋‚˜ ๋ช…ํ™•ํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๋Š” ๋‚ฎ์€ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๊ธฐ์ค€์„ ์™„์ „ํžˆ ์ถฉ์กฑํ•˜์ง€ ์•Š์œผ๋ฉด ํ•ด๋‹น ํ•ญ๋ชฉ์— ๋Œ€ํ•œ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์ง€ ๋งˆ์‹ญ์‹œ์˜ค.
์›Œํฌํ”Œ๋กœ์šฐ JSON ํŒŒ์ผ์„ ๋ถ„์„ํ•˜์—ฌ ์—…์Šคํ…Œ์ด์ง€ ์ œํ’ˆ ํ™œ์šฉ๋„๋ฅผ ํ‰๊ฐ€ํ•˜์‹ญ์‹œ์˜ค.
## ํ‰๊ฐ€ ํ•ญ๋ชฉ:
### 2-1. API ์‚ฌ์šฉ ์ ํ•ฉ์„ฑ (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ๊ณต์‹ ์—”๋“œํฌ์ธํŠธ ์‚ฌ์šฉ
- credential ๋ณด์•ˆ
- ๋ชฉ์  ์ ํ•ฉ API ์„ ํƒ
#### ๊ณต์‹ ์—”๋“œํฌ์ธํŠธ ์‚ฌ์šฉ (2์ )
- `https://api.upstage.ai/v1/solar/...` ๋˜๋Š” ๊ณต์‹ ๋ฌธ์„œ ๋ช…์‹œ ๊ฒฝ๋กœ
- `/chat/completions`, `/document-ai/...` ๋“ฑ **๊ณต์‹ API ์ŠคํŽ™์— ๋งž๋Š” ๊ฒฝ๋กœ**
- custom endpoint / deprecated path ์‚ฌ์šฉํ•˜๋ฉด ๊ฐ์ 
#### Credential ๋ณด์•ˆ (1์ )
- API Key๋ฅผ ์ฝ”๋“œ์— ํ•˜๋“œ์ฝ”๋”ฉํ•˜์ง€ ์•Š์Œ (`{{ $credentials.upstageApi.apiKey }}` ๋“ฑ credential ์ฐธ์กฐ)
- Authorization ํ—ค๋” ํ˜•์‹ ์ •ํ™•: `Bearer {{API_KEY}}`
#### ๋ชฉ์  ์ ํ•ฉ API ์„ ํƒ (2์ )
- **๋ฌธ์„œ ํŒŒ์‹ฑ** โ†’ Document Parse
- **์ž„๋ฒ ๋”ฉ/๊ฒ€์ƒ‰** โ†’ Embeddings API
- **๋Œ€ํ™”/์ƒ์„ฑ** โ†’ Solar Chat
- **์ด๋ฏธ์ง€ ์ถ”์ถœ** โ†’ Layout Analysis
- ๋ชฉ์ -API ๋งค์นญ์ด ์ ์ ˆํ•œ์ง€ ํ‰๊ฐ€
- ์˜ˆ: "OCR์ด ํ•„์š”ํ•œ๋ฐ Chat API๋งŒ ์“ด" ๊ฒฝ์šฐ ๊ฐ์ 
### 2-2. Prompt / System ์„ค๊ณ„ (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ๋ช…ํ™•ํ•œ ์—ญํ• (system)
- JSON/schema ๊ฐ•์ œ๋ ฅ
- ๊น”๋”ํ•œ ํ”„๋กฌํ”„ํŠธ
#### ๋ช…ํ™•ํ•œ ์—ญํ•  (system message) (2์ )
- `messages[0].role === "system"`์— ์—ญํ• /์ œ์•ฝ/์ถœ๋ ฅ ํ˜•์‹ ์ •์˜
- ์˜ˆ: "๋„ˆ๋Š” ํšŒ์˜๋ก ์ž‘์„ฑ ์ „๋ฌธ๊ฐ€์ž…๋‹ˆ๋‹ค. ์•„๋ž˜ ํ˜•์‹์œผ๋กœ๋งŒ ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค."
- system message ์—†์œผ๋ฉด ๊ฐ์ 
#### JSON / schema ๊ฐ•์ œ๋ ฅ (2์ )
- `response_format: { type: "json_object" }` ๋˜๋Š” prompt์— "JSON๋งŒ ์ถœ๋ ฅ" ๋ช…์‹œ
- Structured Output ์‚ฌ์šฉ ์‹œ `schema` ํ•„๋“œ ์ •์˜ ์—ฌ๋ถ€
- ์˜ˆ: ์˜๋„ ๋ถ„๋ฅ˜, ํ•„๋“œ ์ถ”์ถœ ์‹œ "๋ฐ˜๋“œ์‹œ {intent: ..., entities: ...} ํ˜•์‹" ์ง€์‹œ
#### ๊น”๋”ํ•œ ํ”„๋กฌํ”„ํŠธ (๋ถˆํ•„์š” ๋ฐ˜๋ณต X) (1์ )
- ๋™์ผ ์ง€์‹œ ์ค‘๋ณต ์—†์Œ (์˜ˆ: "JSON์œผ๋กœ ์ถœ๋ ฅํ•˜์„ธ์š”"๋ฅผ system/user ์–‘์ชฝ์— ๋ฐ˜๋ณต X)
- ํ”„๋กฌํ”„ํŠธ๊ฐ€ "๋ช…ํ™•ยท๊ฐ„๊ฒฐยท๊ตฌ์กฐํ™”"๋˜์–ด ์žˆ์Œ
- Few-shot ์˜ˆ์‹œ๊ฐ€ ์žˆ๋‹ค๋ฉด ์ผ๊ด€๋œ ํ˜•์‹
### 2-3. ๊ธฐ๋Šฅ ์กฐํ•ฉยท์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜ (5์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- DPโ†’IEโ†’Solar ๋“ฑ ๋‹จ๊ณ„์  ํŒŒ์ดํ”„๋ผ์ธ
- ์ „/ํ›„์ฒ˜๋ฆฌ ์ ์ ˆ
- ์ค‘๋ณต ํ˜ธ์ถœ ์—†์Œ
#### ๋‹จ๊ณ„์  ํŒŒ์ดํ”„๋ผ์ธ (2์ )
- ๋ฌธ์„œ ์—…๋กœ๋“œ โ†’ DP๋กœ ํŒŒ์‹ฑ โ†’ ํ…์ŠคํŠธ ์ถ”์ถœ โ†’ Solar Chat์œผ๋กœ ์š”์•ฝ/๋ถ„์„ ๊ฐ™์€ **๋…ผ๋ฆฌ์  ํ๋ฆ„**
- ๊ฐ ๋‹จ๊ณ„ output์ด ๋‹ค์Œ ๋‹จ๊ณ„ input์œผ๋กœ ์ž์—ฐ์Šค๋Ÿฝ๊ฒŒ ์ „๋‹ฌ
- ์˜ˆ: DP ๊ฒฐ๊ณผ `text`๋ฅผ Chat API `user` ๋ฉ”์‹œ์ง€์— ์‚ฝ์ž…
#### ์ „/ํ›„์ฒ˜๋ฆฌ ์ ์ ˆ (2์ )
- API ํ˜ธ์ถœ ์ „: ํ…์ŠคํŠธ ์ •์ œ, ํ•„๋“œ ๋งคํ•‘, validation (Code/Set ๋…ธ๋“œ)
- API ํ˜ธ์ถœ ํ›„: JSON ํŒŒ์‹ฑ, ํ•„์š” ํ•„๋“œ ์ถ”์ถœ, ์—๋Ÿฌ ํ•ธ๋“ค๋ง
- ์˜ˆ: Chat ์‘๋‹ต์—์„œ `choices[0].message.content`๋ฅผ ์ถ”์ถœ โ†’ JSON ํŒŒ์‹ฑ โ†’ ๋‹ค์Œ ๋‹จ๊ณ„ ์ „๋‹ฌ
#### ์ค‘๋ณต ํ˜ธ์ถœ ์—†์Œ (1์ )
- ๋™์ผ ์ž…๋ ฅ์— ๋Œ€ํ•ด ๊ฐ™์€ API๋ฅผ ์—ฌ๋Ÿฌ ๋ฒˆ ํ˜ธ์ถœํ•˜์ง€ ์•Š์Œ
- ๋ถˆํ•„์š”ํ•œ "loop ๋‚ด ๋ฐ˜๋ณต API ํ˜ธ์ถœ" X
- batch ์ฒ˜๋ฆฌ ๋˜๋Š” 1ํšŒ concatenate ํ›„ ํ˜ธ์ถœ ๊ฐ™์€ ์ตœ์ ํ™” ์‹œ๋„ ์‹œ ๊ฐ€์ 
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{
"API_์‚ฌ์šฉ_์ ํ•ฉ์„ฑ": {
"๊ณต์‹_์—”๋“œํฌ์ธํŠธ_์‚ฌ์šฉ": <0-2>,
"credential_๋ณด์•ˆ": <0-1>,
"๋ชฉ์ _์ ํ•ฉ_API_์„ ํƒ": <0-2>,
"์‚ฌ์šฉ๋œ_API_๋ชฉ๋ก": ["<API1>", "<API2>"],
"์†Œ๊ณ„": <0-5>,
"ํ‰๊ฐ€": "<API ์‚ฌ์šฉ ์ ํ•ฉ์„ฑ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"Prompt_System_์„ค๊ณ„": {
"๋ช…ํ™•ํ•œ_์—ญํ• _system_message": <0-2>,
"JSON_schema_๊ฐ•์ œ๋ ฅ": <0-2>,
"๊น”๋”ํ•œ_ํ”„๋กฌํ”„ํŠธ": <0-1>,
"์†Œ๊ณ„": <0-5>,
"ํ‰๊ฐ€": "<Prompt/System ์„ค๊ณ„ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"๊ธฐ๋Šฅ_์กฐํ•ฉ_์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜": {
"๋‹จ๊ณ„์ _ํŒŒ์ดํ”„๋ผ์ธ": <0-2>,
"์ „ํ›„์ฒ˜๋ฆฌ_์ ์ ˆ": <0-2>,
"์ค‘๋ณต_ํ˜ธ์ถœ_์—†์Œ": <0-1>,
"์†Œ๊ณ„": <0-5>,
"ํ‰๊ฐ€": "<๊ธฐ๋Šฅ ์กฐํ•ฉ/์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์ด์ ": <0-15>,
"์ข…ํ•ฉ_ํ‰๊ฐ€": "<์ „์ฒด์ ์ธ ํ‰๊ฐ€ ์š”์•ฝ>"
}
"""
# =============================================================================
# 3. ์‹ค์šฉ์„ฑ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (30์ )
# =============================================================================
PRACTICALITY_CRITERIA = """
# ์‹ค์šฉ์„ฑ ์‹ฌ์‚ฌ ๊ธฐ์ค€ (30์ )
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ํ‰๊ฐ€ ์›์น™**
- ์ ์ˆ˜๋Š” ๋งค์šฐ ์—„๊ฒฉํ•˜๊ฒŒ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค. ์™„๋ฒฝํ•˜์ง€ ์•Š์œผ๋ฉด ๊ฐ์ ํ•˜์‹ญ์‹œ์˜ค.
- "๊ฑฐ์˜ ์™„์„ฑ" ๋˜๋Š” "๋Œ€๋ถ€๋ถ„ ์ข‹์Œ"์€ ์ถฉ๋ถ„ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๊ธฐ์ค€์„ ์ •ํ™•ํžˆ ์ถฉ์กฑํ•ด์•ผ๋งŒ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๋ถˆํ™•์‹คํ•˜๊ฑฐ๋‚˜ ๋ช…ํ™•ํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๋Š” ๋‚ฎ์€ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๊ธฐ์ค€์„ ์™„์ „ํžˆ ์ถฉ์กฑํ•˜์ง€ ์•Š์œผ๋ฉด ํ•ด๋‹น ํ•ญ๋ชฉ์— ๋Œ€ํ•œ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์ง€ ๋งˆ์‹ญ์‹œ์˜ค.
์ œ์ถœ๋œ ํ”„๋กœ์ ํŠธ ์„ค๋ช…์„ ๋ถ„์„ํ•˜์—ฌ ๋‹ค์Œ ๊ธฐ์ค€์œผ๋กœ ํ‰๊ฐ€ํ•˜์‹ญ์‹œ์˜ค.
## ํ‰๊ฐ€ ํ•ญ๋ชฉ:
### 3-1. ์‹ค์ œ ์—…๋ฌด ์ ์šฉ ๊ฐ€๋Šฅ์„ฑ (10์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ๋ฐ˜๋ณต ์—…๋ฌด ์ œ๊ฑฐ
- ์‹ค์ œ ์‚ฌ์šฉ ์‹œ๋‚˜๋ฆฌ์˜ค ์„ ๋ช…
- ์‹œ๊ฐ„ ์ ˆ๊ฐ ํšจ๊ณผ ๋ช…ํ™•
**ํ•ต์‹ฌ ์งˆ๋ฌธ:**
- ์ด ์›Œํฌํ”Œ๋กœ์šฐ๋Š” ์‹ค์ œ๋กœ **์‚ฌ๋žŒ์ด ํ•˜๋˜ ์ˆ˜์ž‘์—…์„ ์–ผ๋งˆ๋‚˜ ์ค„์ด๋Š”๊ฐ€?**
- "์–ธ์ œ/๋ˆ„๊ฐ€/๋ฌด์—‡์„" ์ค„์ผ ์ˆ˜ ์žˆ๋Š”์ง€ ๋ช…ํ™•ํ•œ๊ฐ€?
- ์ž๋™ํ™”์˜ **์‹œ๊ฐ„ ์ ˆ๊ฐยท์˜ค๋ฅ˜ ๊ฐ์†Œ** ํšจ๊ณผ๊ฐ€ ๋ถ„๋ช…ํ•œ๊ฐ€?
#### ๋ฐ˜๋ณต ์—…๋ฌด ์ œ๊ฑฐ (3์ )
- "๋งค์ผ/๋งค์ฃผ ๋ฐ˜๋ณต๋˜๋Š” ์ž‘์—…"์„ ์ž๋™ํ™”ํ•˜๋Š”์ง€ ๋ช…ํ™•
- ์˜ˆ: ์ผ์ผ ๋ฆฌํฌํŠธ ์ƒ์„ฑ, ๋ฌธ์˜ ์ž๋™ ๋ถ„๋ฅ˜, ์ •๊ธฐ ๋ฐ์ดํ„ฐ ์ง‘๊ณ„ ๋“ฑ
- **๋‹จ๋ฐœ์„ฑ ์ž‘์—…์ด ์•„๋‹Œ, ์ง€์† ์‚ฌ์šฉ ๊ฐ€๋Šฅ**
#### ์‹ค์ œ ์‚ฌ์šฉ ์‹œ๋‚˜๋ฆฌ์˜ค ์„ ๋ช… (4์ )
- "๋ˆ„๊ฐ€ / ์–ธ์ œ / ์–ด๋–ป๊ฒŒ ์‚ฌ์šฉํ•˜๋Š”์ง€" ๊ตฌ์ฒด์  ์„œ์ˆ 
- ์˜ˆ: "๊ณ ๊ฐ์„ผํ„ฐ ์ง์›์ด ๋ฌธ์˜ ์ ‘์ˆ˜ ์‹œ ์ฆ‰์‹œ ์‹คํ–‰", "๋งˆ์ผ€ํŒ… ํŒ€์ด ์ฃผ๊ฐ„ ๋ณด๊ณ ์„œ ์ž‘์„ฑ ์‹œ ํ™œ์šฉ"
- ์‚ฌ์šฉ flow๊ฐ€ ๋ช…ํ™• (trigger ์กฐ๊ฑด, input ์ถœ์ฒ˜, output ํ™œ์šฉ์ฒ˜)
#### ์‹œ๊ฐ„ ์ ˆ๊ฐ ํšจ๊ณผ ๋ช…ํ™• (3์ )
- "Before (์ˆ˜๋™ ์ž‘์—… N์‹œ๊ฐ„) โ†’ After (์ž๋™ํ™” M๋ถ„)" ๊ฐ™์€ **์ •๋Ÿ‰์  ํšจ๊ณผ** ์ œ์‹œ
- ๋˜๋Š” "์˜ค๋ฅ˜์œจ ๊ฐ์†Œ / ์ผ๊ด€์„ฑ ํ–ฅ์ƒ" ๋“ฑ **์งˆ์  ๊ฐœ์„ ** ๋ช…์‹œ
- ํšจ๊ณผ๊ฐ€ ์ฒด๊ฐ ๊ฐ€๋Šฅํ•œ ์ˆ˜์ค€
### 3-2. ์ž๋™ํ™” ํšจ๊ณผ ๋ฐ ์‚ฌ์šฉ์ž ํŽธ์˜์„ฑ (10์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ์ž๋™ํ™” ํšจ๊ณผ ๋ฐ ํ•™์Šต ํŽธ์˜์„ฑ
- ์žฌ์‚ฌ์šฉ ๋˜๋Š” ์ผ๋ฐ˜ํ™” ๊ฐ€๋Šฅํ•œ ๊ตฌ์กฐ
**ํ•ต์‹ฌ ์งˆ๋ฌธ:**
- "์ด ์ž๋™ํ™” ๊ตฌ์กฐ๊ฐ€ ๋‹ค๋ฅธ ์‚ฌ๋žŒยท๋‹ค๋ฅธ ์กฐ์งยท๋‹ค๋ฅธ ๋ฐ์ดํ„ฐ์—์„œ๋„ ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ๊ฐ€?"
- "ํŠน์ • ์ƒํ™ฉ์—๋งŒ ๋™์ž‘ํ•˜๋Š” ํ•ดํ‚น ๊ฐ™์€ ๊ตฌ์กฐ๊ฐ€ ์•„๋‹Œ๊ฐ€?"
- "์ž…๋ ฅ ๊ตฌ์กฐยท์ถ”์ƒํ™”ยท๊ตฌ์„ฑ ์š”์†Œ๊ฐ€ ํ™•์žฅ์„ฑ์„ ๊ณ ๋ คํ•˜๋Š”๊ฐ€?"
#### ์žฌ์‚ฌ์šฉยท์ผ๋ฐ˜ํ™” ๊ฐ€๋Šฅ์„ฑ (5์ )
- ํŠน์ • ํšŒ์‚ฌ/ํŒ€์—๋งŒ ์ข…์†๋œ ํ•˜๋“œ์ฝ”๋”ฉ ์ตœ์†Œํ™”
- ์˜ˆ: "์šฐ๋ฆฌ ํšŒ์‚ฌ ํŠน์ • DB ์Šคํ‚ค๋งˆ์—๋งŒ ๋™์ž‘" โ†’ ๋‚ฎ์€ ์ ์ˆ˜
- ์ž…๋ ฅ ํฌ๋งท๋งŒ ๋งž์ถ”๋ฉด ๋‹ค๋ฅธ ๊ณณ์—์„œ๋„ ๋ฐ”๋กœ ์‚ฌ์šฉ ๊ฐ€๋Šฅ โ†’ ๋†’์€ ์ ์ˆ˜
- workflow๊ฐ€ **์ผ๋ฐ˜ํ™”๋œ ๊ตฌ์กฐ** (์˜ˆ: ํŒŒ์ผ ์—…๋กœ๋“œ โ†’ ํŒŒ์‹ฑ โ†’ ๋ถ„์„ โ†’ ์ถœ๋ ฅ)
#### ์ž๋™ํ™” ํšจ๊ณผ ๋ฐ ํ•™์Šต ํŽธ์˜์„ฑ (5์ )
- ์ž๋™ํ™”๊ฐ€ ์‹ค์ œ๋กœ ์‹œ๊ฐ„์„ ์ ˆ์•ฝํ•˜๊ฑฐ๋‚˜ ํšจ์œจ์„ ๋†’์ด๋Š”๊ฐ€?
- ์‚ฌ์šฉ์ž๊ฐ€ ์‰ฝ๊ฒŒ ์ดํ•ดํ•˜๊ณ  ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๊ตฌ์กฐ์ธ๊ฐ€?
- ํ•™์Šต ํ๋ฆ„์„ ๋ฐฉํ•ดํ•˜์ง€ ์•Š๊ณ  ์ž์—ฐ์Šค๋Ÿฝ๊ฒŒ ๋ณด์กฐํ•˜๋Š” ๊ธฐ๋Šฅ์ธ๊ฐ€?
### 3-3. ์‚ฌ์šฉ์ž ํŽธ์˜์„ฑ (10์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ์ตœ์ข… ์‚ฌ์šฉ์ž ์ž…์žฅ์—์„œ ์ง๊ด€์ ์œผ๋กœ ์“ธ ์ˆ˜ ์žˆ๋Š”๊ฐ€?
**ํ•ต์‹ฌ ์งˆ๋ฌธ:**
- "๊ธฐ์ˆ  ์ง€์‹์ด ์—†๋Š” ์‚ฌ์šฉ์ž๋„ ์‰ฝ๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š”๊ฐ€?"
- "์ž…๋ ฅ์ด ๊ฐ„๋‹จํ•˜๊ณ  ์ถœ๋ ฅ์ด ๋ช…ํ™•ํ•œ๊ฐ€?"
- "์—๋Ÿฌ ๋ฐœ์ƒ ์‹œ ์‚ฌ์šฉ์ž๊ฐ€ ์ดํ•ดํ•  ์ˆ˜ ์žˆ๋Š” ๋ฉ”์‹œ์ง€๊ฐ€ ์ œ๊ณต๋˜๋Š”๊ฐ€?"
#### ์ง๊ด€์ ์ธ ์ž…๋ ฅ ๋ฐฉ์‹ (4์ )
- ์‚ฌ์šฉ์ž๊ฐ€ ์ œ๊ณตํ•ด์•ผ ํ•˜๋Š” ์ž…๋ ฅ์ด ๋ช…ํ™•ํ•˜๊ณ  ๊ฐ„๋‹จ
- ์˜ˆ: "ํŒŒ์ผ ์—…๋กœ๋“œ๋งŒ ํ•˜๋ฉด ๋จ", "๊ฐ„๋‹จํ•œ ์งˆ๋ฌธ ์ž…๋ ฅ"
- ๋ณต์žกํ•œ ์„ค์ •์ด๋‚˜ ๊ธฐ์ˆ  ์ง€์‹ ๋ถˆํ•„์š”
#### ๋ช…ํ™•ํ•œ ์ถœ๋ ฅ ๋ฐ ํ”ผ๋“œ๋ฐฑ (3์ )
- ๊ฒฐ๊ณผ๋ฌผ์ด ์‚ฌ์šฉ์ž๊ฐ€ ์ดํ•ดํ•˜๊ธฐ ์‰ฌ์šด ํ˜•ํƒœ๋กœ ์ œ๊ณต
- ์˜ˆ: "์š”์•ฝ๋œ ํ…์ŠคํŠธ", "๋ถ„๋ฅ˜ ๊ฒฐ๊ณผ์™€ ์‹ ๋ขฐ๋„", "๊ตฌ์กฐํ™”๋œ ๋ฐ์ดํ„ฐ"
- ์ง„ํ–‰ ์ƒํ™ฉ์ด๋‚˜ ์™„๋ฃŒ ์•Œ๋ฆผ ์ œ๊ณต
#### ์—๋Ÿฌ ์ฒ˜๋ฆฌ ๋ฐ ๊ฐ€์ด๋“œ (3์ )
- ์—๋Ÿฌ ๋ฐœ์ƒ ์‹œ ์‚ฌ์šฉ์ž ์นœํ™”์ ์ธ ๋ฉ”์‹œ์ง€ ์ œ๊ณต
- ์˜ˆ: "ํŒŒ์ผ ํ˜•์‹์ด ์˜ฌ๋ฐ”๋ฅด์ง€ ์•Š์Šต๋‹ˆ๋‹ค. PDF ๋˜๋Š” DOCX๋ฅผ ์—…๋กœ๋“œํ•ด์ฃผ์„ธ์š”"
- ์‚ฌ์šฉ ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ๊ฐ„๋‹จํ•œ ๊ฐ€์ด๋“œ๋‚˜ ์˜ˆ์‹œ ์ œ๊ณต
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{
"์‹ค์ œ_์—…๋ฌด_์ ์šฉ_๊ฐ€๋Šฅ์„ฑ": {
"๋ฐ˜๋ณต_์—…๋ฌด_์ œ๊ฑฐ": <0-3>,
"์‹ค์ œ_์‚ฌ์šฉ_์‹œ๋‚˜๋ฆฌ์˜ค_์„ ๋ช…": <0-4>,
"์‹œ๊ฐ„_์ ˆ๊ฐ_ํšจ๊ณผ_๋ช…ํ™•": <0-3>,
"์†Œ๊ณ„": <0-10>,
"ํ‰๊ฐ€": "<์‹ค์ œ ์—…๋ฌด ์ ์šฉ ๊ฐ€๋Šฅ์„ฑ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์ž๋™ํ™”_ํšจ๊ณผ_๋ฐ_์‚ฌ์šฉ์ž_ํŽธ์˜์„ฑ": {
"์žฌ์‚ฌ์šฉ_์ผ๋ฐ˜ํ™”_๊ฐ€๋Šฅ์„ฑ": <0-5>,
"์ž๋™ํ™”_ํšจ๊ณผ_๋ฐ_ํ•™์Šต_ํŽธ์˜์„ฑ": <0-5>,
"์†Œ๊ณ„": <0-10>,
"ํ‰๊ฐ€": "<์ž๋™ํ™” ํšจ๊ณผ ๋ฐ ์‚ฌ์šฉ์ž ํŽธ์˜์„ฑ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์‚ฌ์šฉ์ž_ํŽธ์˜์„ฑ": {
"์ง๊ด€์ ์ธ_์ž…๋ ฅ_๋ฐฉ์‹": <0-4>,
"๋ช…ํ™•ํ•œ_์ถœ๋ ฅ_๋ฐ_ํ”ผ๋“œ๋ฐฑ": <0-3>,
"์—๋Ÿฌ_์ฒ˜๋ฆฌ_๋ฐ_๊ฐ€์ด๋“œ": <0-3>,
"์†Œ๊ณ„": <0-10>,
"ํ‰๊ฐ€": "<์‚ฌ์šฉ์ž ํŽธ์˜์„ฑ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์ด์ ": <0-30>,
"์ข…ํ•ฉ_ํ‰๊ฐ€": "<์ „์ฒด์ ์ธ ํ‰๊ฐ€ ์š”์•ฝ>"
}
"""
# =============================================================================
# 4. ๋ฌธ์ œ ํ•ด๊ฒฐ ์ ‘๊ทผ๋ฒ• ์‹ฌ์‚ฌ ๊ธฐ์ค€ (30์ )
# =============================================================================
PROBLEM_SOLVING_CRITERIA = """
# ์ฐฝ์˜์„ฑ / ๋ฌธ์ œ ํ•ด๊ฒฐ ์ ‘๊ทผ๋ฒ• ์‹ฌ์‚ฌ ๊ธฐ์ค€ (20์ )
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ํ‰๊ฐ€ ์›์น™**
- ์ ์ˆ˜๋Š” ๋งค์šฐ ์—„๊ฒฉํ•˜๊ฒŒ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค. ์™„๋ฒฝํ•˜์ง€ ์•Š์œผ๋ฉด ๊ฐ์ ํ•˜์‹ญ์‹œ์˜ค.
- "๊ฑฐ์˜ ์™„์„ฑ" ๋˜๋Š” "๋Œ€๋ถ€๋ถ„ ์ข‹์Œ"์€ ์ถฉ๋ถ„ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๊ธฐ์ค€์„ ์ •ํ™•ํžˆ ์ถฉ์กฑํ•ด์•ผ๋งŒ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๋ถˆํ™•์‹คํ•˜๊ฑฐ๋‚˜ ๋ช…ํ™•ํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๋Š” ๋‚ฎ์€ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๊ธฐ์ค€์„ ์™„์ „ํžˆ ์ถฉ์กฑํ•˜์ง€ ์•Š์œผ๋ฉด ํ•ด๋‹น ํ•ญ๋ชฉ์— ๋Œ€ํ•œ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์ง€ ๋งˆ์‹ญ์‹œ์˜ค.
์ œ์ถœ๋œ ํ”„๋กœ์ ํŠธ ์„ค๋ช…์„ ๋ถ„์„ํ•˜์—ฌ ๋‹ค์Œ ๊ธฐ์ค€์œผ๋กœ ํ‰๊ฐ€ํ•˜์‹ญ์‹œ์˜ค.
## ํ‰๊ฐ€ ํ•ญ๋ชฉ:
### 4-1. ๋ฌธ์ œ ์ •์˜์˜ ๋…์ฐฝ์„ฑ (10์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ์ž๋™ํ™” ๋Œ€์ƒ ์„ ์ •์ด ์ƒˆ๋กญ๊ฑฐ๋‚˜, ์‹ค์งˆ์  ๋ฌธ์ œ๋ฅผ ์ž˜ ํฌ์ฐฉํ–ˆ๋Š”๊ฐ€?
**ํ•ต์‹ฌ ์งˆ๋ฌธ:**
- "์ด ๋ฌธ์ œ๋Š” ์‹ค์ œ ์‚ฌ์šฉ์žยท์กฐ์ง์ด ๊ฒช๋Š” '์ง„์งœ ๋ฌธ์ œ'์ธ๊ฐ€?"
- "๋ฌธ์ œ์˜ ์ •์˜๊ฐ€ ์‹ค์ œ ์ƒํ™ฉ๊ณผ ๋งž์•„๋–จ์–ด์ง€๋Š”๊ฐ€?"
- "๋ฌธ์ œ ์ •์˜์˜ ๋ฐฉ์‹์ด๋‚˜ ๊ด€์ ์—์„œ ๋…์ฐฝ์„ฑ์ด ์žˆ๋Š”๊ฐ€?"
#### ์‹ค์งˆ์  ๋ฌธ์ œ ํฌ์ฐฉ (5์ )
- "~๊ฐ€ ๋ถˆํŽธํ•˜๋‹ค / ์‹œ๊ฐ„์ด ์˜ค๋ž˜ ๊ฑธ๋ฆฐ๋‹ค / ์‹ค์ˆ˜๊ฐ€ ์žฆ๋‹ค" ๊ฐ™์€ **๊ตฌ์ฒด์  ๋ฌธ์ œ ์ƒํ™ฉ** ์ œ์‹œ
- ์ถ”์ƒ์ ("ํšจ์œจ ํ–ฅ์ƒ") ์•„๋‹Œ, ์ธก์ • ๊ฐ€๋Šฅํ•œ ๋ฌธ์ œ (์˜ˆ: "ํ•˜๋ฃจ 2์‹œ๊ฐ„ ์†Œ์š”", "์˜ค๋ฅ˜์œจ 30%")
- ๋ฌธ์ œ ์ •์˜๊ฐ€ ๋ช…ํ™•ํ•˜๊ณ  ๊ณต๊ฐ ๊ฐ€๋Šฅ
- ์‹ค์ œ ์‚ฌ์šฉ์ž์˜ pain point๋ฅผ ์ •ํ™•ํžˆ ํŒŒ์•…
#### ์ฐจ๋ณ„ํ™”๋œ ๊ด€์  ๋ฐ ๋…์ฐฝ์„ฑ (5์ )
- ๋‹จ์ˆœ "์ด๋Ÿฐ ๊ธฐ๋Šฅ ์žˆ์œผ๋ฉด ์ข‹๊ฒ ๋‹ค"๊ฐ€ ์•„๋‹Œ, **์™œ ์ง€๊ธˆ ๋ฐฉ์‹์ด ํ•œ๊ณ„์ธ์ง€ ๋ถ„์„**
- ๊ฒฝ์Ÿ ์„œ๋น„์Šค/๊ธฐ์กด ๋ฐฉ๋ฒ• ๋Œ€๋น„ ์ฐจ๋ณ„์  ์ œ์‹œ
- ๋ฐ์ดํ„ฐ๋‚˜ ์‚ฌ๋ก€ ๊ธฐ๋ฐ˜ ๊ทผ๊ฑฐ (์˜ˆ: "ํ˜„์žฌ ์ˆ˜๊ธฐ ์ž‘์—… ์‹œ 20๋ถ„ ์†Œ์š”, ์ž๋™ํ™” ์‹œ 2๋ถ„ ์˜ˆ์ƒ")
- ๋ฌธ์ œ๋ฅผ ๋ฐ”๋ผ๋ณด๋Š” ์ƒˆ๋กœ์šด ์‹œ๊ฐ์ด๋‚˜ ์ ‘๊ทผ ๋ฐฉ์‹
### 4-2. ์†”๋ฃจ์…˜ ๊ตฌ์„ฑ์˜ ์ฐธ์‹ ํ•จ (10์ )
**ํ•ต์‹ฌ ํ‰๊ฐ€ ํฌ์ธํŠธ:**
- ๊ธฐ์กด ๋ฐฉ๋ฒ•๊ณผ ์ฐจ๋ณ„ํ™”๋œ ์ ‘๊ทผ์„ ํ–ˆ๋Š”๊ฐ€?
- ์ œํ•œ๋œ ๋…ธ๋“œ/๋„๊ตฌ ๋‚ด์—์„œ ์ฐฝ์˜์  ํ•ด๊ฒฐ์„ ์‹œ๋„ํ–ˆ๋Š”๊ฐ€?
**ํ•ต์‹ฌ ์งˆ๋ฌธ:**
- "๋‹จ์ˆœํžˆ API๋ฅผ ๋ฌถ์–ด๋†“์€ ๊ตฌ์กฐ์ธ๊ฐ€?"
- "๋ฌธ์ œ๋ฅผ ์–ด๋–ค ๋ฐฉ์‹์œผ๋กœ ๋ฐ”๋ผ๋ณด๊ณ  ํ•ด๊ฒฐํ•˜๋ ค ํ–ˆ๋Š”์ง€ ๋…์ฐฝ์„ฑ์ด ์žˆ๋Š”๊ฐ€?"
- "๊ธฐ์ˆ  ๊ตฌ์„ฑ์˜ ์กฐํ•ฉ์ด๋‚˜ ์‹œ๊ฐ์ด ์ƒˆ๋กญ๊ณ  ์„ค๋“๋ ฅ์ด ์žˆ๋Š”๊ฐ€?"
#### ์ฐจ๋ณ„ํ™”๋œ ์ ‘๊ทผ (5์ )
- "DP ํ˜ธ์ถœ + Chat ํ˜ธ์ถœ"๋งŒ์œผ๋กœ ๋๋‚˜๋Š” ๊ฒŒ ์•„๋‹Œ, **์ „๋žต์  ์กฐํ•ฉ**
- ์˜ˆ: โ‘  ๋ฌธ์„œ ํŒŒ์‹ฑ โ†’ โ‘ก ํ‚ค์›Œ๋“œ ์ถ”์ถœ โ†’ โ‘ข ์™ธ๋ถ€ DB ๊ฒ€์ƒ‰ โ†’ โ‘ฃ ์ตœ์ข… ๋ณด๊ณ ์„œ ์ƒ์„ฑ ๊ฐ™์€ **๋‹ค๋‹จ๊ณ„ ๋กœ์ง**
- ๋˜๋Š” "์กฐ๊ฑด ๋ถ„๊ธฐ + ๋ฐ˜๋ณต + ๋ณ‘ํ•ฉ" ๊ฐ™์€ **๋ณต์žก๋„ ์žˆ๋Š” ๊ตฌ์กฐ**
- ๊ฐ ๋‹จ๊ณ„๊ฐ€ ๋…ผ๋ฆฌ์ ์œผ๋กœ ์—ฐ๊ฒฐ๋˜๊ณ  ํ•„์š”์„ฑ์ด ๋ช…ํ™•
#### ์ฐฝ์˜์  ํ•ด๊ฒฐ (5์ )
- ๊ธฐ์กด ํ…œํ”Œ๋ฆฟ/์˜ˆ์ œ๋ฅผ ๊ทธ๋Œ€๋กœ ๊ฐ€์ ธ์˜จ ๊ฒŒ ์•„๋‹Œ, **๋ฌธ์ œ ํ•ด๊ฒฐ์— ํŠนํ™”๋œ ์„ค๊ณ„**
- ์˜ˆ: "intent ๋ถ„๋ฅ˜ ํ›„ ๊ฐ intent๋ณ„ ๋งž์ถค ํ”„๋กฌํ”„ํŠธ" / "๋ฌธ์„œ ํƒ€์ž…๋ณ„ ๋‹ค๋ฅธ ํŒŒ์‹ฑ ์ „๋žต" ๋“ฑ
- ์ฐฝ์˜์  ์•„์ด๋””์–ด๋‚˜ ๋…ํŠนํ•œ workflow ํŒจํ„ด (์˜ˆ: ํ”ผ๋“œ๋ฐฑ ๋ฃจํ”„, ๋‹ค์ค‘ ์—์ด์ „ํŠธ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ๋“ฑ)
- ์ œํ•œ๋œ ๋„๊ตฌ ๋‚ด์—์„œ ์ฐฝ์˜์  ํ•ด๊ฒฐ์„ ์‹œ๋„ํ–ˆ๋Š”๊ฐ€?
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{
"๋ฌธ์ œ_์ •์˜์˜_๋…์ฐฝ์„ฑ": {
"์‹ค์งˆ์ _๋ฌธ์ œ_ํฌ์ฐฉ": <0-5>,
"์ฐจ๋ณ„ํ™”๋œ_๊ด€์ _๋ฐ_๋…์ฐฝ์„ฑ": <0-5>,
"์†Œ๊ณ„": <0-10>,
"ํ‰๊ฐ€": "<๋ฌธ์ œ ์ •์˜์˜ ๋…์ฐฝ์„ฑ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์†”๋ฃจ์…˜_๊ตฌ์„ฑ์˜_์ฐธ์‹ ํ•จ": {
"์ฐจ๋ณ„ํ™”๋œ_์ ‘๊ทผ": <0-5>,
"์ฐฝ์˜์ _ํ•ด๊ฒฐ": <0-5>,
"์†Œ๊ณ„": <0-10>,
"ํ‰๊ฐ€": "<์†”๋ฃจ์…˜ ๊ตฌ์„ฑ์˜ ์ฐธ์‹ ํ•จ ์ „์ฒด์— ๋Œ€ํ•œ ๊ตฌ์ฒด์  ํ‰๊ฐ€>"
},
"์ด์ ": <0-20>,
"์ข…ํ•ฉ_ํ‰๊ฐ€": "<์ „์ฒด์ ์ธ ํ‰๊ฐ€ ์š”์•ฝ>"
}
"""
# =============================================================================
# ํ‰๊ฐ€ ์žฌ๊ฒ€ํ†  ๊ธฐ์ค€
# =============================================================================
REVIEW_CRITERIA = """
# ํ‰๊ฐ€ ์žฌ๊ฒ€ํ†  ๊ธฐ์ค€
๋‹ค์Œ ํ‰๊ฐ€ ๊ฒฐ๊ณผ๋ฅผ ์žฌ๊ฒ€ํ† ํ•˜๊ณ , ์ ์ˆ˜๊ฐ€ ์ ์ ˆํ•œ์ง€ ํ™•์ธํ•˜์‹ญ์‹œ์˜ค.
## ์›๋ณธ ๋ฐ์ดํ„ฐ:
{data}
## ์‹ฌ์‚ฌ ๊ธฐ์ค€:
{criteria}
## ์ดˆ๊ธฐ ํ‰๊ฐ€ ๊ฒฐ๊ณผ:
{initial_score}
## ์žฌ๊ฒ€ํ†  ์š”์ฒญ์‚ฌํ•ญ:
โš ๏ธ **์ค‘์š”: ์—„๊ฒฉํ•œ ์žฌ๊ฒ€ํ†  ์›์น™**
- ์ดˆ๊ธฐ ํ‰๊ฐ€๊ฐ€ ๋„ˆ๋ฌด ํ›„ํ–ˆ๋‹ค๋ฉด ๋ฐ˜๋“œ์‹œ ์ ์ˆ˜๋ฅผ ๋‚ฎ์ถ”์‹ญ์‹œ์˜ค.
- "๊ฑฐ์˜ ์™„์„ฑ" ๋˜๋Š” "๋Œ€๋ถ€๋ถ„ ์ข‹์Œ"์€ ์ถฉ๋ถ„ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๊ธฐ์ค€์„ ์™„์ „ํžˆ ์ถฉ์กฑํ•ด์•ผ๋งŒ ์ ์ˆ˜๋ฅผ ๋ถ€์—ฌํ•˜์‹ญ์‹œ์˜ค.
- ๋ถˆํ™•์‹คํ•˜๊ฑฐ๋‚˜ ๋ช…ํ™•ํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๋Š” ๋ณด์ˆ˜์ ์œผ๋กœ ๋‚ฎ์€ ์ ์ˆ˜๋ฅผ ์œ ์ง€ํ•˜๊ฑฐ๋‚˜ ๋” ๋‚ฎ์ถ”์‹ญ์‹œ์˜ค.
1. ์ ์ˆ˜๊ฐ€ ์‹ฌ์‚ฌ ๊ธฐ์ค€์— ๋ถ€ํ•ฉํ•˜๋Š”๊ฐ€? (๋„ˆ๋ฌด ํ›„ํ•˜์ง€ ์•Š์€๊ฐ€?)
2. ํ‰๊ฐ€๊ฐ€ ๋„ˆ๋ฌด ๊ด€๋Œ€ํ•˜์ง€ ์•Š์€๊ฐ€? (์—„๊ฒฉํ•˜๊ฒŒ ํ‰๊ฐ€๋˜์—ˆ๋Š”๊ฐ€?)
3. ๋ชจ๋“  ํ•ญ๋ชฉ์ด ๊ณต์ •ํ•˜๊ณ  ์—„๊ฒฉํ•˜๊ฒŒ ํ‰๊ฐ€๋˜์—ˆ๋Š”๊ฐ€?
4. ํ‰๊ฐ€ ๊ทผ๊ฑฐ๊ฐ€ ์ถฉ๋ถ„ํžˆ ๊ตฌ์ฒด์ ์ธ๊ฐ€?
5. ์„ธ๋ถ€ ํ•ญ๋ชฉ ์ ์ˆ˜์˜ ํ•ฉ๊ณ„๊ฐ€ ์†Œ๊ณ„/์ด์ ๊ณผ ์ผ์น˜ํ•˜๋Š”๊ฐ€?
6. ์ดˆ๊ธฐ ํ‰๊ฐ€๊ฐ€ ๋„ˆ๋ฌด ํ›„ํ–ˆ๋‹ค๋ฉด ๋ฐ˜๋“œ์‹œ ์ ์ˆ˜๋ฅผ ๋‚ฎ์ถ”์—ˆ๋Š”๊ฐ€?
## ์ถœ๋ ฅ ํ˜•์‹:
JSON ํ˜•์‹์œผ๋กœ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ถœ๋ ฅํ•˜์‹ญ์‹œ์˜ค:
{{
"์žฌ๊ฒ€ํ† _๊ฒฐ๊ณผ": "<์œ ์ง€|์ˆ˜์ •>",
"์ˆ˜์ •_ํ•„์š”_์ด์œ ": "<์ˆ˜์ •์ด ํ•„์š”ํ•œ ๊ฒฝ์šฐ, ๊ทธ ์ด์œ >",
"์ตœ์ข…_์ ์ˆ˜": {{
... (์›๋ณธ๊ณผ ๋™์ผํ•œ ํ˜•์‹์˜ ์ ์ˆ˜ ๊ฐ์ฒด, ํ•„์š”์‹œ ์ˆ˜์ •๋œ ์ ์ˆ˜)
}},
"์žฌ๊ฒ€ํ† _์˜๊ฒฌ": "<์žฌ๊ฒ€ํ†  ์˜๊ฒฌ>"
}}
"""