Hugging Face's logo

Spaces:
ahaahaaha
/
adaptive_rag
Paused

App Files Community
adaptive_rag / nli_hallucination_detection_explained.py
lanny xu
resolve conflict
7edc5f6
raw
history blame contribute delete
31.6 kB
 """
NLI ๅนป่ง‰ๆฃ€ๆต‹ๅŽŸ็†่ฏฆ่งฃ
cross-encoder/nli-deberta-v3-xsmall ๆจกๅž‹ๅฆ‚ไฝ•ๆฃ€ๆต‹ RAG ไธญ็š„ๅนป่ง‰
"""
print("=" * 80)
print("NLI ๅนป่ง‰ๆฃ€ๆต‹ๅŽŸ็† - ไปŽ้›ถๅผ€ๅง‹็†่งฃ")
print("=" * 80)
# ============================================================================
# Part 1: ไป€ไนˆๆ˜ฏ NLI (Natural Language Inference)?
# ============================================================================
print("\n" + "=" * 80)
print("๐Ÿ“š Part 1: ไป€ไนˆๆ˜ฏ NLI (่‡ช็„ถ่ฏญ่จ€ๆŽจ็†)?")
print("=" * 80)
print("""
NLI ็š„ๆ ธๅฟƒไปปๅŠก๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
็ป™ๅฎšไธคๆฎตๆ–‡ๆœฌ๏ผš
- Premise (ๅ‰ๆ): ๅทฒ็Ÿฅ็š„ไบ‹ๅฎž/ๆ–‡ๆกฃ
- Hypothesis (ๅ‡่ฎพ): ๅพ…้ชŒ่ฏ็š„้™ˆ่ฟฐ
ๅˆคๆ–ญๅฎƒไปฌไน‹้—ด็š„ๅ…ณ็ณป๏ผš
1๏ธโƒฃ Entailment (่•ดๅซ): Hypothesis ๅฏไปฅไปŽ Premise ๆŽจๅฏผๅ‡บๆฅ
2๏ธโƒฃ Contradiction (็Ÿ›็›พ): Hypothesis ไธŽ Premise ็Ÿ›็›พ
3๏ธโƒฃ Neutral (ไธญ็ซ‹): ๆ— ๆณ•็กฎๅฎš๏ผŒๆ–‡ๆกฃไธญๆฒกๆœ‰่ถณๅคŸไฟกๆฏ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ไพ‹ๅญ 1: Entailment (่•ดๅซ) โœ…
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
Premise: "ๅฐๆ˜ŽไปŠๅนด 18 ๅฒ๏ผŒๆญฃๅœจๆธ…ๅŽๅคงๅญฆ่ฏป่ฎก็ฎ—ๆœบไธ“ไธšใ€‚"
Hypothesis: "ๅฐๆ˜Žๆ˜ฏไธ€ๅๅคงๅญฆ็”Ÿใ€‚"
ๅ…ณ็ณป: Entailment โœ…
่งฃ้‡Š: ไปŽ"ๆญฃๅœจๆธ…ๅŽๅคงๅญฆ่ฏป่ฎก็ฎ—ๆœบไธ“ไธš"ๅฏไปฅๆŽจๅฏผๅ‡บ"ๆ˜ฏๅคงๅญฆ็”Ÿ"
ไพ‹ๅญ 2: Contradiction (็Ÿ›็›พ) โŒ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
Premise: "ๅฐๆ˜ŽไปŠๅนด 18 ๅฒ๏ผŒๆญฃๅœจๆธ…ๅŽๅคงๅญฆ่ฏป่ฎก็ฎ—ๆœบไธ“ไธšใ€‚"
Hypothesis: "ๅฐๆ˜Žๅทฒ็ป 30 ๅฒไบ†ใ€‚"
ๅ…ณ็ณป: Contradiction โŒ
่งฃ้‡Š: "18ๅฒ" ไธŽ "30ๅฒ" ็Ÿ›็›พ
ไพ‹ๅญ 3: Neutral (ไธญ็ซ‹) โšช
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
Premise: "ๅฐๆ˜ŽไปŠๅนด 18 ๅฒ๏ผŒๆญฃๅœจๆธ…ๅŽๅคงๅญฆ่ฏป่ฎก็ฎ—ๆœบไธ“ไธšใ€‚"
Hypothesis: "ๅฐๆ˜Žๅ–œๆฌขๆ‰“็ฏฎ็ƒใ€‚"
ๅ…ณ็ณป: Neutral โšช
่งฃ้‡Š: ๆ–‡ๆกฃไธญๆฒกๆœ‰ๆๅˆฐๅฐๆ˜Žๆ˜ฏๅฆๅ–œๆฌขๆ‰“็ฏฎ็ƒ๏ผŒๆ— ๆณ•ๅˆคๆ–ญ
""")
# ============================================================================
# Part 2: NLI ๅฆ‚ไฝ•็”จไบŽๅนป่ง‰ๆฃ€ๆต‹?
# ============================================================================
print("\n" + "=" * 80)
print("๐Ÿ” Part 2: NLI ๅฆ‚ไฝ•ๆฃ€ๆต‹ๅนป่ง‰?")
print("=" * 80)
print("""
ๆ ธๅฟƒๆ€ๆƒณ๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅฐ† RAG ็š„ๅนป่ง‰ๆฃ€ๆต‹่ฝฌๅŒ–ไธบ NLI ไปปๅŠก๏ผš
Premise (ๅ‰ๆ) = ๆฃ€็ดขๅˆฐ็š„ๆ–‡ๆกฃ (Documents)
Hypothesis (ๅ‡่ฎพ) = LLM ็”Ÿๆˆ็š„็ญ”ๆกˆ (Generation)
ๅˆคๆ–ญ้€ป่พ‘๏ผš
- Entailment โ†’ ็ญ”ๆกˆๅŸบไบŽๆ–‡ๆกฃ โ†’ โœ… ๆ— ๅนป่ง‰
- Contradiction โ†’ ็ญ”ๆกˆไธŽๆ–‡ๆกฃ็Ÿ›็›พ โ†’ โŒ ๆœ‰ๅนป่ง‰
- Neutral โ†’ ็ญ”ๆกˆๆ–‡ๆกฃไธญๆฒกๆœ‰ โ†’ โš ๏ธ ๅฏ่ƒฝๆ˜ฏๅนป่ง‰
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
RAG ๅœบๆ™ฏ็คบไพ‹๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
็”จๆˆท้—ฎ้ข˜: "AlphaCodium ่ฎบๆ–‡่ฎฒ็š„ๆ˜ฏไป€ไนˆ๏ผŸ"
ๆฃ€็ดขๅˆฐ็š„ๆ–‡ๆกฃ (Premise):
"AlphaCodium ๆ˜ฏไธ€็งๅŸบไบŽไปฃ็ ็”Ÿๆˆ็š„ๆ–ฐๆ–นๆณ•๏ผŒ้€š่ฟ‡่ฟญไปฃๆ”น่ฟ›
ๆๅ‡ LLM ็š„ไปฃ็ ่ƒฝๅŠ›ใ€‚่ฏฅๆ–นๆณ•ๅœจ CodeContests ๆ•ฐๆฎ้›†ไธŠ
่พพๅˆฐไบ† state-of-the-art ็š„ๆ€ง่ƒฝใ€‚"
LLM ็”Ÿๆˆ็š„็ญ”ๆกˆ (Hypothesis):
"AlphaCodium ๆ˜ฏไธ€็งๆ”น่ฟ› LLM ไปฃ็ ็”Ÿๆˆ่ƒฝๅŠ›็š„่ฟญไปฃๆ–นๆณ•ใ€‚"
NLI ๅˆคๆ–ญ:
Premise + Hypothesis โ†’ NLI ๆจกๅž‹ โ†’ Entailment โœ…
โ†’ ็ญ”ๆกˆๅŸบไบŽๆ–‡ๆกฃ๏ผŒๆ— ๅนป่ง‰
ๅไพ‹ - ๆœ‰ๅนป่ง‰็š„ๆƒ…ๅ†ต๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๆฃ€็ดขๅˆฐ็š„ๆ–‡ๆกฃ (Premise):
"AlphaCodium ๆ˜ฏไธ€็งๅŸบไบŽไปฃ็ ็”Ÿๆˆ็š„ๆ–ฐๆ–นๆณ•..."
LLM ็”Ÿๆˆ็š„็ญ”ๆกˆ (Hypothesis):
"AlphaCodium ๆ˜ฏ Google ๅœจ 2024 ๅนดๅ‘ๅธƒ็š„..."
โ†‘ ๆ–‡ๆกฃไธญๆฒกๆœ‰ๆๅˆฐ Google ๅ’Œ 2024
NLI ๅˆคๆ–ญ:
Premise + Hypothesis โ†’ NLI ๆจกๅž‹ โ†’ Neutral/Contradiction โš ๏ธ
โ†’ ็ญ”ๆกˆๅŒ…ๅซๆ–‡ๆกฃไธญๆฒกๆœ‰็š„ไฟกๆฏ๏ผŒๅฏ่ƒฝๆ˜ฏๅนป่ง‰
""")
# ============================================================================
# Part 3: cross-encoder/nli-deberta-v3-xsmall ๆจกๅž‹ๆžถๆž„
# ============================================================================
print("\n" + "=" * 80)
print("๐Ÿค– Part 3: cross-encoder/nli-deberta-v3-xsmall ๆจกๅž‹ๆžถๆž„")
print("=" * 80)
print("""
ๆจกๅž‹ไฟกๆฏ๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅ็งฐ: cross-encoder/nli-deberta-v3-xsmall
ๅŸบ็ก€ๆจกๅž‹: DeBERTa-v3 (Decoding-enhanced BERT with disentangled attention)
ๅคงๅฐ: 40MB (่ถ…่ฝป้‡็บง)
ๅ‚ๆ•ฐ้‡: 22M
่ฎญ็ปƒๆ•ฐๆฎ: SNLI + MultiNLI (็™พไธ‡็บงๅฅๅญๅฏน)
่พ“ๅ‡บ: 3 ไธช็ฑปๅˆซ็š„ๆฆ‚็Ž‡ [Entailment, Neutral, Contradiction]
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๆžถๆž„ๅ›พ๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
่พ“ๅ…ฅๆ–‡ๆœฌ๏ผš
Premise: "AlphaCodium ๆ˜ฏไธ€็งไปฃ็ ็”Ÿๆˆๆ–นๆณ•..."
Hypothesis: "AlphaCodium ๆ˜ฏ Google ๅ‘ๅธƒ็š„..."
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 1: ่พ“ๅ…ฅๆ‹ผๆŽฅ โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ [CLS] Premise [SEP] Hypothesis [SEP] โ”‚
โ”‚ โ”‚
โ”‚ ๅฎž้™…: [CLS] AlphaCodium ๆ˜ฏไธ€็งไปฃ็ ็”Ÿๆˆๆ–นๆณ• [SEP] โ”‚
โ”‚ AlphaCodium ๆ˜ฏ Google ๅ‘ๅธƒ็š„ [SEP] โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 2: Tokenization โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ ๅˆ†่ฏๅนถ่ฝฌๆขไธบ Token IDs โ”‚
โ”‚ [101, 2945, 3421, ..., 4532, 102, 2945, 3421, ...] โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 3: DeBERTa Encoder (12 ๅฑ‚) โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ โ”‚
โ”‚ Layer 1-12: Disentangled Attention โ”‚
โ”‚ - Content-to-Content Attention โ”‚
โ”‚ - Content-to-Position Attention โ”‚
โ”‚ - Position-to-Content Attention โ”‚
โ”‚ โ”‚
โ”‚ ็‰น็‚น๏ผšไฝ็ฝฎไฟกๆฏๅ’Œๅ†…ๅฎนไฟกๆฏๅˆ†็ฆปๅค„็† โ”‚
โ”‚ ๆฏ” BERT ๆ›ดๅฅฝๅœฐ็†่งฃ้•ฟ่ท็ฆปไพ่ต– โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 4: [CLS] Token ็š„ๅ‘้‡่กจ็คบ โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ [CLS] ็š„ๅ‘้‡ๅŒ…ๅซไบ†ๆ•ดไธช่พ“ๅ…ฅๅฏน็š„่ฏญไน‰ไฟกๆฏ โ”‚
โ”‚ Vector: [0.234, -0.567, 0.890, ..., 0.123] (768็ปด) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 5: ๅˆ†็ฑปๅคด (Classification Head) โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ ๅ…จ่ฟžๆŽฅๅฑ‚: 768 โ†’ 3 โ”‚
โ”‚ โ”‚
โ”‚ Logits: [2.3, -1.5, 0.8] โ”‚
โ”‚ โ†‘ โ†‘ โ†‘ โ”‚
โ”‚ Entail Neutral Contra โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 6: Softmax ๅฝ’ไธ€ๅŒ– โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ Probabilities: โ”‚
โ”‚ Entailment: 0.15 (15%) โ”‚
โ”‚ Neutral: 0.05 (5%) โ”‚
โ”‚ Contradiction: 0.80 (80%) โ† ๆœ€้ซ˜๏ผ โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
ๆœ€็ปˆ่พ“ๅ‡บ:
Label: "Contradiction"
Score: 0.80
่งฃ้‡Š: ๆจกๅž‹่ฎคไธบ็ญ”ๆกˆไธŽๆ–‡ๆกฃ็Ÿ›็›พ๏ผŒ็ฝฎไฟกๅบฆ 80%
โ†’ ๆฃ€ๆต‹ๅˆฐๅนป่ง‰๏ผ
""")
# ============================================================================
# Part 4: ไฝ ็š„้กน็›ฎไธญ็š„ๅฎž้™…ๆฃ€ๆต‹ๆต็จ‹
# ============================================================================
print("\n" + "=" * 80)
print("๐Ÿ’ป Part 4: ไฝ ็š„้กน็›ฎไธญ็š„ๅฎž้™…ๆฃ€ๆต‹ๆต็จ‹")
print("=" * 80)
print("""
ๅฎŒๆ•ดๆฃ€ๆต‹ๆต็จ‹๏ผˆhallucination_detector.py๏ผ‰๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
่พ“ๅ…ฅ:
- generation: LLM ็”Ÿๆˆ็š„ๅฎŒๆ•ด็ญ”ๆกˆ
- documents: ๆฃ€็ดขๅˆฐ็š„ๆ–‡ๆกฃ
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 1: ๅฅๅญๅˆ†ๅ‰ฒ โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ ๅฐ† LLM ็”Ÿๆˆ็š„็ญ”ๆกˆๅˆ†ๅ‰ฒๆˆๅคšไธชๅฅๅญ โ”‚
โ”‚ โ”‚
โ”‚ ไพ‹ๅฆ‚: โ”‚
โ”‚ "AlphaCodium ๆ˜ฏไธ€็งไปฃ็ ็”Ÿๆˆๆ–นๆณ•ใ€‚ๅฎƒ็”ฑ Google ๅผ€ๅ‘ใ€‚" โ”‚
โ”‚ โ†“ โ”‚
โ”‚ ["AlphaCodium ๆ˜ฏไธ€็งไปฃ็ ็”Ÿๆˆๆ–นๆณ•ใ€‚", โ”‚
โ”‚ "ๅฎƒ็”ฑ Google ๅผ€ๅ‘ใ€‚"] โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 2: ้€ๅฅ NLI ๆฃ€ๆต‹ โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ โ”‚
โ”‚ for ๆฏไธชๅฅๅญ in ็ญ”ๆกˆ: โ”‚
โ”‚ result = nli_model( โ”‚
โ”‚ premise=documents[:500], # ๆ–‡ๆกฃ๏ผˆๆˆชๆ–ญๅˆฐ500ๅญ—็ฌฆ๏ผ‰โ”‚
โ”‚ hypothesis=sentence # ๅฝ“ๅ‰ๅฅๅญ โ”‚
โ”‚ ) โ”‚
โ”‚ โ”‚
โ”‚ if "contradiction" in result.label: โ”‚
โ”‚ contradiction_count += 1 โ”‚
โ”‚ elif "neutral" in result.label: โ”‚
โ”‚ neutral_count += 1 โ”‚
โ”‚ elif "entailment" in result.label: โ”‚
โ”‚ entailment_count += 1 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 3: ็ปŸ่ฎกๅˆ†ๆž โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ โ”‚
โ”‚ ๅ‡่ฎพๆฃ€ๆต‹ไบ† 10 ไธชๅฅๅญ: โ”‚
โ”‚ - Entailment: 7 ไธช (70%) โ”‚
โ”‚ - Neutral: 2 ไธช (20%) โ”‚
โ”‚ - Contradiction: 1 ไธช (10%) โ”‚
โ”‚ โ”‚
โ”‚ total_sentences = 10 โ”‚
โ”‚ contradiction_ratio = 1/10 = 0.1 (10%) โ”‚
โ”‚ neutral_ratio = 2/10 = 0.2 (20%) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Step 4: ๅˆคๆ–ญๆ˜ฏๅฆๆœ‰ๅนป่ง‰ โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚ โ”‚
โ”‚ ไฝ ็š„้กน็›ฎ้…็ฝฎ: (ๅทฒไผ˜ๅŒ–) โ”‚
โ”‚ โ”‚
โ”‚ has_hallucination = ( โ”‚
โ”‚ contradiction_ratio > 0.3 OR # ็Ÿ›็›พ่ถ…่ฟ‡ 30% โ”‚
โ”‚ neutral_ratio > 0.8 # ไธญ็ซ‹่ถ…่ฟ‡ 80% โ”‚
โ”‚ ) โ”‚
โ”‚ โ”‚
โ”‚ ไธŠไพ‹ไธญ: โ”‚
โ”‚ contradiction_ratio = 0.1 (10%) โœ… < 30% โ”‚
โ”‚ neutral_ratio = 0.2 (20%) โœ… < 80% โ”‚
โ”‚ โ†’ has_hallucination = False โœ… โ”‚
โ”‚ โ†’ ๆœชๆฃ€ๆต‹ๅˆฐๅนป่ง‰ โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ†“
่พ“ๅ‡บ็ป“ๆžœ:
{
"has_hallucination": False,
"contradiction_count": 1,
"neutral_count": 2,
"entailment_count": 7,
"problematic_sentences": [] # ๅชๅŒ…ๅซ็Ÿ›็›พ็š„ๅฅๅญ
}
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅฎž้™…ไปฃ็ ๏ผˆhallucination_detector.py ็ฌฌ 187-241 ่กŒ๏ผ‰๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
for sentence in sentences:
if len(sentence) < 10:
continue
try:
# Cross-encoder ๆ ผๅผ
result = self.nli_model(
f"{documents[:500]} [SEP] {sentence}",
truncation=True,
max_length=512
)
label = result[0]['label'].lower()
if 'contradiction' in label:
contradiction_count += 1
problematic_sentences.append(sentence)
elif 'neutral' in label:
neutral_count += 1
elif 'entailment' in label:
entailment_count += 1
except Exception as e:
print(f"โš ๏ธ NLI ๆฃ€ๆต‹ๅฅๅญๅคฑ่ดฅ: {str(e)[:100]}")
# ๅˆคๆ–ญ้€ป่พ‘
total_sentences = contradiction_count + neutral_count + entailment_count
if total_sentences > 0:
contradiction_ratio = contradiction_count / total_sentences
neutral_ratio = neutral_count / total_sentences
has_hallucination = (contradiction_ratio > 0.3) or (neutral_ratio > 0.8)
""")
# ============================================================================
# Part 5: ไธบไป€ไนˆ่ฟ™ไธชๆ–นๆณ•ๆœ‰ๆ•ˆ?
# ============================================================================
print("\n" + "=" * 80)
print("๐ŸŽฏ Part 5: ไธบไป€ไนˆ NLI ๆฃ€ๆต‹ๅนป่ง‰ๆœ‰ๆ•ˆ?")
print("=" * 80)
print("""
ๆ ธๅฟƒไผ˜ๅŠฟ๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
1๏ธโƒฃ ไธ“้—จ่ฎญ็ปƒ็š„ไปปๅŠกๅฏน้ฝ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
NLI ๆจกๅž‹ๅœจ็™พไธ‡็บงๅฅๅญๅฏนไธŠ่ฎญ็ปƒ๏ผŒไธ“้—จๅญฆไน ๅˆคๆ–ญ้€ป่พ‘ๅ…ณ็ณป๏ผš
- ่ฎญ็ปƒๆ•ฐๆฎ: SNLI (570K) + MultiNLI (433K)
- ไปปๅŠก: ๅˆคๆ–ญ Premise ๆ˜ฏๅฆๆ”ฏๆŒ Hypothesis
- ่ฟ™ๆญฃๆ˜ฏๅนป่ง‰ๆฃ€ๆต‹้œ€่ฆ็š„่ƒฝๅŠ›๏ผ
ไผ ็ปŸ LLM:
"่ฏทๅˆคๆ–ญ่ฟ™ไธช็ญ”ๆกˆๆ˜ฏๅฆๅŸบไบŽๆ–‡ๆกฃ..."
โ†’ LLM ้œ€่ฆ็†่งฃๆŒ‡ไปคใ€ๆŽจ็†ใ€็”Ÿๆˆ็ญ”ๆกˆ
โ†’ ๅฎนๆ˜“ๅ‡บ้”™๏ผŒไธๅคŸไธ“ๆณจ
NLI ๆจกๅž‹:
Input: [Premise, Hypothesis]
โ†’ ็›ดๆŽฅ่พ“ๅ‡บๆฆ‚็Ž‡: [Entail, Neutral, Contra]
โ†’ ไธ“ๆณจไธ”ๅ‡†็กฎ
2๏ธโƒฃ ็ป†็ฒ’ๅบฆ็š„ๅฅๅญ็บงๆฃ€ๆต‹
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
้€ๅฅๆฃ€ๆต‹ๅฏไปฅ็ฒพๅ‡†ๅฎšไฝ้—ฎ้ข˜๏ผš
ๆ•ดไฝ“ๆฃ€ๆต‹๏ผˆLLM๏ผ‰:
"ๆ•ดไธช็ญ”ๆกˆๆ˜ฏๅฆๅŸบไบŽๆ–‡ๆกฃ๏ผŸ"
โ†’ ้šพไปฅๅˆคๆ–ญๅ“ช้ƒจๅˆ†ๆœ‰้—ฎ้ข˜
ๅฅๅญ็บงๆฃ€ๆต‹๏ผˆNLI๏ผ‰:
ๅฅๅญ1: "AlphaCodium ๆ˜ฏไปฃ็ ็”Ÿๆˆๆ–นๆณ•" โ†’ Entailment โœ…
ๅฅๅญ2: "็”ฑ Google ๅผ€ๅ‘" โ†’ Contradiction โŒ โ† ็ฒพๅ‡†ๅฎšไฝ๏ผ
ๅฅๅญ3: "ๅœจ CodeContests ไธŠ่กจ็Žฐๅฅฝ" โ†’ Entailment โœ…
3๏ธโƒฃ ้€Ÿๅบฆๅ’Œๆˆๆœฌไผ˜ๅŠฟ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๆจกๅž‹ๅฏนๆฏ”๏ผš
ไผ ็ปŸ LLM (ๅฆ‚ GPT-3.5):
- ๆŽจ็†ๆ—ถ้—ด: 500-1000ms
- ๆˆๆœฌ: ๆฏๆฌกๆฃ€ๆต‹็บฆ $0.001
- ๅ‚ๆ•ฐ้‡: 175B
NLI ๆจกๅž‹ (cross-encoder/nli-deberta-v3-xsmall):
- ๆŽจ็†ๆ—ถ้—ด: 50-100ms โ† ๅฟซ 10 ๅ€๏ผ
- ๆˆๆœฌ: ๆœฌๅœฐ่ฟ่กŒ๏ผŒๆŽฅ่ฟ‘ $0 โ† ็œ 100 ๅ€๏ผ
- ๅ‚ๆ•ฐ้‡: 22M โ† ๅฐ 7900 ๅ€๏ผ
4๏ธโƒฃ ๅฏ่งฃ้‡Šๆ€ง
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
NLI ่พ“ๅ‡บๆธ…ๆ™ฐ็š„ๆฆ‚็Ž‡ๅˆ†ๅธƒ๏ผš
่พ“ๅ‡บ็คบไพ‹:
{
"label": "Contradiction",
"scores": {
"entailment": 0.05,
"neutral": 0.15,
"contradiction": 0.80 โ† 80% ็กฎๅฎšๆ˜ฏ็Ÿ›็›พ
}
}
vs LLM ่พ“ๅ‡บ:
"ๆˆ‘่ฎคไธบ่ฟ™ไธช็ญ”ๆกˆๅฏ่ƒฝๆœ‰ไบ›้—ฎ้ข˜..." โ† ๆจก็ณŠไธๆธ…
""")
# ============================================================================
# Part 6: DeBERTa vs BERT ็š„ๅ…ณ้”ฎๆ”น่ฟ›
# ============================================================================
print("\n" + "=" * 80)
print("โšก Part 6: DeBERTa vs BERT - ไธบไป€ไนˆ้€‰ DeBERTa?")
print("=" * 80)
print("""
DeBERTa ็š„ๆ ธๅฟƒๅˆ›ๆ–ฐ๏ผšDisentangled Attention
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
BERT ็š„้—ฎ้ข˜๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅœจ BERT ไธญ๏ผŒๆฏไธช token ็š„่กจ็คบ = ๅ†…ๅฎน embedding + ไฝ็ฝฎ embedding
ไพ‹ๅฆ‚๏ผš
"AlphaCodium" ๅœจไฝ็ฝฎ 5:
Token Embedding = Content + Position
= [0.1, 0.2, ...] + [0.3, 0.4, ...]
= [0.4, 0.6, ...]
้—ฎ้ข˜๏ผšๅ†…ๅฎนๅ’Œไฝ็ฝฎๆททๅœจไธ€่ตท๏ผŒๆจกๅž‹้šพไปฅๅŒบๅˆ†๏ผš
- "ๆ˜ฏ" ๅœจไฝ็ฝฎ 3 ็š„้‡่ฆๆ€ง
- "ๆ˜ฏ" ่ฟ™ไธช่ฏๆœฌ่บซ็š„่ฏญไน‰
DeBERTa ็š„่งฃๅ†ณๆ–นๆกˆ๏ผšๅˆ†็ฆปๆณจๆ„ๅŠ›
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅฐ†ๆณจๆ„ๅŠ›ๅˆ†่งฃไธบ 3 ไธช้ƒจๅˆ†๏ผš
1. Content-to-Content Attention
่ฏ i ็š„ๅ†…ๅฎน ๅ…ณๆณจ ่ฏ j ็š„ๅ†…ๅฎน
"AlphaCodium" ๅ…ณๆณจ "ไปฃ็ ็”Ÿๆˆ"
2. Content-to-Position Attention
่ฏ i ็š„ๅ†…ๅฎน ๅ…ณๆณจ ่ฏ j ็š„ไฝ็ฝฎ
"ๆ˜ฏ" ๅ…ณๆณจ ไฝ็ฝฎ 10 (ไธŠไธ‹ๆ–‡)
3. Position-to-Content Attention
่ฏ i ็š„ไฝ็ฝฎ ๅ…ณๆณจ ่ฏ j ็š„ๅ†…ๅฎน
ไฝ็ฝฎ 5 ๅ…ณๆณจ "ๆ–นๆณ•" ่ฟ™ไธช่ฏ
ๅ…ฌๅผ:
Attention(Q, K, V) = softmax(
Q_c ร— K_c^T / โˆšd + # Content-to-Content
Q_c ร— K_p^T / โˆšd + # Content-to-Position
Q_p ร— K_c^T / โˆšd # Position-to-Content
) ร— V
ไผ˜ๅŠฟ:
โœ… ๆ›ดๅฅฝๅœฐ็†่งฃ้•ฟ่ท็ฆปไพ่ต–
โœ… ๆ›ดๅ‡†็กฎ็š„่ฏญไน‰็†่งฃ
โœ… NLI ไปปๅŠกไธŠๆ€ง่ƒฝๆๅ‡ 2-3%
โœ… ๅŒๆ ทๅ‚ๆ•ฐไธ‹ๆ•ˆๆžœๆ›ดๅฅฝ
ๆจกๅž‹ๅฏนๆฏ”๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
BERT-base:
- ๅ‚ๆ•ฐ: 110M
- MNLI ๅ‡†็กฎ็Ž‡: 84.6%
DeBERTa-v3-xsmall:
- ๅ‚ๆ•ฐ: 22M โ† ๅฐ 5 ๅ€
- MNLI ๅ‡†็กฎ็Ž‡: 82.1% โ† ๅช้™ไฝŽ 2.5%
- ๆŽจ็†้€Ÿๅบฆ: ๅฟซ 3 ๅ€
ๆ€งไปทๆฏ”ๆœ€้ซ˜๏ผ่ฟ™ๅฐฑๆ˜ฏไธบไป€ไนˆไฝ ็š„้กน็›ฎ้€‰ๆ‹ฉๅฎƒ
""")
# ============================================================================
# Part 7: ๅฎž้™…ๆกˆไพ‹ๆผ”็คบ
# ============================================================================
print("\n" + "=" * 80)
print("๐Ÿ“ Part 7: ๅฎž้™…ๆกˆไพ‹ๆผ”็คบ")
print("=" * 80)
print("""
ๆกˆไพ‹ 1: ๆญฃๅธธ็ญ”ๆกˆ๏ผˆๆ— ๅนป่ง‰๏ผ‰
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๆ–‡ๆกฃ (Premise):
"Prompt Engineering ๆ˜ฏไธ€็ง้€š่ฟ‡ไผ˜ๅŒ–่พ“ๅ…ฅๆ็คบๆฅๅผ•ๅฏผ
่ฏญ่จ€ๆจกๅž‹่กŒไธบ็š„ๆ–นๆณ•๏ผŒๆ— ้œ€ไฟฎๆ”นๆจกๅž‹ๆƒ้‡ใ€‚"
LLM ็”Ÿๆˆ (Hypothesis):
"Prompt Engineering ๆ˜ฏไธ€็งไผ˜ๅŒ–่พ“ๅ…ฅๆ็คบ็š„ๆ–นๆณ•๏ผŒ
็”จไบŽๅผ•ๅฏผ่ฏญ่จ€ๆจกๅž‹็š„่กŒไธบใ€‚"
NLI ๆฃ€ๆต‹่ฟ‡็จ‹:
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅฅๅญ 1: "Prompt Engineering ๆ˜ฏไธ€็งไผ˜ๅŒ–่พ“ๅ…ฅๆ็คบ็š„ๆ–นๆณ•"
โ†’ NLI: Entailment (0.92) โœ…
โ†’ ๆ–‡ๆกฃไธญๆœ‰๏ผš"้€š่ฟ‡ไผ˜ๅŒ–่พ“ๅ…ฅๆ็คบ"
ๅฅๅญ 2: "็”จไบŽๅผ•ๅฏผ่ฏญ่จ€ๆจกๅž‹็š„่กŒไธบ"
โ†’ NLI: Entailment (0.88) โœ…
โ†’ ๆ–‡ๆกฃไธญๆœ‰๏ผš"ๅผ•ๅฏผ่ฏญ่จ€ๆจกๅž‹่กŒไธบ"
็ปŸ่ฎก:
Entailment: 2/2 = 100%
Neutral: 0/2 = 0%
Contradiction: 0/2 = 0%
ๅˆคๆ–ญ:
contradiction_ratio = 0% < 30% โœ…
neutral_ratio = 0% < 80% โœ…
โ†’ ๆ— ๅนป่ง‰ โœ…
ๆกˆไพ‹ 2: ๅŒ…ๅซๅนป่ง‰็š„็ญ”ๆกˆ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๆ–‡ๆกฃ (Premise):
"AlphaCodium ๆ˜ฏไธ€็งไปฃ็ ็”Ÿๆˆๆ–นๆณ•๏ผŒ้€š่ฟ‡่ฟญไปฃๆ”น่ฟ›
ๆๅ‡ LLM ็š„ไปฃ็ ่ƒฝๅŠ›ใ€‚"
LLM ็”Ÿๆˆ (Hypothesis):
"AlphaCodium ๆ˜ฏ Google ๅœจ 2024 ๅนดๅ‘ๅธƒ็š„ไปฃ็ ็”Ÿๆˆๅทฅๅ…ทใ€‚
ๅฎƒไฝฟ็”จๅผบๅŒ–ๅญฆไน ๆฅ่ฎญ็ปƒๆจกๅž‹ใ€‚"
NLI ๆฃ€ๆต‹่ฟ‡็จ‹:
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅฅๅญ 1: "AlphaCodium ๆ˜ฏ Google ๅœจ 2024 ๅนดๅ‘ๅธƒ็š„ไปฃ็ ็”Ÿๆˆๅทฅๅ…ท"
โ†’ NLI: Neutral (0.75) โš ๏ธ
โ†’ ๆ–‡ๆกฃไธญๆฒกๆœ‰ๆๅˆฐ Google ๅ’Œ 2024
ๅฅๅญ 2: "ๅฎƒไฝฟ็”จๅผบๅŒ–ๅญฆไน ๆฅ่ฎญ็ปƒๆจกๅž‹"
โ†’ NLI: Neutral (0.68) โš ๏ธ
โ†’ ๆ–‡ๆกฃไธญๆฒกๆœ‰ๆๅˆฐๅผบๅŒ–ๅญฆไน 
็ปŸ่ฎก:
Entailment: 0/2 = 0%
Neutral: 2/2 = 100%
Contradiction: 0/2 = 0%
ๅˆคๆ–ญ:
contradiction_ratio = 0% < 30% โœ…
neutral_ratio = 100% > 80% โŒ โ† ่งฆๅ‘๏ผ
โ†’ ๆฃ€ๆต‹ๅˆฐๅนป่ง‰ โš ๏ธ
ๆกˆไพ‹ 3: ๆ˜Žๆ˜พ็Ÿ›็›พ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๆ–‡ๆกฃ (Premise):
"่ฟ™็ฏ‡่ฎบๆ–‡ๅ‘่กจไบŽ 2023 ๅนดใ€‚"
LLM ็”Ÿๆˆ (Hypothesis):
"่ฟ™็ฏ‡่ฎบๆ–‡ๆ˜ฏ 2021 ๅนดๅ‘่กจ็š„ใ€‚"
NLI ๆฃ€ๆต‹:
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ๅฅๅญ 1: "่ฟ™็ฏ‡่ฎบๆ–‡ๆ˜ฏ 2021 ๅนดๅ‘่กจ็š„"
โ†’ NLI: Contradiction (0.95) โŒ
โ†’ 2023 โ‰  2021๏ผŒๆ˜Žๆ˜พ็Ÿ›็›พ๏ผ
็ปŸ่ฎก:
Entailment: 0/1 = 0%
Neutral: 0/1 = 0%
Contradiction: 1/1 = 100%
ๅˆคๆ–ญ:
contradiction_ratio = 100% > 30% โŒ โ† ่งฆๅ‘๏ผ
โ†’ ๆฃ€ๆต‹ๅˆฐๅนป่ง‰ โŒ
""")
# ============================================================================
# Part 8: ไผ˜็ผบ็‚นๅˆ†ๆž
# ============================================================================
print("\n" + "=" * 80)
print("โš–๏ธ Part 8: NLI ๅนป่ง‰ๆฃ€ๆต‹็š„ไผ˜็ผบ็‚น")
print("=" * 80)
print("""
ไผ˜็‚น โœ…
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
1. ๅ‡†็กฎ็Ž‡้ซ˜
- ไธ“้—จ่ฎญ็ปƒ็š„ NLI ๆจกๅž‹๏ผŒๅœจ้€ป่พ‘ๆŽจ็†ไธŠๅ‡†็กฎ็Ž‡ 85-95%
- ๆฏ”้€š็”จ LLM ๅˆคๆ–ญๅ‡†็กฎ 15-20%
2. ้€Ÿๅบฆๅฟซ
- ่ฝป้‡็บงๆจกๅž‹ (22M ๅ‚ๆ•ฐ)
- ๆŽจ็†ๆ—ถ้—ด 50-100ms
- ๆฏ” LLM ๅฟซ 10 ๅ€
3. ๆˆๆœฌไฝŽ
- ๆœฌๅœฐ่ฟ่กŒ๏ผŒๆ— ้œ€ API ่ฐƒ็”จ
- ๆˆๆœฌๆŽฅ่ฟ‘ 0
- ๆฏ” LLM ็œ 100 ๅ€
4. ๅฏ่งฃ้‡Šๆ€งๅผบ
- ่พ“ๅ‡บๆธ…ๆ™ฐ็š„ๆฆ‚็Ž‡ๅˆ†ๅธƒ
- ๅฏไปฅๅฎšไฝๅ…ทไฝ“้—ฎ้ข˜ๅฅๅญ
- ๆ–นไพฟ่ฐƒ่ฏ•ๅ’Œไผ˜ๅŒ–
5. ็ป†็ฒ’ๅบฆๆŽงๅˆถ
- ้€ๅฅๆฃ€ๆต‹
- ๅฏ่‡ชๅฎšไน‰้˜ˆๅ€ผ (30%, 80%)
- ็ตๆดป่ฐƒๆ•ดไธฅๆ ผ็จ‹ๅบฆ
็ผบ็‚น โŒ
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
1. ๅฏน Neutral ็š„ๅˆคๆ–ญไธๅคŸ็ฒพๅ‡†
- Neutral ๆ—ขๅฏ่ƒฝๆ˜ฏๅนป่ง‰๏ผŒไนŸๅฏ่ƒฝๆ˜ฏๅˆ็†ๆŽจ็†
- ้œ€่ฆ่ฎพ็ฝฎๅˆ็†็š„้˜ˆๅ€ผ๏ผˆไฝ ็š„้กน็›ฎ่ฎพไธบ 80%๏ผ‰
2. ไพ่ต–ๅฅๅญๅˆ†ๅ‰ฒ่ดจ้‡
- ๅˆ†ๅ‰ฒ้”™่ฏฏไผšๅฝฑๅ“ๆฃ€ๆต‹
- ไพ‹ๅฆ‚๏ผš"Mr. Smith went to U.S.A." ๅฏ่ƒฝ่ขซ้”™่ฏฏๅˆ†ๅ‰ฒ
3. ไธŠไธ‹ๆ–‡็†่งฃๆœ‰้™
- ๅช็œ‹ 500 ๅญ—็ฌฆ็š„ๆ–‡ๆกฃ
- ๅฏ่ƒฝ้”™่ฟ‡้•ฟๆ–‡ๆกฃไธญ็š„็›ธๅ…ณไฟกๆฏ
4. ่ฏญ่จ€ไพ่ต–
- ไธป่ฆๅœจ่‹ฑๆ–‡ๆ•ฐๆฎไธŠ่ฎญ็ปƒ
- ไธญๆ–‡ๆ•ˆๆžœๅฏ่ƒฝ็•ฅๅทฎ๏ผˆไฝ† DeBERTa-v3 ๅฏนๅคš่ฏญ่จ€ๆ”ฏๆŒ่พƒๅฅฝ๏ผ‰
5. ๆ— ๆณ•ๆฃ€ๆต‹้šๆ€งๅนป่ง‰
- ๅช่ƒฝๆฃ€ๆต‹ๆ˜พๅผ็š„็Ÿ›็›พๆˆ–็ผบๅคฑ
- ๆ— ๆณ•ๆฃ€ๆต‹ๆŽจ็†้”™่ฏฏๆˆ–้€ป่พ‘ๆผๆดž
ๆ”น่ฟ›ๅปบ่ฎฎ ๐Ÿ’ก
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
1. ็ป“ๅˆๅคš็งๆ–นๆณ•
ไฝ ็š„้กน็›ฎๅทฒ็ปๅšไบ†๏ผšVectara + NLI ๆททๅˆๆฃ€ๆต‹ โœ…
2. ๅŠจๆ€่ฐƒๆ•ด้˜ˆๅ€ผ
ๆ นๆฎๅบ”็”จๅœบๆ™ฏ่ฐƒๆ•ด contradiction/neutral ้˜ˆๅ€ผ
3. ๅขžๅŠ ๆ–‡ๆกฃ้•ฟๅบฆ
ไปŽ 500 ๅญ—็ฌฆๅขžๅŠ ๅˆฐ 1000 ๅญ—็ฌฆ๏ผˆ้œ€่ฆๆ›ดๅคš่ฎก็ฎ—๏ผ‰
4. ไฝฟ็”จๆ›ดๅคง็š„ๆจกๅž‹
ๅฆ‚ๆžœๅ‡†็กฎ็Ž‡ไธๅคŸ๏ผŒๅฏๅ‡็บงๅˆฐ cross-encoder/nli-deberta-v3-base
""")
# ============================================================================
# Part 9: ๆ€ป็ป“
# ============================================================================
print("\n" + "=" * 80)
print("๐Ÿ“š Part 9: ๆ ธๅฟƒ่ฆ็‚นๆ€ป็ป“")
print("=" * 80)
print("""
NLI ๅนป่ง‰ๆฃ€ๆต‹ๅŽŸ็†๏ผš
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
1. ๆ ธๅฟƒๆ€ๆƒณ
ๅฐ†ๅนป่ง‰ๆฃ€ๆต‹่ฝฌๅŒ–ไธบ NLI ไปปๅŠก๏ผš
Premise (ๆ–‡ๆกฃ) + Hypothesis (็ญ”ๆกˆ) โ†’ Entailment/Neutral/Contradiction
2. ๆจกๅž‹ๆžถๆž„
cross-encoder/nli-deberta-v3-xsmall (22M ๅ‚ๆ•ฐ, 40MB)
- DeBERTa-v3: ๅˆ†็ฆปๆณจๆ„ๅŠ›ๆœบๅˆถ
- Cross-Encoder: ่”ๅˆ็ผ–็  Premise ๅ’Œ Hypothesis
- 3 ๅˆ†็ฑปๅคด: Entailment/Neutral/Contradiction
3. ๆฃ€ๆต‹ๆต็จ‹
Step 1: ๅˆ†ๅฅ
Step 2: ้€ๅฅ NLI ๅˆคๆ–ญ
Step 3: ็ปŸ่ฎก Entailment/Neutral/Contradiction ๆฏ”ไพ‹
Step 4: ๆ นๆฎ้˜ˆๅ€ผๅˆคๆ–ญๆ˜ฏๅฆๆœ‰ๅนป่ง‰
- contradiction > 30% โ†’ ๅนป่ง‰
- neutral > 80% โ†’ ๅนป่ง‰
4. ๅ…ณ้”ฎไผ˜ๅŠฟ
โœ… ๅ‡†็กฎ็Ž‡: 85-95% (ๆฏ” LLM ้ซ˜ 15-20%)
โœ… ้€Ÿๅบฆ: 50-100ms (ๆฏ” LLM ๅฟซ 10 ๅ€)
โœ… ๆˆๆœฌ: ๆœฌๅœฐ่ฟ่กŒ (ๆฏ” LLM ็œ 100 ๅ€)
โœ… ๅฏ่งฃ้‡Š: ่พ“ๅ‡บๆธ…ๆ™ฐๆฆ‚็Ž‡ๅˆ†ๅธƒ
5. ไฝ ็š„้กน็›ฎ้…็ฝฎ
โœ… ๆจกๅž‹: cross-encoder/nli-deberta-v3-xsmall
โœ… ้˜ˆๅ€ผ: contradiction > 30% or neutral > 80%
โœ… ๆททๅˆๆฃ€ๆต‹: Vectara + NLI
โœ… ไผ˜ๅŒ–: ่‡ชๅŠจ้™็บงใ€้”™่ฏฏๅค„็†ใ€method_used ไธ€่‡ดๆ€ง
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
ไฝ ็š„้กน็›ฎไฝฟ็”จไบ†ไธš็•Œๆœ€ไฝณๅฎž่ทต๏ผ๐Ÿ†
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”
""")
print("\n" + "=" * 80)
print("โœ… NLI ๅนป่ง‰ๆฃ€ๆต‹ๅŽŸ็†่ฎฒ่งฃๅฎŒๆฏ•๏ผ")
print("=" * 80)
print()