Ram-090 Claude Opus 4.6 (1M context) commited on
Commit
00a7178
·
1 Parent(s): f433f81

Relax verification for text documents - support flexible claim matching

Browse files

- Verification now accepts: high similarity + neutral NLI, moderate similarity
+ entailment, or very high similarity alone
- Lowered default similarity threshold from 0.75 to 0.6 for paraphrased text
- Lowered firewall threshold from 0.8 to 0.6 for text document queries
- Fixes issue where valid PDF answers were marked as unsupported

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. config/settings.py +2 -2
  2. core/verifier.py +5 -3
config/settings.py CHANGED
@@ -22,11 +22,11 @@ GROQ_API_KEY = os.getenv("GROQ_API_KEY", "")
22
 
23
  # Semantic similarity threshold (theta_sim)
24
  # Claims with similarity below this are considered unsupported
25
- SIMILARITY_THRESHOLD = 0.75
26
 
27
  # Firewall threshold (tau)
28
  # Responses with SupportRatio below this trigger regeneration
29
- FIREWALL_THRESHOLD = 0.8
30
 
31
  # =============================================================================
32
  # DOCUMENT INGESTION PARAMETERS
 
22
 
23
  # Semantic similarity threshold (theta_sim)
24
  # Claims with similarity below this are considered unsupported
25
+ SIMILARITY_THRESHOLD = 0.6
26
 
27
  # Firewall threshold (tau)
28
  # Responses with SupportRatio below this trigger regeneration
29
+ FIREWALL_THRESHOLD = 0.6
30
 
31
  # =============================================================================
32
  # DOCUMENT INGESTION PARAMETERS
core/verifier.py CHANGED
@@ -282,10 +282,12 @@ class ClaimVerifier:
282
  hypothesis=claim.text
283
  )
284
 
285
- # Apply verification rule
 
286
  is_supported = (
287
- similarity_score >= self.similarity_threshold and
288
- entailment_label == 'ENTAILED'
 
289
  )
290
 
291
  # Update claim object
 
282
  hypothesis=claim.text
283
  )
284
 
285
+ # Apply verification rule:
286
+ # Supported if EITHER high similarity OR entailment confirms it
287
  is_supported = (
288
+ (similarity_score >= self.similarity_threshold and entailment_label in ('ENTAILED', 'NEUTRAL')) or
289
+ (similarity_score >= 0.5 and entailment_label == 'ENTAILED') or
290
+ (similarity_score >= 0.85)
291
  )
292
 
293
  # Update claim object