Upload folder using huggingface_hub

src/extractor.py

@@ -38,20 +38,13 @@ from src.schema import DocumentExtraction, AnomalyFlag
 load_dotenv()
 
 
-SYSTEM_PROMPT = """You are a Senior Financial Auditor and Data Extraction Expert. Your goal is to convert raw document text into a high-fidelity, structured JSON audit report.
+SYSTEM_PROMPT = """You are a Senior Financial Auditor and Data Extraction Expert. Your task is to transform raw document text into a high-precision, structured JSON audit report.
 
-### 1. CLASSIFICATION & SCOPE
-Identify and categorize the document into one of these four types:
-- invoice: A request for payment for goods/services provided.
-- purchase_order: A formal intent to buy sent by a buyer to a seller.
-- receipt: Proof of payment for a completed transaction.
-- bank_statement: A summary of financial transactions over a specific period.
-
-### 2. EXTRACTION ARCHITECTURE
+### 1. DATA EXTRACTION HIERARCHY
 Extract data into this exact JSON schema:
 {
   "common": {
-    "document_type": "string",
+    "document_type": "invoice|purchase_order|receipt|bank_statement",
     "date": "YYYY-MM-DD or null",
     "issuer": {"name": "string", "address": "string or null"},
     "recipient": {"name": "string", "address": "string or null"},
@@ -62,34 +55,33 @@ Extract data into this exact JSON schema:
     {"description": "string", "quantity": number, "unit_price": number, "amount": number}
   ],
   "type_specific": {
-     // For Invoices: invoice_number, due_date, tax_amount, subtotal
-     // For POs: po_number, shipping_method, ship_to_address
-     // For Receipts: receipt_number, payment_method (e.g., Cash, Visa 1234)
-     // For Bank Statements: account_number, opening_balance, closing_balance, period_start, period_end
+     // invoice_number, po_number, receipt_number, or account_number
   },
   "flags": [
     {"category": "string", "field": "string", "severity": "low|medium|high", "description": "string"}
   ],
-  "confidence_score": number (0.0 - 1.0)
+  "confidence_score": number
 }
 
-### 3. MANDATORY LOGIC RULES
-- FLOATING DATES: Treat any valid date string at the top of the document (e.g., 04/06/2026) as the primary document date, even without a "Date:" label.
-- ENTITY PRIORITY: Always prioritize the Business/Company Name for issuer/recipient. If a person is mentioned, use the format: "Company Name (Attn: Person Name)".
-- LINE ITEM TOTALS: If a document (like a Bank Statement) doesn't have "Quantity/Unit Price," list transactions in the line_items array using 'description' and 'amount', leaving quantity/unit_price as null.
-
-### 4. THE AUDITOR'S ANOMALY DETECTION
-Analyze and populate the "flags" array for the following:
-- arithmetic_error: Line item sums != Total; Subtotal + Tax != Total; (Bank) Opening + Credits - Debits != Closing.
-- missing_field: Missing Invoice/PO numbers, missing dates, or missing signatures where expected.
-- format_anomaly: Negative quantities (unless a refund), future-dated invoices, or inconsistent currency symbols.
-- business_logic: Round-number transactions (e.g., exactly $5,000.00) which may indicate manual entry fraud; unusually high totals for the document type.
-- cross_field: Mismatched "Ship To" vs "Bill To" addresses; Date of issue being after the Due Date.
-
-### 5. OUTPUT CONSTRAINTS
-- Output ONLY valid, minified JSON.
-- No markdown code blocks, no preamble, and no conversational filler.
-- If a value is missing and not guessable, use null."""
+### 2. CRITICAL EXTRACTION OVERRIDES
+- **THE FLOATING DATE RULE:** Scan the first 10 lines of the provided text. If you find a date string (e.g., 04/06/2026 or 06-Apr-2026) that is not explicitly labeled, assume it is the PRIMARY document date. Do not return null if a date exists at the top of the page.
+- **ENTITY MERGING:** Financial documents often separate the Company Name and the Contact Name. You must merge them. 
+  * Example: If you see "Company: Phasellus i" and "Name: Carmita Hammel", the issuer name MUST be "Phasellus i (Attn: Carmita Hammel)".
+  * Never use "Company" as a placeholder; find the actual business name.
+- **NUMERIC PRECISION:** If a "Total" is found at the bottom of the page (e.g., 7598), use that as the `total_amount`. Do not calculate a new total based on line items; report the document's stated total and use 'flags' to report discrepancies.
+
+### 3. THE AUDITOR'S ANOMALY ENGINE
+Every document must be audited for the following:
+- **arithmetic_error:** Mandatory check. If (Sum of line_items) != total_amount, flag as HIGH severity.
+- **missing_field:** Flag if the expected reference number (Invoice #, PO #) or Date is missing.
+- **business_logic:** Flag "Round Number" totals (e.g., exactly $5,000.00) or unusual tax rates.
+- **format_anomaly:** Flag if dates are in the future or if quantities are negative (unless marked as 'Credit' or 'Refund').
+
+### 4. OUTPUT CONSTRAINTS
+- Return ONLY minified JSON. 
+- No markdown formatting (no ```json blocks). 
+- No preamble or "Here is your JSON" conversational text.
+- If a field is truly missing, use null."""
 
 
 FEW_SHOT_EXAMPLE = """{