Spaces:
Sleeping
Sleeping
Commit History
-
Heat code extractor v6: EasyOCR fallback for codes Tesseract can't read
Avinash NallaEasyOCR added as final Strategy 7 fallback in heat_code.py. Lazy- imported so the heavy torch dependency stays optional — code falls back gracefully if easyocr isn't installed at runtime. Reader is cached at module scope so model-load cost is paid once per process. Triggered only when (a) cheaper strategies have ≤2 candidates and (b) text_layer is thin, keeping perf intact for the common case. Target case: Flotite G18M. Tesseract cannot read this code at any DPI or PSM combination — verified locally across psm 3/4/6/7/8/11/12. EasyOCR's CRNN recognizer handles thin-font multi-column tables better than Tesseract's classical pipeline. Tradeoff: container image grows ~2GB (torch + opencv + scipy). User has authorized. Note: EasyOCR fallback path not tested in dev sandbox due to disk and proxy constraints. Validation must happen on HF rebuild against Flotite document.3453432 -
Heat code extractor v5: targeted cell-region OCR + aggressive false-positive filtering
Avinash NallaMajor recovery — MTR W000009810's '1ZZWN' is now extracted (was previously unrecoverable). Aggressive false-positive filtering brings WOI and Flotite to clean output. New: targeted cell-region OCR (Strategy 6). - Renders PDF at 600 DPI. - Uses image_to_data to find bbox of 'HEAT' words paired with 'CODE' / 'NO' / 'NR' / 'NUMBER' / '#' indicators (avoids cropping around 'HEAT TREATMENT' headers). - Crops ~1200x400 px adjacent regions. - Applies autocontrast + 3x contrast + double sharpen preprocessing. - OCRs each crop at psm 6. Recovers 1ZZWN from MTR W000009810 which no other strategy could. New filters: - Phone numbers (\d{3}-\d{3}-\d{4}). - Order numbers (single letter + 6+ digits, e.g. W000009810). - Section-break stop (unconditional) when scanning past 'HEAT' label encounters MECHANICAL / CHEMICAL / TENSILE / HARDNESS / address- block keywords (STREET / AVENUE / etc.). Stops false positives from leaking in from adjacent sections. - Pure-nu…b54d77d
-
Heat code extractor v4: kill A370-12a, N021-1 truncation, YYYYMMDD dates
Avinash NallaThree targeted polish fixes from the v3 staging result: 1) ASTM revision suffix: A370-12a was leaking through because the standards regex stopped at the revision number and didn't allow the trailing letter (ASTM revisions like '-12a' / '-15b' are common). Extended the ASTM and ASME patterns to consume an optional single trailing letter. 2) N021-1 was the OCR-truncated form of JN021-1 (Tesseract dropped the leading J). The substring dedup only caught strict-prefix duplicates; added strict-suffix dedup so 'N021-1' gets dropped when 'JN021-1' is also a candidate. Helps with any single-char OCR drop at the start of a code. 3) 20060101 (YYYYMMDD-style date) was getting accepted because my year regex only handles 4-digit standalone years. Added a rule to reject pure-numeric tokens with 8+ digits — dates, order numbers, batch IDs. Real heat codes top out at ~7 digits when pure numeric (10720 is 5). KB419-1 forced-OCR result is now 6 real codes plus 2 ac…d34c94f -
Heat code extractor v3: capture all 6 KB419-1 codes; reject OCR-mangled English; normalize O→0
Avinash NallaThree issues fixed from the staging test on KB419-1: 1) Missing KG416-1 and KS182-1: the OCR column heuristic was only scanning 3 lines after a 'Heat No.' header. KB419-1's table has 6 rows, so the last 3 were dropped. Bumped both the pdfplumber bbox-based scan (6 → 20 lines) and the OCR-text heuristic (3 → 20 non-empty lines) so multi-row tables are captured in full. 2) CON-REDUCER false positive: the previous 'isalpha()' check missed tokens with dashes. Replaced with a 'must contain at least one digit' rule that catches both pure-alpha (DIMENSION) and alpha+dash (CON-REDUCER, WP304-S) tokens with no digits. 3) JNO21-1 vs JN021-1 cosmetic OCR drift: Tesseract reads the digit 0 as letter O. For OCR-sourced candidates, replace O→0 in-place. None of our 14 confirmed real heat codes use letter O. Side effect of (1) was that the wider lookahead also caught a wave of OCR-mangled English words (ACC0RDING, BEL0W, FR0M, D0NE, ABO̧VE, etc.) where Tesseract turned …f9d6ac3
-
Tighten heat_code extractor: reject standards numbers + material specs
Avinash NallaProduction extractor was returning 8 codes from 10720.pdf when only 2 are real heat codes. False positives were fragments of standards references that the OCR fallback was grabbing because they appear near 'Heat Code' headers in the doc text: - 0103 from 'NACE MR 0103' - 0175 from 'MR 0175' - 2015 from 'ISO 9001:2015' - 15156 from 'ISO 15156' - 17945 from 'ISO 17945' - A106B from '1/2 X 4 XH A106B SMLS TBE NIPPLE' (material spec) Three fixes: 1) Expand year regex from 20[2-3]\d to 20[0-3]\d (catches 2015 etc). 2) Add material-spec pattern A\d{3}[A-Z] for A106B / A105N / A234B. Distinct from real heat codes like A8633 (4 digits, no trailing letter) so no regression on those. 3) New _collect_standards_numbers() scans the document text for ISO/NACE/ASTM/ASME/EN/DIN/API/AISI/JIS/MSS/PED prefixes and adds the trailing standard numbers to a reject set. The filter rejects candidates that match any of those tokens. Validated against all 10 sample MT…cd10e70
-
Phase 2+4: PO/Invoice client folder routing and subject-line MTR trigger
Avinash NallaPhase 2 — Client folder routing: - New table client_domains (domain primary key, client_name, audit fields). - New module app/lib/utils/client_routing.py with resolve_client_name() and add_or_update_client_domain() helpers. - processor.py: PO/Invoice uploads now insert {client_name}/ between the type path and the YYYY-MM folder. Unknown sender domains route to {type_path}/_unmatched/ and add a daily_brief row with reason 'client_unmatched' so ops can add the mapping without redeploy. - Daily brief gets a new 'Unmatched Sender Domains' section. - Alembic migration 0002 + matching DDL in database.py. - Seed script: scripts/seed_client_domains.{json,py} for initial mappings. Phase 4 — Subject-line MTR trigger: - Per project brief: any PDF attachment whose subject or filename contains 'MTR' (case-insensitive, word-boundary) routes as MTR without keyword scoring. Heat code extraction still runs.c55abce -
Phase 1: heat code extraction and multi-folder MTR routing
Avinash Nallad618a6e
-
Replace "tie" logic with "ambiguous" logic so that daily brief manual reviews happen when there is more than 1 doc type that meets confidence threshold
189d6c8
-
More classification keywords
a28aeac
-
Verbose logging to debug classification and new classification keywords and weights
a6c05d2 -
Update classification logic for purchase orders
7f6c50b
-
Error handling, confidence scoring, daily brief, deduplicate webhook notifications
0a367ef