Mohibullah commited on
Commit
4919e03
ยท
1 Parent(s): 8170b9a

Fix OCR prompt: prioritize drug names over Bengali headers, multilingual prescription handling

Browse files
Files changed (1) hide show
  1. gradio_pharmacopilot_demo.py +39 -29
gradio_pharmacopilot_demo.py CHANGED
@@ -92,47 +92,57 @@ def is_controlled_substance(drug_name: str) -> bool:
92
 
93
  # โ”€โ”€ Prompts โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
94
  # Pass 1: MiniCPM-V reads ALL text from the prescription image
95
- FULL_OCR_PROMPT = """You are an OCR engine for medical prescriptions.
96
-
97
- Read ALL text visible in this prescription image. Include everything:
98
- - Printed headers, clinic names, hospital names, logo text
99
- - Patient information: name, address, date of birth, phone number
100
- - Prescriber/Doctor information: name, credentials, address, DEA number, NPI number, phone number, signature presence
101
- - Date of the prescription
102
- - ALL drug/medication names with their strengths and dosage forms
103
- - Directions for use (Sig) exactly as written - do NOT translate abbreviations
104
- - Quantity prescribed (numeric and written)
105
- - Number of refills authorized
106
- - Whether "Dispense As Written" or "No Substitution" is checked
107
- - Any other stamps, markings, or text
108
-
109
- Rules:
110
- - Output ALL text exactly as written on the prescription
111
- - Preserve the layout structure using line breaks
112
- - Do NOT interpret, correct spelling, or translate medical abbreviations
113
- - If text is illegible, write [ILLEGIBLE] in its place
114
- - If a section appears to be a signature, note it as [SIGNATURE PRESENT]
115
- - Include field labels (e.g., "Patient:", "Rx:", "Sig:") if visible
116
-
117
- Return the complete text extraction now."""
 
 
 
 
 
 
118
 
119
  # Pass 2: Nemotron structures the raw OCR into the clinical JSON schema
120
  STRUCTURING_PROMPT_TEMPLATE = """You are a HIPAA-compliant Clinical Data Extraction Agent.
121
 
122
- You have been given raw OCR text extracted from a medical prescription image. Your task is to parse this text into a structured JSON format.
123
 
124
  STRICT RULES:
125
- 1. ZERO HALLUCINATION: This is life-critical medical data. If a field is not found in the text, output null for its value. Do NOT guess or infer.
126
- 2. NO CLINICAL TRANSLATION: Extract the Sig (directions) EXACTLY as written. Do not expand abbreviations.
127
- 3. Assign a confidence_score (0.00 to 1.00) to every field based on how clearly it appeared in the OCR text.
128
- 4. Determine if the drug is a Controlled Substance (DEA Schedules II-V).
 
 
 
 
129
 
130
  RAW OCR TEXT:
131
  ---
132
  {ocr_text}
133
  ---
134
 
135
- Return ONLY a valid JSON object with this exact structure (no markdown, no explanation):
136
  {{
137
  "document_metadata": {{
138
  "is_controlled_substance": false,
 
92
 
93
  # โ”€โ”€ Prompts โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
94
  # Pass 1: MiniCPM-V reads ALL text from the prescription image
95
+ FULL_OCR_PROMPT = """You are a medical prescription OCR engine. This prescription may contain text in multiple scripts (e.g., Bengali, Hindi, Arabic, English). Your job is to read and transcribe ALL text.
96
+
97
+ PRIORITY ORDER โ€” read these sections first:
98
+ 1. HANDWRITTEN CONTENT: Drug names (Tab., Cap., Inj., Syp.), dosages (mg, ml), frequencies (1+0+1, BD, TDS), durations
99
+ 2. Patient name and age/date of birth (often near top, after "Name:" or similar)
100
+ 3. Date of prescription
101
+ 4. Doctor/Prescriber name and credentials (often printed at top or stamped at bottom)
102
+ 5. Clinic/Hospital name, address, phone numbers
103
+ 6. Any other printed or stamped text
104
+
105
+ OUTPUT FORMAT โ€” structure your output like this:
106
+ DOCTOR: [doctor name and credentials]
107
+ CLINIC: [clinic/hospital name and address]
108
+ PATIENT: [patient name]
109
+ DATE: [prescription date]
110
+ Rx:
111
+ 1) [drug name] [strength] โ€” [frequency/dosage instructions]
112
+ 2) [drug name] [strength] โ€” [frequency/dosage instructions]
113
+ ...
114
+ ADVICE: [any additional instructions, follow-up notes]
115
+ SIGNATURE: [PRESENT/NOT VISIBLE]
116
+
117
+ CRITICAL RULES:
118
+ - Drug names are almost always written in English/Latin script (e.g., Tab. Diclofenac, Cap. Omeprazole) even on non-English prescriptions. READ THEM CAREFULLY.
119
+ - Dosage patterns like "1+0+1", "2+0+2", "0+0+1" mean morning+afternoon+night doses
120
+ - Transcribe EXACTLY as written โ€” do NOT translate, correct spelling, or expand abbreviations
121
+ - Read ALL numbered items (โ‘ , โ‘ก, โ‘ข or 1), 2), 3) etc.)
122
+ - If text is illegible, write [ILLEGIBLE]
123
+ - Include ALL drugs โ€” prescriptions often have 3-10 medications listed"""
124
 
125
  # Pass 2: Nemotron structures the raw OCR into the clinical JSON schema
126
  STRUCTURING_PROMPT_TEMPLATE = """You are a HIPAA-compliant Clinical Data Extraction Agent.
127
 
128
+ You have been given raw OCR text extracted from a medical prescription image. Parse this text into structured JSON.
129
 
130
  STRICT RULES:
131
+ 1. ZERO HALLUCINATION: If a field is not found, output null. Do NOT guess.
132
+ 2. NO CLINICAL TRANSLATION: Extract Sig/directions EXACTLY as written (e.g., "2+0+2", "1 tab PO BID"). Do NOT expand.
133
+ 3. Assign confidence (0.00 to 1.00) based on clarity in the OCR text.
134
+ 4. For drug_name: extract the FIRST/PRIMARY drug prescribed (e.g., "Tab. Diclofenac" โ†’ "Diclofenac"). If multiple drugs, use the first one.
135
+ 5. For directions_sig: include the dosage pattern (e.g., "2+0+2" or "1+0+1") and any duration mentioned.
136
+ 6. Dosage forms: Tab. = tablets, Cap. = capsules, Syp. = syrup, Inj. = injection, Susp. = suspension.
137
+ 7. Look for patient name after "Name:" or "เฆจเฆพเฆฎ:" fields. Look for date after "Date:" or "เฆคเฆพเฆฐเฆฟเฆ–:".
138
+ 8. Doctor name is usually printed at the top or bottom of the prescription.
139
 
140
  RAW OCR TEXT:
141
  ---
142
  {ocr_text}
143
  ---
144
 
145
+ Return ONLY valid JSON (no markdown, no explanation):
146
  {{
147
  "document_metadata": {{
148
  "is_controlled_substance": false,