Add a document scanning endpoint with AI enhancements

Add a new `/docscan` endpoint to the API that performs auto-cropping, perspective correction, alignment, contrast enhancement, noise reduction, sharpening, and optional HD upscaling of document images.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: dc097ae8-2157-4d92-8d04-6b44128d6d7c
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: dd0bd260-40d9-4e6b-8be5-962fa7796efb
Replit-Commit-Screenshot-Url: https://storage.googleapis.com/screenshot-production-us-central1/01531b1e-f634-49fa-b952-38b1db7203b1/dc097ae8-2157-4d92-8d04-6b44128d6d7c/BHf9clb

.replit

@@ -37,3 +37,7 @@ externalPort = 80
 [[ports]]
 localPort = 38887
 externalPort = 3000
+
+[[ports]]
+localPort = 44343
+externalPort = 3001

README.md

@@ -10,13 +10,14 @@ license: mit
 
 # AI Image Processing API
 
-A comprehensive image processing API with multiple AI-powered features including super-resolution, background removal, and noise reduction.
+A comprehensive image processing API with multiple AI-powered features including super-resolution, background removal, noise reduction, and document scanning.
 
 ## Features
 
 - **Image Enhancement**: Upscale images 2x or 4x using Real-ESRGAN
 - **Background Removal**: Remove backgrounds using BiRefNet AI model via rembg
 - **Noise Reduction**: Reduce image noise using OpenCV Non-Local Means Denoising
+- **Document Scanning**: Auto-crop, align, and enhance document photos with AI
 - **RESTful API**: Full API with automatic OpenAPI/Swagger documentation
 - **Web Interface**: Simple drag-and-drop interface for testing
 
@@ -46,6 +47,24 @@ Reduce image noise using Non-Local Means Denoising.
 - `file`: Image file
 - `strength`: Denoising strength (1-30, default: 10)
 
+### Document Scanning
+#### `POST /docscan`
+Scan and enhance document images with AI-powered processing.
+
+**Features:**
+- Auto-detection of document edges
+- Auto-crop and perspective correction
+- Alignment and straightening
+- CLAHE contrast enhancement
+- Bilateral noise reduction (preserves edges)
+- Unsharp mask sharpening
+- Optional HD upscaling with Real-ESRGAN
+
+**Parameters:**
+- `file`: Document image (PNG, JPG, JPEG, WebP, BMP)
+- `enhance_hd`: Enable AI HD enhancement (default: true)
+- `scale`: Upscale factor 1-4 (default: 2)
+
 ### Other Endpoints
 - `GET /docs` - Interactive Swagger UI documentation
 - `GET /redoc` - ReDoc documentation
@@ -59,6 +78,7 @@ Reduce image noise using Non-Local Means Denoising.
 | Super Resolution | Real-ESRGAN x4plus | State-of-the-art image upscaling |
 | Background Removal | BiRefNet-general | High-accuracy segmentation via rembg |
 | Noise Reduction | OpenCV NLM | Non-Local Means Denoising |
+| Document Scanning | OpenCV + Real-ESRGAN | Edge detection, perspective correction, HD enhancement |
 
 ## Local Development
 
@@ -126,6 +146,21 @@ with open("denoised.png", "wb") as f:
     f.write(response.content)
 ```
 
+### Python - Document Scanning
+```python
+import requests
+
+with open("document_photo.jpg", "rb") as f:
+    response = requests.post(
+        "https://your-space.hf.space/docscan",
+        files={"file": f},
+        params={"enhance_hd": True, "scale": 2}
+    )
+    
+with open("scanned_document.png", "wb") as f:
+    f.write(response.content)
+```
+
 ### cURL Examples
 ```bash
 # Enhance image
@@ -139,6 +174,10 @@ curl -X POST "https://your-space.hf.space/remove-background?bgcolor=transparent"
 # Denoise image
 curl -X POST "https://your-space.hf.space/denoise?strength=10" \
   -F "file=@noisy.jpg" -o denoised.png
+
+# Scan document
+curl -X POST "https://your-space.hf.space/docscan?enhance_hd=true&scale=2" \
+  -F "file=@document.jpg" -o scanned.png
 ```
 
 ## License

app.py

@@ -25,6 +25,7 @@ A comprehensive image processing API with multiple AI-powered features.
 - **Image Upscaling**: Enhance image resolution up to 4x using Real-ESRGAN
 - **Background Removal**: Remove backgrounds using rembg with BiRefNet model
 - **Noise Reduction**: Reduce image noise using advanced denoising algorithms
+- **Document Scanning**: Auto-crop, align, and enhance document photos with AI
 - **Quality Enhancement**: Improve image clarity and reduce artifacts
 
 ### Supported Formats:
@@ -34,8 +35,9 @@ A comprehensive image processing API with multiple AI-powered features.
 - **Super Resolution**: Real-ESRGAN x4plus
 - **Background Removal**: rembg with BiRefNet-massive model
 - **Noise Reduction**: OpenCV Non-Local Means Denoising
+- **Document Scanner**: OpenCV edge detection + Real-ESRGAN upscaling
     """,
-    version="2.0.0",
+    version="2.1.0",
     docs_url="/docs",
     redoc_url="/redoc",
 )
@@ -87,7 +89,7 @@ async def health_check():
     return {
         "status": "healthy", 
         "version": "2.0.0",
-        "features": ["enhance", "remove-background", "denoise"]
+        "features": ["enhance", "remove-background", "denoise", "docscan"]
     }
 
 @app.get("/model-info")
@@ -110,6 +112,12 @@ async def model_info():
                 "name": "Non-Local Means Denoising",
                 "description": "Advanced noise reduction algorithm",
                 "source": "OpenCV"
+            },
+            "document_scanner": {
+                "name": "AI Document Scanner",
+                "description": "Auto-crop, perspective correction, alignment, and HD enhancement",
+                "features": ["edge detection", "perspective transform", "CLAHE contrast", "bilateral denoising", "unsharp masking", "Real-ESRGAN upscaling"],
+                "source": "OpenCV + Real-ESRGAN"
             }
         },
         "supported_formats": ["png", "jpg", "jpeg", "webp", "bmp"],
@@ -476,6 +484,148 @@ async def denoise_image_base64(
     except Exception as e:
         raise HTTPException(status_code=500, detail=f"Error denoising image: {str(e)}")
 
+
+doc_scanner = None
+
+def get_doc_scanner():
+    global doc_scanner
+    if doc_scanner is None:
+        from document_scanner import get_document_scanner
+        doc_scanner = get_document_scanner()
+    return doc_scanner
+
+
+@app.post("/docscan")
+async def scan_document(
+    file: UploadFile = File(..., description="Document image to scan (PNG, JPG, JPEG, WebP, BMP)"),
+    enhance_hd: bool = Query(default=True, description="Apply HD enhancement using AI (Real-ESRGAN)"),
+    scale: int = Query(default=2, ge=1, le=4, description="Upscale factor for HD enhancement (1-4)")
+):
+    """
+    Scan and enhance a document image with AI-powered processing.
+    
+    This endpoint performs:
+    - **Auto-detection**: Finds document edges automatically using edge detection
+    - **Auto-crop**: Removes background and crops to document boundaries
+    - **Perspective correction**: Straightens tilted or skewed documents
+    - **Alignment**: Ensures the document is properly aligned
+    - **Contrast enhancement**: Applies CLAHE for improved readability
+    - **Noise reduction**: Uses bilateral filtering to reduce noise while preserving edges
+    - **Sharpening**: Applies unsharp masking for crisp text without artifacts
+    - **HD upscaling**: Optionally uses Real-ESRGAN for high-definition output
+    
+    Parameters:
+    - **file**: Upload a photo of a document (supports various angles and lighting)
+    - **enhance_hd**: Enable AI-powered HD enhancement (default: True)
+    - **scale**: Upscaling factor 1-4 (default: 2 for balanced quality/size)
+    
+    Returns the scanned document as a high-quality PNG file.
+    """
+    allowed_types = ["image/png", "image/jpeg", "image/jpg", "image/webp", "image/bmp"]
+    if file.content_type not in allowed_types:
+        raise HTTPException(
+            status_code=400, 
+            detail=f"Invalid file type. Allowed types: {', '.join(allowed_types)}"
+        )
+    
+    try:
+        contents = await file.read()
+        input_image = Image.open(io.BytesIO(contents))
+        
+        if input_image.mode != "RGB":
+            input_image = input_image.convert("RGB")
+        
+        max_size = 2048
+        if input_image.width > max_size or input_image.height > max_size:
+            ratio = min(max_size / input_image.width, max_size / input_image.height)
+            new_size = (int(input_image.width * ratio), int(input_image.height * ratio))
+            input_image = input_image.resize(new_size, Image.LANCZOS)
+        
+        original_size = {"width": input_image.width, "height": input_image.height}
+        
+        scanner = get_doc_scanner()
+        scanned_image = scanner.process_document(input_image, enhance_hd=enhance_hd, scale=scale)
+        
+        file_id = str(uuid.uuid4())
+        output_path = OUTPUT_DIR / f"{file_id}_scanned.png"
+        scanned_image.save(output_path, "PNG", optimize=True)
+        
+        return FileResponse(
+            output_path,
+            media_type="image/png",
+            filename=f"scanned_{file.filename.rsplit('.', 1)[0]}.png"
+        )
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Error scanning document: {str(e)}")
+
+
+@app.post("/docscan/base64")
+async def scan_document_base64(
+    file: UploadFile = File(..., description="Document image to scan"),
+    enhance_hd: bool = Query(default=True, description="Apply HD enhancement using AI"),
+    scale: int = Query(default=2, ge=1, le=4, description="Upscale factor for HD enhancement (1-4)")
+):
+    """
+    Scan and enhance a document image, returning the result as base64.
+    
+    Same processing as /docscan but returns base64-encoded image data.
+    Useful for integrations that prefer base64 over file downloads.
+    """
+    import base64
+    
+    allowed_types = ["image/png", "image/jpeg", "image/jpg", "image/webp", "image/bmp"]
+    if file.content_type not in allowed_types:
+        raise HTTPException(
+            status_code=400, 
+            detail=f"Invalid file type. Allowed types: {', '.join(allowed_types)}"
+        )
+    
+    try:
+        contents = await file.read()
+        input_image = Image.open(io.BytesIO(contents))
+        
+        if input_image.mode != "RGB":
+            input_image = input_image.convert("RGB")
+        
+        max_size = 2048
+        if input_image.width > max_size or input_image.height > max_size:
+            ratio = min(max_size / input_image.width, max_size / input_image.height)
+            new_size = (int(input_image.width * ratio), int(input_image.height * ratio))
+            input_image = input_image.resize(new_size, Image.LANCZOS)
+        
+        original_size = {"width": input_image.width, "height": input_image.height}
+        
+        scanner = get_doc_scanner()
+        scanned_image = scanner.process_document(input_image, enhance_hd=enhance_hd, scale=scale)
+        
+        buffer = io.BytesIO()
+        scanned_image.save(buffer, format="PNG", optimize=True)
+        buffer.seek(0)
+        
+        img_base64 = base64.b64encode(buffer.getvalue()).decode("utf-8")
+        
+        return JSONResponse({
+            "success": True,
+            "image_base64": img_base64,
+            "original_size": original_size,
+            "scanned_size": {"width": scanned_image.width, "height": scanned_image.height},
+            "enhance_hd": enhance_hd,
+            "scale_factor": scale,
+            "processing": {
+                "auto_crop": True,
+                "perspective_correction": True,
+                "contrast_enhancement": "CLAHE",
+                "noise_reduction": "bilateral_filter",
+                "sharpening": "unsharp_mask",
+                "hd_upscaling": "Real-ESRGAN" if enhance_hd else "disabled"
+            }
+        })
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Error scanning document: {str(e)}")
+
+
 if __name__ == "__main__":
     import uvicorn
     uvicorn.run(app, host="0.0.0.0", port=7860)

document_scanner.py

@@ -0,0 +1,200 @@
+import cv2
+import numpy as np
+from PIL import Image, ImageEnhance, ImageFilter
+
+class DocumentScanner:
+    def __init__(self):
+        pass
+    
+    def order_points(self, pts):
+        rect = np.zeros((4, 2), dtype="float32")
+        s = pts.sum(axis=1)
+        rect[0] = pts[np.argmin(s)]
+        rect[2] = pts[np.argmax(s)]
+        diff = np.diff(pts, axis=1)
+        rect[1] = pts[np.argmin(diff)]
+        rect[3] = pts[np.argmax(diff)]
+        return rect
+
+    def four_point_transform(self, image, pts):
+        rect = self.order_points(pts)
+        (tl, tr, br, bl) = rect
+        
+        widthA = np.sqrt(((br[0] - bl[0]) ** 2) + ((br[1] - bl[1]) ** 2))
+        widthB = np.sqrt(((tr[0] - tl[0]) ** 2) + ((tr[1] - tl[1]) ** 2))
+        maxWidth = max(int(widthA), int(widthB))
+        
+        heightA = np.sqrt(((tr[0] - br[0]) ** 2) + ((tr[1] - br[1]) ** 2))
+        heightB = np.sqrt(((tl[0] - bl[0]) ** 2) + ((tl[1] - bl[1]) ** 2))
+        maxHeight = max(int(heightA), int(heightB))
+        
+        dst = np.array([
+            [0, 0],
+            [maxWidth - 1, 0],
+            [maxWidth - 1, maxHeight - 1],
+            [0, maxHeight - 1]], dtype="float32")
+        
+        M = cv2.getPerspectiveTransform(rect, dst)
+        warped = cv2.warpPerspective(image, M, (maxWidth, maxHeight))
+        return warped
+
+    def detect_document(self, image):
+        orig = image.copy()
+        height, width = image.shape[:2]
+        
+        ratio = height / 500.0
+        new_width = int(width / ratio)
+        resized = cv2.resize(image, (new_width, 500))
+        
+        gray = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
+        
+        blurred = cv2.GaussianBlur(gray, (5, 5), 0)
+        
+        edged = cv2.Canny(blurred, 50, 200)
+        
+        kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
+        edged = cv2.dilate(edged, kernel, iterations=1)
+        
+        contours, _ = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
+        contours = sorted(contours, key=cv2.contourArea, reverse=True)[:10]
+        
+        screen_cnt = None
+        for c in contours:
+            peri = cv2.arcLength(c, True)
+            approx = cv2.approxPolyDP(c, 0.02 * peri, True)
+            
+            if len(approx) == 4:
+                screen_cnt = approx
+                break
+        
+        if screen_cnt is None:
+            edge_margin = 0.02
+            h, w = resized.shape[:2]
+            margin_x = int(w * edge_margin)
+            margin_y = int(h * edge_margin)
+            screen_cnt = np.array([
+                [[margin_x, margin_y]],
+                [[w - margin_x, margin_y]],
+                [[w - margin_x, h - margin_y]],
+                [[margin_x, h - margin_y]]
+            ])
+        
+        return screen_cnt.reshape(4, 2) * ratio
+
+    def auto_crop_and_align(self, image):
+        if isinstance(image, Image.Image):
+            image = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
+        
+        doc_contour = self.detect_document(image)
+        
+        warped = self.four_point_transform(image, doc_contour)
+        
+        return warped
+
+    def enhance_sharpness(self, image, amount=1.5):
+        if isinstance(image, np.ndarray):
+            pil_image = Image.fromarray(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
+        else:
+            pil_image = image
+        
+        blurred = pil_image.filter(ImageFilter.GaussianBlur(radius=1))
+        
+        blurred_np = np.array(blurred).astype(np.float32)
+        original_np = np.array(pil_image).astype(np.float32)
+        
+        sharpened = original_np + (original_np - blurred_np) * amount
+        sharpened = np.clip(sharpened, 0, 255).astype(np.uint8)
+        
+        return Image.fromarray(sharpened)
+
+    def adaptive_contrast(self, image):
+        if isinstance(image, Image.Image):
+            image = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
+        
+        lab = cv2.cvtColor(image, cv2.COLOR_BGR2LAB)
+        l, a, b = cv2.split(lab)
+        
+        clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
+        l = clahe.apply(l)
+        
+        lab = cv2.merge([l, a, b])
+        result = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)
+        
+        return result
+
+    def denoise_preserve_details(self, image, strength=3):
+        if isinstance(image, Image.Image):
+            image = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
+        
+        denoised = cv2.bilateralFilter(image, 9, strength * 10, strength * 10)
+        
+        return denoised
+
+    def process_document(self, pil_image, enhance_hd=True, scale=2):
+        img_array = np.array(pil_image)
+        if len(img_array.shape) == 2:
+            img_array = cv2.cvtColor(img_array, cv2.COLOR_GRAY2BGR)
+        else:
+            img_array = cv2.cvtColor(img_array, cv2.COLOR_RGB2BGR)
+        
+        cropped = self.auto_crop_and_align(img_array)
+        
+        denoised = self.denoise_preserve_details(cropped, strength=2)
+        
+        contrasted = self.adaptive_contrast(denoised)
+        
+        result_rgb = cv2.cvtColor(contrasted, cv2.COLOR_BGR2RGB)
+        result_pil = Image.fromarray(result_rgb)
+        
+        sharpened = self.enhance_sharpness(result_pil, amount=0.8)
+        
+        enhancer = ImageEnhance.Brightness(sharpened)
+        brightened = enhancer.enhance(1.05)
+        
+        if enhance_hd:
+            try:
+                from enhancer import ImageEnhancer
+                ai_enhancer = ImageEnhancer()
+                hd_image = ai_enhancer.enhance(brightened, scale=scale)
+                return hd_image
+            except Exception as e:
+                print(f"AI enhancement not available: {e}")
+                new_size = (brightened.width * scale, brightened.height * scale)
+                hd_image = brightened.resize(new_size, Image.LANCZOS)
+                return self.enhance_sharpness(hd_image, amount=0.5)
+        
+        return brightened
+
+
+class FallbackDocumentScanner:
+    def process_document(self, pil_image, enhance_hd=True, scale=2):
+        if pil_image.mode != "RGB":
+            pil_image = pil_image.convert("RGB")
+        
+        enhancer = ImageEnhance.Contrast(pil_image)
+        contrasted = enhancer.enhance(1.15)
+        
+        enhancer = ImageEnhance.Sharpness(contrasted)
+        sharpened = enhancer.enhance(1.3)
+        
+        enhancer = ImageEnhance.Brightness(sharpened)
+        brightened = enhancer.enhance(1.05)
+        
+        if enhance_hd:
+            new_size = (brightened.width * scale, brightened.height * scale)
+            hd_image = brightened.resize(new_size, Image.LANCZOS)
+            
+            enhancer = ImageEnhance.Sharpness(hd_image)
+            final = enhancer.enhance(1.2)
+            return final
+        
+        return brightened
+
+
+def get_document_scanner():
+    try:
+        import cv2
+        return DocumentScanner()
+    except ImportError:
+        print("OpenCV not available, using fallback scanner")
+        return FallbackDocumentScanner()

replit.md

@@ -5,6 +5,7 @@ An AI-powered image processing API with multiple features:
 - Image enhancement/upscaling using Real-ESRGAN
 - Background removal using BiRefNet via rembg
 - Noise reduction using OpenCV Non-Local Means Denoising
+- Document scanning with auto-crop, alignment, and HD enhancement
 - FastAPI backend with automatic Swagger API documentation
 - Simple web frontend for testing
 
@@ -18,6 +19,7 @@ An AI-powered image processing API with multiple features:
 ├── app.py              # Full FastAPI app for Hugging Face deployment
 ├── app_local.py        # Lightweight local preview server
 ├── enhancer.py         # Real-ESRGAN model wrapper (for HF deployment)
+├── document_scanner.py # Document scanning with OpenCV (auto-crop, align, enhance)
 ├── templates/
 │   └── index.html      # Frontend interface
 ├── requirements.txt    # Dependencies for Hugging Face Spaces
@@ -38,14 +40,33 @@ An AI-powered image processing API with multiple features:
 - `POST /remove-background/base64` - Remove background (returns base64)
 - `POST /denoise` - Reduce image noise (OpenCV NLM)
 - `POST /denoise/base64` - Denoise image (returns base64)
+- `POST /docscan` - Scan document (auto-crop, align, HD enhance)
+- `POST /docscan/base64` - Scan document (returns base64)
+
+## Document Scanner Features
+The `/docscan` endpoint provides:
+- **Auto-detection**: Edge detection using Canny algorithm
+- **Auto-crop**: Contour detection and perspective correction
+- **Alignment**: Four-point perspective transform
+- **Contrast**: CLAHE (Contrast Limited Adaptive Histogram Equalization)
+- **Denoising**: Bilateral filter (preserves edges while reducing noise)
+- **Sharpening**: Unsharp masking for crisp text
+- **HD Upscaling**: Optional Real-ESRGAN enhancement (1-4x scale)
 
 ## Deploying to Hugging Face Spaces
 1. Create a new Space on Hugging Face
 2. Select "Docker" as the SDK
-3. Upload all files: `app.py`, `enhancer.py`, `templates/`, `requirements.txt`, `Dockerfile`, `README.md`
+3. Upload all files: `app.py`, `enhancer.py`, `document_scanner.py`, `templates/`, `requirements.txt`, `Dockerfile`, `README.md`
 4. The Space will auto-build the container and download AI models
 
 ## Recent Changes
+- 2025-11-28: Added document scanning feature
+  - Auto-crop with edge detection and contour finding
+  - Perspective correction for skewed documents
+  - CLAHE contrast enhancement
+  - Bilateral filter denoising (preserves details)
+  - Unsharp mask sharpening
+  - Optional HD upscaling with Real-ESRGAN
 - 2025-11-28: Added background removal and noise reduction features
   - BiRefNet integration via rembg for background removal
   - OpenCV Non-Local Means Denoising