Upload folder using huggingface_hub

DEPLOY.md

@@ -0,0 +1,181 @@
+# Deploying LearningStudio Wrapper to Hugging Face
+
+This guide explains how to deploy the LearningStudio callout detection wrapper to a HuggingFace Inference Endpoint.
+
+## Prerequisites
+
+1. **HuggingFace Account**: Create an account at [huggingface.co](https://huggingface.co)
+2. **HuggingFace CLI**: Install the CLI tool
+3. **AWS Infrastructure**: The callout detection Lambda stack must be deployed
+
+### Install HuggingFace CLI
+
+```bash
+pip install huggingface_hub
+```
+
+### Login to HuggingFace
+
+```bash
+huggingface-cli login
+```
+
+Follow the prompts to enter your HuggingFace token.
+
+## Step 1: Get AWS API Gateway Info
+
+After deploying the callout detection Lambda stack, get the API Gateway URL and key:
+
+```bash
+cd callout-detection-lambda
+
+# Get the API Gateway endpoint URL
+aws cloudformation describe-stacks \
+    --stack-name callout-detection-dev \
+    --query "Stacks[0].Outputs[?OutputKey=='ServiceEndpoint'].OutputValue" \
+    --output text
+
+# Get the API key
+aws apigateway get-api-keys \
+    --name-query "learningstudio-key-dev" \
+    --include-values \
+    --query "items[0].value" \
+    --output text
+```
+
+Save these values - you'll need them when configuring the HF endpoint.
+
+## Step 2: Create HuggingFace Model Repository
+
+First time only - create the model repository:
+
+```bash
+huggingface-cli repo create YOUR_USERNAME/learningstudio-callout-wrapper --type model
+```
+
+Or create via the HuggingFace web interface at https://huggingface.co/new
+
+## Step 3: Upload Wrapper Files
+
+Navigate to the wrapper directory and upload files:
+
+```bash
+cd callout-detection-lambda/hf_inference/learningstudio_wrapper
+
+# Upload all files to the repository
+huggingface-cli upload YOUR_USERNAME/learningstudio-callout-wrapper \
+    handler.py inference.py requirements.txt README.md \
+    --repo-type model
+```
+
+## Step 4: Create Inference Endpoint
+
+1. Go to https://ui.endpoints.huggingface.co/
+2. Click "New endpoint"
+3. Select your model repository (`YOUR_USERNAME/learningstudio-callout-wrapper`)
+4. Configure the endpoint:
+   - **Instance type**: CPU (this wrapper doesn't need GPU)
+   - **Region**: Choose a region close to your API Gateway
+   - **Scaling**: Start with 1 replica
+
+## Step 5: Configure Secrets
+
+In the HuggingFace Inference Endpoint settings, add environment variables:
+
+1. Go to your endpoint settings
+2. Click "Settings" or "Environment Variables"
+3. Add the following secrets:
+
+| Name | Value |
+|------|-------|
+| `API_GATEWAY_URL` | `https://xxx.execute-api.us-east-1.amazonaws.com/dev` |
+| `API_KEY` | Your API key from Step 1 |
+
+## Step 6: Test the Endpoint
+
+Once the endpoint is running, test it:
+
+```bash
+# Set your HuggingFace token
+export HF_TOKEN="your-hf-token"
+
+# Test with a URL
+curl -X POST https://YOUR_ENDPOINT.endpoints.huggingface.cloud \
+  -H "Authorization: Bearer $HF_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"inputs": "https://example.com/test-drawing.png"}'
+```
+
+Expected response:
+
+```json
+{
+  "predictions": [
+    {
+      "id": 1,
+      "label": "callout",
+      "class_id": 0,
+      "confidence": 0.95,
+      "bbox": {"x1": 100, "y1": 200, "x2": 300, "y2": 400}
+    }
+  ],
+  "total_detections": 1,
+  "image": "...",
+  "image_width": 1920,
+  "image_height": 1080
+}
+```
+
+## Updating the Wrapper
+
+To update the wrapper code:
+
+```bash
+cd callout-detection-lambda/hf_inference/learningstudio_wrapper
+
+# Upload updated files
+huggingface-cli upload YOUR_USERNAME/learningstudio-callout-wrapper \
+    handler.py inference.py requirements.txt README.md \
+    --repo-type model
+```
+
+The endpoint will automatically pick up the changes on the next request (after a brief cold start).
+
+## Rotating API Keys
+
+To rotate the API key without touching the HF endpoint:
+
+1. Create a new API key in AWS API Gateway
+2. Update the `API_KEY` secret in HF endpoint settings
+3. Delete the old API key in AWS
+
+## Troubleshooting
+
+### "API_GATEWAY_URL and API_KEY must be set"
+
+The environment variables are not configured. Go to your endpoint settings and add the secrets.
+
+### Timeout errors
+
+The callout detection pipeline takes 30-120 seconds typically. If you're getting timeouts:
+- Check that the Lambda stack is deployed and working
+- Verify the API Gateway URL is correct
+- Check CloudWatch logs for the Lambda functions
+
+### Authentication errors
+
+- Verify the API key is correct
+- Check that the key hasn't been deleted or rotated
+- Ensure the key is associated with the usage plan
+
+### Connection refused
+
+- Verify the API Gateway URL is correct
+- Check that the endpoint is in the right region
+- Ensure the Lambda stack is deployed
+
+## Monitoring
+
+- **HuggingFace**: Check endpoint logs in the HF dashboard
+- **AWS CloudWatch**: Monitor Lambda function logs and metrics
+- **API Gateway**: View API Gateway metrics for request counts and errors

README.md

@@ -0,0 +1,150 @@
+---
+tags:
+- object-detection
+- callout-detection
+- architectural-drawings
+- wrapper
+library_name: custom
+task: object-detection
+license: apache-2.0
+---
+
+# LearningStudio Callout Detection Wrapper
+
+Wrapper for the Lambda-based callout detection pipeline, providing EMCO-compatible API format for LearningStudio integration.
+
+## Overview
+
+This wrapper:
+1. Accepts image input in multiple formats (URL, base64, data URL)
+2. Gets a presigned S3 URL from API Gateway
+3. Uploads the image directly to S3 (avoids API Gateway data transfer costs)
+4. Starts the detection job via API Gateway (small JSON payload)
+5. Polls for completion
+6. Transforms results to EMCO-compatible format
+
+## Architecture
+
+```
+HF Wrapper
+    │
+    ├─1─▶ GET /upload-url (get presigned S3 URL)
+    │
+    ├─2─▶ PUT image directly to S3 (free, bypasses API Gateway)
+    │
+    ├─3─▶ POST /detect {"job_id", "s3_url"} (tiny payload)
+    │
+    └─4─▶ GET /status/{job_id} (poll until complete)
+```
+
+## API Format
+
+### Input
+
+Accepts images in multiple formats:
+
+```json
+// HTTP URL
+{"inputs": "https://example.com/image.jpg"}
+
+// Data URL (base64 encoded)
+{"inputs": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."}
+
+// Raw base64
+{"inputs": "iVBORw0KGgoAAAANSUhEUgAAAAUA..."}
+```
+
+### Output
+
+Returns EMCO-compatible format:
+
+```json
+{
+  "predictions": [
+    {
+      "id": 1,
+      "label": "callout",
+      "class_id": 0,
+      "confidence": 0.95,
+      "bbox": {
+        "x1": 100,
+        "y1": 200,
+        "x2": 300,
+        "y2": 400
+      }
+    }
+  ],
+  "total_detections": 1,
+  "image": "base64_encoded_image",
+  "image_width": 1920,
+  "image_height": 1080
+}
+```
+
+### Bounding Box Format
+
+- **Input from Lambda**: `[x, y, width, height]` (xywh format)
+- **Output to LearningStudio**: `{"x1", "y1", "x2", "y2"}` (xyxy format)
+
+The wrapper automatically converts between these formats.
+
+## Configuration
+
+This endpoint requires the following secrets to be configured in HuggingFace Inference Endpoint settings:
+
+| Secret | Description |
+|--------|-------------|
+| `API_GATEWAY_URL` | Full URL of the API Gateway endpoint (e.g., `https://xxx.execute-api.us-east-1.amazonaws.com/dev`) |
+| `API_KEY` | API key for authentication |
+
+## Usage
+
+### Python
+
+```python
+import requests
+
+HF_ENDPOINT = "https://your-endpoint.endpoints.huggingface.cloud"
+HF_TOKEN = "your-hf-token"
+
+response = requests.post(
+    HF_ENDPOINT,
+    headers={"Authorization": f"Bearer {HF_TOKEN}"},
+    json={"inputs": "https://example.com/architectural-drawing.png"}
+)
+
+result = response.json()
+print(f"Found {result['total_detections']} callouts")
+for pred in result["predictions"]:
+    print(f"  Callout {pred['id']}: {pred['bbox']}, confidence={pred['confidence']}")
+```
+
+### cURL
+
+```bash
+curl -X POST https://your-endpoint.endpoints.huggingface.cloud \
+  -H "Authorization: Bearer $HF_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"inputs": "https://example.com/architectural-drawing.png"}'
+```
+
+## Processing Time
+
+Typical processing time is 30-120 seconds depending on image size and complexity. The wrapper polls the backend every 5 seconds with a maximum timeout of 15 minutes.
+
+## Error Handling
+
+Errors are returned in a consistent format:
+
+```json
+{
+  "error": "Description of the error",
+  "predictions": [],
+  "total_detections": 0,
+  "image": ""
+}
+```
+
+## License
+
+Apache 2.0

handler.py

@@ -0,0 +1,74 @@
+"""
+HuggingFace Inference Endpoint Handler for LearningStudio Callout Detection.
+
+This wrapper provides an EMCO-compatible API format for LearningStudio integration,
+calling the AWS Lambda-based callout detection pipeline via API Gateway.
+"""
+
+from typing import Dict, Any, List, Union
+from inference import inference, normalize_to_base64
+
+
+class EndpointHandler:
+    """
+    HuggingFace Inference Endpoint Handler.
+
+    This class provides the interface expected by HuggingFace Inference Endpoints.
+    It wraps the callout detection pipeline and transforms outputs to EMCO format.
+    """
+
+    def __init__(self, path: str = ""):
+        """
+        Initialize the endpoint handler.
+
+        Args:
+            path: Model path (unused for this wrapper, but required by HF interface)
+        """
+        # No model to load - this is a wrapper for an external API
+        pass
+
+    def __call__(self, data: Dict[str, Any]) -> Dict[str, Any]:
+        """
+        Process an inference request.
+
+        Args:
+            data: Request data with format:
+                {
+                    "inputs": "image_url_or_base64",
+                    "parameters": {...}  # Optional parameters
+                }
+
+        Returns:
+            EMCO-compatible response:
+            {
+                "predictions": [
+                    {
+                        "id": 1,
+                        "label": "callout",
+                        "class_id": 0,
+                        "confidence": 0.95,
+                        "bbox": {"x1": 100, "y1": 200, "x2": 300, "y2": 400}
+                    },
+                    ...
+                ],
+                "total_detections": N,
+                "image": "base64_encoded_image"
+            }
+        """
+        # Extract input
+        inputs = data.get("inputs")
+        if inputs is None:
+            return {
+                "error": "Missing 'inputs' field",
+                "predictions": [],
+                "total_detections": 0,
+                "image": ""
+            }
+
+        # Extract optional parameters
+        parameters = data.get("parameters", {})
+
+        # Call the inference function
+        result = inference(inputs, parameters)
+
+        return result

inference.py

@@ -0,0 +1,364 @@
+"""
+Inference module for LearningStudio Callout Detection wrapper.
+
+This module:
+1. Normalizes input to bytes (handles URLs, data URLs, raw base64)
+2. Gets presigned S3 URL from API Gateway
+3. Uploads image directly to S3 (bypasses API Gateway for large payloads)
+4. Calls API Gateway to start detection job
+5. Polls for completion
+6. Transforms callouts to EMCO format
+"""
+
+import os
+import base64
+import time
+import logging
+from typing import Dict, Any, List, Optional, Tuple
+
+import requests
+
+# Configure logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Environment variables (set in HF Inference Endpoint secrets)
+API_GATEWAY_URL = os.environ.get("API_GATEWAY_URL", "")
+API_KEY = os.environ.get("API_KEY", "")
+
+# Polling configuration
+MAX_WAIT_SECONDS = 900  # 15 minutes
+POLL_INTERVAL_SECONDS = 5
+
+
+def normalize_to_bytes(image_input: str) -> Tuple[bytes, str]:
+    """
+    Normalize image input to bytes.
+
+    Handles:
+    - HTTP/HTTPS URLs: Downloads image
+    - Data URLs (data:image/png;base64,...): Decodes base64
+    - Raw base64: Decodes to bytes
+
+    Args:
+        image_input: Image URL, data URL, or base64 string
+
+    Returns:
+        Tuple of (image_bytes, filename)
+    """
+    # Check if it's a URL
+    if image_input.startswith(("http://", "https://")):
+        logger.info(f"Downloading image from URL: {image_input[:100]}...")
+        response = requests.get(image_input, timeout=60)
+        response.raise_for_status()
+
+        # Try to get filename from URL
+        from urllib.parse import urlparse
+        parsed = urlparse(image_input)
+        filename = os.path.basename(parsed.path) or "image.png"
+
+        return response.content, filename
+
+    # Check if it's a data URL
+    if image_input.startswith("data:"):
+        # Parse data URL: data:image/png;base64,<data>
+        try:
+            header, encoded = image_input.split(",", 1)
+            # Extract extension from mime type
+            mime_part = header.split(";")[0].replace("data:", "")
+            ext = mime_part.split("/")[-1] if "/" in mime_part else "png"
+            return base64.b64decode(encoded), f"image.{ext}"
+        except ValueError:
+            raise ValueError("Invalid data URL format")
+
+    # Assume it's already base64
+    try:
+        return base64.b64decode(image_input), "image.png"
+    except Exception as e:
+        raise ValueError(f"Invalid base64 string: {e}")
+
+
+def get_upload_url(filename: str = "image.png") -> Dict[str, str]:
+    """
+    Get presigned S3 URL for image upload.
+
+    Args:
+        filename: Original filename for the image
+
+    Returns:
+        Dict with job_id, upload_url, s3_url
+    """
+    if not API_GATEWAY_URL or not API_KEY:
+        raise ValueError(
+            "API_GATEWAY_URL and API_KEY must be set in environment variables. "
+            "Configure these in your HF Inference Endpoint secrets."
+        )
+
+    url = f"{API_GATEWAY_URL.rstrip('/')}/upload-url"
+    headers = {"x-api-key": API_KEY}
+    params = {"filename": filename}
+
+    logger.info(f"Getting upload URL from {url}")
+    response = requests.get(url, headers=headers, params=params, timeout=30)
+    response.raise_for_status()
+
+    result = response.json()
+    logger.info(f"Got upload URL for job_id={result.get('job_id')}")
+    return result
+
+
+def upload_to_s3(upload_url: str, image_bytes: bytes) -> None:
+    """
+    Upload image directly to S3 using presigned URL.
+
+    Args:
+        upload_url: Presigned PUT URL
+        image_bytes: Image data to upload
+    """
+    logger.info(f"Uploading {len(image_bytes)} bytes to S3...")
+    response = requests.put(
+        upload_url,
+        data=image_bytes,
+        headers={"Content-Type": "image/png"},
+        timeout=60
+    )
+    response.raise_for_status()
+    logger.info("Upload complete")
+
+
+def start_detection_job(job_id: str, s3_url: str, params: Optional[Dict] = None) -> str:
+    """
+    Start a detection job via API Gateway.
+
+    Args:
+        job_id: Job ID from get_upload_url
+        s3_url: S3 URL from get_upload_url
+        params: Optional processing parameters
+
+    Returns:
+        Job ID for polling
+    """
+    url = f"{API_GATEWAY_URL.rstrip('/')}/detect"
+    headers = {
+        "x-api-key": API_KEY,
+        "Content-Type": "application/json"
+    }
+    payload = {
+        "job_id": job_id,
+        "s3_url": s3_url
+    }
+    if params:
+        payload["params"] = params
+
+    logger.info(f"Starting detection job {job_id}")
+    response = requests.post(url, headers=headers, json=payload, timeout=30)
+    response.raise_for_status()
+
+    result = response.json()
+    logger.info(f"Detection job started: {result.get('status')}")
+    return job_id
+
+
+def poll_for_completion(job_id: str) -> Dict[str, Any]:
+    """
+    Poll API Gateway for job completion.
+
+    Args:
+        job_id: Job ID to poll
+
+    Returns:
+        Final result with callouts
+    """
+    url = f"{API_GATEWAY_URL.rstrip('/')}/status/{job_id}"
+    headers = {"x-api-key": API_KEY}
+
+    elapsed = 0
+    while elapsed < MAX_WAIT_SECONDS:
+        logger.info(f"Polling job {job_id} (elapsed: {elapsed}s)")
+
+        response = requests.get(url, headers=headers, timeout=30)
+        response.raise_for_status()
+
+        result = response.json()
+        status = result.get("status")
+
+        if status == "SUCCEEDED":
+            logger.info(f"Job {job_id} completed successfully")
+            return result
+
+        if status in ("FAILED", "TIMED_OUT", "ABORTED"):
+            error_msg = result.get("error", f"Job {status.lower()}")
+            logger.error(f"Job {job_id} failed: {error_msg}")
+            return {
+                "status": status,
+                "error": error_msg,
+                "callouts": []
+            }
+
+        # Still running, wait and retry
+        time.sleep(POLL_INTERVAL_SECONDS)
+        elapsed += POLL_INTERVAL_SECONDS
+
+    # Timeout
+    logger.error(f"Job {job_id} timed out after {MAX_WAIT_SECONDS}s")
+    return {
+        "status": "TIMEOUT",
+        "error": f"Timeout waiting for results after {MAX_WAIT_SECONDS}s",
+        "callouts": []
+    }
+
+
+def transform_to_emco_format(
+    callouts: List[Dict],
+    image_base64: str,
+    image_width: int = 0,
+    image_height: int = 0
+) -> Dict[str, Any]:
+    """
+    Transform callouts from Lambda format to EMCO format.
+
+    Lambda format:
+        {"bbox": [x, y, w, h], "score": 0.95, ...}  # xywh
+
+    EMCO format:
+        {"bbox": {"x1": x, "y1": y, "x2": x+w, "y2": y+h}, "confidence": 0.95, ...}  # xyxy
+
+    Args:
+        callouts: List of callouts from Lambda
+        image_base64: Original image as base64
+        image_width: Image width
+        image_height: Image height
+
+    Returns:
+        EMCO-compatible response dict
+    """
+    predictions = []
+
+    for i, callout in enumerate(callouts):
+        bbox = callout.get("bbox", [0, 0, 0, 0])
+
+        # Convert from [x, y, w, h] to {x1, y1, x2, y2}
+        x, y, w, h = bbox[0], bbox[1], bbox[2], bbox[3]
+
+        prediction = {
+            "id": i + 1,
+            "label": "callout",
+            "class_id": 0,
+            "confidence": callout.get("score", callout.get("confidence", 1.0)),
+            "bbox": {
+                "x1": int(x),
+                "y1": int(y),
+                "x2": int(x + w),
+                "y2": int(y + h)
+            }
+        }
+
+        # Include optional fields if present
+        if "text" in callout:
+            prediction["text"] = callout["text"]
+
+        predictions.append(prediction)
+
+    return {
+        "predictions": predictions,
+        "total_detections": len(predictions),
+        "image": image_base64,
+        "image_width": image_width,
+        "image_height": image_height
+    }
+
+
+def inference(image_input: str, parameters: Optional[Dict] = None) -> Dict[str, Any]:
+    """
+    Run inference on an image.
+
+    This is the main entry point for the HF wrapper.
+
+    Flow:
+    1. Normalize input to bytes
+    2. Get presigned S3 URL
+    3. Upload image directly to S3
+    4. Start detection job (small JSON payload)
+    5. Poll for completion
+    6. Transform results to EMCO format
+
+    Args:
+        image_input: Image URL, data URL, or base64 string
+        parameters: Optional processing parameters
+
+    Returns:
+        EMCO-compatible response with predictions
+    """
+    try:
+        # 1. Normalize input to bytes
+        logger.info("Normalizing input...")
+        image_bytes, filename = normalize_to_bytes(image_input)
+
+        # Keep base64 for response
+        image_base64 = base64.b64encode(image_bytes).decode("utf-8")
+
+        # 2. Get presigned upload URL
+        logger.info("Getting upload URL...")
+        upload_info = get_upload_url(filename)
+        job_id = upload_info["job_id"]
+        upload_url = upload_info["upload_url"]
+        s3_url = upload_info["s3_url"]
+
+        # 3. Upload image directly to S3
+        logger.info("Uploading to S3...")
+        upload_to_s3(upload_url, image_bytes)
+
+        # 4. Start detection job
+        logger.info("Starting detection job...")
+        start_detection_job(job_id, s3_url, parameters)
+
+        # 5. Poll for completion
+        logger.info("Polling for completion...")
+        result = poll_for_completion(job_id)
+
+        # 6. Check for errors
+        if result.get("status") in ("FAILED", "TIMED_OUT", "ABORTED", "TIMEOUT"):
+            return {
+                "error": result.get("error", "Unknown error"),
+                "predictions": [],
+                "total_detections": 0,
+                "image": image_base64
+            }
+
+        # 7. Transform to EMCO format
+        logger.info("Transforming results to EMCO format...")
+        callouts = result.get("callouts", [])
+        image_width = result.get("image_width", 0)
+        image_height = result.get("image_height", 0)
+
+        return transform_to_emco_format(
+            callouts,
+            image_base64,
+            image_width,
+            image_height
+        )
+
+    except requests.exceptions.RequestException as e:
+        logger.error(f"Request error: {e}")
+        return {
+            "error": f"Request error: {str(e)}",
+            "predictions": [],
+            "total_detections": 0,
+            "image": ""
+        }
+    except ValueError as e:
+        logger.error(f"Validation error: {e}")
+        return {
+            "error": str(e),
+            "predictions": [],
+            "total_detections": 0,
+            "image": ""
+        }
+    except Exception as e:
+        logger.error(f"Unexpected error: {e}", exc_info=True)
+        return {
+            "error": f"Unexpected error: {str(e)}",
+            "predictions": [],
+            "total_detections": 0,
+            "image": ""
+        }

requirements.txt

@@ -0,0 +1,2 @@
+requests>=2.31.0
+Pillow>=10.0.0