README_API.md

# 📊 Dense Captioning Platform API Documentation

## Overview

The Dense Captioning Platform provides comprehensive chart analysis through a Gradio-based API. It can classify chart types, detect chart elements, and segment data points from uploaded images.

## API Access

**Base URL:** `https://hanszhu-dense-captioning-platform.hf.space`

**API Type:** Gradio Client API (not RESTful)

## Installation

### Prerequisites

```bash
pip install gradio-client
```

### Quick Start

### Python Client (Recommended)

```python
from gradio_client import Client, handle_file

# Initialize client with direct URL
client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Analyze a chart image using file path
result = client.predict(
    image=handle_file('path/to/your/chart.png'),
    fn_index=0
)

print(result)
```

### Using a URL

```python
from gradio_client import Client, handle_file

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Use a publicly accessible image URL
result = client.predict(
    image=handle_file("https://example.com/chart.png"),
    fn_index=0
)

print(result)
```

## Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `image` | File/URL | Yes | Chart image to analyze (PNG, JPG, JPEG supported) |

## Important Notes

### ✅ Working Approach
- **Use `fn_index=0`** instead of `api_name="/predict"`
- **Use direct URL** `"https://hanszhu-dense-captioning-platform.hf.space"`
- **Always use `handle_file()`** for both local files and URLs
- **This is a Gradio Client API**, not a RESTful API

### ❌ What Doesn't Work
- Direct HTTP POST requests to `/predict`
- Using `api_name="/predict"` with this setup
- Using `Client("hanszhu/Dense-Captioning-Platform")` (use direct URL instead)

## Output Format

The API returns a JSON object with the following structure:

```json
{
    "chart_type_id": 4,
    "chart_type_label": "Bar plot",
    "element_result": {
        "bboxes": [...],
        "segments": [...]
    },
    "datapoint_result": {
        "bboxes": [...],
        "segments": [...]
    },
    "status": "Full analysis completed",
    "processing_time": 2.345
}
```

### Output Fields

| Field | Type | Description |
|-------|------|-------------|
| `chart_type_id` | int | Numeric identifier for chart type (0-27) |
| `chart_type_label` | string | Human-readable chart type name |
| `element_result` | object/string | Detected chart elements (titles, axes, legends, etc.) |
| `datapoint_result` | object/string | Segmented data points and regions |
| `status` | string | Processing status message |
| `processing_time` | float | Time taken for analysis in seconds |

## Supported Chart Types

The platform can classify 28 different chart types:

| ID | Chart Type | ID | Chart Type |
|----|------------|----|------------|
| 0 | Line graph | 14 | Histogram |
| 1 | Natural image | 15 | Box plot |
| 2 | Table | 16 | Vector plot |
| 3 | 3D object | 17 | Pie chart |
| 4 | Bar plot | 18 | Surface plot |
| 5 | Scatter plot | 19 | Algorithm |
| 6 | Medical image | 20 | Contour plot |
| 7 | Sketch | 21 | Tree diagram |
| 8 | Geographic map | 22 | Bubble chart |
| 9 | Flow chart | 23 | Polar plot |
| 10 | Heat map | 24 | Area chart |
| 11 | Mask | 25 | Pareto chart |
| 12 | Block diagram | 26 | Radar chart |
| 13 | Venn diagram | 27 | Confusion matrix |

## Chart Elements Detected

The element detection model identifies:

- **Titles & Labels**: Chart title, subtitle, axis labels
- **Axes**: X-axis, Y-axis, tick labels
- **Legend**: Legend title, legend items, legend text
- **Data Elements**: Data points, data lines, data bars, data areas
- **Structural Elements**: Grid lines, plot areas

## Error Handling

The API returns error messages in the response fields when issues occur:

```json
{
    "chart_type_id": "Error: Model not available",
    "chart_type_label": "Error: Model not available",
    "element_result": "Error: Invalid image format",
    "datapoint_result": "Error: Processing failed",
    "status": "Error in chart classification",
    "processing_time": 0.0
}
```

## Rate Limits

- **Free Tier**: Limited requests per hour
- **Processing Time**: Typically 2-5 seconds per image
- **Image Size**: Recommended max 10MB

## Complete Working Example

Here's a complete example that demonstrates all the working patterns:

```python
from gradio_client import Client, handle_file
import json

def analyze_chart(image_path_or_url):
    """
    Analyze a chart image using the Dense Captioning Platform API
    
    Args:
        image_path_or_url (str): Path to local image file or URL to image
    
    Returns:
        dict: Analysis results with chart type, elements, and data points
    """
    try:
        # Initialize client with direct URL
        client = Client("https://hanszhu-dense-captioning-platform.hf.space")
        
        # Make prediction using the working approach
        result = client.predict(
            image=handle_file(image_path_or_url),
            fn_index=0
        )
        
        return result
        
    except Exception as e:
        return {
            "error": f"API call failed: {str(e)}",
            "status": "Error",
            "processing_time": 0.0
        }

# Example usage
if __name__ == "__main__":
    # Test with a local file
    local_result = analyze_chart("path/to/your/chart.png")
    print("Local file result:", json.dumps(local_result, indent=2))
    
    # Test with a URL
    url_result = analyze_chart("https://example.com/chart.png")
    print("URL result:", json.dumps(url_result, indent=2))
```

## Examples

### Example 1: Bar Chart Analysis

```python
from gradio_client import Client, handle_file

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Analyze a bar chart
result = client.predict(
    image=handle_file('bar_chart.png'),
    fn_index=0
)

print(f"Chart Type: {result['chart_type_label']}")
print(f"Processing Time: {result['processing_time']}s")
```

### Example 2: Batch Processing

```python
from gradio_client import Client, handle_file
import os

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Process multiple charts
chart_files = ['chart1.png', 'chart2.png', 'chart3.png']
results = []

for chart_file in chart_files:
    if os.path.exists(chart_file):
        result = client.predict(
            image=handle_file(chart_file),
            fn_index=0
        )
        results.append(result)
        print(f"Processed {chart_file}: {result['chart_type_label']}")
```

### Example 3: Test with Public Image

```python
from gradio_client import Client, handle_file

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Test with a public image URL
result = client.predict(
    image=handle_file("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png"),
    fn_index=0
)

print("✅ API Test Successful!")
print(f"Chart Type: {result['chart_type_label']}")
print(f"Status: {result['status']}")
```

## Troubleshooting

### Common Issues

1. **"Model not available"**: The models are still loading, wait a moment and retry
2. **"Invalid image format"**: Ensure the image is in PNG, JPG, or JPEG format
3. **"Processing failed"**: The image might be corrupted or too large
4. **"Expecting value: line 1 column 1"**: Use `fn_index=0` instead of `api_name="/predict"`
5. **"Cannot find a function with api_name"**: Use direct URL and `fn_index=0`

### Best Practices

1. **Image Quality**: Use clear, high-resolution images for best results
2. **Format**: PNG or JPG formats work best
3. **Size**: Keep images under 10MB for faster processing
4. **Client Setup**: Always use direct URL and `fn_index=0`
5. **File Handling**: Always use `handle_file()` for both local files and URLs
6. **Retry Logic**: Implement retry logic for failed requests

### Quick Test

To verify the API is working, run this test:

```python
from gradio_client import Client, handle_file

try:
    client = Client("https://hanszhu-dense-captioning-platform.hf.space")
    result = client.predict(
        image=handle_file("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png"),
        fn_index=0
    )
    print("✅ API is working!")
    print(f"Chart Type: {result['chart_type_label']}")
except Exception as e:
    print(f"❌ API test failed: {e}")
```

## Support

For issues or questions:
- Check the [Hugging Face Space](https://huggingface.co/spaces/hanszhu/Dense-Captioning-Platform)
- Review the error messages in the API response
- Ensure your image format and size are within limits