---
title: Room Object Segmentation
emoji: 🏠
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
pinned: false
license: apache-2.0
---

# Room Object Segmentation API

🚧 **Placeholder version** for frontend development. Returns mock detection data.

Real Grounding DINO + SAM2 inference coming soon with ZeroGPU optimization.

Developed by [Kreatazz Innovation Technology Solutions](https://huggingface.co/kreatazz-tech)

## Features

✨ **Zero-shot Detection** - No training required, works with natural language prompts  
🎯 **High Precision** - State-of-the-art segmentation with SAM 2.1  
⚡ **GPU Acceleration** - ZeroGPU A10G for fast inference  
💾 **Smart Caching** - Automatic result caching to minimize GPU usage  
🔌 **REST API** - Easy integration with any frontend  

## Live Demo

Upload a room image and specify objects to detect (e.g., `chair . table . sofa . lamp`)

The system will return:
- Annotated image with bounding boxes, masks, and labels
- JSON with detection details (bboxes, scores, areas, segments)
- Cache status (whether result was cached)

---

## API Documentation

### Endpoint

```
POST https://kreatazz-tech-room-object-segmentation.hf.space/api/predict
```

### Authentication

This Space is publicly accessible. For production use, consider implementing API key authentication.

### Request Format

**Headers:**
```
Content-Type: application/json
```

**Body:**
```json
{
  "data": [
    "<base64_encoded_image>",
    "chair . table . sofa . lamp . bed",
    0.35,
    0.25
  ]
}
```

**Parameters:**
- `data[0]` (string): Base64-encoded image OR numpy array
- `data[1]` (string): Dot-separated object labels (e.g., "chair . table . sofa")
- `data[2]` (float): Box threshold (0.0-1.0, default: 0.35)
- `data[3]` (float): Text threshold (0.0-1.0, default: 0.25)

### Response Format

```json
{
  "data": [
    "<base64_annotated_image>",
    {
      "image_shape": {"height": 1080, "width": 1920},
      "num_detections": 5,
      "cached": true,
      "cache_key": "a3f2c1b5e8d9f0a1",
      "detections": [
        {
          "id": 0,
          "label": "chair",
          "confidence": 0.8523,
          "sam2_score": 0.9234,
          "bbox_xyxy": [120.5, 340.2, 450.8, 680.1],
          "bbox_xywh": [120.5, 340.2, 330.3, 339.9],
          "area_px": 89234
        }
      ]
    }
  ],
  "duration": 2.34
}
```

**Response Fields:**
- `data[0]`: Base64-encoded annotated image with bounding boxes and masks
- `data[1].num_detections`: Number of objects detected
- `data[1].cached`: Whether result was served from cache (saves GPU time)
- `data[1].detections`: Array of detected objects with:
  - `label`: Object class name
  - `confidence`: Detection confidence (Grounding DINO)
  - `sam2_score`: Segmentation quality score (SAM 2.1)
  - `bbox_xyxy`: Bounding box [x1, y1, x2, y2]
  - `bbox_xywh`: Bounding box [x, y, width, height]
  - `area_px`: Segmentation mask area in pixels

---

## Usage Examples

### JavaScript/TypeScript (Frontend)

```javascript
async function detectRoomObjects(imageFile, objects = "chair . table . sofa") {
  // Convert image to base64
  const base64Image = await fileToBase64(imageFile);
  
  const response = await fetch(
    "https://kreatazz-tech-room-object-segmentation.hf.space/api/predict",
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        data: [base64Image, objects, 0.35, 0.25]
      })
    }
  );
  
  const result = await response.json();
  return result.data[1]; // Returns detection JSON
}

function fileToBase64(file) {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.onload = () => resolve(reader.result.split(',')[1]);
    reader.onerror = reject;
    reader.readAsDataURL(file);
  });
}
```

### Python

```python
import requests
import base64
from pathlib import Path

def detect_objects(image_path: str, objects: str = "chair . table . sofa"):
    # Read and encode image
    img_bytes = Path(image_path).read_bytes()
    img_b64 = base64.b64encode(img_bytes).decode()
    
    # Call API
    response = requests.post(
        "https://kreatazz-tech-room-object-segmentation.hf.space/api/predict",
        json={
            "data": [img_b64, objects, 0.35, 0.25]
        }
    )
    
    result = response.json()
    return result["data"][1]  # Returns detection JSON

# Example usage
detections = detect_objects("room.jpg", "chair . table . lamp")
print(f"Found {detections['num_detections']} objects")
print(f"Cached: {detections['cached']}")
```

### cURL

```bash
# Prepare base64 image
IMAGE_B64=$(base64 -w 0 room.jpg)

# Call API
curl -X POST https://kreatazz-tech-room-object-segmentation.hf.space/api/predict \
  -H "Content-Type: application/json" \
  -d "{
    \"data\": [
      \"$IMAGE_B64\",
      \"chair . table . sofa . lamp\",
      0.35,
      0.25
    ]
  }"
```

---

## Performance & Caching

### GPU Usage Optimization

The Space implements **intelligent caching** to minimize GPU costs:

- First request for an image → GPU inference (~2-5 seconds)
- Subsequent identical requests → Cached result (~0.1 seconds)
- Cache key: Hash of image + prompt + thresholds
- Cache size: Up to 100 recent results (LRU eviction)

**Cost Savings:** Repeat requests use **zero GPU time**

### Recommended Thresholds

- **Box Threshold** (0.35): Higher = fewer, more confident detections
- **Text Threshold** (0.25): Higher = stricter text-image matching

For furniture detection in typical room photos:
- Box: 0.30-0.40
- Text: 0.20-0.30

---

## Supported Object Classes

This model supports **open-vocabulary detection** - you can specify any object name in natural language!

**Common furniture classes:**
```
chair . table . sofa . bed . lamp . desk . shelf . cabinet . 
tv . monitor . plant . rug . curtain . door . window . picture
```

**Tips:**
- Use simple, common object names
- Separate objects with ` . ` (space-dot-space)
- Be specific (e.g., "office chair" vs "chair" for better results)

---

## Integration with Azure Backend

If you're using this Space with an Azure-hosted backend:

1. **Update Azure environment:**
   ```bash
   # In .env.azure
   HF_SPACE_API_URL=https://kreatazz-tech-room-object-segmentation.hf.space/api/predict
   ```

2. **Call from backend:**
   ```python
   import requests
   import os
   
   HF_API_URL = os.getenv("HF_SPACE_API_URL")
   
   def segment_via_hf(image_b64: str, prompt: str):
       response = requests.post(
           HF_API_URL,
           json={"data": [image_b64, prompt, 0.35, 0.25]}
       )
       return response.json()["data"][1]
   ```

---

## Technical Details

**Models:**
- **Grounding DINO Tiny**: IDEA-Research/grounding-dino-tiny
- **SAM 2.1 Hiera Large**: facebook/sam2.1-hiera-large

**Hardware:**
- ZeroGPU A10G (24GB VRAM)
- Auto-scaling based on usage
- 120-second GPU timeout per request

**Frameworks:**
- Gradio 5.29.0
- PyTorch 2.x
- Transformers
- Supervision

---

## Support & Contact

**Issues:** Report bugs via GitHub Issues  
**Organization:** [Kreatazz Innovation Technology Solutions](https://huggingface.co/kreatazz-tech)  
**License:** Apache 2.0

---

## Rate Limits

**Free tier:**
- No hard rate limits
- Shared ZeroGPU allocation
- May experience queuing during high traffic

**For production use:**
- Consider upgrading to dedicated GPU
- Implement client-side caching
- Contact HuggingFace for enterprise plans