README.md

metadata

title: Room Object Segmentation
emoji: 🏠
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
pinned: false
license: apache-2.0

Room Object Segmentation API

🚧 Placeholder version for frontend development. Returns mock detection data.

Real Grounding DINO + SAM2 inference coming soon with ZeroGPU optimization.

Developed by Kreatazz Innovation Technology Solutions

Features

✨ Zero-shot Detection - No training required, works with natural language prompts
🎯 High Precision - State-of-the-art segmentation with SAM 2.1
⚡ GPU Acceleration - ZeroGPU A10G for fast inference
💾 Smart Caching - Automatic result caching to minimize GPU usage
🔌 REST API - Easy integration with any frontend

Live Demo

Upload a room image and specify objects to detect (e.g., chair . table . sofa . lamp)

The system will return:

• Annotated image with bounding boxes, masks, and labels
• JSON with detection details (bboxes, scores, areas, segments)
• Cache status (whether result was cached)

---

API Documentation

Endpoint

POST https://kreatazz-tech-room-object-segmentation.hf.space/api/predict

Authentication

This Space is publicly accessible. For production use, consider implementing API key authentication.

Request Format

Headers:

Content-Type: application/json

Body:

{
  "data": [
    "<base64_encoded_image>",
    "chair . table . sofa . lamp . bed",
    0.35,
    0.25
  ]
}

Parameters:

• data[0] (string): Base64-encoded image OR numpy array
• data[1] (string): Dot-separated object labels (e.g., "chair . table . sofa")
• data[2] (float): Box threshold (0.0-1.0, default: 0.35)
• data[3] (float): Text threshold (0.0-1.0, default: 0.25)

Response Format

{
  "data": [
    "<base64_annotated_image>",
    {
      "image_shape": {"height": 1080, "width": 1920},
      "num_detections": 5,
      "cached": true,
      "cache_key": "a3f2c1b5e8d9f0a1",
      "detections": [
        {
          "id": 0,
          "label": "chair",
          "confidence": 0.8523,
          "sam2_score": 0.9234,
          "bbox_xyxy": [120.5, 340.2, 450.8, 680.1],
          "bbox_xywh": [120.5, 340.2, 330.3, 339.9],
          "area_px": 89234
        }
      ]
    }
  ],
  "duration": 2.34
}

Response Fields:

• data[0]: Base64-encoded annotated image with bounding boxes and masks
• data[1].num_detections: Number of objects detected
• data[1].cached: Whether result was served from cache (saves GPU time)
• data[1].detections: Array of detected objects with:
  • label: Object class name
  • confidence: Detection confidence (Grounding DINO)
  • sam2_score: Segmentation quality score (SAM 2.1)
  • bbox_xyxy: Bounding box [x1, y1, x2, y2]
  • bbox_xywh: Bounding box [x, y, width, height]
  • area_px: Segmentation mask area in pixels

---

Usage Examples

JavaScript/TypeScript (Frontend)

async function detectRoomObjects(imageFile, objects = "chair . table . sofa") {
  // Convert image to base64
  const base64Image = await fileToBase64(imageFile);
  
  const response = await fetch(
    "https://kreatazz-tech-room-object-segmentation.hf.space/api/predict",
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        data: [base64Image, objects, 0.35, 0.25]
      })
    }
  );
  
  const result = await response.json();
  return result.data[1]; // Returns detection JSON
}

function fileToBase64(file) {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.onload = () => resolve(reader.result.split(',')[1]);
    reader.onerror = reject;
    reader.readAsDataURL(file);
  });
}

Python

import requests
import base64
from pathlib import Path

def detect_objects(image_path: str, objects: str = "chair . table . sofa"):
    # Read and encode image
    img_bytes = Path(image_path).read_bytes()
    img_b64 = base64.b64encode(img_bytes).decode()
    
    # Call API
    response = requests.post(
        "https://kreatazz-tech-room-object-segmentation.hf.space/api/predict",
        json={
            "data": [img_b64, objects, 0.35, 0.25]
        }
    )
    
    result = response.json()
    return result["data"][1]  # Returns detection JSON

# Example usage
detections = detect_objects("room.jpg", "chair . table . lamp")
print(f"Found {detections['num_detections']} objects")
print(f"Cached: {detections['cached']}")

cURL

# Prepare base64 image
IMAGE_B64=$(base64 -w 0 room.jpg)

# Call API
curl -X POST https://kreatazz-tech-room-object-segmentation.hf.space/api/predict \
  -H "Content-Type: application/json" \
  -d "{
    \"data\": [
      \"$IMAGE_B64\",
      \"chair . table . sofa . lamp\",
      0.35,
      0.25
    ]
  }"

---

Performance & Caching

GPU Usage Optimization

The Space implements intelligent caching to minimize GPU costs:

• First request for an image → GPU inference (~2-5 seconds)
• Subsequent identical requests → Cached result (~0.1 seconds)
• Cache key: Hash of image + prompt + thresholds
• Cache size: Up to 100 recent results (LRU eviction)

Cost Savings: Repeat requests use zero GPU time

Recommended Thresholds

• Box Threshold (0.35): Higher = fewer, more confident detections
• Text Threshold (0.25): Higher = stricter text-image matching

For furniture detection in typical room photos:

• Box: 0.30-0.40
• Text: 0.20-0.30

---

Supported Object Classes

This model supports open-vocabulary detection - you can specify any object name in natural language!

Common furniture classes:

chair . table . sofa . bed . lamp . desk . shelf . cabinet . 
tv . monitor . plant . rug . curtain . door . window . picture

Tips:

• Use simple, common object names
• Separate objects with . (space-dot-space)
• Be specific (e.g., "office chair" vs "chair" for better results)

---

Integration with Azure Backend

If you're using this Space with an Azure-hosted backend:

1. Update Azure environment:

   # In .env.azure
   HF_SPACE_API_URL=https://kreatazz-tech-room-object-segmentation.hf.space/api/predict
   

2. Call from backend:

   import requests
   import os
   
   HF_API_URL = os.getenv("HF_SPACE_API_URL")
   
   def segment_via_hf(image_b64: str, prompt: str):
       response = requests.post(
           HF_API_URL,
           json={"data": [image_b64, prompt, 0.35, 0.25]}
       )
       return response.json()["data"][1]
   

---

Technical Details

Models:

• Grounding DINO Tiny: IDEA-Research/grounding-dino-tiny
• SAM 2.1 Hiera Large: facebook/sam2.1-hiera-large

Hardware:

• ZeroGPU A10G (24GB VRAM)
• Auto-scaling based on usage
• 120-second GPU timeout per request

Frameworks:

• Gradio 5.29.0
• PyTorch 2.x
• Transformers
• Supervision

---

Support & Contact

Issues: Report bugs via GitHub Issues
Organization: Kreatazz Innovation Technology Solutions
License: Apache 2.0

---

Rate Limits

Free tier:

• No hard rate limits
• Shared ZeroGPU allocation
• May experience queuing during high traffic

For production use:

• Consider upgrading to dedicated GPU
• Implement client-side caching
• Contact HuggingFace for enterprise plans