README.md

---
title: Babelbox Edge Vision Translator
emoji: 🌍📸
colorFrom: gray
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
---

# Babelbox

Babelbox is a webcam object-detection demo that overlays English object labels and simple target-language translations in the browser.

This repository is structured to work as both:

- the **GitHub source repo**
- the **Hugging Face Docker Space repo**

Hugging Face reads the YAML block above, builds `Dockerfile`, and starts `backend.py` on port 7860.

## Detection engines

The app has two selectable detection engines:

- **RF-DETR backend**: `index.html` sends camera frames to the local Python backend in `backend.py`, which runs `Roboflow/rf-detr-medium`.
- **Legacy browser DETR**: `index.html` uses `worker.js` to run `Xenova/detr-resnet-50` in a Web Worker.

On Hugging Face Docker Spaces, `backend.py` serves both the browser app and the RF-DETR `/detect` API from the same origin.

Translations are dictionary-based. No translation model is loaded in the hot path.

## Requirements

### RF-DETR backend mode

- Python 3.11+
- `pip`
- A Torch-supported CPU/GPU/MPS runtime. RF-DETR works on CPU, but latency is much better with acceleration.

### Legacy browser DETR mode

- A modern browser
- Camera access over `localhost` or HTTPS
- WebGPU is preferred when available, with WASM fallback handled by Transformers.js

## Local setup

Install the Python dependencies:

```bash
python3 -m pip install -r requirements.txt
```

Start the RF-DETR backend and frontend server:

```bash
python3 backend.py
```

Open `http://127.0.0.1:7860`. The first detection request may take a while because the model is downloaded and loaded on demand.

Optional: you can still serve the frontend separately for static UI development:

```bash
python3 -m http.server 8000
```

When served on port 8000, the frontend calls the backend at `http://127.0.0.1:7860/detect`.

## Hugging Face Space deployment

Use this same repository as the Space repo. The important files are:

```text
README.md   Space config and project docs
Dockerfile  Hugging Face Space runtime
backend.py  Serves the browser UI and RF-DETR API
index.html  Browser UI
worker.js   Browser DETR worker
```

Add the Space as a Git remote, then push:

```bash
git remote add hf https://huggingface.co/spaces/malen-h/babelbox
git push hf main
```

If your branch is not named `main`, push it to the Space's `main` branch:

```bash
git push hf HEAD:main
```

## Backend API

### Health check

```text
GET http://127.0.0.1:7860/health
```

Returns JSON with the backend status, model ID, and whether the model is loaded.

### Detection

```text
POST http://127.0.0.1:7860/detect
```

The frontend sends JPEG image bytes. The backend returns:

```json
{
    "results": [
        {
            "label": "bottle",
            "score": 0.92,
            "box": {
                "xmin": 100,
                "ymin": 120,
                "xmax": 220,
                "ymax": 360
            }
        }
    ]
}
```

Optional request headers:

- `X-Confidence-Threshold`: detection threshold, default `0.75`
- `X-Focus-Mode`: when `true`, filters out people and returns the strongest object

## Supported languages

The app uses hard-coded label dictionaries for:

- Spanish (`spa_Latn`)
- French (`fra_Latn`)
- Chinese Simplified (`zho_Hans`)
- Hindi (`hin_Deva`)
- Afrikaans (`afr_Latn`)
- Zulu (`zul_Latn`)

Unknown labels fall back to English. To add or improve translations, update the dictionaries in both `index.html` and `worker.js`.

## Project structure

```text
backend.py        RF-DETR backend and /detect API
Dockerfile        Hugging Face Docker Space runtime
requirements.txt  Python dependencies
index.html        Browser UI, webcam, engine selector, overlays
worker.js         Legacy browser DETR worker
```

## Notes

- Run the frontend through a local server; camera APIs are unreliable from `file://`.
- The default RF-DETR mode requires `backend.py` to be running.
- Static hosting can run the frontend, but it cannot run the Python backend. Hugging Face RF-DETR deployment uses the Docker Space config in this repo.
- Model files are cached by Hugging Face tooling outside the repository, commonly under `~/.cache/huggingface`.