README.md

metadata

title: Babelbox Edge Vision Translator
emoji: 🌍📸
colorFrom: gray
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

Babelbox

Babelbox is a webcam object-detection demo that overlays English object labels and simple target-language translations in the browser.

This repository is structured to work as both:

• the GitHub source repo
• the Hugging Face Docker Space repo

Hugging Face reads the YAML block above, builds Dockerfile, and starts backend.py on port 7860.

Detection engines

The app has two selectable detection engines:

• RF-DETR backend: index.html sends camera frames to the local Python backend in backend.py, which runs Roboflow/rf-detr-medium.
• Legacy browser DETR: index.html uses worker.js to run Xenova/detr-resnet-50 in a Web Worker.

On Hugging Face Docker Spaces, backend.py serves both the browser app and the RF-DETR /detect API from the same origin.

Translations are dictionary-based. No translation model is loaded in the hot path.

Requirements

RF-DETR backend mode

• Python 3.11+
• pip
• A Torch-supported CPU/GPU/MPS runtime. RF-DETR works on CPU, but latency is much better with acceleration.

Legacy browser DETR mode

• A modern browser
• Camera access over localhost or HTTPS
• WebGPU is preferred when available, with WASM fallback handled by Transformers.js

Local setup

Install the Python dependencies:

python3 -m pip install -r requirements.txt

Start the RF-DETR backend and frontend server:

python3 backend.py

Open http://127.0.0.1:7860. The first detection request may take a while because the model is downloaded and loaded on demand.

Optional: you can still serve the frontend separately for static UI development:

python3 -m http.server 8000

When served on port 8000, the frontend calls the backend at http://127.0.0.1:7860/detect.

Hugging Face Space deployment

Use this same repository as the Space repo. The important files are:

README.md   Space config and project docs
Dockerfile  Hugging Face Space runtime
backend.py  Serves the browser UI and RF-DETR API
index.html  Browser UI
worker.js   Browser DETR worker

Add the Space as a Git remote, then push:

git remote add hf https://huggingface.co/spaces/malen-h/babelbox
git push hf main

If your branch is not named main, push it to the Space's main branch:

git push hf HEAD:main

Backend API

Health check

GET http://127.0.0.1:7860/health

Returns JSON with the backend status, model ID, and whether the model is loaded.

Detection

POST http://127.0.0.1:7860/detect

The frontend sends JPEG image bytes. The backend returns:

{
    "results": [
        {
            "label": "bottle",
            "score": 0.92,
            "box": {
                "xmin": 100,
                "ymin": 120,
                "xmax": 220,
                "ymax": 360
            }
        }
    ]
}

Optional request headers:

• X-Confidence-Threshold: detection threshold, default 0.75
• X-Focus-Mode: when true, filters out people and returns the strongest object

Supported languages

The app uses hard-coded label dictionaries for:

• Spanish (spa_Latn)
• French (fra_Latn)
• Chinese Simplified (zho_Hans)
• Hindi (hin_Deva)
• Afrikaans (afr_Latn)
• Zulu (zul_Latn)

Unknown labels fall back to English. To add or improve translations, update the dictionaries in both index.html and worker.js.

Project structure

backend.py        RF-DETR backend and /detect API
Dockerfile        Hugging Face Docker Space runtime
requirements.txt  Python dependencies
index.html        Browser UI, webcam, engine selector, overlays
worker.js         Legacy browser DETR worker

Notes

• Run the frontend through a local server; camera APIs are unreliable from file://.
• The default RF-DETR mode requires backend.py to be running.
• Static hosting can run the frontend, but it cannot run the Python backend. Hugging Face RF-DETR deployment uses the Docker Space config in this repo.
• Model files are cached by Hugging Face tooling outside the repository, commonly under ~/.cache/huggingface.