# Hand Tracking Drawing

Draw in the air with your hand, then view the drawing as a rotatable 3D object.

![Fig.1: Example of drawing](example/HAND_TRACK_EXAMPLE_2.png)


**[DOCKER IMAGE HERE](https://hub.docker.com/r/flugmaschine/hand-tracking-drawing)**

## Model

Uses the **MediaPipe Hand Landmarker** (`models/hand_landmarker.task`), which returns
21 keypoints per hand. Gestures are classified from ratios and joint angles between
those keypoints, so recognition is independent of hand size, distance, or handedness.

## Logic

- **One index finger** - draw
- **Index + middle** - move cursor without drawing
- **Fist (hold)** - grab the drawing; move the fist to rotate it in 3D
- **Two fists** - scale (canvas in 2D, object in 3D)
- **Open palm** - release, back to drawing
- **OK (hold)** - clear the canvas
- **Pinky (hold)** - undo the last stroke
- **Middle finger** - shows `:((` for a second
- **Both palms waving** - say goodbye, closes the app

Keys: `S` save, `C` clear, `U` undo, `1..6` color, `[` / `]` thickness,
`SPACE` 2D/3D, `R` reset view, `K` skeleton, `D` debug, `H` help, `Q` quit.

## 3D Drawing

The **3D Drawing** mode uses the YOLO depth model (`models/yolo26s-depth.pt`), which
returns a per pixel depth map in meters. Every drawn point takes its Z from the metric
depth under the fingertip, so moving the hand closer or further really draws into the
scene: the stroke is a 3D curve in space, not a flat picture placed on a board. The
first drawn point sets the zero plane, everything after that is relative to it.
Depth runs in a background thread, so tracking and drawing stay at camera speed.

Raw monocular depth flickers, so the pen does not follow it directly: the global scale
drift of the map is cancelled against the static background, the fingertip depth goes
through a one euro filter with a noise deadband and a speed limit, and the newest points
of a stroke are low passed along the curve. A circle drawn without moving the hand comes
out flat, while a deliberate push of 25 cm is kept in full.

By default strokes are colored by a depth gradient - warm near the camera, cool further
away. `1..6` switch to a solid color, `G` goes back to the gradient.

Gestures and keys are the same as in 2D, with two additions:

- **Two ring fingers (hold)** - toggle depth colors for the whole frame, drawing keeps working
- **Open palm (hold)** - reset the view

In 3D the fist rotates the drawing itself and two fists scale it, `SPACE` toggles
depth colors, `R` resets the view.

## Run

```bash
pip install -r requirements.txt
python src/main.py
```

Pick the mode in the menu, or skip it:

```bash
python src/main.py --no-menu --mode 3d
python src/main.py --no-menu --mode 3d --depth-size 448 --depth-every 1
```

by Salimli Ayzek (Салимли Айзек): https://mathematiclove.github.io