Spaces:
Sleeping
Sleeping
Krishna Venkatesh commited on
Commit ·
223db34
1
Parent(s): e923c09
chore: verify Docker and update project checklist
Browse files- .dockerignore +16 -0
- .gitignore +1 -0
- README.md +7 -1
- docs/next_steps.md +13 -8
- docs/todo.md +40 -55
- scripts/augment_data.py +9 -2
.dockerignore
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.git
|
| 2 |
+
.venv
|
| 3 |
+
__pycache__
|
| 4 |
+
*.py[cod]
|
| 5 |
+
*.egg-info
|
| 6 |
+
.pytest_cache
|
| 7 |
+
.coverage
|
| 8 |
+
htmlcov
|
| 9 |
+
data/raw/*
|
| 10 |
+
data/processed/*
|
| 11 |
+
data/annotations/*
|
| 12 |
+
PROJECT_STATUS.md
|
| 13 |
+
*.mp4
|
| 14 |
+
*.mov
|
| 15 |
+
docs/demo.gif
|
| 16 |
+
|
.gitignore
CHANGED
|
@@ -13,6 +13,7 @@ data/annotations/*
|
|
| 13 |
!data/raw/.gitkeep
|
| 14 |
!data/processed/.gitkeep
|
| 15 |
!data/annotations/.gitkeep
|
|
|
|
| 16 |
*.mp4
|
| 17 |
*.mov
|
| 18 |
PROJECT_STATUS.md
|
|
|
|
| 13 |
!data/raw/.gitkeep
|
| 14 |
!data/processed/.gitkeep
|
| 15 |
!data/annotations/.gitkeep
|
| 16 |
+
data/samples/upload_*
|
| 17 |
*.mp4
|
| 18 |
*.mov
|
| 19 |
PROJECT_STATUS.md
|
README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
| 1 |
# BrailleVision
|
| 2 |
|
|
|
|
|
|
|
| 3 |
BrailleVision is a computer-vision demo that reads camera or uploaded images of Braille and returns English text with an annotated detection overlay. The current implementation is validated on synthetic Braille images and is ready for the next phase: real embossed Braille image collection, tuning, and benchmarking.
|
| 4 |
|
| 5 |
## Status
|
|
@@ -8,7 +10,9 @@ BrailleVision is a computer-vision demo that reads camera or uploaded images of
|
|
| 8 |
- Coverage from the latest local run: `72%`
|
| 9 |
- Formatting and linting are clean with Black, isort, and flake8
|
| 10 |
- Flask demo runs locally at `http://127.0.0.1:7860`
|
| 11 |
-
- Docker
|
|
|
|
|
|
|
| 12 |
|
| 13 |
## Features
|
| 14 |
|
|
@@ -172,6 +176,8 @@ python scripts/benchmark.py --testset data/processed/
|
|
| 172 |
|
| 173 |
The benchmark script looks for annotation JSON files next to each processed image and in `data/annotations/`.
|
| 174 |
|
|
|
|
|
|
|
| 175 |
Report results as character error rate or accuracy:
|
| 176 |
|
| 177 |
```text
|
|
|
|
| 1 |
# BrailleVision
|
| 2 |
|
| 3 |
+
[](https://github.com/sriksven/braillevision/actions/workflows/ci.yml)
|
| 4 |
+
|
| 5 |
BrailleVision is a computer-vision demo that reads camera or uploaded images of Braille and returns English text with an annotated detection overlay. The current implementation is validated on synthetic Braille images and is ready for the next phase: real embossed Braille image collection, tuning, and benchmarking.
|
| 6 |
|
| 7 |
## Status
|
|
|
|
| 10 |
- Coverage from the latest local run: `72%`
|
| 11 |
- Formatting and linting are clean with Black, isort, and flake8
|
| 12 |
- Flask demo runs locally at `http://127.0.0.1:7860`
|
| 13 |
+
- Docker build, health check, UI load, and sample upload were verified locally
|
| 14 |
+
- 10 public real Braille photos have been downloaded locally for smoke testing
|
| 15 |
+
- 80 augmented real-image variants were generated locally
|
| 16 |
|
| 17 |
## Features
|
| 18 |
|
|
|
|
| 176 |
|
| 177 |
The benchmark script looks for annotation JSON files next to each processed image and in `data/annotations/`.
|
| 178 |
|
| 179 |
+
Current status: the benchmark command runs, but accuracy is blocked until real-image annotation JSON exists.
|
| 180 |
+
|
| 181 |
Report results as character error rate or accuracy:
|
| 182 |
|
| 183 |
```text
|
docs/next_steps.md
CHANGED
|
@@ -6,7 +6,11 @@ For the full checklist, see [TODO](todo.md).
|
|
| 6 |
|
| 7 |
## 1. Docker Verification
|
| 8 |
|
| 9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
|
| 11 |
```bash
|
| 12 |
docker compose up --build
|
|
@@ -14,10 +18,10 @@ curl http://localhost:7860/health
|
|
| 14 |
docker compose down
|
| 15 |
```
|
| 16 |
|
| 17 |
-
If Docker reports that it cannot connect to the daemon, start Docker Desktop and retry.
|
| 18 |
-
|
| 19 |
## 2. Real Braille Images
|
| 20 |
|
|
|
|
|
|
|
| 21 |
Done when uploading real Braille photos through the UI produces correct or near-correct text.
|
| 22 |
|
| 23 |
- Save 20 to 30 real photos in `data/raw/`.
|
|
@@ -53,6 +57,8 @@ Example file in `data/annotations/`:
|
|
| 53 |
|
| 54 |
## 4. Augmentation
|
| 55 |
|
|
|
|
|
|
|
| 56 |
Done when `data/processed/` contains variants for the real-image set.
|
| 57 |
|
| 58 |
```bash
|
|
@@ -63,6 +69,8 @@ The script creates eight variants per image.
|
|
| 63 |
|
| 64 |
## 5. Benchmark
|
| 65 |
|
|
|
|
|
|
|
| 66 |
Done when README has a benchmark table with real numbers.
|
| 67 |
|
| 68 |
```bash
|
|
@@ -77,12 +85,9 @@ accuracy = 1 - CER
|
|
| 77 |
|
| 78 |
## 6. Public Repo and CI
|
| 79 |
|
| 80 |
-
|
| 81 |
|
| 82 |
-
|
| 83 |
-
- Confirm `.github/workflows/ci.yml` runs.
|
| 84 |
-
- Fix any CI-only failures.
|
| 85 |
-
- Add the CI badge to `README.md` after the first successful run.
|
| 86 |
|
| 87 |
## 7. Public Demo
|
| 88 |
|
|
|
|
| 6 |
|
| 7 |
## 1. Docker Verification
|
| 8 |
|
| 9 |
+
Status: done locally.
|
| 10 |
+
|
| 11 |
+
`docker compose up --build` starts, `http://localhost:7860/health` returns OK, and sample upload through `/upload` returned `hello`.
|
| 12 |
+
|
| 13 |
+
Keep this command set for future verification:
|
| 14 |
|
| 15 |
```bash
|
| 16 |
docker compose up --build
|
|
|
|
| 18 |
docker compose down
|
| 19 |
```
|
| 20 |
|
|
|
|
|
|
|
| 21 |
## 2. Real Braille Images
|
| 22 |
|
| 23 |
+
Status: partially done. Ten public real Braille photos are in ignored `data/raw/`, and the current pipeline has been smoke-tested against them.
|
| 24 |
+
|
| 25 |
Done when uploading real Braille photos through the UI produces correct or near-correct text.
|
| 26 |
|
| 27 |
- Save 20 to 30 real photos in `data/raw/`.
|
|
|
|
| 57 |
|
| 58 |
## 4. Augmentation
|
| 59 |
|
| 60 |
+
Status: done for the current 10-image local set. `data/processed/` has 80 generated variants.
|
| 61 |
+
|
| 62 |
Done when `data/processed/` contains variants for the real-image set.
|
| 63 |
|
| 64 |
```bash
|
|
|
|
| 69 |
|
| 70 |
## 5. Benchmark
|
| 71 |
|
| 72 |
+
Status: blocked until annotation JSON exists.
|
| 73 |
+
|
| 74 |
Done when README has a benchmark table with real numbers.
|
| 75 |
|
| 76 |
```bash
|
|
|
|
| 85 |
|
| 86 |
## 6. Public Repo and CI
|
| 87 |
|
| 88 |
+
Status: done.
|
| 89 |
|
| 90 |
+
GitHub repo is public at `https://github.com/sriksven/braillevision`, Actions is green, and the CI badge is in `README.md`.
|
|
|
|
|
|
|
|
|
|
| 91 |
|
| 92 |
## 7. Public Demo
|
| 93 |
|
docs/todo.md
CHANGED
|
@@ -40,89 +40,60 @@ Status: done.
|
|
| 40 |
|
| 41 |
## 4. GitHub Repository
|
| 42 |
|
| 43 |
-
Status:
|
| 44 |
|
| 45 |
Done when the public GitHub repo has a green CI badge.
|
| 46 |
|
| 47 |
Tasks:
|
| 48 |
|
| 49 |
-
-
|
| 50 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
-
|
| 53 |
-
git remote add origin https://github.com/YOUR_USERNAME/braillevision
|
| 54 |
-
git branch -M main
|
| 55 |
-
git push -u origin main
|
| 56 |
-
```
|
| 57 |
-
|
| 58 |
-
- Confirm GitHub Actions starts under the Actions tab
|
| 59 |
-
- Wait for CI to pass
|
| 60 |
-
- Fix anything CI flags
|
| 61 |
-
- Add CI badge to `README.md`
|
| 62 |
-
- Commit and push README update:
|
| 63 |
|
| 64 |
```bash
|
| 65 |
-
git
|
| 66 |
-
git commit -m "docs: add CI badge"
|
| 67 |
-
git push
|
| 68 |
```
|
| 69 |
|
| 70 |
## 5. Docker
|
| 71 |
|
| 72 |
-
Status:
|
| 73 |
|
| 74 |
Done when the app runs cleanly inside Docker and health check passes.
|
| 75 |
|
| 76 |
Tasks:
|
| 77 |
|
| 78 |
-
-
|
| 79 |
-
-
|
| 80 |
-
|
| 81 |
-
```
|
| 82 |
-
|
| 83 |
-
``
|
|
|
|
| 84 |
|
| 85 |
-
|
| 86 |
|
| 87 |
```bash
|
|
|
|
| 88 |
docker compose up --build
|
| 89 |
-
```
|
| 90 |
-
|
| 91 |
-
- Open `http://localhost:7860`
|
| 92 |
-
- Check health:
|
| 93 |
-
|
| 94 |
-
```bash
|
| 95 |
curl http://localhost:7860/health
|
| 96 |
-
```
|
| 97 |
-
|
| 98 |
-
Expected:
|
| 99 |
-
|
| 100 |
-
```json
|
| 101 |
-
{"status": "ok", "version": "1.0"}
|
| 102 |
-
```
|
| 103 |
-
|
| 104 |
-
- Upload a test image through the UI inside Docker
|
| 105 |
-
- Stop container:
|
| 106 |
-
|
| 107 |
-
```bash
|
| 108 |
docker compose down
|
| 109 |
```
|
| 110 |
|
| 111 |
## 6. Real Braille Images
|
| 112 |
|
| 113 |
-
Status:
|
| 114 |
|
| 115 |
Done when uploading a real Braille photo via the UI produces correct or near-correct English text.
|
| 116 |
|
| 117 |
### 6a. Collect Images
|
| 118 |
|
| 119 |
-
-
|
| 120 |
-
|
| 121 |
-
- `braille book page`
|
| 122 |
-
- `braille label`
|
| 123 |
-
- `braille sign`
|
| 124 |
-
- Download 20 to 30 real photos
|
| 125 |
-
- Save them to `data/raw/`
|
| 126 |
|
| 127 |
### 6b. Test In The App
|
| 128 |
|
|
@@ -137,7 +108,11 @@ Open:
|
|
| 137 |
http://127.0.0.1:7860
|
| 138 |
```
|
| 139 |
|
| 140 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
|
| 142 |
- expected text
|
| 143 |
- actual pipeline output
|
|
@@ -179,7 +154,7 @@ Example:
|
|
| 179 |
|
| 180 |
## 7. Augmentation
|
| 181 |
|
| 182 |
-
Status:
|
| 183 |
|
| 184 |
Done when `data/processed/` has at least 80 images from 10 real images and eight variants each.
|
| 185 |
|
|
@@ -191,9 +166,14 @@ python scripts/augment_data.py --input data/raw/ --output data/processed/
|
|
| 191 |
|
| 192 |
Then spot-check outputs. They should look like realistic degraded images, not broken samples.
|
| 193 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 194 |
## 8. Benchmark
|
| 195 |
|
| 196 |
-
Status:
|
| 197 |
|
| 198 |
Done when README has a benchmark table with real numbers.
|
| 199 |
|
|
@@ -203,6 +183,12 @@ Run:
|
|
| 203 |
python scripts/benchmark.py --testset data/processed/
|
| 204 |
```
|
| 205 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 206 |
Record accuracy by condition:
|
| 207 |
|
| 208 |
- normal
|
|
@@ -380,4 +366,3 @@ Avoid claiming 90%+ accuracy until real-image benchmarks prove it.
|
|
| 380 |
| 5 | Record demo video | Required for submissions |
|
| 381 |
| 6 | Benchmark and README GIF | Improves judge confidence |
|
| 382 |
| 7 | Submit Devpost entries | Final deliverable |
|
| 383 |
-
|
|
|
|
| 40 |
|
| 41 |
## 4. GitHub Repository
|
| 42 |
|
| 43 |
+
Status: done.
|
| 44 |
|
| 45 |
Done when the public GitHub repo has a green CI badge.
|
| 46 |
|
| 47 |
Tasks:
|
| 48 |
|
| 49 |
+
- Done: public GitHub repo exists at `https://github.com/sriksven/braillevision`
|
| 50 |
+
- Done: `origin` remote is configured
|
| 51 |
+
- Done: `main` branch pushed
|
| 52 |
+
- Done: GitHub Actions started
|
| 53 |
+
- Done: CI completed successfully
|
| 54 |
+
- Done: CI badge added to `README.md`
|
| 55 |
|
| 56 |
+
Commands used:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
```bash
|
| 59 |
+
git push -u origin main
|
|
|
|
|
|
|
| 60 |
```
|
| 61 |
|
| 62 |
## 5. Docker
|
| 63 |
|
| 64 |
+
Status: done.
|
| 65 |
|
| 66 |
Done when the app runs cleanly inside Docker and health check passes.
|
| 67 |
|
| 68 |
Tasks:
|
| 69 |
|
| 70 |
+
- Done: Docker Desktop daemon started
|
| 71 |
+
- Done: image built with `docker compose up --build`
|
| 72 |
+
- Done: UI returned HTTP 200 at `http://localhost:7860`
|
| 73 |
+
- Done: `/health` returned `{"status":"ok","version":"1.0"}`
|
| 74 |
+
- Done: uploaded `data/samples/hello.png` through `/upload`; pipeline returned `hello`
|
| 75 |
+
- Done: container stopped with `docker compose down`
|
| 76 |
+
- Done: `.dockerignore` added to keep `.venv`, raw data, processed data, and local artifacts out of Docker context
|
| 77 |
|
| 78 |
+
Commands used:
|
| 79 |
|
| 80 |
```bash
|
| 81 |
+
docker info
|
| 82 |
docker compose up --build
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
curl http://localhost:7860/health
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
docker compose down
|
| 85 |
```
|
| 86 |
|
| 87 |
## 6. Real Braille Images
|
| 88 |
|
| 89 |
+
Status: partially done. This is still the highest-priority remaining product work.
|
| 90 |
|
| 91 |
Done when uploading a real Braille photo via the UI produces correct or near-correct English text.
|
| 92 |
|
| 93 |
### 6a. Collect Images
|
| 94 |
|
| 95 |
+
- Done: downloaded 10 real public Braille photos from Wikimedia Commons into ignored `data/raw/`
|
| 96 |
+
- Not done: collect the full 20 to 30 image set
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
|
| 98 |
### 6b. Test In The App
|
| 99 |
|
|
|
|
| 108 |
http://127.0.0.1:7860
|
| 109 |
```
|
| 110 |
|
| 111 |
+
Done: current pipeline was run over all 10 downloaded real images.
|
| 112 |
+
|
| 113 |
+
Baseline result: dot detection fires on most images, but decoded text is mostly wrong or incomplete. This confirms real-image tuning is still required.
|
| 114 |
+
|
| 115 |
+
Continue recording:
|
| 116 |
|
| 117 |
- expected text
|
| 118 |
- actual pipeline output
|
|
|
|
| 154 |
|
| 155 |
## 7. Augmentation
|
| 156 |
|
| 157 |
+
Status: done for the current 10 real-image set.
|
| 158 |
|
| 159 |
Done when `data/processed/` has at least 80 images from 10 real images and eight variants each.
|
| 160 |
|
|
|
|
| 166 |
|
| 167 |
Then spot-check outputs. They should look like realistic degraded images, not broken samples.
|
| 168 |
|
| 169 |
+
Current local result:
|
| 170 |
+
|
| 171 |
+
- 10 real images in `data/raw/`
|
| 172 |
+
- 80 generated image variants in `data/processed/`
|
| 173 |
+
|
| 174 |
## 8. Benchmark
|
| 175 |
|
| 176 |
+
Status: blocked on real annotation JSON.
|
| 177 |
|
| 178 |
Done when README has a benchmark table with real numbers.
|
| 179 |
|
|
|
|
| 183 |
python scripts/benchmark.py --testset data/processed/
|
| 184 |
```
|
| 185 |
|
| 186 |
+
Current result:
|
| 187 |
+
|
| 188 |
+
```text
|
| 189 |
+
No annotated images found.
|
| 190 |
+
```
|
| 191 |
+
|
| 192 |
Record accuracy by condition:
|
| 193 |
|
| 194 |
- normal
|
|
|
|
| 366 |
| 5 | Record demo video | Required for submissions |
|
| 367 |
| 6 | Benchmark and README GIF | Improves judge confidence |
|
| 368 |
| 7 | Submit Devpost entries | Final deliverable |
|
|
|
scripts/augment_data.py
CHANGED
|
@@ -5,11 +5,16 @@ from __future__ import annotations
|
|
| 5 |
import argparse
|
| 6 |
from pathlib import Path
|
| 7 |
|
| 8 |
-
from PIL import Image, ImageEnhance, ImageFilter
|
|
|
|
|
|
|
| 9 |
|
| 10 |
|
| 11 |
def augment_image(path: Path, out_dir: Path) -> None:
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
| 13 |
variants = {
|
| 14 |
"normal": img,
|
| 15 |
"dim": ImageEnhance.Brightness(img).enhance(0.65),
|
|
@@ -37,6 +42,8 @@ def main() -> None:
|
|
| 37 |
)
|
| 38 |
args = parser.parse_args()
|
| 39 |
for image_path in sorted(args.src.glob("*.*")):
|
|
|
|
|
|
|
| 40 |
augment_image(image_path, args.out)
|
| 41 |
|
| 42 |
|
|
|
|
| 5 |
import argparse
|
| 6 |
from pathlib import Path
|
| 7 |
|
| 8 |
+
from PIL import Image, ImageEnhance, ImageFilter, UnidentifiedImageError
|
| 9 |
+
|
| 10 |
+
IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".bmp", ".tif", ".tiff", ".webp"}
|
| 11 |
|
| 12 |
|
| 13 |
def augment_image(path: Path, out_dir: Path) -> None:
|
| 14 |
+
try:
|
| 15 |
+
img = Image.open(path).convert("RGB")
|
| 16 |
+
except UnidentifiedImageError:
|
| 17 |
+
return
|
| 18 |
variants = {
|
| 19 |
"normal": img,
|
| 20 |
"dim": ImageEnhance.Brightness(img).enhance(0.65),
|
|
|
|
| 42 |
)
|
| 43 |
args = parser.parse_args()
|
| 44 |
for image_path in sorted(args.src.glob("*.*")):
|
| 45 |
+
if image_path.suffix.lower() not in IMAGE_EXTENSIONS:
|
| 46 |
+
continue
|
| 47 |
augment_image(image_path, args.out)
|
| 48 |
|
| 49 |
|