Update MediaPipe MVP model card
Browse files
README.md
CHANGED
|
@@ -15,11 +15,11 @@ license: other
|
|
| 15 |
|
| 16 |
# KSL Word Recognition Model Repo
|
| 17 |
|
| 18 |
-
This folder is a Hugging Face-compatible model package for word-level Korean Sign Language recognition. It contains the model structure
|
| 19 |
|
| 20 |
## Current MVP Artifact
|
| 21 |
|
| 22 |
-
The repository
|
| 23 |
|
| 24 |
Current MVP vocabulary:
|
| 25 |
|
|
@@ -30,15 +30,24 @@ Current MVP vocabulary:
|
|
| 30 |
Measured locally on the development machine:
|
| 31 |
|
| 32 |
```text
|
| 33 |
-
Validation accuracy:
|
| 34 |
-
|
| 35 |
-
|
|
|
|
| 36 |
```
|
| 37 |
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
```powershell
|
| 41 |
-
python training\
|
| 42 |
```
|
| 43 |
|
| 44 |
The trained `mediapipe_mvp.joblib` file is intentionally not tracked in GitHub. It should be distributed through Hugging Face.
|
|
@@ -65,6 +74,7 @@ This MVP artifact does not contain the AIHub video dataset. It only contains the
|
|
| 65 |
```text
|
| 66 |
RGB webcam frame
|
| 67 |
-> MediaPipe MVP recognizer when mediapipe_mvp.joblib exists
|
|
|
|
| 68 |
- extracts pose and hand landmarks
|
| 69 |
- classifies a short keypoint sequence with a lightweight classifier
|
| 70 |
-> otherwise YoloRoiExtractor
|
|
|
|
| 15 |
|
| 16 |
# KSL Word Recognition Model Repo
|
| 17 |
|
| 18 |
+
This folder is a Hugging Face-compatible model package for word-level Korean Sign Language recognition. It contains the model structure, inference contract, and the MediaPipe MVP runtime used by the backend.
|
| 19 |
|
| 20 |
## Current MVP Artifact
|
| 21 |
|
| 22 |
+
The repository loads a lightweight MediaPipe MVP artifact when `mediapipe_mvp.joblib` is present. This artifact is trained for a controlled one-person proof-of-concept, not for production-quality Korean Sign Language translation.
|
| 23 |
|
| 24 |
Current MVP vocabulary:
|
| 25 |
|
|
|
|
| 30 |
Measured locally on the development machine:
|
| 31 |
|
| 32 |
```text
|
| 33 |
+
Validation accuracy: 98.0% on a 50-sample held-out signer split
|
| 34 |
+
Training samples: 900 balanced samples, 10 labels, 5 camera angles
|
| 35 |
+
Mean inference latency: about 112.3 ms/frame on CPU
|
| 36 |
+
Approximate throughput: about 8.9 fps
|
| 37 |
```
|
| 38 |
|
| 39 |
+
Important limitation: this metric is from controlled AIHub word clips and a small 10-word vocabulary. It is suitable for a one-person demo, but it is not evidence of robust real-world meeting performance.
|
| 40 |
+
|
| 41 |
+
Training command used for the current artifact:
|
| 42 |
+
|
| 43 |
+
```powershell
|
| 44 |
+
python training\train_mediapipe_mvp.py --data-root "D:\수어 영상" --cache-dir "D:\ksl_cache\mediapipe_mvp_features" --max-per-label 90 --sequence-length 16 --angles F D U L R --classifier extra_trees --confidence-threshold 0.45
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
Benchmark command:
|
| 48 |
|
| 49 |
```powershell
|
| 50 |
+
python training\benchmark_mediapipe_mvp.py --data-root "D:\수어 영상" --cache-dir "D:\ksl_cache\mediapipe_mvp_features" --max-frames 120
|
| 51 |
```
|
| 52 |
|
| 53 |
The trained `mediapipe_mvp.joblib` file is intentionally not tracked in GitHub. It should be distributed through Hugging Face.
|
|
|
|
| 74 |
```text
|
| 75 |
RGB webcam frame
|
| 76 |
-> MediaPipe MVP recognizer when mediapipe_mvp.joblib exists
|
| 77 |
+
- resizes large RGB frames to max width 640 before MediaPipe
|
| 78 |
- extracts pose and hand landmarks
|
| 79 |
- classifies a short keypoint sequence with a lightweight classifier
|
| 80 |
-> otherwise YoloRoiExtractor
|