| | --- |
| | title: MTG Card Segmentation |
| | emoji: 🎴 |
| | colorFrom: blue |
| | colorTo: purple |
| | sdk: static |
| | pinned: false |
| | license: mit |
| | short_description: MTG Real-time Card Semantic Segmentation |
| | --- |
| | |
| | # Card Segmentation - Real-time Detection |
| |
|
| | A real-time card segmentation application that uses semantic segmentation to detect and highlight cards in camera feeds. Built with ONNX Runtime Web for in-browser AI inference. |
| |
|
| | - Pre-generated dataset in this [link](https://huggingface.co/datasets/dhvazquez/mtg_synthetic_cards_semantic_segmentation) |
| | - Trained model in this [link](https://huggingface.co/dhvazquez/mtg_semantic_segmentation) |
| | - Source Code in this [link](https://github.com/diegovazquez/mtg_card_image_segmentation) |
| |
|
| | ## Features |
| |
|
| | - **Real-time Detection**: Continuous card segmentation on live camera feed |
| | - **Browser-based AI**: All processing happens in your browser using ONNX Runtime Web |
| | - **Multi-camera Support**: Automatic camera detection and selection |
| | - **Visual Feedback**: Transparent cyan overlay showing detected card regions |
| | - **Performance Monitoring**: Real-time FPS and inference statistics |
| |
|
| | ## How It Works |
| |
|
| | 1. **Camera Selection**: Choose from available cameras on your device |
| | 2. **Video Processing**: Live video feed with automatic resolution handling |
| | 3. **AI Inference**: Real-time semantic segmentation using a lightweight MobileNetV3 model |
| | 4. **Visual Overlay**: Detected cards are highlighted with a transparent cyan mask |
| |
|
| | ## Technical Details |
| |
|
| | - **Model**: LRASPP MobileNetV3 Large architecture |
| | - **Input Size**: 480x640 pixels with ImageNet normalization |
| | - **Output**: 2-class segmentation (Background, Card) |
| | - **Runtime**: ONNX Runtime Web with GPU acceleration support |
| | - **Performance**: Optimized for real-time inference with automatic provider selection |
| |
|
| | ### GPU Acceleration |
| |
|
| | The demo automatically detects and uses the best available acceleration in priority order: |
| |
|
| | 1. **WebGPU**: Next-generation GPU acceleration (primary choice for best performance) |
| | 2. **WebGL**: Widely supported GPU acceleration (fallback with 2-5x performance boost when compatible) |
| | 3. **WebAssembly**: CPU fallback ensuring universal compatibility |
| |
|
| | **Note on GPU Compatibility**: Some models may use operators not supported by GPU providers (like 'HardSigmoid' in WebGL). In such cases, the demo automatically falls back to the next available provider while clearly indicating the reason in the acceleration status. |
| |
|
| | The active acceleration method is displayed in the interface, showing which provider is being used, GPU information when available, and the reason for any fallbacks. |
| |
|
| | ## Usage |
| |
|
| | 1. Grant camera permissions when prompted |
| | 2. Select your preferred camera from the dropdown |
| | 3. Click "Start Camera" to begin video feed |
| | 4. Click "Start Detection" to enable real-time card segmentation |
| | 5. Point your camera at cards to see the segmentation overlay |
| |
|
| | ## Privacy |
| |
|
| | - All processing happens locally in your browser |
| | - No data is sent to external servers |
| | - Camera access is only used for real-time processing |
| |
|
| | ## Performance Tips |
| |
|
| | - Use good lighting for better detection accuracy |
| | - Keep cards clearly visible and unobstructed |
| | - Close other browser tabs for better performance |
| |
|
| | ## Troubleshooting |
| |
|
| | **Camera not detected:** |
| | - Ensure camera permissions are granted |
| | - Try refreshing the page |
| | - Check if camera is being used by another application |
| |
|
| | **Low performance:** |
| | - Close other browser tabs and applications |
| | - Use a device with better hardware acceleration |
| | - Ensure good lighting conditions |
| |
|
| | **Model not loading:** |
| | - Check internet connection for initial model download |
| | - Clear browser cache and reload |
| | - Try a different browser |
| |
|
