README.md

metadata

title: MTG Card Segmentation
emoji: 🎴
colorFrom: blue
colorTo: purple
sdk: static
pinned: false
license: mit
short_description: MTG Real-time Card Semantic Segmentation

Card Segmentation - Real-time Detection

A real-time card segmentation application that uses semantic segmentation to detect and highlight cards in camera feeds. Built with ONNX Runtime Web for in-browser AI inference.

• Pre-generated dataset in this link
• Trained model in this link
• Source Code in this link

Features

• Real-time Detection: Continuous card segmentation on live camera feed
• Browser-based AI: All processing happens in your browser using ONNX Runtime Web
• Multi-camera Support: Automatic camera detection and selection
• Visual Feedback: Transparent cyan overlay showing detected card regions
• Performance Monitoring: Real-time FPS and inference statistics

How It Works

1. Camera Selection: Choose from available cameras on your device
2. Video Processing: Live video feed with automatic resolution handling
3. AI Inference: Real-time semantic segmentation using a lightweight MobileNetV3 model
4. Visual Overlay: Detected cards are highlighted with a transparent cyan mask

Technical Details

• Model: LRASPP MobileNetV3 Large architecture
• Input Size: 480x640 pixels with ImageNet normalization
• Output: 2-class segmentation (Background, Card)
• Runtime: ONNX Runtime Web with GPU acceleration support
• Performance: Optimized for real-time inference with automatic provider selection

GPU Acceleration

The demo automatically detects and uses the best available acceleration in priority order:

1. WebGPU: Next-generation GPU acceleration (primary choice for best performance)
2. WebGL: Widely supported GPU acceleration (fallback with 2-5x performance boost when compatible)
3. WebAssembly: CPU fallback ensuring universal compatibility

Note on GPU Compatibility: Some models may use operators not supported by GPU providers (like 'HardSigmoid' in WebGL). In such cases, the demo automatically falls back to the next available provider while clearly indicating the reason in the acceleration status.

The active acceleration method is displayed in the interface, showing which provider is being used, GPU information when available, and the reason for any fallbacks.

Usage

1. Grant camera permissions when prompted
2. Select your preferred camera from the dropdown
3. Click "Start Camera" to begin video feed
4. Click "Start Detection" to enable real-time card segmentation
5. Point your camera at cards to see the segmentation overlay

Privacy

• All processing happens locally in your browser
• No data is sent to external servers
• Camera access is only used for real-time processing

Performance Tips

• Use good lighting for better detection accuracy
• Keep cards clearly visible and unobstructed
• Close other browser tabs for better performance

Troubleshooting

Camera not detected:

• Ensure camera permissions are granted
• Try refreshing the page
• Check if camera is being used by another application

Low performance:

• Close other browser tabs and applications
• Use a device with better hardware acceleration
• Ensure good lighting conditions

Model not loading:

• Check internet connection for initial model download
• Clear browser cache and reload
• Try a different browser