Update README.md
Browse files
README.md
CHANGED
|
@@ -37,6 +37,34 @@ This huggingface repository includes video storyboard classification models, fra
|
|
| 37 |
</table>
|
| 38 |
</div>
|
| 39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
The video processing pipeline leading to the clean videos in the Surg-3M dataset is as follows:
|
| 41 |
<div align="center">
|
| 42 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/67d9504a41d31cc626fcecc8/yj2S0GMJm2C2AYwbr1p6G.png"> </img>
|
|
|
|
| 37 |
</table>
|
| 38 |
</div>
|
| 39 |
|
| 40 |
+
##Video classification model
|
| 41 |
+
|
| 42 |
+
```python
|
| 43 |
+
import torch
|
| 44 |
+
from PIL import Image
|
| 45 |
+
from model_loader import build_model
|
| 46 |
+
|
| 47 |
+
# Load the model
|
| 48 |
+
net = build_model(mode='classify')
|
| 49 |
+
model_path = 'Video storyboard classification models'
|
| 50 |
+
|
| 51 |
+
# Enable multi-GPU support
|
| 52 |
+
net = torch.nn.DataParallel(net)
|
| 53 |
+
torch.backends.cudnn.benchmark = True
|
| 54 |
+
state = torch.load(model_path, map_location=torch.device('cuda'))
|
| 55 |
+
net.load_state_dict(state['net'])
|
| 56 |
+
net.eval()
|
| 57 |
+
|
| 58 |
+
# Load the video storyboard and convert it to a PyTorch tensor
|
| 59 |
+
img_path = 'path/to/your/image.jpg'
|
| 60 |
+
img = Image.open(img_path)
|
| 61 |
+
img = img.resize((224, 224))
|
| 62 |
+
img_tensor = torch.tensor(np.array(img)).unsqueeze(0).to('cuda')
|
| 63 |
+
|
| 64 |
+
# Extract features from the image using the ResNet50 model
|
| 65 |
+
outputs = net(img_tensor)
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
The video processing pipeline leading to the clean videos in the Surg-3M dataset is as follows:
|
| 69 |
<div align="center">
|
| 70 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/67d9504a41d31cc626fcecc8/yj2S0GMJm2C2AYwbr1p6G.png"> </img>
|