wip

README.md

@@ -44,6 +44,12 @@ cog push
 
 ## Training
 
+download the training data
+
+```bash
+gdown https://drive.google.com/uc?id=11M6nSuSuvoU2wpcV_-6KFqCzEMGP75q6?usp=drive_link -O ./data/
+```
+
 ```bash
 # Run training with default configuration
 python scripts/train.py
@@ -105,7 +111,11 @@ To run predictions with cog or locally on an existing checkpoint, you can find a
 
 ## License
 
-[Your License Here]
+MIT License
+
+Copyright (c) 2024 Bryant Wolf
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
 
 ## Citation
 
@@ -113,4 +123,4 @@ If you use this model in your research, please cite:
 
 ```bibtex
 [Your Citation Here]
-```
+```

model-card.md

@@ -0,0 +1,97 @@
+---
+language: en
+tags:
+  - clip
+  - breakdance
+  - video-classification
+  - dance
+license: MIT
+datasets:
+  - custom
+---
+
+# CLIP-Based Break Dance Move Classifier
+
+This model is a fine-tuned version of CLIP (ViT-Large/14) specialized in classifying break dance power moves from video frames, including windmills, halos, and swipes.
+
+## Model Description
+
+- **Model Type:** Fine-tuned CLIP model
+- **Base Model:** ViT-Large/14
+- **Task:** Video Classification
+- **Training Data:** Custom break dance video dataset
+- **Output:** 3 classes of break dance moves
+
+## Usage
+
+```python
+from transformers import CLIPProcessor, CLIPModel
+import torch
+import cv2
+from PIL import Image
+
+# Load model and processor
+processor = CLIPProcessor.from_pretrained("[your-username]/clip-breakdance-classifier")
+model = CLIPModel.from_pretrained("[your-username]/clip-breakdance-classifier")
+
+# Load video and process frames
+video = cv2.VideoCapture("breakdance_move.mp4")
+predictions = []
+
+while video.isOpened():
+    ret, frame = video.read()
+    if not ret:
+        break
+
+    # Convert BGR to RGB and to PIL Image
+    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+    frame_pil = Image.fromarray(frame_rgb)
+
+    # Process frame
+    inputs = processor(images=frame_pil, return_tensors="pt")
+    outputs = model(**inputs)
+    predictions.append(outputs)
+
+video.release()
+```
+
+## Limitations
+
+- Model performance may vary with video quality and lighting conditions
+- Best results are achieved with clear, centered shots of the dance moves
+- May have difficulty distinguishing between similar power moves
+- Performance may be affected by unusual camera angles or partial views
+- Currently only supports three specific power moves (windmills, halos, and swipes)
+
+## Training Procedure
+
+- Fine-tuned on CLIP ViT-Large/14 architecture
+- Training dataset: Custom dataset of break dance videos
+- Dataset size: [specify number] frames from [specify number] different videos
+- Training epochs: [specify number]
+- Learning rate: [specify rate]
+- Batch size: [specify size]
+- Hardware used: [specify GPU/CPU details]
+
+## Evaluation Results
+
+- Overall accuracy: [specify %]
+  Per-class performance:
+- Windmills: [specify precision/recall]
+- Halos: [specify precision/recall]
+- Swipes: [specify precision/recall]
+
+## Citation
+
+If you use this model in your research or project, please cite:
+
+```bibtex
+@misc{clip-breakdance-classifier,
+  author = {Bryant Wolf},
+  title = {CLIP-Based Break Dance Move Classifier},
+  year = {2024},
+  publisher = {Hugging Face},
+  journal = {Hugging Face Model Hub},
+  howpublished = {\url{https://huggingface.co/[your-username]/clip-breakdance-classifier}}
+}
+```

{script → scripts}/hyperparameter_tuning.py

@@ -8,7 +8,7 @@ import math
 
 import sys
 sys.path.append(os.path.dirname(os.path.dirname(__file__)))
-from script.train import train_and_evaluate
+from scripts.train import train_and_evaluate
 from src.utils.utils import create_run_directory
 
 def create_hyperparam_directory():

scripts/upload_to_hub.py

@@ -0,0 +1,17 @@
+from transformers import CLIPProcessor, CLIPModel
+from huggingface_hub import HfApi
+
+def upload_model_to_hub():
+    # Initialize huggingface api
+    api = HfApi()
+    
+    # Load your fine-tuned model
+    model = CLIPModel.from_pretrained("./checkpoints/")
+    processor = CLIPProcessor.from_pretrained("openai/clip-vit-large-patch14")
+    
+    # Push to hub
+    model.push_to_hub("[your-username]/clip-breakdance-classifier")
+    processor.push_to_hub("[your-username]/clip-breakdance-classifier")
+
+if __name__ == "__main__":
+    upload_model_to_hub()