Upload README.md

README.md

@@ -0,0 +1,340 @@
+---
+license: mit
+datasets:
+- Bingsu/Gameplay_Images
+language:
+- en
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+- roc_auc
+- confusion_matrix
+base_model:
+- google/efficientnet-b0
+pipeline_tag: image-classification
+tags:
+- game-detection
+- image-classification
+- efficientnet
+- hashtag-generation
+- computer-vision
+- gaming
+---
+
+# Game_Detection
+
+### Automated Video Game Recognition for Hashtag Suggestion on Live Streaming Platforms
+
+A 10-class image classifier that identifies which video game is being played from a gameplay
+screenshot. Built on a fine-tuned [`google/efficientnet-b0`](https://huggingface.co/google/efficientnet-b0)
+backbone, trained at a custom, aspect-ratio-preserving **180×320** input resolution (instead of the
+standard 224×224 square crop) on the [`Bingsu/Gameplay_Images`](https://huggingface.co/datasets/Bingsu/Gameplay_Images)
+dataset.
+
+This model was built as part of a university course project (AI Lab, SE334) — *"Automated Video Game
+Recognition and Hashtag Suggestion for Live Streaming Platforms Using Image Classification"* — and
+powers the [GameSense](https://gamesense-h456.onrender.com/) demo app.
+
+**Authors:** Afrim Hossen Khan
+
+## Model Details
+
+- **Base model:** `google/efficientnet-b0`
+- **Task:** Multi-class image classification (10 classes)
+- **License:** MIT
+- **Architecture:** EfficientNet-B0 backbone (ImageNet-pretrained), fine-tuned end-to-end with the
+  final classifier layer replaced for 10 output classes. Trained at a custom **180×320** input
+  resolution — half of the source dataset's native 640×360, preserving the true 16:9 aspect ratio —
+  made possible without architectural changes since EfficientNet's `AdaptiveAvgPool2d` head is
+  resolution-agnostic.
+- **Fine-tuning objective:** Cross-entropy loss with label smoothing (0.1), `sklearn` balanced class
+  weights applied in the loss (the source dataset is already perfectly balanced at 1,000 images/class)
+- **Training regime:** Mixed-precision (AMP) training on dual CUDA T4 GPUs, AdamW optimizer with a
+  OneCycleLR schedule, up to 25 epochs with early stopping (patience = 6, monitored on validation loss)
+
+## Classes
+
+`Among Us, Apex Legends, Fortnite, Forza Horizon, Free Fire, Genshin Impact, God of War, Minecraft,
+Roblox, Terraria`
+
+## Intended Use
+
+This model is intended for identifying which video game is shown in a gameplay screenshot. Example use
+cases:
+
+- Auto-generating hashtags/tags for gameplay clips, stream thumbnails, and social posts
+- Categorizing or organizing gameplay footage/screenshots by game on a content platform
+- A component in a larger stream metadata or content-tagging pipeline
+- Research and coursework on multi-class visual classification
+
+**Out of scope:** This model only recognizes the 10 games listed above — any other game will be forced
+into one of these 10 labels rather than correctly rejected. It has been evaluated on one dataset only,
+and has not been validated against real-world production streaming footage, unusual camera angles,
+menu/loading screens, or extensive in-game cosmetic content (e.g. crossover skins) that may visually
+resemble a different game in the label set.
+
+## How to Use
+
+This model is distributed in two formats — pick whichever fits your stack.
+
+### Option A: ONNX (lightweight, CPU-friendly)
+
+Download both files and keep them in the same folder — the `.onnx` graph loads its weights from the
+`.onnx.data` file alongside it at runtime:
+
+- [`efficientnet_b0_gameplay.onnx`](https://huggingface.co/afrim3000/Game/resolve/main/efficientnet_b0_gameplay.onnx) — the ONNX graph
+- [`efficientnet_b0_gameplay.onnx.data`](https://huggingface.co/afrim3000/Game/resolve/main/efficientnet_b0_gameplay.onnx.data) — the external weights file
+
+Install dependencies:
+
+```bash
+pip install onnxruntime huggingface_hub pillow numpy
+```
+
+#### Single-image prediction
+
+```python
+import numpy as np
+import onnxruntime as ort
+from PIL import Image
+from huggingface_hub import hf_hub_download
+
+REPO_ID = "afrim3000/Game"
+IMG_SIZE = (320, 180)  # PIL resize takes (width, height)
+IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32)
+IMAGENET_STD  = np.array([0.229, 0.224, 0.225], dtype=np.float32)
+CLASS_NAMES = ['Among Us', 'Apex Legends', 'Fortnite', 'Forza Horizon', 'Free Fire',
+               'Genshin Impact', 'God of War', 'Minecraft', 'Roblox', 'Terraria']
+
+# Downloads both files into the same local cache folder — required, since the
+# .onnx graph references .onnx.data by relative path at load time.
+onnx_path = hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay.onnx")
+hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay.onnx.data")
+
+session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
+input_name = session.get_inputs()[0].name
+output_name = session.get_outputs()[0].name
+
+def preprocess_pil(img: Image.Image) -> np.ndarray:
+    img = img.convert("RGB").resize(IMG_SIZE)
+    arr = np.asarray(img, dtype=np.float32) / 255.0        # HWC, [0,1]
+    arr = (arr - IMAGENET_MEAN) / IMAGENET_STD              # normalize, same stats as training
+    return arr.transpose(2, 0, 1)                           # HWC -> CHW
+
+def softmax(x: np.ndarray) -> np.ndarray:
+    e = np.exp(x - x.max(axis=1, keepdims=True))
+    return e / e.sum(axis=1, keepdims=True)
+
+def predict(image_path: str):
+    image = Image.open(image_path)
+    x = preprocess_pil(image)[np.newaxis, ...].astype(np.float32)
+    logits = session.run([output_name], {input_name: x})[0]
+    probs = softmax(logits)[0]
+    top_idx = int(probs.argmax())
+    return CLASS_NAMES[top_idx], probs
+
+label, probs = predict("path/to/screenshot.jpg")
+print(f"Prediction: {label}")
+for name, p in sorted(zip(CLASS_NAMES, probs), key=lambda t: -t[1]):
+    print(f"  {name:<16} {p*100:5.1f}%")
+```
+
+#### Batch prediction
+
+```python
+image_paths = ["shot1.jpg", "shot2.jpg", "shot3.jpg"]
+
+batch = np.stack([preprocess_pil(Image.open(p)) for p in image_paths]).astype(np.float32)
+logits = session.run([output_name], {input_name: batch})[0]
+probs = softmax(logits)
+preds = probs.argmax(axis=1)
+
+for path, pred, p in zip(image_paths, preds, probs):
+    print(f"{path}: {CLASS_NAMES[int(pred)]}  ({p[int(pred)]*100:.1f}%)")
+```
+
+> For GPU inference, install `onnxruntime-gpu` instead and pass
+> `providers=["CUDAExecutionProvider", "CPUExecutionProvider"]` when creating the session.
+
+### Option B: PyTorch (.pth checkpoint)
+
+Download the checkpoint:
+
+- [`efficientnet_b0_gameplay_final.pth`](https://huggingface.co/afrim3000/Game/resolve/main/efficientnet_b0_gameplay_final.pth)
+
+Install dependencies:
+
+```bash
+pip install torch torchvision huggingface_hub pillow numpy
+```
+
+#### Single-image prediction
+
+```python
+import torch
+import torch.nn as nn
+import numpy as np
+from torchvision import models, transforms
+from PIL import Image
+from huggingface_hub import hf_hub_download
+
+REPO_ID = "afrim3000/Game"
+IMG_SIZE = (180, 320)  # (H, W) — torchvision transforms convention
+CLASS_NAMES = ['Among Us', 'Apex Legends', 'Fortnite', 'Forza Horizon', 'Free Fire',
+               'Genshin Impact', 'God of War', 'Minecraft', 'Roblox', 'Terraria']
+
+ckpt_path = hf_hub_download(repo_id=REPO_ID, filename="efficientnet_b0_gameplay_final.pth")
+checkpoint = torch.load(ckpt_path, map_location="cpu")
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+model = models.efficientnet_b0(weights=None)
+in_features = model.classifier[1].in_features
+model.classifier[1] = nn.Linear(in_features, len(CLASS_NAMES))
+model.load_state_dict(checkpoint["model_state_dict"])
+model.to(device).eval()
+
+transform = transforms.Compose([
+    transforms.Resize(IMG_SIZE),
+    transforms.ToTensor(),
+    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
+])
+
+@torch.no_grad()
+def predict(image_path: str):
+    image = Image.open(image_path).convert("RGB")
+    x = transform(image).unsqueeze(0).to(device)
+    logits = model(x)
+    probs = torch.softmax(logits, dim=1)[0]
+    top_idx = int(probs.argmax())
+    return CLASS_NAMES[top_idx], probs.cpu().numpy()
+
+label, probs = predict("path/to/screenshot.jpg")
+print(f"Prediction: {label}")
+for name, p in sorted(zip(CLASS_NAMES, probs), key=lambda t: -t[1]):
+    print(f"  {name:<16} {p*100:5.1f}%")
+```
+
+#### Batch prediction
+
+```python
+from torch.utils.data import Dataset, DataLoader
+
+class ImageListDataset(Dataset):
+    def __init__(self, paths, transform):
+        self.paths = paths
+        self.transform = transform
+
+    def __len__(self):
+        return len(self.paths)
+
+    def __getitem__(self, i):
+        img = Image.open(self.paths[i]).convert("RGB")
+        return self.transform(img), self.paths[i]
+
+image_paths = ["shot1.jpg", "shot2.jpg", "shot3.jpg"]
+loader = DataLoader(ImageListDataset(image_paths, transform), batch_size=8)
+
+model.eval()
+with torch.no_grad():
+    for images, paths in loader:
+        images = images.to(device)
+        logits = model(images)
+        probs = torch.softmax(logits, dim=1)
+        preds = probs.argmax(dim=1)
+        for path, pred, p in zip(paths, preds, probs):
+            print(f"{path}: {CLASS_NAMES[int(pred)]}  ({p[int(pred)]*100:.1f}%)")
+```
+
+## Training Data
+
+The model was fine-tuned on the [`Bingsu/Gameplay_Images`](https://huggingface.co/datasets/Bingsu/Gameplay_Images)
+dataset — 10,000 gameplay screenshots (1,000 per class) at native 640×360 resolution, PNG format.
+
+- **Labels:** 10 classes (see [Classes](#classes) above)
+- **Splits:** Stratified 70 / 15 / 15 train / validation / test (the source dataset ships a single
+  `train` split only; the split above was carved out manually, preserving per-class balance)
+- **Preprocessing:** Resize to 180×320 (custom, aspect-ratio-preserving resolution), ImageNet
+  normalization (mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`)
+- **Training augmentation:** Random horizontal flip, color jitter, random rotation (±8°), random
+  erasing
+- **Class balancing:** The dataset is already perfectly balanced (1,000 images/class); `sklearn`
+  balanced class weights are still computed and applied in the loss as a safeguard
+
+## Training Procedure
+
+<!-- PLACEHOLDER: training curves (loss/accuracy per epoch) — image to be uploaded -->
+
+![training_curves](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/lBckExVfa7nctNBbRq36B.png)
+
+
+- **Framework:** PyTorch
+- **Hardware:** Kaggle free-tier T4 x2 GPUs
+- **Loss:** Cross-entropy with label smoothing (0.1)
+- **Mixed precision:** Enabled (AMP)
+
+
+## Evaluation
+
+Evaluated on the held-out test split (n = 1,500) at a decision threshold of 0.5.
+
+### Classification Report
+
+| Class          | Precision | Recall | F1-score | Support |
+|----------------|:---------:|:------:|:--------:|:-------:|
+| Among Us       | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Apex Legends   | 1.0000    | 0.9933 | 0.9967   | 150     |
+| Fortnite       | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Forza Horizon  | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Free Fire      | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Genshin Impact | 0.9934    | 1.0000 | 0.9967   | 150     |
+| God of War     | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Minecraft      | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Roblox         | 1.0000    | 1.0000 | 1.0000   | 150     |
+| Terraria       | 1.0000    | 1.0000 | 1.0000   | 150     |
+| **accuracy**   |           |        | **0.9993** | 1,500 |
+| macro avg      | 0.9993    | 0.9993 | 0.9993   | 1,500   |
+| weighted avg   | 0.9993    | 0.9993 | 0.9993   | 1,500   |
+
+**Test ROC-AUC:** 1.0000 (macro average; per-class AUC is also 1.0000 across all 10 classes)
+
+### Confusion Matrix
+
+<!-- PLACEHOLDER: image to be uploaded -->
+
+
+![confusion_matrix](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/GgkCX5ir4A12Iip96hrQy.png)
+
+### ROC Curve
+
+<!-- PLACEHOLDER: image to be uploaded -->
+
+
+![roc_auc_curves](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/Q0J0IoHtaHhkq_IaTP-Qf.png)
+
+## Limitations
+
+- Performance is reported on a single dataset; generalization to other capture sources, image
+  qualities, camera angles, or game versions/UI updates is not guaranteed.
+- The classifier is closed-set — it will always assign one of the 10 trained classes, even to games or
+  content it has never seen, rather than rejecting out-of-distribution input.
+- Confidence can be lower on visually ambiguous content, such as games with extensive cosmetic/skin
+  systems whose art style can resemble another class in the label set.
+- The model has not been evaluated as a standalone production guardrail; low-confidence predictions
+  should be handled with a confidence threshold or human review rather than trusted outright.
+
+## Citation
+
+If you use this model, please cite this repository and reference this course project:
+
+```
+@misc{game-detection-classifier,
+  title  = {Automated Video Game Recognition and Hashtag Suggestion for Live Streaming Platforms
+            Using Image Classification},
+  author = {Afrim Hossen Khan},
+  year   = {2026},
+  note   = {Course project, AI Lab (SE334), Daffodil International University}
+}
+```