Upload 6 files

README.md

@@ -1,15 +1,210 @@
-# Ice Cream Counter Annotator — Fixed23
+---
+title: Ice Cream Dataset + Counter
+emoji: 🍦
+colorFrom: yellow
+colorTo: red
+sdk: gradio
+sdk_version: 6.5.1
+app_file: app.py
+pinned: false
+---
 
-This version fixes the RT-DETR training crash:
+# 🍦 Ice Cream Dataset + Counter — Gradio Space
 
-`RuntimeError: Expected all tensors to be on the same device, but got index is on cpu, different from other tensors on cuda:0`
+This is the **Gradio + ZeroGPU-compatible** version of the uploaded Ice Cream Counter project. It does **not** use Docker, FastAPI, Uvicorn, or a custom HTML frontend.
 
-The issue occurs in Hugging Face RT-DETR's optional contrastive denoising path. Fixed23 disables that auxiliary path **before model construction** and also replaces the helper with the official `(None, None, None, None)` no-denoising return as a final safeguard.
+## What it does
 
-The normal RT-DETR detector loss and 10-class fine-tuning remain enabled.
+1. Upload training freezer photos.
+2. Define your product classes.
+3. Annotate each ice cream with bounding boxes.
+4. Train an RT-DETR object detector.
+5. Upload one new freezer image.
+6. Get the total count, per-product counts, confidence scores, and an annotated result image.
 
-The `80 -> 10` checkpoint shape messages are expected when adapting the COCO checkpoint to 10 custom classes.
+The model is still:
 
-## Install
+`PekingU/rtdetr_r50vd`
 
-Replace the files in the Hugging Face Space with this ZIP and restart/rebuild the Space.
+No YOLO is used.
+
+## Create the Hugging Face Space
+
+Create a new Space and choose:
+
+- **SDK:** Gradio
+- **Hardware:** ZeroGPU or a dedicated GPU is recommended for training
+
+The app uses `@spaces.GPU` for training and counting, so it also boots correctly when the Space hardware is **ZeroGPU**.
+
+Then upload these files/folders:
+
+```text
+app.py
+requirements.txt
+README.md
+training/
+```
+
+You do **not** need:
+
+```text
+Dockerfile
+FastAPI
+Uvicorn
+static/index.html
+app/main.py
+```
+
+## Persistent dataset and model
+
+The app stores:
+
+```text
+images/
+dataset.json
+model/
+generated_dataset/
+```
+
+When `/data` is available and writable, the app automatically uses:
+
+```text
+/data/icecream_counter/
+```
+
+You can also explicitly set:
+
+```text
+DATA_DIR=/data/icecream_counter
+```
+
+in the Space variables.
+
+### Important Hugging Face storage point
+
+A normal Space filesystem is not permanent storage across every rebuild/restart. If you need the dataset and trained model to survive Space restarts/rebuilds, attach **persistent storage** to the Space or move the data/model to an external persistent service.
+
+The Gradio conversion itself does not change this storage rule.
+
+## Annotation workflow
+
+In the **Annotate** tab the actual uploaded training image is displayed directly using Gradio's Image component. This avoids browser canvas/JavaScript issues that can make the preview appear black.
+
+1. Select a training image.
+2. The real image appears in the preview.
+3. Read the image dimensions shown below it.
+4. Select the product class.
+5. Enter the bounding box in original-image pixel coordinates:
+   - X (left)
+   - Y (top)
+   - Width
+   - Height
+6. Click **Save Box**.
+7. Saved boxes are drawn in red on the real image.
+8. Repeat for every ice cream.
+9. Use **Delete Box** or **Clear All Boxes** when needed.
+
+This version prioritizes a reliable visible image over the previous JavaScript canvas approach.
+
+## Training
+
+The app creates an 80/20 COCO train/validation split from the annotated images and starts the existing RT-DETR training script.
+
+Default settings:
+
+```text
+Epochs: 30
+Batch size: 2
+Learning rate: 1e-5
+```
+
+For an initial test on a small dataset, use fewer epochs such as 2–5. Once everything works, increase the epochs.
+
+A GPU Space is strongly recommended.
+
+
+## Training fix
+
+This release includes a CUDA-device fix for RT-DETR's contrastive-denoising training path. On some
+Transformers/PyTorch combinations, the denoising class-index tensor can remain on CPU while the
+RT-DETR class embedding is on CUDA, producing:
+
+`RuntimeError: Expected all tensors to be on the same device ... cpu ... cuda:0`
+
+The training script now moves nested target tensors explicitly and patches the RT-DETR denoising
+helper so its target tensors follow the class-embedding device. The `num_labels=10` vs. checkpoint
+`80` message is expected when fine-tuning the COCO-pretrained checkpoint for 10 custom classes;
+`ignore_mismatched_sizes=True` intentionally reinitializes the classification heads.
+
+## Counting
+
+After training, open the **Count** tab and upload one image.
+
+The result contains:
+
+```json
+{
+  "total": 31,
+  "counts": {
+    "cone": 6,
+    "correto": 7,
+    "cornetto": 12,
+    "magnum": 6
+  }
+}
+```
+
+The result image also shows the detected bounding boxes and confidence values.
+
+## Default classes
+
+```text
+cornetto
+magnum
+correto
+cone
+cup
+sandwich
+stick
+other
+```
+
+You can change them from the Dataset tab.
+
+## Local test
+
+```bash
+pip install -r requirements.txt
+python app.py
+```
+
+Then open:
+
+```text
+http://localhost:7860
+```
+
+## Recommended Space setup
+
+For the first deployment:
+
+1. Create the Space as **Gradio**.
+2. Upload `app.py`, `requirements.txt`, `README.md`, and `training/`.
+3. Wait for dependencies to install.
+4. Open the Dataset tab.
+5. Upload 2+ training images.
+6. Annotate them.
+7. Start with 2–5 epochs to verify training.
+8. After the model finishes, test the Count tab.
+9. For serious training, attach a GPU and persistent storage.
+
+
+
+### RT-DETR training compatibility
+The trainer disables RT-DETR contrastive denoising by default because some Transformers releases can create CPU class-index tensors while the embedding is on CUDA. The detector's normal supervised detection loss remains enabled. A device-safe denoising patch is also included for future re-enablement.
+
+
+## Fixed22 training patch
+
+This version fixes the RT-DETR CPU/CUDA denoising crash by wrapping the denoising class embedding as a real `torch.nn.Module` and moving its index tensor to the embedding weight device before `nn.Embedding` is called. A plain Python function wrapper is intentionally not used because Transformers expects the embedding to remain a module.

training/train.py

@@ -6,7 +6,7 @@ import torch
 from PIL import Image
 from torch.utils.data import Dataset, DataLoader
 from tqdm import tqdm
-from transformers import RTDetrImageProcessor, RTDetrForObjectDetection, RTDetrConfig
+from transformers import RTDetrImageProcessor, RTDetrForObjectDetection
 
 BASE_MODEL = "PekingU/rtdetr_r50vd"
 
@@ -88,21 +88,70 @@ def evaluate(model, loader, device):
 
 
 def patch_rtdetr_denoising_device():
-    """Disable RT-DETR contrastive denoising completely.
+    """Patch RT-DETR denoising so embedding indices are always on the embedding device.
 
-    The installed Transformers build is entering the denoising helper even
-    though the training targets are on CUDA. Denoising is auxiliary to the
-    detector loss, so return the official no-denoising tuple directly.
+    Important: the wrapper must be an nn.Module, not a plain Python function.
+    Transformers may inspect/use class_embed as a module, and a plain function
+    loses the original module parameters/device information.
     """
-    import transformers.models.rt_detr.modeling_rt_detr as rtdetr_mod
-
-    def disabled_denoising(*args, **kwargs):
-        return None, None, None, None
-
-    disabled_denoising._icecream_denoising_disabled = True
-    rtdetr_mod.get_contrastive_denoising_training_group = disabled_denoising
-    print("RT-DETR contrastive denoising DISABLED")
-
+    try:
+        import transformers.models.rt_detr.modeling_rt_detr as rtdetr_mod
+        import torch.nn as nn
+
+        original = rtdetr_mod.get_contrastive_denoising_training_group
+        if getattr(original, "_icecream_device_patch", False):
+            return
+
+        class DeviceSafeEmbedding(nn.Module):
+            def __init__(self, embedding):
+                super().__init__()
+                self.embedding = embedding
+
+            def forward(self, indices):
+                if torch.is_tensor(indices):
+                    indices = indices.to(self.embedding.weight.device)
+                return self.embedding(indices)
+
+        def wrapped(targets, num_classes, num_queries, class_embed,
+                    num_denoising_queries=100, label_noise_ratio=0.5,
+                    box_noise_scale=1.0, **kwargs):
+            # Move every target tensor to the embedding/model device.
+            try:
+                embed_device = class_embed.weight.device
+            except Exception:
+                try:
+                    embed_device = next(class_embed.parameters()).device
+                except Exception:
+                    embed_device = None
+
+            if embed_device is not None:
+                fixed_targets = []
+                for target in targets:
+                    if isinstance(target, dict):
+                        target = dict(target)
+                        for key, value in list(target.items()):
+                            if torch.is_tensor(value):
+                                target[key] = value.to(embed_device)
+                    fixed_targets.append(target)
+                targets = fixed_targets
+                class_embed = DeviceSafeEmbedding(class_embed)
+
+            return original(
+                targets=targets,
+                num_classes=num_classes,
+                num_queries=num_queries,
+                class_embed=class_embed,
+                num_denoising_queries=num_denoising_queries,
+                label_noise_ratio=label_noise_ratio,
+                box_noise_scale=box_noise_scale,
+                **kwargs,
+            )
+
+        wrapped._icecream_device_patch = True
+        rtdetr_mod.get_contrastive_denoising_training_group = wrapped
+        print("RT-DETR denoising device patch installed")
+    except Exception as exc:
+        raise RuntimeError(f"Could not install RT-DETR denoising device patch: {exc}") from exc
 
 def main():
     p=argparse.ArgumentParser()
@@ -126,25 +175,22 @@ def main():
     if len(train.category_id_to_label)!=len(classes) or len(val.category_id_to_label)!=len(classes):
         raise ValueError("COCO categories do not match classes.txt. Rebuild the dataset after saving the classes.")
 
-    # Build from a config with denoising disabled BEFORE model construction.
-    cfg = RTDetrConfig.from_pretrained(BASE_MODEL)
-    if hasattr(cfg, "num_denoising"):
-        cfg.num_denoising = 0
-    if hasattr(cfg, "num_denoising_queries"):
-        cfg.num_denoising_queries = 0
-    cfg.num_labels = len(classes)
-    cfg.id2label = id2label
-    cfg.label2id = label2id
-
     model=RTDetrForObjectDetection.from_pretrained(
-        BASE_MODEL, config=cfg,
+        BASE_MODEL,num_labels=len(classes),id2label=id2label,label2id=label2id,
         ignore_mismatched_sizes=True
     )
-    # Belt-and-suspenders: if this Transformers build still calls the helper,
-    # return the official no-denoising tuple instead of entering F.embedding.
-    patch_rtdetr_denoising_device()
+    # Some Transformers RT-DETR releases enter the denoising path whenever
+    # training, regardless of num_denoising. Keep the config disabled where
+    # supported, but also install the device-safe embedding patch below.
+    for cfg_owner in (model, getattr(model, "model", None)):
+        cfg = getattr(cfg_owner, "config", None)
+        if cfg is not None:
+            for name in ("num_denoising", "num_denoising_queries"):
+                if hasattr(cfg, name):
+                    setattr(cfg, name, 0)
     device=torch.device("cuda" if torch.cuda.is_available() else "cpu")
     model.to(device)
+    patch_rtdetr_denoising_device()
 
     tr=DataLoader(train,batch_size=a.batch_size,shuffle=True,num_workers=0,collate_fn=collate_fn)
     va=DataLoader(val,batch_size=a.batch_size,shuffle=False,num_workers=0,collate_fn=collate_fn)