Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,134 @@
+---
+license: apache-2.0
+library_name: pytorch
+base_model: google/siglip2-base-patch16-224
+tags:
+- nutrition-estimation
+- calorie-estimation
+- food
+- nutrition5k
+- image-regression
+datasets:
+- Nutrition5k
+language:
+- en
+metrics:
+- mae
+---
+
+# kcalorie — food-photo calorie & macro estimation
+
+A single RGB food photo in, five numbers out: **calories (kcal), mass (g), protein (g), carbohydrate (g), fat (g)**.
+kcalorie is a SigLIP2-base vision tower with a small regression head, fine-tuned end-to-end and evaluated
+under a strict, leakage-audited Nutrition5k protocol.
+
+![Comparison on the locked Nutrition5k test set](comparison.png)
+
+## Headline result — locked Nutrition5k test set (608 dishes, official test split, never seen)
+
+| Metric | kcalorie | CalorieCLIP (same 608 dishes) |
+|---|---|---|
+| Calorie MAE | **51.4 kcal** (95% CI 35.8–78.9) | 181.2 kcal |
+| Median absolute error | **25.9 kcal** | 137.0 kcal |
+| Within ±50 kcal | **76.8%** | 21.4% |
+| Within ±100 kcal | **92.4%** | 40.6% |
+| Bias | **−0.7 kcal** | +134.8 kcal |
+| Calibrated 90% interval | **±105.9 kcal** (actual coverage 93.6%) | — |
+| Invalid predictions | 0 / 608 | 0 / 608 |
+
+Every test image is a deterministic central frame from the official Nutrition5k camera-A side-angle
+video of an official *test-split* dish — selected by dish ID, never by image quality. Evaluation is
+dish-level (`physical_meal_id`).
+
+## Comparison with open VLMs (paired 304-dish subset, seed 17)
+
+Each model answered the same calorie question on the same dishes
+(prompt: *"Estimate the total calories in this food image. Return exactly one integer number in kcal…"*).
+
+| Model | n | MAE (kcal) | Median AE | Within ±100 kcal | Bias |
+|---|---|---|---|---|---|
+| **kcalorie (ours)** | 304 | **64.4** | **23.5** | **92.4%** | −13.6 |
+| Qwen3.5-2B | 304 | 116.4 | 65.0 | 69.4% | −48.1 |
+| MiniCPM-V 4.6 (1.3B) | 304 | 163.3 | 97.0 | 52.6% | +75.5 |
+| Gemma-4-12B † | 100 | 176.8 | 90.5 | 58.0% | −43.3 |
+| CalorieCLIP | 304 | 196.2 | 133.2 | 42.1% | +121.1 |
+
+† Gemma-4-12B was scored on a seeded 100-dish paired subset of the same 304 dishes (compute budget);
+all parse rates were 100%.
+
+## Training method
+
+- **Architecture**: `google/siglip2-base-patch16-224` vision tower (92M params) → LayerNorm →
+  MLP head (768→768→256→5) with softplus outputs. Trained with Huber loss, per-task, quality-weighted.
+- **Schedule**: 15 epochs × 20,000 samples, batch 256 (bf16), head LR 1e-4 / trunk LR 2e-5,
+  cosine schedule with 5% warmup, first 5% head-only, last 4 vision blocks unfrozen, weight decay 0.05, seed 17.
+- **Checkpoint selection**: best validation MAE on 80 held-out camera-A dishes (validation MAE 39.2 kcal).
+  Training ran ~34 minutes on one RTX PRO 6000 Blackwell (CUDA 13).
+- **Calibration**: a conformal radius fitted on 86 held-out calibration dishes gives 90% prediction
+  intervals of ±105.9 kcal, with 93.6% actual coverage on the locked test set (`calibration.json`).
+
+![Training loss](loss_curve.png)
+
+## Training data (20,000 samples, leakage-audited)
+
+| Source | Samples | Notes |
+|---|---|---|
+| [`pinkieseb/nutrition_dataset`](https://huggingface.co/datasets/pinkieseb/nutrition_dataset) (Nutrition5k video frames) | 19,734 | only dishes provably in the **official Nutrition5k train split** (matched by exact nutrition signature), ≤16 frames per dish |
+| [`aryachakraborty/Food_Calorie_Dataset`](https://huggingface.co/datasets/aryachakraborty/Food_Calorie_Dataset) | 266 | human-estimated labels, down-weighted (0.4) |
+
+Leakage controls, all enforced programmatically before training:
+- **Group separation**: splits partition by physical dish (`physical_meal_id`); no dish appears in two splits.
+- **Exact-media**: SHA-256 disjointness across fit / validation / calibration / test.
+- **Near-duplicate**: perceptual-hash disjointness; 2 fit images visually colliding with held-out frames were dropped and backfilled.
+- **Official-split proof**: a Nutrition5k registry maps nutrition signatures to official train/test IDs;
+  ambiguous signatures are excluded. The audit trail (SHA-256 of every manifest and the source tree) ships in the results.
+
+## Usage
+
+```python
+import torch, torch.nn as nn, torch.nn.functional as F
+from transformers import AutoImageProcessor, SiglipVisionModel
+from huggingface_hub import hf_hub_download
+from PIL import Image
+
+class NutritionHead(nn.Module):
+    def __init__(self, input_dim: int = 768):
+        super().__init__()
+        self.net = nn.Sequential(
+            nn.LayerNorm(input_dim), nn.Linear(input_dim, 768), nn.GELU(), nn.Dropout(.15),
+            nn.Linear(768, 256), nn.GELU(), nn.Dropout(.10), nn.Linear(256, 5))
+    def forward(self, x): return F.softplus(self.net(x))
+
+MODEL_ID = "google/siglip2-base-patch16-224"
+processor = AutoImageProcessor.from_pretrained(MODEL_ID)
+vision = SiglipVisionModel.from_pretrained(MODEL_ID)
+head = NutritionHead()
+state = torch.load(hf_hub_download("Dralkh/kcalorie", "kcalorie.pt"), map_location="cpu", weights_only=False)
+vision.load_state_dict(state["vision_state"]); head.load_state_dict(state["head_state"])
+vision.eval(); head.eval()
+
+image = Image.open("meal.jpg").convert("RGB")
+pixels = processor(images=image, return_tensors="pt").pixel_values
+with torch.inference_mode():
+    out = vision(pixel_values=pixels)
+    feats = out.pooler_output if out.pooler_output is not None else out.last_hidden_state[:, 0]
+    kcal, mass_g, protein_g, carb_g, fat_g = head(feats.float())[0].tolist()
+print(f"{kcal:.0f} kcal ± 106 (90% interval), {mass_g:.0f} g, P{protein_g:.0f}/C{carb_g:.0f}/F{fat_g:.0f}")
+```
+
+## Limitations
+
+- **Domain**: trained and evaluated on Nutrition5k-style plated single dishes photographed from the side;
+  accuracy on multi-plate scenes, packaged food, or drinks is untested.
+- **Heavy tail**: RMSE (327) is far above MAE (51) — very high-calorie dishes are occasionally missed badly.
+  Use the calibrated interval, not the point estimate, for anything that matters.
+- **Frames, not meals**: training frames come from videos of ~1,250 distinct dishes; visual diversity is
+  narrower than the sample count suggests.
+- The VLM baselines answered a zero-shot prompt; instruction-tuned prompting or few-shot could improve them.
+
+## Provenance
+
+Full pipeline (data prep, leakage audit, training, evaluation, benchmark) and all raw results with
+SHA-256 checksums: [`Dralkh/ccalorie-siglip2-b0-strict`](https://huggingface.co/datasets/Dralkh/ccalorie-siglip2-b0-strict).
+Trained 2026-07-12 for ≈ $1.20 of GPU time. Base checkpoint: CalorieCLIP comparison uses
+[`jc-builds/CalorieCLIP`](https://huggingface.co/jc-builds/CalorieCLIP) (OpenAI CLIP ViT-B/32 + MLP regressor).