Upload README.md with huggingface_hub

README.md

@@ -1,53 +1,44 @@
----
-license: apache-2.0
-library_name: pytorch
-base_model: google/siglip2-base-patch16-224
-tags:
-- nutrition-estimation
-- calorie-estimation
-- food
-- nutrition5k
-- image-regression
-datasets:
-- Nutrition5k
-language:
-- en
-metrics:
-- mae
----
-
 # CalorieK — food-photo calorie & macro estimation
 
 A single RGB food photo in, five numbers out: **calories (kcal), mass (g), protein (g), carbohydrate (g), fat (g)**.
 CalorieK is a SigLIP2-base vision tower with a small regression head, fine-tuned end-to-end and evaluated
 under a strict, leakage-audited Nutrition5k protocol.
 
-![Comparison on the locked Nutrition5k test set](comparison.png)
+[![Comparison on the locked Nutrition5k test set](https://huggingface.co/Dralkh/CalorieK/resolve/main/comparison.png)](https://huggingface.co/Dralkh/CalorieK/blob/main/comparison.png)
 
 ## Headline result — locked Nutrition5k test set (608 dishes, official test split, never seen)
 
 | Metric | CalorieK | CalorieCLIP (same 608 dishes) |
-|---|---|---|
-| Calorie MAE | **51.4 kcal** (95% CI 35.8–78.9) | 181.2 kcal |
-| Median absolute error | **25.9 kcal** | 137.0 kcal |
-| Within ±50 kcal | **76.8%** | 21.4% |
-| Within ±100 kcal | **92.4%** | 40.6% |
-| Bias | **−0.7 kcal** | +134.8 kcal |
+| --- | --- | --- |
+| Calorie MAE | **49.8 kcal** (95% CI 34.4–78.0) | 181.2 kcal |
+| Median absolute error | **24.5 kcal** | 137.0 kcal |
+| Within ±50 kcal | **78.5%** | 21.4% |
+| Within ±100 kcal | **94.2%** | 40.6% |
+| Bias | **−6.6 kcal** | +134.8 kcal |
 | Calibrated 90% interval | **±105.9 kcal** (actual coverage 93.6%) | — |
 | Invalid predictions | 0 / 608 | 0 / 608 |
 
 Every test image is a deterministic central frame from the official Nutrition5k camera-A side-angle
-video of an official *test-split* dish — selected by dish ID, never by image quality. Evaluation is
+video of an official _test-split_ dish — selected by dish ID, never by image quality. Evaluation is
 dish-level (`physical_meal_id`).
 
+### Macronutrient accuracy (same 608 dishes)
+
+| Task | MAE | Median AE | Bias |
+| --- | --- | --- | --- |
+| Mass | **90.4 g** | 34.5 g | −47.3 g |
+| Protein | **7.8 g** | 3.9 g | −1.9 g |
+| Carbohydrate | **5.8 g** | 2.8 g | −1.2 g |
+| Fat | **7.2 g** | 2.2 g | −3.1 g |
+
 ## Comparison with open VLMs (paired 304-dish subset, seed 17)
 
 Each model answered the same calorie question on the same dishes
-(prompt: *"Estimate the total calories in this food image. Return exactly one integer number in kcal…"*).
+(prompt: _"Estimate the total calories in this food image. Return exactly one integer number in kcal…"_).
 
 | Model | n | MAE (kcal) | Median AE | Within ±100 kcal | Bias |
-|---|---|---|---|---|---|
-| **CalorieK (ours)** | 304 | **64.4** | **23.5** | **92.4%** | −13.6 |
+| --- | --- | --- | --- | --- | --- |
+| **CalorieK (ours)** | 304 | **62.7** | **24.0** | **94.1%** | −11.0 |
 | Qwen3.5-2B | 304 | 116.4 | 65.0 | 69.4% | −48.1 |
 | Qwen3.5-9B (Q8_0, thinking off) ‡ | 304 | 117.2 | 64.5 | 65.8% | +6.1 |
 | MiniCPM-V 4.6 (1.3B) | 304 | 163.3 | 97.0 | 52.6% | +75.5 |
@@ -58,8 +49,6 @@ Each model answered the same calorie question on the same dishes
 † Gemma-4-12B was scored on a seeded 100-dish paired subset of the same 304 dishes (compute budget);
 all parse rates were 100%.
 ‡ Qwen3.5-9B ran as an 8-bit GGUF (Q8_0) via llama.cpp with 4k context and thinking disabled.
-Notably it matches the 2B (117.2 vs 116.4 MAE) — 4.5× more parameters bought no accuracy on this task,
-suggesting the zero-shot VLM error floor is portion-mass ambiguity, not model capacity.
 MiniMax M3 (a hosted reasoning model, chain-of-thought enabled) reinforces the point from the other
 direction: extensive per-dish reasoning did not help — it lands second-to-last with a strong +104 kcal
 systematic overestimate. Reasoning does not fix calibration.
@@ -67,53 +56,58 @@ systematic overestimate. Reasoning does not fix calibration.
 ## Out-of-distribution evaluation (MM-Food-100K, 500 wild food photos)
 
 To measure generalization beyond Nutrition5k plating, both regression models were scored on a seeded
-500-image sample of [MM-Food-100K](https://huggingface.co/datasets/Humanbased-AI/MM-Food-100K)
-(restaurant/home photos; `food_prob ≥ 0.9`). **Caveat: these labels are crowd/AI estimates, not weighed
-measurements** — absolute errors include label noise no model can beat, so the relative comparison is
+500-image sample of [MM-Food-100K](https://huggingface.co/datasets/Codatta/MM-Food-100K)
+(restaurant/home photos; `food_prob ≥ 0.9`). **Caveat: these labels are crowd/AI estimates, not weighed**
+**measurements** — absolute errors include label noise no model can beat, so the relative comparison is
 the meaningful signal.
 
-| | CalorieK | CalorieCLIP | |
-|---|---|---|---|
-| MAE | **164.9 kcal** | 206.4 kcal | 1.25× better |
-| Median error | **139.1 kcal** | 179.9 kcal | 1.3× better |
-| Within ±100 kcal | **36.0%** | 25.6% | |
-| Bias | **−4.8 kcal** | +98.3 kcal | |
-| R² | **+0.24** | −0.08 | |
+|  | CalorieK | CalorieCLIP |
+| --- | --- | --- |
+| MAE | **162.5 kcal** | 206.4 kcal |
+| Median error | **135.0 kcal** | 179.9 kcal |
+| Within ±100 kcal | **38.8%** | 25.6% |
+| Bias | **−8.1 kcal** | +98.3 kcal |
+| R² | **+0.24** | −0.08 |
 
 Findings, stated honestly:
+
 - CalorieK beats CalorieCLIP out-of-distribution on every metric, keeps **near-zero bias** on food it
-  has never seen, and retains real signal (R² +0.24) where CalorieCLIP is worse than predicting the
-  mean (R² −0.08).
-- But CalorieK degrades ~3× versus in-distribution (51.4 → 164.9 kcal MAE): it is substantially a
-  specialist in Nutrition5k-style plated dishes. For wild-photo deployment, mixing diverse weakly
-  labeled data into training is the indicated next step.
+has never seen, and retains real signal (R² +0.24) where CalorieCLIP is worse than predicting the
+mean (R² −0.08).
+- But CalorieK degrades ~3× versus in-distribution (49.8 → 162.5 kcal MAE): it is substantially a
+specialist in Nutrition5k-style plated dishes. For wild-photo deployment, mixing diverse weakly
+labeled data into training is the indicated next step.
 
-## Training method
+## Training method (improved)
 
 - **Architecture**: `google/siglip2-base-patch16-224` vision tower (92M params) → LayerNorm →
-  MLP head (768→768→256→5) with softplus outputs. Trained with Huber loss, per-task, quality-weighted.
-- **Schedule**: 15 epochs × 20,000 samples, batch 256 (bf16), head LR 1e-4 / trunk LR 2e-5,
-  cosine schedule with 5% warmup, first 5% head-only, last 4 vision blocks unfrozen, weight decay 0.05, seed 17.
-- **Checkpoint selection**: best validation MAE on 80 held-out camera-A dishes (validation MAE 39.2 kcal).
-  Training ran ~34 minutes on one RTX PRO 6000 Blackwell (CUDA 13).
+MLP head (768→768→256→5) with softplus outputs. Trained with composite Huber loss, per-task weighted.
+- **Augmentation**: MixUp + CutMix (both α=0.4, 50% probability per batch), plus standard
+RandomResizedCrop, HorizontalFlip, Rotation, ColorJitter.
+- **Schedule**: 20 epochs × 20,000 samples, effective batch 256 (bf16), head LR 1e-4 / trunk LR 2e-5,
+cosine schedule with restarts, 5% warmup, first 5% head-only, last 4 vision blocks unfrozen,
+weight decay 0.05, seed 17.
+- **Checkpoint selection**: best validation MAE on 80 held-out camera-A dishes (validation MAE 39.3 kcal).
+Training ran ~12 minutes on one RTX 4080 (16 GB).
 - **Calibration**: a conformal radius fitted on 86 held-out calibration dishes gives 90% prediction
-  intervals of ±105.9 kcal, with 93.6% actual coverage on the locked test set (`calibration.json`).
+intervals of ±105.9 kcal, with 93.6% actual coverage on the locked test set (`calibration.json`).
 
-![Training loss](loss_curve.png)
+[![Training loss](https://huggingface.co/Dralkh/CalorieK/resolve/main/loss_curve.png)](https://huggingface.co/Dralkh/CalorieK/blob/main/loss_curve.png)
 
 ## Training data (20,000 samples, leakage-audited)
 
 | Source | Samples | Notes |
-|---|---|---|
+| --- | --- | --- |
 | [`pinkieseb/nutrition_dataset`](https://huggingface.co/datasets/pinkieseb/nutrition_dataset) (Nutrition5k video frames) | 19,734 | only dishes provably in the **official Nutrition5k train split** (matched by exact nutrition signature), ≤16 frames per dish |
 | [`aryachakraborty/Food_Calorie_Dataset`](https://huggingface.co/datasets/aryachakraborty/Food_Calorie_Dataset) | 266 | human-estimated labels, down-weighted (0.4) |
 
 Leakage controls, all enforced programmatically before training:
+
 - **Group separation**: splits partition by physical dish (`physical_meal_id`); no dish appears in two splits.
 - **Exact-media**: SHA-256 disjointness across fit / validation / calibration / test.
 - **Near-duplicate**: perceptual-hash disjointness; 2 fit images visually colliding with held-out frames were dropped and backfilled.
 - **Official-split proof**: a Nutrition5k registry maps nutrition signatures to official train/test IDs;
-  ambiguous signatures are excluded. The audit trail (SHA-256 of every manifest and the source tree) ships in the results.
+ambiguous signatures are excluded. The audit trail (SHA-256 of every manifest and the source tree) ships in the results.
 
 ## Usage
 
@@ -151,9 +145,20 @@ print(f"{kcal:.0f} kcal ± 106 (90% interval), {mass_g:.0f} g, P{protein_g:.0f}/
 ## Limitations
 
 - **Domain**: trained and evaluated on Nutrition5k-style plated single dishes photographed from the side;
-  accuracy on multi-plate scenes, packaged food, or drinks is untested.
-- **Heavy tail**: RMSE (327) is far above MAE (51) — very high-calorie dishes are occasionally missed badly.
-  Use the calibrated interval, not the point estimate, for anything that matters.
+accuracy on multi-plate scenes, packaged food, or drinks is untested.
+- **Heavy tail**: RMSE (326) is far above MAE (50) — very high-calorie dishes are occasionally missed badly.
+Use the calibrated interval, not the point estimate, for anything that matters.
 - **Frames, not meals**: training frames come from videos of ~1,250 distinct dishes; visual diversity is
-  narrower than the sample count suggests.
+narrower than the sample count suggests.
 - The VLM baselines answered a zero-shot prompt; instruction-tuned prompting or few-shot could improve them.
+
+## Citation
+
+```bibtex
+@misc{caloriek2025,
+  title={CalorieK: Accurate Food Calorie Estimation from a Single RGB Image},
+  author={Dralkh},
+  year={2025},
+  url={https://huggingface.co/Dralkh/CalorieK}
+}
+```