Model card: drop calibration warning
Browse files
README.md
CHANGED
|
@@ -18,17 +18,12 @@ language:
|
|
| 18 |
# DPF-Nutrition (RGB-D) β weights & Core ML
|
| 19 |
|
| 20 |
Trained weights for **DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion**
|
| 21 |
-
(Han et al., *Foods* 2023, [arXiv:2310.11702](https://arxiv.org/abs/2310.11702)),
|
| 22 |
-
**
|
| 23 |
|
| 24 |
- π¬ Research repo (training & reproduction): https://github.com/T0MYYY/Nutrition5k
|
| 25 |
- π± iOS deployment app (Core ML): https://github.com/T0MYYY/CalBro
|
| 26 |
-
|
| 27 |
-
## β οΈ Important β not calibrated for real use
|
| 28 |
-
|
| 29 |
-
These weights are a **research/student reproduction**. The backbone is **not** rigorously calibrated for
|
| 30 |
-
phone capture or real plated-food ground truth. **Outputs have no reference value** and must **not** be used
|
| 31 |
-
for medical, dietary, or clinical decisions.
|
| 32 |
|
| 33 |
## Contents
|
| 34 |
|
|
@@ -37,22 +32,21 @@ for medical, dietary, or clinical decisions.
|
|
| 37 |
| `dpf_nutrition_food2k.pt` | DPF-Nutrition PyTorch checkpoint (Food2K ResNet-101 backbone, RGB-D fusion head) |
|
| 38 |
| `food2k_resnet101.pth` | Food2K ResNet-101 pretrained backbone used by DPF |
|
| 39 |
| `coreml/DPFNutritionRGBDepth.mlpackage` | DPF-Nutrition converted to Core ML (rgb+depth β [cal, mass, fat, carb, protein]) |
|
| 40 |
-
| `coreml/DepthAnythingV2SmallF16P6.mlpackage` | Depth Anything V2 (Small) Core ML, 518Γ392,
|
| 41 |
|
| 42 |
## Pipeline
|
| 43 |
|
| 44 |
`RGB β Depth Anything V2 (predicted depth) β DPF-Nutrition RGB-D fusion β [calories, mass, fat, carbs, protein]`
|
| 45 |
|
| 46 |
-
On LiDAR iPhones the hardware depth can replace the monocular depth stage
|
| 47 |
|
| 48 |
## Citation
|
| 49 |
|
| 50 |
```bibtex
|
| 51 |
@article{han2023dpfnutrition,
|
| 52 |
-
title
|
| 53 |
-
author
|
| 54 |
-
journal = {Foods}, volume = {12}, number = {23}, pages = {4293}, year = {2023},
|
| 55 |
-
doi = {10.3390/foods12234293}
|
| 56 |
}
|
| 57 |
@inproceedings{thames2021nutrition5k,
|
| 58 |
title = {Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food},
|
|
|
|
| 18 |
# DPF-Nutrition (RGB-D) β weights & Core ML
|
| 19 |
|
| 20 |
Trained weights for **DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion**
|
| 21 |
+
(Han et al., *Foods* 2023, [arXiv:2310.11702](https://arxiv.org/abs/2310.11702)), trained and evaluated
|
| 22 |
+
on **Nutrition5k** under the official train/test splits, and converted to Core ML for on-device use.
|
| 23 |
|
| 24 |
- π¬ Research repo (training & reproduction): https://github.com/T0MYYY/Nutrition5k
|
| 25 |
- π± iOS deployment app (Core ML): https://github.com/T0MYYY/CalBro
|
| 26 |
+
- π€ Reproduction & ablation checkpoints: https://huggingface.co/T0MYYY/nutrition5k-experiments
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## Contents
|
| 29 |
|
|
|
|
| 32 |
| `dpf_nutrition_food2k.pt` | DPF-Nutrition PyTorch checkpoint (Food2K ResNet-101 backbone, RGB-D fusion head) |
|
| 33 |
| `food2k_resnet101.pth` | Food2K ResNet-101 pretrained backbone used by DPF |
|
| 34 |
| `coreml/DPFNutritionRGBDepth.mlpackage` | DPF-Nutrition converted to Core ML (rgb+depth β [cal, mass, fat, carb, protein]) |
|
| 35 |
+
| `coreml/DepthAnythingV2SmallF16P6.mlpackage` | Depth Anything V2 (Small) Core ML, 518Γ392, depth stage used in CalBro |
|
| 36 |
|
| 37 |
## Pipeline
|
| 38 |
|
| 39 |
`RGB β Depth Anything V2 (predicted depth) β DPF-Nutrition RGB-D fusion β [calories, mass, fat, carbs, protein]`
|
| 40 |
|
| 41 |
+
On LiDAR iPhones the hardware depth can replace the monocular depth stage.
|
| 42 |
|
| 43 |
## Citation
|
| 44 |
|
| 45 |
```bibtex
|
| 46 |
@article{han2023dpfnutrition,
|
| 47 |
+
title = {DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion},
|
| 48 |
+
author = {Han, Yuzhe and Cheng, Qimin and Wu, Wenjin and Huang, Ziyang},
|
| 49 |
+
journal = {Foods}, volume = {12}, number = {23}, pages = {4293}, year = {2023}, doi = {10.3390/foods12234293}
|
|
|
|
| 50 |
}
|
| 51 |
@inproceedings{thames2021nutrition5k,
|
| 52 |
title = {Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food},
|