| --- |
| language: en |
| tags: |
| - mlx |
| - qwen3.5 |
| - nutrition |
| - food.com |
| pipeline_tag: image-text-to-text |
| library_name: mlx |
| license: apache-2.0 |
| base_model: |
| - Qwen/Qwen3.5-2B |
| --- |
| |
| # Qwen3.5 Nutrition Finte Tune |
|
|
| This is a highly specialized, local-first Vision-Language Model (VLM) designed to estimate nutritional macros and physical weight from images of food. The model is fine-tuned to output strict, minified JSON payloads containing calories, macros (protein, fat, carbohydrates), and estimated dry/physical weight in grams. |
|
|
| It has been natively converted to Apple's MLX format and quantized to 4-bit (INT4), reducing its memory footprint to roughly ~1.5GB. This allows for lightning-fast, entirely offline, on-device inference on Apple Silicon hardware (macOS, iOS, iPadOS) using the Neural Engine. |
|
|
| - **Base Architecture:** [Qwen3.5-2B](https://huggingface.co/Qwen) (Omni/Unified Vision-Language architecture) |
| - **Framework:** MLX (`mlx-vlm`) |
| - **Quantization:** INT4 (4-bit) |
| - **Fine-Tuning Method:** LoRA (via Unsloth), merged to 16-bit prior to MLX conversion. |
| - **Primary Task:** Multimodal Image-to-JSON parsing. |
|
|
| ## Intended Use |
| - **Primary Use Case:** Seamless integration into native Swift/Apple ecosystem applications requiring on-device visual food analysis without relying on cloud APIs. |
| - **Input:** A user prompt (e.g. food description) + a standard RGB food image. |
| - **Output:** A standardized JSON schema containing absolute nutritional values. |
|
|
| ## Training Data & Credit |
| This model was fine-tuned on a custom dataset synthesized from a massive corpus of recipe data and user-uploaded images. |
| - **Dataset Source:** Modified from the [Food.com Recipes and Reviews](https://www.kaggle.com/datasets/irkaal/foodcom-recipes-and-reviews) dataset. The text macros were mathematically processed to derive estimated physical weights, combined with dynamically downloaded web images, and formatted into an OpenAI-style JSONL conversation structure. |
| - **Dataset License:** Public Domain (CC0) |
|
|
| ## Licensing & Attribution |
| - **Model License:** [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) (Inherited from the Qwen 3.5 2B base model). |
| - **Attribution:** All base architectural credit belongs to the Qwen team at Alibaba Cloud. |
|
|
| ## Usage Example (MLX) |
| ```python |
| from mlx_vlm import load, generate |
| from mlx_vlm.prompt_utils import get_message_profile |
| from mlx_vlm.utils import load_image |
| |
| # Load the INT4 MLX model |
| model, processor = load("alexsofonea/Qwen3.5_2B_MLX_Nutrition") |
| |
| # Format input |
| image = load_image("path/to/food.jpg") |
| prompt = "Output JSON.\nEstimate." |
| |
| # Generate JSON |
| output = generate(model, processor, prompt, image, max_tokens=150, temperature=0.0) |
| print(output) |
| ``` |
|
|
| ## About Developer |
| ```JSON |
| { |
| "developer": { |
| "name": "Alex", |
| "age": 19, |
| "location": "Munchen, Germany", |
| "company": "Tecky", |
| "role": "Founder", |
| "background": [ |
| "Developer since age 7", |
| "Filmmaker" |
| ] |
| }, |
| "mission": "Building local-first AI agents and integrating modern cinematic design into native software." |
| } |
| ``` |
| Contact: |
| - [alex@alexsofonea.com](mailto:alex@alexsofonea.com) |
| - [LinkedIn](https://www.linkedin.com/in/alexsofonea/) |
| - [Website](https://alexsofonea.com) |
|
|
|
|
