Changes nemotron model

README.md

@@ -16,13 +16,17 @@ tags:
   - small-models
   - gradio
   - build-small-hackathon
+  - track:backyard
+  - sponsor:nvidia
+  - achievement:offbrand
+  - achievement:sharing
   - backyard-ai
   - leaflet
   - cosine-similarity
 models:
-  - nvidia/nemotron-3-nano-omni-30b-a3b-reasoning
-  - sentence-transformers/all-MiniLM-L6-v2
   - meta-llama/Llama-3.2-11B-Vision-Instruct
+  - nvidia/Nemotron-Mini-4B-Instruct
+  - sentence-transformers/all-MiniLM-L6-v2
 ---
 
 # PawMap 🐾 — Collaborative stray animal mapping with AI
@@ -34,6 +38,9 @@ Built for the [**Build Small Hackathon 2026**](https://huggingface.co/build-smal
 
 **Try it:** [live Space](https://huggingface.co/spaces/build-small-hackathon/viralata-mapper) · **Traces:** [dataset on the Hub](https://huggingface.co/datasets/build-small-hackathon/viralata-mapper-storage)
 
+> 🎬 **Demo video:** `TODO — paste your YouTube / Vimeo / Loom link here before submitting`
+> 📣 **Social post:** `TODO — paste your X / LinkedIn / Bluesky / Mastodon post link here before submitting`
+
 ---
 
 ## The Problem
@@ -54,10 +61,10 @@ There are two AI-powered flows: **registering a sighting** and **recording help*
 
 ```mermaid
 flowchart TD
-    A["📷 Photo from phone"] --> B["🤖 Vision AI\nLlama-3.2-11B or Nemotron Omni\n(serverless API)"]
+    A["📷 Photo from phone"] --> B["🤖 Vision AI\nLlama-3.2-11B-Vision\n(HF Serverless)"]
     B -->|"is it an animal?"| C{Animal detected?}
     C -->|"No"| ERR["❌ Error message\n'No dog or cat identified'"]
-    C -->|"Yes"| D["📝 Structured JSON\nspecies · breed · color · condition · marks"]
+    C -->|"Yes"| D["📝 Structured JSON\nNemotron Mini 4B Instruct normalizes\nspecies · breed · color · condition · marks"]
     D --> E["🔢 Semantic embedding\nall-MiniLM-L6-v2 · 384 dim\n(local, 22M params)"]
     E --> F{Cosine similarity\n≥ 0.80?}
     F -->|"Yes — known animal"| G["➕ New sighting\nadded to existing animal"]
@@ -99,14 +106,14 @@ The map uses color to signal urgency: **🟢 green** = dog · **🟠 orange** =
 
 | Component | Model / Library | Where it runs |
 |---|---|---|
-| Visual identification | **Nemotron Omni 30B** via NVIDIA NIM | remote API |
-| Vision fallback | **Llama-3.2-11B-Vision-Instruct** via HF Serverless | remote API |
-| Semantic embedding | **all-MiniLM-L6-v2** (384 dim) | CPU, local |
+| Visual identification | **Llama-3.2-11B-Vision-Instruct** (11B) via HF Serverless | remote API |
+| Text reasoning / JSON | **Nemotron Mini 4B Instruct** via NVIDIA NIM | remote API |
+| Semantic embedding | **all-MiniLM-L6-v2** (384 dim, 22M params) | CPU, local |
 | Animal matching | Cosine similarity (threshold 0.80) | CPU, local |
 | Database | SQLite | local / persistent |
 | Frontend | SPA via `gradio.Server` + Leaflet.js + Lucide Icons | browser |
 
-**Total parameters running locally: ~22M.** The vision model runs via serverless API — nothing heavy on the Space machine. The only local model is MiniLM for embeddings.
+**Total parameters running locally: ~22M** (MiniLM). The vision and reasoning models run via serverless API — nothing heavy on the Space machine. **Every model used is well under the 32B hackathon cap:** Llama-3.2-11B-Vision (11B), Nemotron Mini 4B Instruct (4B) and MiniLM-L6-v2 (22M).
 
 ---
 
@@ -129,6 +136,8 @@ This submission earns the following hackathon bonus quests:
 
 - 🎨 **Off-Brand** — Fully custom interface via `gradio.Server`, no default Gradio components visible. SPA frontend with Leaflet.js for the map, Lucide Icons, and custom design.
 - 📡 **Sharing is Caring** — Full traces of every sighting (photo → AI analysis → embedding → matching result) are automatically published as a dataset on the Hub via `/admin/push-traces`.
+- 🤏 **Tiny Titan** — The only on-device model is `all-MiniLM-L6-v2` (22M params, far under 4B): all animal matching runs locally on a tiny embedding model.
+- 🟩 **NVIDIA / Nemotron** — Structured identification is normalized by `Nemotron-Mini-4B-Instruct` via NVIDIA NIM — small (4B), well under the 32B cap.
 
 ---
 
@@ -139,10 +148,10 @@ git clone https://huggingface.co/spaces/build-small-hackathon/viralata-mapper
 cd viralata-mapper
 pip install -r requirements.txt
 
-# with HuggingFace Serverless
+# Vision via HuggingFace Serverless (Llama-3.2-11B-Vision)
 HF_TOKEN=hf_... python app.py
 
-# or with NVIDIA NIM (takes precedence over HF_TOKEN)
+# Add NVIDIA NIM for Nemotron Mini 4B Instruct (structured-output normalization)
 NVIDIA_API_KEY=nvapi_... python app.py
 
 # no key — runs in fallback mode without AI identification
@@ -162,7 +171,7 @@ On first run the app seeds the database with demo animals around Brasília so th
 | Secret | Description |
 |---|---|
 | `HF_TOKEN` | HuggingFace token for Llama-3.2-11B-Vision via Serverless Inference |
-| `NVIDIA_API_KEY` | Alternative: Nemotron Omni via NVIDIA NIM (takes precedence over HF_TOKEN) |
+| `NVIDIA_API_KEY` | NVIDIA NIM key for Nemotron Mini 4B Instruct (structured-output normalization) |
 | `MATCH_THRESHOLD` | Optional. Similarity threshold. Default: `0.80` |
 | `HF_DATASET_ID` | Optional. Hub dataset for trace publishing (e.g. `org/dataset-name`) |
 
@@ -214,7 +223,7 @@ Set up a **Persistent Storage Bucket** in the Space and mount it at `/data/` so
 ## Credits
 
 - **[Meta](https://ai.meta.com/llama/)** — Llama 3.2 11B Vision Instruct (Llama 3.2 Community License).
-- **[NVIDIA](https://build.nvidia.com/)** — Nemotron Omni 30B via NVIDIA NIM.
+- **[NVIDIA](https://build.nvidia.com/)** — Nemotron Mini 4B Instruct via NVIDIA NIM.
 - **[Hugging Face](https://huggingface.co/)** — `sentence-transformers/all-MiniLM-L6-v2`, Serverless Inference, Space hosting.
 - **[Leaflet.js](https://leafletjs.com/)** — interactive map.
 - **[Gradio](https://gradio.app/)** — `gradio.Server` for the custom frontend.
@@ -224,7 +233,7 @@ Set up a **Persistent Storage Bucket** in the Space and mount it at `/data/` so
 
 ## License
 
-MIT for the application code. Models follow their own licenses — see the Meta (Llama 3.2) and NVIDIA (Nemotron Omni) model cards for full terms.
+MIT for the application code. Models follow their own licenses — see the Meta (Llama 3.2) and NVIDIA (Nemotron Mini 4B Instruct) model cards for full terms.
 
 ---
 

core/ai.py

@@ -1,12 +1,21 @@
 """
-ai.py — Visao via HuggingFace InferenceClient ou NVIDIA NIM.
+ai.py — Visao via HuggingFace InferenceClient + normalizacao via NVIDIA NIM.
 
-Variaveis de ambiente:
-  HF_TOKEN       — token HuggingFace (usa HF Serverless Inference)
-  NVIDIA_API_KEY — chave NVIDIA NIM (free tier em build.nvidia.com)
+Arquitetura (todos os modelos < 32B):
+  - Visao  : meta-llama/Llama-3.2-11B-Vision-Instruct  (11B, HF Serverless)  -> precisa HF_TOKEN
+  - Texto  : nvidia/nemotron-mini-4b-instruct                  (4B,  NVIDIA NIM)     -> precisa NVIDIA_API_KEY (opcional)
+  - Embed  : sentence-transformers/all-MiniLM-L6-v2     (22M, local/CPU)
+
+O Llama enxerga a foto e devolve a descricao bruta. Se NVIDIA_API_KEY estiver
+configurada, o Nemotron Mini 4B Instruct (texto puro) normaliza esse JSON para o schema
+estrito (raca da lista permitida, cores simples, condicao, frase de matching).
+Sem NVIDIA_API_KEY, usa-se direto a saida do Llama.
 
-Modelo HF:  meta-llama/Llama-3.2-11B-Vision-Instruct  (disponivel no HF Serverless)
-Modelo NIM: nvidia/nemotron-3-nano-omni-30b-a3b         (Nemotron Omni via NVIDIA)
+Variaveis de ambiente:
+  HF_TOKEN        — token HuggingFace (HF Serverless Inference) — obrigatorio p/ visao
+  NVIDIA_API_KEY  — chave NVIDIA NIM (free tier em build.nvidia.com) — opcional
+  HF_VISION_MODEL — override do modelo de visao (default Llama-3.2-11B-Vision)
+  NVIDIA_MODEL    — override do modelo NIM (default nvidia/nemotron-mini-4b-instruct)
 """
 import base64
 import io
@@ -20,7 +29,9 @@ import numpy as np
 log = logging.getLogger(__name__)
 
 _HF_MODEL  = "meta-llama/Llama-3.2-11B-Vision-Instruct"
-_NIM_MODEL = "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning"  # Nemotron Omni VLM, NVIDIA NIM
+# String do modelo na API NVIDIA NIM (integrate.api.nvidia.com).
+# Confirme o id exato em build.nvidia.com; pode sobrescrever via NVIDIA_MODEL.
+_NIM_MODEL = "nvidia/nemotron-mini-4b-instruct"  # Nemotron Mini 4B Instruct (texto, 4B)
 
 PROMPT = (
     "Examine this image carefully.\n"
@@ -47,41 +58,56 @@ PROMPT = (
     " \"description_text\": \"one concise English sentence describing this specific animal for identity matching\"}"
 )
 
+# Nemotron Mini 4B Instruct normaliza a saida da visao para o schema estrito (texto puro).
+NORMALIZE_PROMPT = (
+    "You normalize raw animal descriptions for a stray-animal database.\n"
+    "Given the input JSON, return ONLY a cleaned JSON object with the SAME keys.\n"
+    "Rules:\n"
+    "- breed_estimate must be the single best match from the allowed dog/cat breed lists; "
+    "use SRD (dog) or Domestic Shorthair (cat) only if truly unidentifiable.\n"
+    "- species is 'dog' or 'cat'; size is 'small', 'medium' or 'large'.\n"
+    "- primary_color and secondary_colors are simple lowercase words.\n"
+    "- condition is exactly one of: healthy, thin, injured.\n"
+    "- description_text is ONE concise English sentence for identity matching.\n"
+    "- Do NOT invent details not present in the input. No markdown, no explanation.\n\n"
+    "Input JSON:\n"
+)
+
 
 class AnimalAI:
     def __init__(self):
-        self.mode   = None  # "hf" | "nim"
-        self.model  = None
-        self.client = None  # OpenAI (NIM) ou InferenceClient (HF)
+        self.vision_model  = _HF_MODEL
+        self.vision_client = None  # InferenceClient (HF) — visao
+        self.nim_model     = _NIM_MODEL
+        self.nim_client    = None  # OpenAI client -> NVIDIA NIM — texto
 
         hf_token   = os.environ.get("HF_TOKEN", "")
         nvidia_key = os.environ.get("NVIDIA_API_KEY", "")
 
+        # Visao: Llama-3.2-11B-Vision via HF Serverless (obrigatorio para identificar foto)
+        if hf_token:
+            try:
+                from huggingface_hub import InferenceClient
+                self.vision_model  = os.environ.get("HF_VISION_MODEL", _HF_MODEL)
+                self.vision_client = InferenceClient(model=self.vision_model, token=hf_token)
+                log.info("Visao: %s via HF InferenceClient", self.vision_model)
+            except ImportError:
+                log.warning("huggingface_hub nao instalado")
+        else:
+            log.warning("Sem HF_TOKEN — visao desabilitada. Configure HF_TOKEN nos Secrets.")
+
+        # Texto: Nemotron Mini 4B Instruct via NVIDIA NIM (opcional — normaliza o JSON)
         if nvidia_key:
             try:
                 from openai import OpenAI
-                self.mode   = "nim"
-                self.model  = os.environ.get("NVIDIA_MODEL", _NIM_MODEL)
-                self.client = OpenAI(
+                self.nim_model  = os.environ.get("NVIDIA_MODEL", _NIM_MODEL)
+                self.nim_client = OpenAI(
                     base_url="https://integrate.api.nvidia.com/v1",
                     api_key=nvidia_key,
                 )
-                log.info("AI: Nemotron Omni via NVIDIA NIM (%s)", self.model)
-            except ImportError:
-                log.warning("openai nao instalado")
-
-        elif hf_token:
-            try:
-                from huggingface_hub import InferenceClient
-                self.mode   = "hf"
-                self.model  = os.environ.get("NVIDIA_MODEL", _HF_MODEL)
-                self.client = InferenceClient(model=self.model, token=hf_token)
-                log.info("AI: %s via HF InferenceClient", self.model)
+                log.info("Texto: Nemotron Mini 4B Instruct via NVIDIA NIM (%s)", self.nim_model)
             except ImportError:
-                log.warning("huggingface_hub nao instalado")
-
-        else:
-            log.warning("Sem chave de API — IA desabilitada. Configure HF_TOKEN nos Secrets.")
+                log.warning("openai nao instalado — normalizacao Nemotron desabilitada")
 
         self.embedder = None
         try:
@@ -93,50 +119,26 @@ class AnimalAI:
 
     def analyze_image(self, image) -> dict:
         """Analisa imagem. _ai_success=False indica que a IA nao foi usada."""
-        if self.client is None:
+        if self.vision_client is None:
             return self._fallback()
 
         try:
             img_b64 = self._to_b64(image)
 
-            if self.mode == "hf":
-                # HuggingFace InferenceClient — suporte nativo a multimodal
-                resp = self.client.chat_completion(
-                    messages=[{
-                        "role": "user",
-                        "content": [
-                            {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + img_b64}},
-                            {"type": "text", "text": PROMPT},
-                        ],
-                    }],
-                    max_tokens=400,
-                    temperature=0.1,
-                )
-                raw = resp.choices[0].message.content or ""
-            else:
-                # NVIDIA NIM — Nemotron Omni (reasoning model)
-                resp = self.client.chat.completions.create(
-                    model=self.model,
-                    messages=[{
-                        "role": "user",
-                        "content": [
-                            {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + img_b64}},
-                            {"type": "text", "text": PROMPT},
-                        ],
-                    }],
-                    max_tokens=1024,
-                    temperature=0.6,
-                    top_p=0.95,
-                    extra_body={
-                        "chat_template_kwargs": {"enable_thinking": True},
-                        "reasoning_budget": 512,
-                    },
-                )
-                msg = resp.choices[0].message
-                # Nemotron Omni: resposta pode estar em content ou reasoning_content
-                raw = (msg.content or "") or (getattr(msg, "reasoning_content", "") or "")
-
-            log.info("AI resposta: %s", raw[:200])
+            # Visao multimodal — Llama-3.2-11B-Vision (HF InferenceClient)
+            resp = self.vision_client.chat_completion(
+                messages=[{
+                    "role": "user",
+                    "content": [
+                        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + img_b64}},
+                        {"type": "text", "text": PROMPT},
+                    ],
+                }],
+                max_tokens=400,
+                temperature=0.1,
+            )
+            raw = resp.choices[0].message.content or ""
+            log.info("Visao resposta: %s", raw[:200])
             result = self._parse(raw)
 
             # Rejeição explícita: a IA não viu nenhum animal
@@ -144,14 +146,44 @@ class AnimalAI:
                 log.info("IA: nenhum animal detectado na imagem.")
                 return {"is_animal": False, "_ai_success": True}
 
+            # Normalização opcional via Nemotron Mini 4B Instruct (texto puro, NVIDIA NIM)
+            if self.nim_client is not None:
+                result = self._normalize_with_nemotron(result)
+
             result["is_animal"] = True
             result["_ai_success"] = True
             return result
 
         except Exception as e:
-            log.error("Erro na API de IA: %s", e)
+            log.error("Erro na API de visao: %s", e)
             return self._fallback()
 
+    def _normalize_with_nemotron(self, result: dict) -> dict:
+        """Coage o JSON da visao para o schema estrito usando Nemotron Mini 4B Instruct (texto)."""
+        try:
+            payload = json.dumps(result, ensure_ascii=False)
+            resp = self.nim_client.chat.completions.create(
+                model=self.nim_model,
+                messages=[{"role": "user", "content": NORMALIZE_PROMPT + payload}],
+                max_tokens=400,
+                temperature=0.2,
+                top_p=0.9,
+            )
+            msg = resp.choices[0].message
+            raw = (msg.content or "") or (getattr(msg, "reasoning_content", "") or "")
+            cleaned = self._extract_json(raw)
+            if cleaned:
+                for k in ("species", "breed_estimate", "size", "primary_color",
+                          "secondary_colors", "distinctive_marks", "condition",
+                          "description_text"):
+                    if k in cleaned and cleaned[k] not in (None, "", []):
+                        result[k] = cleaned[k]
+                log.info("Nemotron 4B normalizou os campos.")
+            return result
+        except Exception as e:
+            log.warning("Nemotron normalize falhou, usando saida da visao: %s", e)
+            return result
+
     def get_embedding(self, description: dict) -> list:
         """Embedding da descricao. Aleatorio se IA falhou (evita falsos matches)."""
         if not description.get("_ai_success", True):
@@ -177,13 +209,21 @@ class AnimalAI:
         return base64.b64encode(buf.getvalue()).decode()
 
     @staticmethod
-    def _parse(raw: str) -> dict:
+    def _extract_json(raw: str):
+        """Retorna dict do primeiro bloco JSON valido, ou None."""
         m = re.search(r"\{.*\}", raw, re.DOTALL)
         if m:
             try:
                 return json.loads(m.group())
             except json.JSONDecodeError:
-                pass
+                return None
+        return None
+
+    @staticmethod
+    def _parse(raw: str) -> dict:
+        parsed = AnimalAI._extract_json(raw)
+        if parsed is not None:
+            return parsed
         log.warning("JSON nao parseado — fallback")
         return AnimalAI._fallback()