Sync GitHub snapshot to Hugging Face

.dockerignore

@@ -0,0 +1,13 @@
+.git
+.github
+.agents
+.codex
+.env
+.venv
+venv
+__pycache__
+*.pyc
+chroma_db_docs
+chroma_db_sql
+chroma_db_sql_backup_*
+MODELS

.gitattributes

@@ -0,0 +1,14 @@
+*.mp4 filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.jpg filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text
+*.jpeg filter=lfs diff=lfs merge=lfs -text
+*.gif filter=lfs diff=lfs merge=lfs -text
+*.webp filter=lfs diff=lfs merge=lfs -text
+*.pdf filter=lfs diff=lfs merge=lfs -text
+*.sqlite3 filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+# Hugging Face's GitHub sync may omit LFS-only directories. Keep the packaged
+# Chroma store as regular Git blobs so it is always present in the Docker build.
+chroma_db_docs/** -filter -diff -merge -text
+chroma_package/** filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,12 @@
+.keras
+__pycache__/
+.venv/
+chroma_db_sql
+chroma_db_sql_backup_*
+chroma_db_docs/
+venv
+.venv/
+.env
+MODELS/
+chroma_db_sql_backup_20260710_211319
+.jpg

Dockerfile

@@ -0,0 +1,35 @@
+FROM python:3.11-slim
+
+# Crea y asigna un usuario para HF Spaces (recomendado)
+RUN useradd -m -u 1000 user
+ENV PATH="/home/user/.local/bin:$PATH"
+ENV PYTHONUNBUFFERED=1
+ENV PORT=7860
+ENV ANONYMIZED_TELEMETRY=False
+ENV LOCAL_MODEL_CONTEXT_SIZE=2048
+ENV LOCAL_MODEL_MAX_NEW_TOKENS=256
+ENV LOCAL_MODEL_THREADS=2
+ENV LOCAL_MODEL_BATCH_THREADS=2
+ENV LOCAL_MODEL_BATCH_SIZE=256
+
+WORKDIR /home/user/app
+RUN chown -R user:user /home/user/app
+
+USER user
+
+COPY --chown=user ./requirements.txt requirements.txt
+RUN pip install --no-cache-dir --upgrade -r requirements.txt
+
+COPY --chown=user . /home/user/app
+RUN python download_model.py \
+    --download-only \
+    --repo AlbertiTechnology/qwen3-4b-instruct-gguf \
+    --local-dir MODELS/qwen3-4b-instruct-gguf
+RUN python index_documents_to_chroma.py \
+    --documents-dir documentos/ASTM \
+    --persist-dir chroma_db_docs \
+    --collection document_context \
+    --reset-collection
+
+# Valida la base Chroma generada durante el build y luego inicia la API.
+CMD ["python", "-u", "start_hf.py"]

README.md

@@ -0,0 +1,88 @@
+---
+title: Agent API
+emoji: 🤖
+colorFrom: green
+colorTo: green
+sdk: docker
+pinned: false
+---
+
+# Modelos principales
+ollama pull llama3.2:3b
+ollama pull llava:7b
+ollama pull nomic-embed-text    
+
+## Modelo Qwen3-4B-Instruct GGUF local
+
+Al construir la imagen, Docker descarga el GGUF desde
+`AlbertiTechnology/qwen3-4b-instruct-gguf` y deja una copia local dentro de la
+imagen. Docker ejecuta automaticamente:
+
+```powershell
+python download_model.py --download-only --repo AlbertiTechnology/qwen3-4b-instruct-gguf --local-dir MODELS/qwen3-4b-instruct-gguf
+```
+
+El modelo queda almacenado en
+`/home/user/app/MODELS/qwen3-4b-instruct-gguf/model.gguf` dentro de la imagen y
+la app lo ejecuta localmente mediante llama.cpp.
+
+Esta descarga no requiere un token durante el build porque el repositorio es
+publico.
+
+## Contexto documental (Chroma)
+
+El agente consulta la coleccion `document_context` de `chroma_db_docs`. Durante
+el build de Docker, la base se reconstruye automaticamente usando solamente los
+PDF y archivos de metadata ubicados en `documentos/ASTM`.
+
+Para reconstruirla manualmente fuera de Docker, ejecute:
+
+```powershell
+venv\Scripts\python.exe index_documents_to_chroma.py --documents-dir "documentos/ASTM" --reset-collection
+```
+
+## Despliegue en Hugging Face Spaces
+
+El contenedor ejecuta automaticamente `start_hf.py`. En cada arranque:
+
+1. valida `DATABASE_URL`;
+2. valida el modelo local `MODELS/qwen3-4b-instruct-gguf/model.gguf`;
+3. valida la base Chroma y comprueba que `document_context` no este vacia;
+4. inicia FastAPI/Uvicorn en el puerto `7860`.
+
+Los reinicios del contenedor reutilizan la base creada dentro de la imagen. Cada
+nuevo build vuelve a generarla desde `documentos/ASTM`.
+
+Configure en **Settings > Variables and secrets** del Space:
+
+- `DATABASE_URL`: conexion PostgreSQL usada para conversaciones y mensajes.
+
+Los logs del Space muestran el proceso completo sin imprimir secrets:
+
+```text
+[2026-07-10T12:00:00Z] [BOOT:START] Starting Hugging Face Space initialization.
+[2026-07-10T12:00:00Z] [BOOT:ENV] Required secrets are present (values are not printed).
+[2026-07-10T12:00:00Z] [BOOT:CHROMA] Using build-generated document_context collection (119 chunks).
+[2026-07-10T12:00:00Z] [BOOT:MODEL] Local Qwen model is available at /home/user/app/MODELS/qwen3-4b-instruct-gguf.
+[2026-07-10T12:01:00Z] [BOOT:API] Starting Agent API on 0.0.0.0:7860...
+```
+
+Si falta un secret o la base Chroma empaquetada no es valida, el despliegue se
+detiene con un mensaje `[BOOT:ERROR]` visible en **Logs**. La API no arranca con
+una base incompleta.
+
+## Lanzar el chat del agente
+
+Inicie la API desde la raiz del proyecto:
+
+```powershell
+venv\Scripts\python.exe -m uvicorn main:app --host 0.0.0.0 --port 8011 --reload
+```
+
+Abra el frontend de chat en `http://127.0.0.1:8011/` (o en
+`http://127.0.0.1:8011/ui`). Se usa el puerto 8011 porque en este entorno los
+puertos 8000 y 8010 ya estan ocupados por otras instancias.
+
+La API queda disponible mediante `POST http://127.0.0.1:8011/chat` y por
+WebSocket en `ws://127.0.0.1:8011/ws/chat`. La documentacion interactiva se
+abre en `http://127.0.0.1:8011/docs`.

assemble_chroma.py

@@ -0,0 +1,47 @@
+from __future__ import annotations
+
+import hashlib
+import tarfile
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parent
+CHROMA_DIR = ROOT / "chroma_db_docs"
+PARTS_DIR = ROOT / "chroma_package"
+ARCHIVE_PATH = ROOT / "chroma_db_docs.tar"
+DATABASE_PATH = CHROMA_DIR / "chroma.sqlite3"
+EXPECTED_ARCHIVE_SHA256 = "656a7155238ba2b6ec99baad3fc22698a4765696ba5cad613a6469ab8dba8add"
+EXPECTED_DATABASE_SHA256 = "55b525aca0298422c27393485f9feb146ceb2119fb60f382c81ec44e2b670db6"
+
+
+def main() -> None:
+    parts = sorted(PARTS_DIR.glob("chroma.tar.part*"))
+    if not parts:
+        raise FileNotFoundError(f"No packaged Chroma archive parts found in {PARTS_DIR}")
+
+    digest = hashlib.sha256()
+    with ARCHIVE_PATH.open("wb") as output:
+        for part in parts:
+            payload = part.read_bytes()
+            output.write(payload)
+            digest.update(payload)
+
+    actual_sha256 = digest.hexdigest()
+    if actual_sha256 != EXPECTED_ARCHIVE_SHA256:
+        ARCHIVE_PATH.unlink(missing_ok=True)
+        raise RuntimeError(
+            f"Invalid assembled Chroma archive: expected {EXPECTED_ARCHIVE_SHA256}, got {actual_sha256}"
+        )
+
+    with tarfile.open(ARCHIVE_PATH) as archive:
+        archive.extractall(ROOT, filter="data")
+    ARCHIVE_PATH.unlink()
+
+    database_sha256 = hashlib.sha256(DATABASE_PATH.read_bytes()).hexdigest()
+    if database_sha256 != EXPECTED_DATABASE_SHA256:
+        raise RuntimeError(f"Invalid extracted Chroma database: {database_sha256}")
+    print(f"Assembled and extracted packaged Chroma database from {len(parts)} parts.")
+
+
+if __name__ == "__main__":
+    main()

chat_history.py

@@ -0,0 +1,113 @@
+from __future__ import annotations
+
+import json
+import os
+
+from dotenv import load_dotenv
+from sqlalchemy import create_engine, text
+from sqlalchemy.engine import Engine
+
+
+load_dotenv()
+
+_engine: Engine | None = None
+
+
+def get_engine() -> Engine:
+    global _engine
+    if _engine is None:
+        database_url = os.getenv("DATABASE_URL", "").strip("'\"")
+        if not database_url:
+            raise RuntimeError("DATABASE_URL is missing in .env")
+        _engine = create_engine(database_url, pool_pre_ping=True)
+    return _engine
+
+
+def list_users() -> list[dict]:
+    query = text("SELECT id, username FROM auth_user ORDER BY username")
+    with get_engine().connect() as connection:
+        return [dict(row) for row in connection.execute(query).mappings()]
+
+
+def list_conversations(user_id: int) -> list[dict]:
+    query = text("""
+        SELECT id, title, created_at, updated_at
+        FROM chat_conversation
+        WHERE user_id = :user_id
+        ORDER BY updated_at DESC
+    """)
+    with get_engine().connect() as connection:
+        return [dict(row) for row in connection.execute(query, {"user_id": user_id}).mappings()]
+
+
+def create_conversation(user_id: int, title: str) -> dict:
+    query = text("""
+        INSERT INTO chat_conversation (user_id, title, created_at, updated_at)
+        VALUES (:user_id, :title, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
+        RETURNING id, title, created_at, updated_at
+    """)
+    with get_engine().begin() as connection:
+        user_exists = connection.execute(
+            text("SELECT 1 FROM auth_user WHERE id = :user_id"),
+            {"user_id": user_id},
+        ).scalar_one_or_none()
+        if not user_exists:
+            raise ValueError("User does not exist")
+        return dict(connection.execute(query, {"user_id": user_id, "title": title[:255]}).mappings().one())
+
+
+def ensure_conversation_owner(conversation_id: int, user_id: int) -> None:
+    query = text("""
+        SELECT 1 FROM chat_conversation
+        WHERE id = :conversation_id AND user_id = :user_id
+    """)
+    with get_engine().connect() as connection:
+        if connection.execute(query, {"conversation_id": conversation_id, "user_id": user_id}).scalar_one_or_none() is None:
+            raise ValueError("Conversation does not exist for this user")
+
+
+def list_messages(conversation_id: int, user_id: int, limit: int = 100) -> list[dict]:
+    ensure_conversation_owner(conversation_id, user_id)
+    query = text("""
+        SELECT id, role, content, metadata, created_at
+        FROM (
+            SELECT id, role, content, metadata, created_at
+            FROM chat_message
+            WHERE conversation_id = :conversation_id
+            ORDER BY created_at DESC, id DESC
+            LIMIT :limit
+        ) recent
+        ORDER BY created_at, id
+    """)
+    with get_engine().connect() as connection:
+        return [dict(row) for row in connection.execute(
+            query,
+            {"conversation_id": conversation_id, "limit": limit},
+        ).mappings()]
+
+
+def add_message(conversation_id: int, role: str, content: str, metadata: dict | None = None) -> dict:
+    if role not in {"user", "assistant", "system", "tool"}:
+        raise ValueError("Invalid message role")
+    query = text("""
+        INSERT INTO chat_message (conversation_id, role, content, metadata, created_at)
+        VALUES (:conversation_id, :role, :content, CAST(:metadata AS jsonb), CURRENT_TIMESTAMP)
+        RETURNING id, role, content, metadata, created_at
+    """)
+    with get_engine().begin() as connection:
+        result = connection.execute(query, {
+            "conversation_id": conversation_id,
+            "role": role,
+            "content": content,
+            "metadata": json.dumps(metadata or {}),
+        }).mappings().one()
+        connection.execute(
+            text("UPDATE chat_conversation SET updated_at = CURRENT_TIMESTAMP WHERE id = :id"),
+            {"id": conversation_id},
+        )
+        return dict(result)
+
+
+def format_history(messages: list[dict]) -> str:
+    labels = {"user": "Usuario", "assistant": "Asistente", "system": "Sistema", "tool": "Herramienta"}
+    return "\n".join(f"{labels.get(item['role'], item['role'])}: {item['content']}" for item in messages)

chroma_package/chroma.tar.part000

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65c35fd47bd4477e6191c8c21089ca964234e3c9828584727900001954967551
+size 1048576

chroma_package/chroma.tar.part001

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa014588b4933dd06d56e4e687a30d7306996b4ec26d36781f28617e32b70646
+size 1048576

chroma_package/chroma.tar.part002

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ba27ec61709167e861a5be3f6201559bae9681dc1486da1d71faa6fd9b419e7
+size 1048576

chroma_package/chroma.tar.part003

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe340cf3d33052d8182c77aeb3348eda508cf3b24d8e3d26062a652e91aa8514
+size 1048576

chroma_package/chroma.tar.part004

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:575558a748fe946d0d0739ab89561d6f0dd6b5a4604f7761d0b02922a95bbe47
+size 1048576

chroma_package/chroma.tar.part005

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45176f26eb2b299c40cf3f9299537da87f832850d1d4919e4b64dbc1eb50f4ad
+size 1048576

chroma_package/chroma.tar.part006

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a1a1a9eb15c41f08b17647435808ae0043d9202834328da808aca3291a9f29c
+size 1048576

chroma_package/chroma.tar.part007

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e980c7b513a6a215a0e695d608e4e458fb032bda66989ab06a3c7143d7aca38f
+size 1048576

chroma_package/chroma.tar.part008

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e0b9f5a70402f580dc5f3e22d81f3cc2974d8b3097616eb862e90d52a994ba0
+size 1048576

chroma_package/chroma.tar.part009

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca2ba0cae914969c2b40b6968cac201e369c4a491ec6d2bb8a51ad8548e1be1d
+size 1048576

chroma_package/chroma.tar.part010

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb03e8b9b6ca1b7ff3ce1036204813c9329a31ee43629597ba9a012fa056d99a
+size 1048576

chroma_package/chroma.tar.part011

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0c9cfe276a0ec34a38e919797cf0f78ea7ef699f07b8d3369f4bea6edf399d1
+size 1048576

chroma_package/chroma.tar.part012

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d15559befdb6feba40ba6f1b1837f89a04fd57d656966e5b6cac80e656f749d2
+size 1048576

chroma_package/chroma.tar.part013

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f07de4d98338f834410f87f9e9b8370dd0a5c5bdc025c4a7a31e15154291741
+size 1048576

chroma_package/chroma.tar.part014

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3511b64d121fb3bf7d153953d337b6ca927e8b5b663ffed966ea2e0c4b4628e
+size 1048576

chroma_package/chroma.tar.part015

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e4f060d4976062708cc284f17f5d7c1abfd96a1a2731906a692ae8e2056f1b0
+size 1048576

chroma_package/chroma.tar.part016

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d31f964daa207c8e01212eec3b65b2b7c0e822230da8050d63071bb48fd8104
+size 1048576

chroma_package/chroma.tar.part017

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:452e566c99d6ae83dcc51f76fa6e05ca973b5252b583197c203cd408503edcaa
+size 1048576

chroma_package/chroma.tar.part018

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d520d26d67d9417e106e476412062397d8da26eeaaff990a28d7d8d4a25f9a5
+size 1048576

chroma_package/chroma.tar.part019

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b44242930fedc80d1f296090ec1c6430cf5ee82d6a831fc0c1ccfab556d8839
+size 1048576

chroma_package/chroma.tar.part020

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ced66eb812af87fdd7d4510dec1d961b11305a0f41f62524908bee06039ae7e
+size 1048576

chroma_package/chroma.tar.part021

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98852c1de48470bb0e376fe2ea1c993bf45f1fe7219d7bb9f6160e6cbfd24fe6
+size 1048576

chroma_package/chroma.tar.part022

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:669b5a07aab5765ae82aac8c8fc9f1983b59c155da1ccde62fbf1f9b9aca5fda
+size 1048576

chroma_package/chroma.tar.part023

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8770292cfbd0bb1f21ac2790430c70bc0f59f9192f7870ee440cc03ef2b71a83
+size 1048576

chroma_package/chroma.tar.part024

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c93c7f96953e1111142a9bc1d4028802b3b23eaca468f1d50dbc7364beeae3c
+size 1048576

chroma_package/chroma.tar.part025

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d36867e83340a7c71db5d24dd58d62e18cae88d3fdad220e594f65d51f558028
+size 1048576

chroma_package/chroma.tar.part026

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3f11fe483337ab16ca169371a7ad9073eba0439d81f510bb38da77b2dbc6a66
+size 1048576

chroma_package/chroma.tar.part027

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1acff85b6b1c5562abcfc1bfcdfde113af93b93c7e21bb3090c77886265b13b1
+size 1048576

chroma_package/chroma.tar.part028

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:caae7161018565b34a3415ba796afc3ce6d416adc88839529bd583b21ddbafc2
+size 1048576

chroma_package/chroma.tar.part029

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa653f4e877833b34c9c0d4872864dede62a7cb3342fe3e30f0a802f6240806c
+size 1048576

chroma_package/chroma.tar.part030

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34a833b0ddfd963065e6495865615c55551ef1e303e1b7d35e445eaf9f132cf9
+size 1048576

chroma_package/chroma.tar.part031

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6a871bad03781ede5e0b2bf8ac942b863fb0d148f8c543173e112c06c3a13e0
+size 1048576

chroma_package/chroma.tar.part032

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bfab78062f40a1584d3d2e770bcb68bf2fe3dfde4748212b05f1ac0b7479e4c
+size 1048576

chroma_package/chroma.tar.part033

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10a8257a9fe92f51f5ed87073e2b2f8bb9581e5f89b21c0d15a4a430a6faacb2
+size 1048576

chroma_package/chroma.tar.part034

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b43d1c0a5ebc09ee9946e27d946f0b330e29395fa4a28d2854c28c0895edc13
+size 1048576

chroma_package/chroma.tar.part035

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4926d441aa6853986161bf90d6f72b4e03e645172c2a3771fd884126991df108
+size 1048576

chroma_package/chroma.tar.part036

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7188e748dd4299c9ffe9a29dfe335f042ceb4a1e013b0136f58f2ee2cf81490
+size 1048576

chroma_package/chroma.tar.part037

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c07f9538a0897a8fc376bde94411119d9d78a488090eba6b899be1099ec2657
+size 1048576

chroma_package/chroma.tar.part038

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:792bf9b178553098f3bb8093f62da2745d7df06e8a6fd83cf78d62fe8d5e6b9f
+size 1048576

chroma_package/chroma.tar.part039

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e54b2490d46a2da6c5624b95a41fbe2d3ebd9ca1abc08aae8b394894692fb94
+size 1048576

chroma_package/chroma.tar.part040

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea33273014fb24f70ee5fab9dc7ea9c5d1d6425aaeb6e5f2b7aa653437e10642
+size 1048576

chroma_package/chroma.tar.part041

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:727da8b64de123485c3fb80e3deefcbb23561773fe36ca2fd6765d38fe0e70b7
+size 1048576

chroma_package/chroma.tar.part042

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47befd62d954e37220dfcb702e0aee94c78337ce74bee676600a91941fbe2ac7
+size 1048576