Upload folder using huggingface_hub

README.md

@@ -15,108 +15,48 @@ language:
 pipeline_tag: text-generation
 ---
 
-# 🧠 Noesis Decoder (AletheiaEngine)
+# Aletheia Noesis Decoder
 
-**Repository:** [gnai-creator/noesis-decoder](https://huggingface.co/gnai-creator/noesis-decoder)
-**Author:** Felipe M. Muniz (`gnai-creator`)
-**License:** Apache-2.0
+Noesis é o decoder proprietário da AletheiaEngine. Ele traduz estados simbólicos \(\psi_s\) em linguagem natural garantindo a coerência epistemológica medida pela **Qualidade da Verdade (Q)**. Diferente de abordagens puramente estatísticas, o Noesis prioriza a fidelidade semântica entre a intenção simbólica e o texto gerado.
 
----
-
-## 🔍 Overview
-
-**Noesis Decoder** is the proprietary symbolic decoder of **AletheiaEngine** — a hybrid symbolic–neural system designed for *philosophical artificial general intelligence*.
-
-Unlike conventional text generators, Noesis translates **symbolic embeddings (ψₛ)** into meaningful language based on *epistemic coherence*, rather than statistical prediction.
+## Arquitetura
 
----
-
-## ⚙️ Model Architecture
+- **Transformer Decoder** com 8 camadas e atenção cruzada condicionada por \(\psi_s\) e estados contínuos.
+- Entradas: vetor de intenção `psi_s`, memória lenta `state` opcional e sequência de tokens.
+- Saídas: `logits`, embedding semântico `z_text` e métrica \(\hat{Q}\).
+- Função de perda combina cross-entropy, coerência Q e penalizações de restrições.
 
-* **Framework:** PyTorch → ONNX Runtime
-* **Files:**
+## API FastAPI
 
-  * `model_infer.onnx` – Inference model (optimized)
-  * `noesis.pt` – PyTorch checkpoint (training artifact)
-  * `inference.py` – Custom ONNX handler
-* **Input:** float32 symbolic vector, shape `[1, D]`
-* **Output:** decoded float or token embeddings (depending on context)
-
----
+O pacote inclui a aplicação `aletheia_decoder.api.app` com duas rotas:
 
-## 🧩 Example Usage
+- `POST /generate`: recebe `psi_s`, `state`, `constraints`, prompts opcionais e parâmetros de decodificação. Retorna texto gerado, tokens, \(\hat{Q}\) e metadados.
+- `POST /train`: permite fine-tuning local a partir de um dataset JSONL.
 
-### 🔹 Python + ONNX Runtime
+### Execução local
 
-```python
-from huggingface_hub import hf_hub_download
-import onnxruntime as ort
-import numpy as np
-
-# Download ONNX model
-onnx_path = hf_hub_download(
-    repo_id="gnai-creator/noesis-decoder",
-    filename="model_infer.onnx",
-    repo_type="model"
-)
-
-# Load runtime
-sess = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
-input_name  = sess.get_inputs()[0].name
-output_name = sess.get_outputs()[0].name
-
-# Example symbolic vector ψₛ
-x = np.random.randn(1, 300).astype("float32")
-
-# Run inference
-y = sess.run([output_name], {input_name: x})[0]
-print("Output shape:", y.shape)
+```bash
+pip install -r requirements.txt
+uvicorn aletheia_decoder.api.app:app --host 0.0.0.0 --port 8000
 ```
 
----
-
-## 💡 Training Data
-
-Trained on **symbolic text pairs** generated from philosophical, logical, and reflective corpora within the AletheiaEngine ecosystem.
-Goal: alignment between **symbolic intention (ψₛ)** and **natural language output**.
+## Treinamento
 
----
-
-## 📊 Metrics (Indicative)
-
-| Metric        | Value        | Description                                |
-| ------------- | ------------ | ------------------------------------------ |
-| Cosine(Q)     | 0.83         | Symbolic alignment measure                 |
-| Perplexity    | 2.41         | Statistical readability proxy              |
-| Latency (CPU) | ~28 ms/token | Inference on Intel Sapphire Rapids (1vCPU) |
-
----
-
-## 🚀 Deployment
-
-This model is compatible with **Hugging Face Inference Endpoints** using the `Default` engine and the included `handler.py` handler.
-
-Recommended hardware:
+Use o script `aletheia_decoder/model/train.py` para treinar o modelo com dados anotados:
 
-* **CPU:** Intel Sapphire Rapids (1vCPU / 2GB)
-* **GPU:** NVIDIA T4 for larger batch inference
-
----
-
-## ⚠️ Limitations
-
-* Not a conventional LLM — requires symbolic vectors as input.
-* Outputs are contextualized to Aletheia’s symbolic reasoning pipeline.
-* Not suited for free-form text generation.
+```bash
+python -m aletheia_decoder.model.train data/dataset.jsonl --epochs 200 --batch-size 4 --lr 3e-5
+```
 
----
+Os checkpoints podem ser salvos com `torch.save(model.state_dict(), "noesis.pt")` e exportados para ONNX se necessário.
 
-## 📜 License
+## Deploy no Hugging Face Spaces
 
-This repository is distributed under the **Apache License 2.0**.
-See [LICENSE](./LICENSE) for details.
+1. Faça fork deste diretório para um repositório separado.
+2. Configure o Space no modo **FastAPI** apontando para `app:app`.
+3. Defina as variáveis de ambiente de acordo com sua infraestrutura (ex.: `DECODER_ENDPOINT`).
+4. Opcionalmente publique checkpoints `.pt` e `.onnx` nos assets do Space.
 
----
+## Slogan
 
-> *“Truth is not imposed; it emerges from alignment.”*
-> — *Felipe M. Muniz (2025)*
+> **Noesis — onde a intenção se torna linguagem.**

handler.py

@@ -18,12 +18,63 @@ from __future__ import annotations
 
 from dataclasses import dataclass
 from pathlib import Path
+import re
 from typing import Any, Mapping, MutableMapping, Optional
 
 import numpy as np
 import onnxruntime as ort
 
 
+_WORD_RE = re.compile(r"\w+", re.UNICODE)
+
+
+class _TextEncoder:
+    """Deterministic text → vector encoder.
+
+    The Hugging Face Inference Endpoints frequently pass user prompts as
+    strings via the ``inputs`` field.  The Noesis decoder, however, expects a
+    symbolic vector (``psi``) as input.  To provide a graceful fallback the
+    handler lazily converts short text prompts into a stable float32 vector by
+    hashing tokens onto a hypersphere.  This mirrors the lightweight
+    ``TextEncoder256`` implementation bundled with the full AletheiaEngine
+    package while avoiding a heavy import dependency inside the endpoint
+    container.
+    """
+
+    def __init__(self, dim: int) -> None:
+        self.dim = dim
+
+    @staticmethod
+    def _tokens(text: str) -> list[str]:
+        return [tok.lower() for tok in _WORD_RE.findall(text)]
+
+    @staticmethod
+    def _seed(tok: str) -> int:
+        # FNV-1a hash for determinism across processes/platforms.
+        value = 2166136261
+        for byte in tok.encode("utf-8"):
+            value ^= byte
+            value = (value * 16777619) & 0xFFFFFFFF
+        return int(value)
+
+    def encode(self, text: str) -> np.ndarray:
+        tokens = self._tokens(text)
+        if not tokens:
+            return np.zeros((1, self.dim), dtype=np.float32)
+
+        vecs = []
+        for tok in tokens:
+            rs = np.random.RandomState(self._seed(tok))
+            embedding = rs.normal(0.0, 1.0, size=(self.dim,)).astype(np.float32)
+            norm = float(np.linalg.norm(embedding)) or 1.0
+            vecs.append(embedding / norm)
+
+        stacked = np.stack(vecs, axis=0)
+        pooled = stacked.mean(axis=0, dtype=np.float32, keepdims=True)
+        pooled_norm = float(np.linalg.norm(pooled)) or 1.0
+        return pooled / pooled_norm
+
+
 @dataclass(frozen=True)
 class _ModelIO:
     """Snapshot of ONNX input and output metadata."""
@@ -42,6 +93,8 @@ class EndpointHandler:
 
         self.primary_input = self.io.inputs[0].name
         self.slow_input = self._find_input("slow_state")
+        self._primary_dim = self._infer_primary_dim()
+        self._text_encoder = _TextEncoder(self._primary_dim)
         self._defaults = {
             node.name: self._zeros_like(node)
             for node in self.io.inputs
@@ -53,13 +106,24 @@ class EndpointHandler:
             self._slow_fallback = None
 
     def _load_session(self) -> ort.InferenceSession:
-        model_path = self.model_dir / "model.onnx"
-        if not model_path.exists():
-            available = ", ".join(sorted(str(p.name) for p in self.model_dir.glob("*.onnx"))) or "<none>"
-            raise FileNotFoundError(
-                "Could not locate 'model.onnx' in %s (available: %s)" % (self.model_dir, available)
-            )
-        return ort.InferenceSession(str(model_path), providers=["CPUExecutionProvider"])
+        """Load the ONNX session, tolerating alternate filenames."""
+
+        preferred_names = ("model.onnx", "model_infer.onnx")
+        for name in preferred_names:
+            candidate = self.model_dir / name
+            if candidate.exists():
+                return ort.InferenceSession(str(candidate), providers=["CPUExecutionProvider"])
+
+        available = sorted(str(p.name) for p in self.model_dir.glob("*.onnx"))
+        if len(available) == 1:
+            # Fall back to the lone ONNX artefact if it has a non-standard name.
+            return ort.InferenceSession(str(self.model_dir / available[0]), providers=["CPUExecutionProvider"])
+
+        choices = ", ".join(available) or "<none>"
+        raise FileNotFoundError(
+            "Could not locate any of %s in %s (available: %s)"
+            % (", ".join(preferred_names), self.model_dir, choices)
+        )
 
     @property
     def _input_map(self) -> Mapping[str, ort.NodeArg]:
@@ -75,6 +139,14 @@ class EndpointHandler:
                 return node.name
         return None
 
+    def _infer_primary_dim(self) -> int:
+        node = self._input_map[self.primary_input]
+        for dim in reversed(node.shape):
+            if isinstance(dim, int) and dim > 0:
+                return dim
+        # Conservative default matching TextEncoder256.
+        return 256
+
     @staticmethod
     def _zeros_like(node: ort.NodeArg) -> np.ndarray:
         shape: list[int] = []
@@ -99,11 +171,19 @@ class EndpointHandler:
     def _prepare_inputs(self, payload: Mapping[str, Any]) -> MutableMapping[str, np.ndarray]:
         psi = payload.get("psi")
         if psi is None:
-            psi = payload.get("vector") or payload.get("psi_s") or payload.get("inputs")
+            psi = (
+                payload.get("vector")
+                or payload.get("psi_s")
+                or payload.get("inputs")
+                or payload.get("prompt")
+                or payload.get("text")
+            )
         if psi is None:
             raise KeyError("Payload must include a 'psi' field containing the symbolic vector.")
 
-        inputs: MutableMapping[str, np.ndarray] = {self.primary_input: self._coerce_array(psi)}
+        inputs: MutableMapping[str, np.ndarray] = {
+            self.primary_input: self._vector_from_payload(psi)
+        }
 
         if self.slow_input is not None:
             slow_value = payload.get("slow_state") or payload.get("slow") or payload.get("state")
@@ -117,6 +197,15 @@ class EndpointHandler:
 
         return inputs
 
+    def _vector_from_payload(self, value: Any) -> np.ndarray:
+        if isinstance(value, str):
+            return self._text_encoder.encode(value)
+
+        if isinstance(value, (list, tuple)) and value and all(isinstance(v, str) for v in value):
+            return self._text_encoder.encode(" ".join(value))
+
+        return self._coerce_array(value)
+
     @staticmethod
     def _format_output(name: str, value: np.ndarray) -> Any:
         value = np.asarray(value, dtype=np.float32)