--- language: - en - fr pipeline_tag: feature-extraction library_name: onnx tags: - embeddings - retrieval - semantic-search - matryoshka - tokenizer-free - byte-level - onnx - anankesearch --- # AnankeEmbed-S (v1) **AnankeEmbed-S** is a compact, **tokenizer-free** UTF-8 byte-level bidirectional embedding model built by **AMEFORGE** for **AnankeSearch**. It maps queries and documents into a shared 256-dimensional semantic space with **Matryoshka Representation Learning (MRL)** support (`64`, `128`, and `256` dimensions). ## Model Architecture | Parameter | Value | | :--- | :--- | | **Input Representation** | Raw UTF-8 bytes (`VOCAB = 259`: `0..255` bytes, `PAD=256`, `QUERY=257`, `DOC=258`) | | **Tokenizer** | **None** (native UTF-8 bytes — handles typos, accents, emojis, code, any language) | | **Local Mixing & Patching** | Local byte convolution + `4x4` patching (`max_bytes=512` → `128` positions) | | **Backbone** | Bidirectional Transformer (`d=384`, `layers=6`, `heads=6`, RMSNorm, RoPE, SwiGLU) | | **Output Dimensions** | `256` with Matryoshka truncation at `[64, 128, 256]` | | **Precision / Format** | ONNX (`ananke-embed.onnx`, ~46.4 MB) + PyTorch (`best.pt`, ~46.3 MB) | ## Evaluation Results Trained on `NVIDIA A100-SXM4-80GB` (`6.71 h`, best checkpoint at step **`38,000`**): ### Retrieval Benchmarks (Matryoshka Dimensions) | Dimension | MRR | Recall@1 | Recall@10 | | :---: | :---: | :---: | :---: | | **64** | **0.7974** | **0.7350** | 0.9155 | | **128** | 0.7960 | 0.7320 | 0.9150 | | **256** | 0.7966 | 0.7335 | **0.9165** | ### Ananke Search Exam - **`examen_r1`**: `0.70` (70% Top-1) - **`examen_mrr`**: `0.791` > **Matryoshka Efficiency:** Truncating embeddings to **64 dimensions** achieves `MRR = 0.7974` (on par with 256d) while reducing vector storage and similarity compute by **4x**. ## Repository Files - `ananke-embed.onnx` — Exported ONNX graph for fast CPU/GPU inference - `ananke-embed.json` — Model configuration (`d`, `layers`, `heads`, `patch`, `max_bytes`, `out_dim`, `matryoshka`) - `best.pt` — Best PyTorch checkpoint (step `38,000`) - `resultats.json` — Detailed evaluation metrics and training curve ## Deployment & Usage ### 1. One-Command Server Install (AnankeSearch / Kaia) ```bash HF_READ_TOKEN=hf_xxx MODEL_REPO=kaia-cod/anankesearch-embed-s bash scripts/ananke_embed_install.sh ``` ### 2. HTTP Embedding Endpoint (`serve.py`) ```bash curl -X POST http://127.0.0.1:8090/embed \ -H "Authorization: Bearer $ANANKE_EMBED_KEY" \ -H "Content-Type: application/json" \ -d '{"texts": ["fastapi background tasks"], "role": "query", "dim": 64}' ``` ## Intended Use & Limitations - Designed for semantic retrieval, reranking, and private tenant indexing inside **AnankeSearch** and **Kaia (SAM-G-Agent)**. - Trained primarily on web/technical retrieval pairs; increasing French/multilingual training pairs is recommended for future versions (`v2+`).