File size: 2,936 Bytes
c86e41c
4efe1d7
 
 
 
 
c86e41c
 
 
4efe1d7
c86e41c
4efe1d7
 
c86e41c
4efe1d7
c86e41c
4efe1d7
c86e41c
 
4efe1d7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c86e41c
4efe1d7
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
language:
- en
- fr
pipeline_tag: feature-extraction
library_name: onnx
tags:
- embeddings
- retrieval
- semantic-search
- matryoshka
- tokenizer-free
- byte-level
- onnx
- anankesearch
---

# AnankeEmbed-S (v1)

**AnankeEmbed-S** is a compact, **tokenizer-free** UTF-8 byte-level bidirectional embedding model built by **AMEFORGE** for **AnankeSearch**. It maps queries and documents into a shared 256-dimensional semantic space with **Matryoshka Representation Learning (MRL)** support (`64`, `128`, and `256` dimensions).

## Model Architecture

| Parameter | Value |
| :--- | :--- |
| **Input Representation** | Raw UTF-8 bytes (`VOCAB = 259`: `0..255` bytes, `PAD=256`, `QUERY=257`, `DOC=258`) |
| **Tokenizer** | **None** (native UTF-8 bytes — handles typos, accents, emojis, code, any language) |
| **Local Mixing & Patching** | Local byte convolution + `4x4` patching (`max_bytes=512` → `128` positions) |
| **Backbone** | Bidirectional Transformer (`d=384`, `layers=6`, `heads=6`, RMSNorm, RoPE, SwiGLU) |
| **Output Dimensions** | `256` with Matryoshka truncation at `[64, 128, 256]` |
| **Precision / Format** | ONNX (`ananke-embed.onnx`, ~46.4 MB) + PyTorch (`best.pt`, ~46.3 MB) |

## Evaluation Results

Trained on `NVIDIA A100-SXM4-80GB` (`6.71 h`, best checkpoint at step **`38,000`**):

### Retrieval Benchmarks (Matryoshka Dimensions)

| Dimension | MRR | Recall@1 | Recall@10 |
| :---: | :---: | :---: | :---: |
| **64** | **0.7974** | **0.7350** | 0.9155 |
| **128** | 0.7960 | 0.7320 | 0.9150 |
| **256** | 0.7966 | 0.7335 | **0.9165** |

### Ananke Search Exam

- **`examen_r1`**: `0.70` (70% Top-1)
- **`examen_mrr`**: `0.791`

> **Matryoshka Efficiency:** Truncating embeddings to **64 dimensions** achieves `MRR = 0.7974` (on par with 256d) while reducing vector storage and similarity compute by **4x**.

## Repository Files

- `ananke-embed.onnx` — Exported ONNX graph for fast CPU/GPU inference
- `ananke-embed.json` — Model configuration (`d`, `layers`, `heads`, `patch`, `max_bytes`, `out_dim`, `matryoshka`)
- `best.pt` — Best PyTorch checkpoint (step `38,000`)
- `resultats.json` — Detailed evaluation metrics and training curve

## Deployment & Usage

### 1. One-Command Server Install (AnankeSearch / Kaia)

```bash
HF_READ_TOKEN=hf_xxx MODEL_REPO=kaia-cod/anankesearch-embed-s bash scripts/ananke_embed_install.sh
```

### 2. HTTP Embedding Endpoint (`serve.py`)

```bash
curl -X POST http://127.0.0.1:8090/embed \
  -H "Authorization: Bearer $ANANKE_EMBED_KEY" \
  -H "Content-Type: application/json" \
  -d '{"texts": ["fastapi background tasks"], "role": "query", "dim": 64}'
```

## Intended Use & Limitations

- Designed for semantic retrieval, reranking, and private tenant indexing inside **AnankeSearch** and **Kaia (SAM-G-Agent)**.
- Trained primarily on web/technical retrieval pairs; increasing French/multilingual training pairs is recommended for future versions (`v2+`).