README.md

---
language:
- en
license: apache-2.0
tags:
- multimodal
- embedding
- trimodal
- retrieval
- image-text-audio
- feature-extraction
library_name: pytorch
pipeline_tag: feature-extraction
datasets:
- custom
---

# ESS-AIST-81M Preview

`ESS-AIST-81M Preview` is the current Cortext trial checkpoint from the ESS line.

- release checkpoint: `ess_aist_full_v9_subjectfix_l4k/best_model.pt`
- exported checkpoint epoch: `3`
- text encoder: `MongoDB/mdbr-leaf-ir`
- image encoder: `mobilenetv4_conv_medium.e180_r384_in12k`
- audio encoder: native `mn20_as` EfficientAT LoRA audio backbone

This preview is the current bridge artifact for Cortext. It keeps the ESS
`semantic / subject / event` slice layout, but the `v9` dataset repair moved
the `subject` slice much closer to the entity signal Cortext actually needs.

GGUF quantizations for this exact release live in:

- `augmem/ESS-AIST-81M-preview-GGUF`

Tradeoff:

- held-out subject/entity separation is much stronger than the earlier `v7` preview
- speech and SALT retrieval are weaker than the earlier `v7` retrieval-max point

For Cortext, this is still the better preview because the entity-side signal is
materially stronger.

## Embedding Layout

Output embedding: `1536d`

- `0:512` semantic
- `512:1024` subject
- `1024:1536` event

Recommended normalized runtime views:

- `semantic_key = l2norm(z[0:512])`
- `subject_key = l2norm(z[512:1024])`
- `event_key = l2norm(z[1024:1536])`
- `full_key = l2norm(z[0:1536])`

## Exact Release Metrics

All numbers below are from the exact published checkpoint state exported from
`ess_aist_full_v9_subjectfix_l4k/best_model.pt` at checkpoint epoch `3`.

Evaluation scope note:

- `SALT` is no longer a clean out-of-training benchmark for this ESS line because SALT-derived rows were used during training.
- `speech holdout` is also train-adjacent rather than fully external because explicit speech/audio-text data was added back into the corpus.
- In this release, those two surfaces should be read as regression gates and in-domain checks, not as contamination-free generalization claims.
- A later full external sweep (`MTEB / MIEB / MAEB`) is still pending.

### 512d Retrieval

Source:
- `retrieval_512_gt1030.json`

Speech holdout:

- `A->T_r1 = 0.3276`
- `T->A_r1 = 0.3202`
- `A->T_r5 = 0.6120`
- `T->A_r5 = 0.6046`

SALT:

- `I->T_r1 = 0.3179`
- `T->I_r1 = 0.3425`
- `A->T_r1 = 0.1226`
- `T->A_r1 = 0.1272`
- `I->A_r1 = 0.1970`
- `A->I_r1 = 0.2148`

### Held-Out ESS Eval

Sources:

- `subject_eval.json`
- `event_eval.json`
- `prefix_eval.json`

Subject / entity surface:

- `subject_key` same/different AUC: `0.9881`
- `subject_key` same-topic-different-subject rejection AUC: `0.9881`

Event / disambiguation surface:

- `subject_key` event same/different AUC: `0.8855`
- `event_key` event same/different AUC: `0.8193`
- `subject_key` same-subject-different-event rejection AUC: `0.7381`
- `event_key` same-subject-different-event rejection AUC: `0.6807`
- `subject_key` topic-shift rejection AUC: `0.9513`
- `event_key` topic-shift rejection AUC: `0.8969`

Interpretation:

- the repaired `v9` held-out surface is no longer near-random on subject/entity
- the current `subject` slice is the strongest entity carrier in the model
- event structure is usable, but still entangled with subject
- this is the right bridge checkpoint for Cortext, not the final `semantic/entity` architecture

## Architecture

This preview is a frozen-encoder / trainable-projector stack:

- text encoder params: `22,861,056`
- image encoder params: `8,434,512`
- audio encoder params: `20,639,974`
- image projection params: `9,975,296`
- audio projection params: `9,975,296`
- text projection params: `8,926,720`
- total exact loaded params: `80,812,854`

The audio path is not the old dual-audio teacher path. It uses the native
audioheavy LoRA EfficientAT backbone.

## Files

| File | Purpose |
|---|---|
| `ESS-AIST-81M.safetensors` | Full preview release artifact |
| `export_metadata.json` | ESS export contract |
| `manifest.json` | Release manifest |
| `parameter_breakdown.json` | Exact parameter accounting |
| `ess_ait_86m_spec.yaml` | Training config used for the release line |
| `retrieval_512_gt1030.json` | Exact 512d retrieval eval for this checkpoint |
| `subject_eval.json` | Exact held-out subject eval for this checkpoint |
| `event_eval.json` | Exact held-out event eval for this checkpoint |
| `prefix_eval.json` | Prefix-level AUC summary |

## Caveats

- This is the current preview checkpoint, not the final Cortext model family.
- The current runtime slices are still named `semantic / subject / event`; the next family will move toward `semantic / entity`.
- Subject/entity is now strong on the repaired `v9` held-out surface, but event remains entangled and the engine still needs attention over active anchors for weak-reference resolution.
- Retrieval on `speech holdout` and `SALT` is lower than the earlier `v7` preview.
- `SALT` and `speech holdout` are useful release gates for this line, but they are no longer fully external benchmarks in the same way they were for the earlier pre-ESS artifacts.
- Use this for internal Cortext trials, not as the final memory-model release.