README.md

---
tags:
- multi-agent-systems
- memory
- routing
---
# Sigma-Mem Checkpoints

Trained Sigma-Mem parameters for five center-model configurations.

## Paper

[Σ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems](https://arxiv.org/abs/2607.27958)

![Sigma-Mem method overview](assets/Method.png)

## What These Checkpoints Contain

These files contain the **Sigma-Mem components** illustrated in the figure above;
they do not contain a fine-tuned copy of the Central Model (CM). For each peer
\\(p\\), Sigma-Mem maintains a real symmetric memory matrix
\\(\mathbf{M}_p \in \mathbb{R}^{r \times r}\\), with
\\(\mathbf{M}_p = \mathbf{M}_p^\top\\). Given the current task direction
\\(\boldsymbol{\phi}(\mathbf{x}_t)\\), the memory readout is

$$
\mathbf{r}_{p,t} = \mathbf{M}_p\boldsymbol{\phi}(\mathbf{x}_t).
$$

The learned projection converts this readout into a peer-specific residual steering
vector:

$$
\boldsymbol{\delta}_{p,t}
= g\,\mathbf{P}\mathbf{M}_p\boldsymbol{\phi}(\mathbf{x}_t).
$$

The steering vector is added only to the upper residual stream while the CM judges
peer \\(p\\)'s response. It therefore influences response aggregation through the
CM's Yes-versus-No utility scores, while all original CM parameters remain frozen and
unchanged. The joint relationship matrix \\(\mathbf{G}\\) is updated separately from
peer correctness patterns and is used by the final peer-selection posterior.

Each checkpoint provides the learned Sigma-Mem address/projection and residual-
steering parameters needed by this process. Runtime \\(\mathbf{M}_p\\) and
\\(\mathbf{G}\\) states are initialized for the evaluation stream and updated only
after the current decision receives external correctness feedback.

## Available Checkpoints

| Directory | Central model |
| --- | --- |
| `Qwen3-0.6B/` | Qwen3-0.6B |
| `Qwen3-4B/` | Qwen3-4B |
| `Qwen3-8B/` | Qwen3-8B |
| `Qwen3.5-4B/` | Qwen3.5-4B |
| `Qwen3.5-9B/` | Qwen3.5-9B |

Each directory contains:

- `sym_memory.pt`: trained Sigma-Mem parameters.
- `train_config.json`: portable training and model metadata.
- `checkpoint_manifest.json`: integrity metadata for the checkpoint.

## Download

```bash
hf download Sssunset/Sigma-Mem --local-dir models/Sigma-Mem
```