File size: 5,618 Bytes

---
license: mit
language:
- en
pipeline_tag: text-generation
tags:
- conversational
- dialogue
- latent-space
- thought-vectors
- research
- pytorch
---

# thoughtvec — conversation in thought-vector space

A ~48M-parameter dialogue system that never does token-level language
modeling in its reasoning loop. Trained from scratch in ~25 GPU-hours on one
consumer GPU (AMD RX 6700 XT, 12 GB) — a 12-hour codec run and a 12-hour
thinker run. (That's the cost of these weights, not of the research: the
ablation record behind them — 86 checkpointed runs — took on the order of
another hundred GPU-hours.)

It works in two parts:

- A **codec** (32.9M params) turns text into a sequence of "thought vectors"
  ordered by importance — any prefix of them decodes back to text, so
  compression ratio is a decode-time choice. Byte-perfect at 4:1, readable
  at 8:1.
- A **thinker** (15.1M params) converses directly in that latent space:
  history turns in, response thoughts out, and the frozen codec decoder
  renders the reply. No tokens anywhere in between.

```
user > i'm feeling really overwhelmed with work lately.
bot  > I can imagine. What's been going on?
user > my boss keeps piling on deadlines.
bot  > That sounds like a lot of pressure. Have you been able to talk to anyone else?
```

That transcript is greedy decoding (temperature 0), reproducible from
`FINAL_12H-best.pt`.

**Try it in the browser**: [thought-vectors-chat](https://huggingface.co/spaces/nochinator/thought-vectors-chat)
— a model picker lets you compare live against the paper's matched
token-LM baseline (§6.5) on the same conversation.

## What to expect — and not

This is a research model, and a small one. It holds coherent, grounded,
context-sensitive small talk: openings, follow-up questions, multi-turn
reference. That is the finding — conversational competence at this level
does not require a token-level LM in the loop.

It is *not* a general assistant. No knowledge tasks, English small talk
only. And it has one well-documented disease: it can reply cheerfully to
bad news, especially after an upbeat turn ("That's good to hear." to
insomnia). The paper traces that failure through three ablation rounds to
a training-data absence — no conversation in any corpus used ever reverses
mood mid-dialogue — and shows that even targeted data patching bought only
partial, topic-gated routing rather than the skill (the patched case-study
model commiserates with illness or job loss, and still cheers a burglary
or a hailstorm). If you probe the model, you will find it; it's a case
study, not a surprise.

## Files

| File | What it is |
|---|---|
| `FINAL_12H-best.pt` | Thinker flagship (use this one to chat) |
| `FINAL2_12H-best.pt` | Register case-study run (+EmpatheticDialogues +reversal splices) |
| `m5_frontier-best.pt` | Codec — needed alongside either thinker |
| `spm16k_bpe.model` | 16K SentencePiece tokenizer — all checkpoints depend on it |

SHA-256 (byte-identical to the [GitHub Release](https://github.com/nochinator/thought-vectors/releases/tag/v1.1.0) assets):

```
4685ab3dae35e7835ee0fdd469d23d39347b0cca0f71c0598bea30443f0d7672  FINAL_12H-best.pt
bc565782b51d74fe56de1c2d9dd4e44c52b1e6ff192bfcdb758291edeaf5240c  FINAL2_12H-best.pt
77bf080cc216f628fa80c260acc2cfe2ffb03c19bc3d44071047ad4377b0e4ad  m5_frontier-best.pt
```

The paper's matched token-LM baseline (`b3_lm_48m_24h-best.pt`, §6.5) is
the comparison model, not part of this system, so the raw checkpoint ships
as a GitHub Release asset only rather than in this repo. An ONNX export of
it is available for live comparison in the [chat demo](https://huggingface.co/spaces/nochinator/thought-vectors-chat)'s
model picker.

## Run it locally

CPU is enough — chat inference runs on CPU by design.

```bash
git clone https://github.com/nochinator/thought-vectors
cd thought-vectors
scripts/setup_env.sh --cpu

# place the files (thinker embeds codec weights but reads codec config
# from the codec file — you need both):
#   FINAL_12H-best.pt   -> checkpoints/FINAL_12H/best.pt
#   m5_frontier-best.pt -> checkpoints/m5_frontier/best.pt

.venv/bin/tv-chat --ckpt checkpoints/FINAL_12H/best.pt --device cpu
```

## Results at a glance

| Run | val_cos ↑ | ref_F1 ↑ | self_rep ↓ | ctx_sens ↓ |
|---|---|---|---|---|
| FINAL_12H (flagship) | **0.428** | **0.297** | 0.188 | **0.146** |
| FINAL2_12H (case study) | 0.415 | 0.278 | **0.154** | 0.206 |

Codec: byte-perfect reconstruction at 4:1 compression through 257 tokens;
graceful degradation at 8:1 and beyond. Every number maps to a named log
file in the repo — the full experimental record, including the negative
rounds and four incidents of models gaming lexical metrics, is in
[RESEARCH_LOG.md](https://github.com/nochinator/thought-vectors/blob/main/RESEARCH_LOG.md).

## Links

- **Paper**: [DOI 10.5281/zenodo.21262842](https://doi.org/10.5281/zenodo.21262842) — also as
  [PDF in the repo](https://github.com/nochinator/thought-vectors/blob/main/paper/main.pdf)
- **Code + logs**: https://github.com/nochinator/thought-vectors
- **Reproduce from scratch** (~25 GPU-hours, one RX 6700 XT):
  [docs/REPRODUCE.md](https://github.com/nochinator/thought-vectors/blob/main/docs/REPRODUCE.md)

## Citation

```bibtex
@misc{nochi2026thoughtvectors,
  author = {nochi},
  title  = {Codecs: Separating Meaning from Language --- Coherent
            Dialogue in Thought-Vector Space from 48M Parameters on One
            Consumer GPU},
  year   = {2026},
  doi    = {10.5281/zenodo.21262842},
  url    = {https://github.com/nochinator/thought-vectors}
}
```