| --- |
| license: mit |
| language: |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - conversational |
| - dialogue |
| - latent-space |
| - thought-vectors |
| - research |
| - pytorch |
| --- |
| |
| # thoughtvec — conversation in thought-vector space |
|
|
| A ~48M-parameter dialogue system that never does token-level language |
| modeling in its reasoning loop. Trained from scratch in ~25 GPU-hours on one |
| consumer GPU (AMD RX 6700 XT, 12 GB) — a 12-hour codec run and a 12-hour |
| thinker run. (That's the cost of these weights, not of the research: the |
| ablation record behind them — 86 checkpointed runs — took on the order of |
| another hundred GPU-hours.) |
|
|
| It works in two parts: |
|
|
| - A **codec** (32.9M params) turns text into a sequence of "thought vectors" |
| ordered by importance — any prefix of them decodes back to text, so |
| compression ratio is a decode-time choice. Byte-perfect at 4:1, readable |
| at 8:1. |
| - A **thinker** (15.1M params) converses directly in that latent space: |
| history turns in, response thoughts out, and the frozen codec decoder |
| renders the reply. No tokens anywhere in between. |
|
|
| ``` |
| user > i'm feeling really overwhelmed with work lately. |
| bot > I can imagine. What's been going on? |
| user > my boss keeps piling on deadlines. |
| bot > That sounds like a lot of pressure. Have you been able to talk to anyone else? |
| ``` |
|
|
| That transcript is greedy decoding (temperature 0), reproducible from |
| `FINAL_12H-best.pt`. |
|
|
| **Try it in the browser**: [thought-vectors-chat](https://huggingface.co/spaces/nochinator/thought-vectors-chat) |
| — a model picker lets you compare live against the paper's matched |
| token-LM baseline (§6.5) on the same conversation. |
|
|
| ## What to expect — and not |
|
|
| This is a research model, and a small one. It holds coherent, grounded, |
| context-sensitive small talk: openings, follow-up questions, multi-turn |
| reference. That is the finding — conversational competence at this level |
| does not require a token-level LM in the loop. |
|
|
| It is *not* a general assistant. No knowledge tasks, English small talk |
| only. And it has one well-documented disease: it can reply cheerfully to |
| bad news, especially after an upbeat turn ("That's good to hear." to |
| insomnia). The paper traces that failure through three ablation rounds to |
| a training-data absence — no conversation in any corpus used ever reverses |
| mood mid-dialogue — and shows that even targeted data patching bought only |
| partial, topic-gated routing rather than the skill (the patched case-study |
| model commiserates with illness or job loss, and still cheers a burglary |
| or a hailstorm). If you probe the model, you will find it; it's a case |
| study, not a surprise. |
|
|
| ## Files |
|
|
| | File | What it is | |
| |---|---| |
| | `FINAL_12H-best.pt` | Thinker flagship (use this one to chat) | |
| | `FINAL2_12H-best.pt` | Register case-study run (+EmpatheticDialogues +reversal splices) | |
| | `m5_frontier-best.pt` | Codec — needed alongside either thinker | |
| | `spm16k_bpe.model` | 16K SentencePiece tokenizer — all checkpoints depend on it | |
|
|
| SHA-256 (byte-identical to the [GitHub Release](https://github.com/nochinator/thought-vectors/releases/tag/v1.1.0) assets): |
|
|
| ``` |
| 4685ab3dae35e7835ee0fdd469d23d39347b0cca0f71c0598bea30443f0d7672 FINAL_12H-best.pt |
| bc565782b51d74fe56de1c2d9dd4e44c52b1e6ff192bfcdb758291edeaf5240c FINAL2_12H-best.pt |
| 77bf080cc216f628fa80c260acc2cfe2ffb03c19bc3d44071047ad4377b0e4ad m5_frontier-best.pt |
| ``` |
|
|
| The paper's matched token-LM baseline (`b3_lm_48m_24h-best.pt`, §6.5) is |
| the comparison model, not part of this system, so the raw checkpoint ships |
| as a GitHub Release asset only rather than in this repo. An ONNX export of |
| it is available for live comparison in the [chat demo](https://huggingface.co/spaces/nochinator/thought-vectors-chat)'s |
| model picker. |
|
|
| ## Run it locally |
|
|
| CPU is enough — chat inference runs on CPU by design. |
|
|
| ```bash |
| git clone https://github.com/nochinator/thought-vectors |
| cd thought-vectors |
| scripts/setup_env.sh --cpu |
| |
| # place the files (thinker embeds codec weights but reads codec config |
| # from the codec file — you need both): |
| # FINAL_12H-best.pt -> checkpoints/FINAL_12H/best.pt |
| # m5_frontier-best.pt -> checkpoints/m5_frontier/best.pt |
| |
| .venv/bin/tv-chat --ckpt checkpoints/FINAL_12H/best.pt --device cpu |
| ``` |
|
|
| ## Results at a glance |
|
|
| | Run | val_cos ↑ | ref_F1 ↑ | self_rep ↓ | ctx_sens ↓ | |
| |---|---|---|---|---| |
| | FINAL_12H (flagship) | **0.428** | **0.297** | 0.188 | **0.146** | |
| | FINAL2_12H (case study) | 0.415 | 0.278 | **0.154** | 0.206 | |
|
|
| Codec: byte-perfect reconstruction at 4:1 compression through 257 tokens; |
| graceful degradation at 8:1 and beyond. Every number maps to a named log |
| file in the repo — the full experimental record, including the negative |
| rounds and four incidents of models gaming lexical metrics, is in |
| [RESEARCH_LOG.md](https://github.com/nochinator/thought-vectors/blob/main/RESEARCH_LOG.md). |
|
|
| ## Links |
|
|
| - **Paper**: [DOI 10.5281/zenodo.21262842](https://doi.org/10.5281/zenodo.21262842) — also as |
| [PDF in the repo](https://github.com/nochinator/thought-vectors/blob/main/paper/main.pdf) |
| - **Code + logs**: https://github.com/nochinator/thought-vectors |
| - **Reproduce from scratch** (~25 GPU-hours, one RX 6700 XT): |
| [docs/REPRODUCE.md](https://github.com/nochinator/thought-vectors/blob/main/docs/REPRODUCE.md) |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{nochi2026thoughtvectors, |
| author = {nochi}, |
| title = {Codecs: Separating Meaning from Language --- Coherent |
| Dialogue in Thought-Vector Space from 48M Parameters on One |
| Consumer GPU}, |
| year = {2026}, |
| doi = {10.5281/zenodo.21262842}, |
| url = {https://github.com/nochinator/thought-vectors} |
| } |
| ``` |
|
|
