---
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: text-generation
tags:
- byte-level
- custom-code
- experimental
- harmonic-gpt
- multiparty-dialogue
---

# Harmonic GPT 128M Byte Agent Multiparty

An experimental 127.6M-parameter causal byte Transformer trained to represent dialogue as a
room of peer agents rather than a privileged user/assistant pair. Turns explicitly encode the
speaker, addressee, audience, and participant registry.

The checkpoint starts from Transformer pretraining step 90,000 (5.898B ordered UTF-8 bytes) and
then receives 500 completion-only SFT steps at peak LR `3e-5`. Its mixture contains 25% No Robots
Q&A, 25% everyday conversations, 10% practical dialogue, 5% reasoning, 25% MultiLIGHT
multiparty dialogue, and 10% pretraining replay.

## Evaluation snapshot

- Held-out ordinary-chat BPB: 1.244777 (starting value 1.294554)
- Held-out multiparty BPB: 1.216836 (starting value 1.382048)
- Matched base-corpus BPB: 0.950660 (starting value 0.940957)
- Response-only linguistic composite: 76.6
- 768-byte repetition probe: 0/6 detected loops under both greedy and T=0.8/top-k 40
- EOS: 5/6 greedy and 6/6 stochastic long-form probes

These are behavioral diagnostics, not claims of factual reliability. The model frequently gives
incorrect or nonsensical answers and must not be used for consequential decisions.

## Native byte framing

```text
<|room|> scope=direct members=p0,p1
<|participants|>
p0=Agent.Ajax
p1=Agent.Steve
<|turn|> p0>p1 audience=p0,p1
Agent.Ajax:
Hello!
<|end|>
<|turn|> p1>p0 audience=p0,p1
Agent.Steve:
```

The learned response terminator is `\n<|end|>\n`. The included handler maps OpenAI-style
user/assistant histories onto the two peer-agent slots while preserving turn boundaries.

The No Robots component makes this release CC BY-NC 4.0. MultiLIGHT is used through its public
ShareGPT conversion and is primarily fantasy roleplay; the checkpoint should be treated as an
exploratory group-dialogue experiment.