Blame for dev/HANDOFF.md

# HANDOFF — DOS LLM (中文 / English)

Read this first. Everything below was checked against the real running page, not assumed.
Project owner language: Traditional Chinese. Replies: short, factual, no emojis, say limitations honestly.

## 1. What it is (and what it is not)

* Space: `HelloSun/dosllm` (static, public). Live: https://hellosun-dosllm.static.hf.space/
* One **FreeDOS 1.4** machine runs in the page on the **v86** x86 emulator. Inside it: Crynwr **NE2000 packet driver** -> mTCP **DHCP.EXE**
  (192.168.86.100) -> **G.EXE**, a real DOS program (Open Watcom + mTCP) that does DNS, TCP and **HTTP** itself.
* Press the page's button -> the page types `SET GRADIO_HOST=...`, `CLS`, `G <question>` into the DOS prompt -> G.EXE asks a public Gradio Space
  and prints `A> <answer>` on the DOS screen. The page then compares the DOS screen text with the answer it captured from Gradio (`DOS = LLM`).
* **Honest limits.** DOS has no TLS and browsers cannot open raw TCP, and a static Space has no server. So `site/bridge.js` relays the guest's
  plain HTTP (port 80) as HTTPS `fetch()`. The DOS side is genuine TCP/HTTP; the last hop to the Space is HTTPS done by the browser.
  The page (not DOS) decides *when* to run G.EXE, by typing into the DOS keyboard. The "LLM" is whatever Gradio Space is configured (`tencent/Hy3`).
* **Chinese (中文) works too.** The floppy also loads **小貓中文 (XiaoMao Chinese)**, a DOS TSR from the owner's `HelloSun/c_dos` Space: Big5 display + Zhuyin input.
  Ask in Chinese (the page, or `Ctrl+Space` + Zhuyin typed directly in DOS) and the answer is printed in Chinese on the DOS screen (section 10).
* js-dos 6.22 was tried first in the sibling Space: it has no TCP/IP at all, which is why v86 is used.

## 2. Data path

```
G.EXE (mTCP TCP/IP stack, DOS)  -- plain HTTP/1.0 on port 80 -->
   packet driver NE2000.COM (INT 0x60, I/O 0xC000, IRQ 10)
   -> v86 emulated NE2000 (PCI)                          DHCP + DNS answered by v86 itself (DNS = DoH to cloudflare-dns.com)
   -> v86 "wisp" network adapter (terminates guest TCP)
   -> WebSocket('wisp://dosllm-bridge.invalid/?m=dos')   <- monkeypatched in bridge.js (Proxy on window.WebSocket)
   -> FakeWispServer (bridge.js): per TCP stream collects ONE HTTP request, then
      fetch('https://<Host header>/gradio_api/...')      (Spaces answer CORS by echoing the caller's Origin, also http://127.0.0.1 and localhost)
   -> writes 'HTTP/1.0 <status>' + Content-Length + body back, then a WISP CLOSE (guest sees FIN after the data)
```
* WISP frames: 1 CONNECT, 2 DATA, 3 CONTINUE (credit; we hand out 128 on stream 0 and per DATA), 4 CLOSE. The wisp adapter reports the destination as an **IP**, so
  the bridge takes the real host from the HTTP `Host:` header (DOS resolved the real name through DoH).
* **Chinese transcoding in the bridge** (`site/big5.js`): DOS shows/handles **Big5**, Gradio speaks UTF-8. A POST body that contains bytes >= 0x80 is decoded with
  `TextDecoder('big5')` before the HTTPS request; JSON/SSE replies (`\uXXXX` escapes) are rewritten to Big5 bytes before DOS sees them (characters Big5 lacks -> `?`).
* **Allow-list in `relay()`**: host must match `*.hf.space`, path must start with `/gradio_api/`, GET/POST only, request body <= 8 KB, no credentials added, 150 s timeout.
  Other ports are refused. It is not an open proxy; keep it that way.

## 3. Layout and how to build

See `README.md` for the tree. `tools/build_all.sh` (a few seconds once deps are cached):

1. `tools/fetch_deps.sh`: v86 0.5.472 (npm), SeaBIOS/VGABIOS, FreeDOS 1.4 floppy, Crynwr NE2000.COM, mTCP 2026-10-10 (bin + src), Open Watcom **1.9** into `deps/`,
   each checked against a pinned SHA-256 (change the hashes on purpose when upgrading). `DEPS=<dir>` reuses a cache.
2. `src/build.sh` -> `build/G.EXE` (16-bit DOS compact model, `-0 -mc -oh -os -s -zp2 -zpw -we`). mTCP sources are mirrored in lowercase first (Linux is case sensitive, mTCP is not).
3. `src/make_image.sh` -> `build/dosllm.img` (FreeDOS boot floppy: installer files removed, boot files + NE2000.COM + DHCP.EXE + G.EXE; ~1 MB free).
3b. `src/build.sh` also builds **`XIAOMAO.EXE`** from `src/xiaomao/` (patched for v86, needs `nasm`; built twice to compute the resident size) and `make_image.sh` adds it with
   `FONT.DAT`, `IME.DAT`, `HIMEMX.EXE` from `third_party/xiaomao/` (taken unmodified from the owner's c_dos build). `FDCONFIG.SYS` loads HimemX (XMS), `FDAUTO.BAT` runs XIAOMAO first.
4. `src/build_assets.py` -> `dist/assets.b64.js` (`window.DOSLLM_ASSETS` = v86.wasm, BIOSes, floppy as **gzip+base64 text**) + `dist/libv86.js`.
   Reason: HF static Spaces answer binary files with a CORS-less redirect; text files are served directly.
5. `build_all.sh` copies `site/*` and `src/` into `dist/`. **`dist/` is exactly what gets uploaded to the Space** (plus `dev/`).

`src/g.cpp` outline: join argv into the question (printable ASCII; `"` and `\` removed) -> `netInit` -> `resolveHost(GRADIO_HOST)` ->
exchange 1: `POST /gradio_api/call/chat` -> read `"event_id":"..."` -> exchange 2: `GET /gradio_api/call/chat/<id>` read as a line stream
(`event: generating` prints a `.`; the `data:` line after `event: complete` is kept) -> decode the first JSON string after `[["`
(`\n` -> space, `\uXXXX` -> ASCII or `?`) -> word-wrap at 78 columns as `A> ...`. Exit codes: 1 usage, 2 stack, 3 DNS, 4 exchange 1, 5 no event_id, 6 exchange 2, 7 no answer.

`site/app.js`: unpack `DOSLLM_ASSETS` (`DecompressionStream`) -> start `V86` -> poll the DOS text screen until `TCP/IP is up` (printed by `FDAUTO.BAT`) -> enable buttons.
`ask()` types the three commands (25 ms per key), waits until the last screen line is the prompt `X:\>` again (deadline 190 s), then `finish()` compares screen vs bridge answer.
Buttons: main ask, 4 example chips, reboot (`vm.restart()`). `window.dosVm` exposes the v86 instance for tests.

## 4. Gradio API facts (verified on `tencent/Hy3`, Gradio 6.12, endpoint `/chat`)

* `GET /gradio_api/info` lists endpoints/parameters. `/chat` has 9 inputs: `message, system_prompt, history, think_level, temperature, max_tokens, top_p, preserved_thinking, functions_json_str`.
* **All 9 parameters are required on the raw `/call` API.** With only `message` the stream answers `event: error / data: null` (the official `gradio_client` fills the defaults).
  G.EXE sends `[question, "Answer in plain ASCII English in at most 40 words.", [], "no_think", null, 0, 0, null, ""]`.
* Two steps: `POST /gradio_api/call/chat {"data":[...]}` -> `{"event_id":"..."}`, then `GET /gradio_api/call/chat/<event_id>` -> SSE: N x `event: generating`
  (cumulative `[[content, reasoning, tool_calls, history]]`), then `event: complete` with the same shape. A whole reply is ~3 KB, the complete frame ~0.5 KB, pure ASCII (JSON `\u` escapes).
* The system prompt keeps replies short (fits the 80x25 DOS screen) and ASCII (FreeDOS has no CJK font). `no_think` avoids long chain-of-thought latency (~2 s model time vs several seconds).
* The Space is only a front end: it needs its own secrets (`HY_API_KEY` etc.) to reach the upstream model. If the owner removes them or the Space sleeps, calls fail
  (symptom: `event: error`). Quick health check without DOS: `python3 -c "from gradio_client import Client; print(Client('tencent/Hy3').predict(message='hi', api_name='/chat'))"`.
* Any other Space works only if its `/chat` has the same parameters; otherwise adapt `body` in `src/g.cpp` using `/gradio_api/info`.

## 5. Hard-won facts (do not rediscover these)

* **NE2000 is NOT at 0x300.** SeaBIOS re-maps the PCI card: I/O **0xC000, IRQ 10**. Load with `NE2000 0x60 10 0xC000`
  (from `emu.v86.cpu.devices.pci.device_spaces`: BAR0 `c001`, dword 0x3c `10a`). Packet interrupt 0x60 must match `PACKETINT 0x60` in `MTCP.CFG`.
* DHCP.EXE rewrites `A:\MTCP.CFG` on the floppy (writable). `MTCPCFG=A:\MTCP.CFG` is set in `FDAUTO.BAT`.
* **v86's fake TCP never retransmits and ignores the guest's receive window.** It sends 1460-byte segments back to back, waiting only for the ACK of the previous one.
  If the guest has no room it drops the segment, never ACKs, and the connection hangs for good (visible in `dosVm.network_adapter.tcp_conn[...]`: `pending: true`, `sendBuf` > 0;
  the screen shows the `.` dots but no `A>` line). With a 2 KB receive buffer G.EXE hung about 1 time in 9 questions (once for 167 s until its own timeout);
  with `setRecvBuffer(8192)` (mTCP allows 512..16384) and 1 KB reads: 0 hangs in 50 consecutive questions (30 in the sibling project, 20 here).
  Any DOS program receiving more than a few hundred bytes must use a big buffer and drain it fast. `tests/stress.py` exists to catch this again.
* **Use Open Watcom 1.9, not 2.0 beta.** EXEs from 2.0 beta end with `End: heap is corrupted!` (mTCP's `_heapchk` returns 3). `-we` bites on struct padding (W880).
* mTCP needs `#include "global.cfg"` lowercase in `g.cfg` on Linux; `g.cfg` is mTCP's `SAMPLE.CFG` with `TCP_MAX_SOCKETS 2`. Rebuilt EXEs differ only by mTCP's embedded build date string.
* **Typing into DOS:** `emu.keyboard_send_text(text, 25)`; without the 25 ms delay FreeCOM drops characters. Give each command >= 250 ms before the next.
  The question is sanitised twice (page and G.EXE): printable ASCII only; `" \ % & | < > ^ !` removed because FreeCOM would interpret them. DOS command line limit is 127 characters.
* **Screen scaling / HiDPI (a real bug I shipped once).** In *graphics* mode v86 sets its canvas CSS size from `devicePixelRatio`. My first version scaled that canvas with a CSS
  `transform` computed from `canvas.width` (640), so on DPR 2 screens (retina laptops, most phones) the screen came out 2x too large inside `overflow:hidden`: only the top-left quarter was
  visible and long answers were cut off. At DPR 1 it showed fine but at 1.34x (uneven stroke widths). **Do not style v86's canvas.** `present()` in `app.js` hides it and draws its pixels
  onto our own `#view` canvas in *device* pixels (see section 10). In text mode (before XiaoMao loads) v86 draws an 80x25 DOM text layer, which `fitText()` scales with a CSS transform.
  `screen_set_scale()` and `use_graphical_text` produced a black/cropped screen: not used.
* G.EXE uses a different local TCP port per connection (`pseudoPort() + counter`). DNS answers rotate between several A records (normal). DOS clock is **UTC**.
* The page reads results from the DOS screen: lines `A> ...` plus 3-space-indented continuation lines. If you change G.EXE's output format, change `dosAnswer()` too.

## 6. Testing (Playwright)

`pip install playwright && playwright install chromium`; Chromium flags `--no-sandbox --disable-gpu --disable-dev-shm-usage`.
* `python3 -m http.server 8805 --directory dist` then
  `python3 tests/ask.py http://127.0.0.1:8805/ "question"` (also works on the live URL): waits for DOS networking, clicks the button, prints the log, the DOS-vs-LLM check, and saves a screenshot.
* `python3 tests/stress.py 20 http://127.0.0.1:8805/`: asks N times and, if one question has not finished after 45 s, prints the DOS screen and v86's TCP connection state.
* `python3 tests/display.py <url> <prefix>`: presenter geometry + screenshots at DPR 1/2/3 (look at the PNGs).
* `python3 tests/multi.py <url> [questions...]`: a list of Chinese/English/edge-case questions on one page; `python3 tests/ime_typing.py <url>`: typing 你好 in DOS with the Zhuyin IME.
* Results when this was written (local, English only, before the Chinese work): main button, example chip, reboot button all fine; **20/20 questions answered, every one `DOS = LLM`**; `17 times 23` -> DOS printed `391`.
* Use `kill <pid>`; `pkill -f <name>` kills your own shell.

## 7. Deploy rules (the owner was angry about breakage before)

1. **Only write to `HelloSun/dosllm`** (and never to `dosmqtt`, `c_dos`, `flare-gamet2` or any other Space from this project).
2. **Never delete remote files.** Upload with `HfApi.create_commit` + `CommitOperationAdd` only; compare `list_repo_files` before/after and report `deleted: []`.
3. Account name is `HelloSun` (case matters: `hellosun/...` gave 403 "no rights to create a space"). Auth: read the token from `/root/.cache/huggingface/token`.
   Env `HF_TOKEN` is invalid. A token was once pasted into chat; it is exposed and lacks create rights: advise revoking it. **Never write a token into any file.**
   The `hf` CLI is broken in this sandbox (typer/click mismatch): use the Python API.
4. HF needs ~1 min to publish after upload; verify with `curl` for a new symbol, then run `tests/ask.py` on the live URL.

```python
from huggingface_hub import HfApi, CommitOperationAdd
from pathlib import Path
api = HfApi(token=Path("/root/.cache/huggingface/token").read_text().strip()); repo = "HelloSun/dosllm"
before = set(api.list_repo_files(repo, repo_type="space"))
ops = [CommitOperationAdd(path_in_repo=str(p.relative_to(root)), path_or_fileobj=str(p)) for p in files]
api.create_commit(repo_id=repo, repo_type="space", operations=ops, commit_message="...")
assert not (before - set(api.list_repo_files(repo, repo_type="space")))
```
Space file set: `index.html app.js bridge.js style.css assets.b64.js libv86.js README.md LICENSE LICENSE-GPL-3.txt NOTICE.md`, `src/` (sources and scripts), `dev/` (`dosllm-dev.zip`, `HANDOFF.md`).

## 8. Licensing

G.EXE links mTCP (GPL-3.0-or-later), so our DOS sources, scripts and web code are GPL-3.0-or-later and the mTCP source archive is shipped in `third_party/`.
XiaoMao Chinese is by **HelloSun** — `Copyright (C) 2026 HelloSun`, v1.0, first version 2026-10-25 — **GPL-3.0-only plus a GPLv3 §7(b) additional term: the author attribution must be preserved** in all copies and modified versions,
including the notice XIAOMAO.EXE prints when it loads and the footer of a page that hosts it (`third_party/xiaomao/LICENSE-xiaomao`; `TYPE AUTHOR.TXT` in DOS). **Never remove or alter it.** Our copy is a *modified* version
(marked in the header of `src/xiaomao/xiaomao.c`; keep that mark). It is a separate program on the floppy ( our patched copy of its source is in `src/xiaomao/`, upstream package in `third_party/xiaomao/`); `FONT.DAT` derives from GNU Unifont
(GPL-2.0-or-later with the font embedding exception), `IME.DAT` from pypinyin/jieba/OpenCC data (MIT/MIT/Apache-2.0), `HIMEMX.EXE` is FDOS HimemX (GPL / Artistic).
FreeDOS (GPL-2.0), v86 (BSD-2-Clause), SeaBIOS (LGPL-3.0) and the Crynwr packet driver (source must stay available: Crynwr 11.x sources are at
https://archive.org/details/PKTD11A_ZIP) are listed in `site/NOTICE.md`. Open Watcom is only a build tool and is not redistributed.
The Gradio Space (`tencent/Hy3`) is a third-party service with its own terms.

## 9. Known limits / ideas

* Reply is delivered when the stream ends (DOS shows dots while it streams); one question per run, no conversation history (G.EXE sends `history=[]`).
  Multi-turn would need G.EXE to keep and resend `history` (the stream's last element), within DOS memory limits.
* Big5 only (see section 10); replies capped by the system prompt at ~40 words / ~80 Chinese characters; `G.EXE` truncates a stream data line at 2000 bytes and an answer at ~1200 characters.
* Only HTTP on port 80 and only `*.hf.space/gradio_api/` are bridged. No TLS in DOS, no auth/token support by design.
* Needs WebAssembly, `DecompressionStream` and internet (DoH + Space). No offline mode.
* Ideas: show the streamed text live (bridge could forward SSE chunks as they arrive), conversation history, a private-Space mode with a user-supplied token kept only in memory,
  sharing the WISP framing code with `dosmqtt`'s `bridge.js`.