|
Download dev/HANDOFF.md from HelloSun/dosllm: direct link, hf CLI and curl.
- Browser
- Download file 15.6 kB
-
https://huggingface.co/spaces/HelloSun/dosllm/resolve/main/dev/HANDOFF.md
- Command line
-
hf download hf://spaces/HelloSun/dosllm/dev/HANDOFF.md
-
curl -L -o HANDOFF.md https://huggingface.co/spaces/HelloSun/dosllm/resolve/main/dev/HANDOFF.md
15.6 kB
| # HANDOFF — DOS LLM (中文 / English) | |
| Read this first. Everything below was checked against the real running page, not assumed. | |
| Project owner language: Traditional Chinese. Replies: short, factual, no emojis, say limitations honestly. | |
| ## 1. What it is (and what it is not) | |
| * Space: `HelloSun/dosllm` (static, public). Live: https://hellosun-dosllm.static.hf.space/ | |
| * One **FreeDOS 1.4** machine runs in the page on the **v86** x86 emulator. Inside it: Crynwr **NE2000 packet driver** -> mTCP **DHCP.EXE** | |
| (192.168.86.100) -> **G.EXE**, a real DOS program (Open Watcom + mTCP) that does DNS, TCP and **HTTP** itself. | |
| * Press the page's button -> the page types `SET GRADIO_HOST=...`, `CLS`, `G <question>` into the DOS prompt -> G.EXE asks a public Gradio Space | |
| and prints `A> <answer>` on the DOS screen. The page then compares the DOS screen text with the answer it captured from Gradio (`DOS = LLM`). | |
| * **Honest limits.** DOS has no TLS and browsers cannot open raw TCP, and a static Space has no server. So `site/bridge.js` relays the guest's | |
| plain HTTP (port 80) as HTTPS `fetch()`. The DOS side is genuine TCP/HTTP; the last hop to the Space is HTTPS done by the browser. | |
| The page (not DOS) decides *when* to run G.EXE, by typing into the DOS keyboard. The "LLM" is whatever Gradio Space is configured (`tencent/Hy3`). | |
| * **Chinese (中文) works too.** The floppy also loads **小貓中文 (XiaoMao Chinese)**, a DOS TSR from the owner's `HelloSun/c_dos` Space: Big5 display + Zhuyin input. | |
| Ask in Chinese (the page, or `Ctrl+Space` + Zhuyin typed directly in DOS) and the answer is printed in Chinese on the DOS screen (section 10). | |
| * js-dos 6.22 was tried first in the sibling Space: it has no TCP/IP at all, which is why v86 is used. | |
| ## 2. Data path | |
| ``` | |
| G.EXE (mTCP TCP/IP stack, DOS) -- plain HTTP/1.0 on port 80 --> | |
| packet driver NE2000.COM (INT 0x60, I/O 0xC000, IRQ 10) | |
| -> v86 emulated NE2000 (PCI) DHCP + DNS answered by v86 itself (DNS = DoH to cloudflare-dns.com) | |
| -> v86 "wisp" network adapter (terminates guest TCP) | |
| -> WebSocket('wisp://dosllm-bridge.invalid/?m=dos') <- monkeypatched in bridge.js (Proxy on window.WebSocket) | |
| -> FakeWispServer (bridge.js): per TCP stream collects ONE HTTP request, then | |
| fetch('https://<Host header>/gradio_api/...') (Spaces answer CORS by echoing the caller's Origin, also http://127.0.0.1 and localhost) | |
| -> writes 'HTTP/1.0 <status>' + Content-Length + body back, then a WISP CLOSE (guest sees FIN after the data) | |
| ``` | |
| * WISP frames: 1 CONNECT, 2 DATA, 3 CONTINUE (credit; we hand out 128 on stream 0 and per DATA), 4 CLOSE. The wisp adapter reports the destination as an **IP**, so | |
| the bridge takes the real host from the HTTP `Host:` header (DOS resolved the real name through DoH). | |
| * **Chinese transcoding in the bridge** (`site/big5.js`): DOS shows/handles **Big5**, Gradio speaks UTF-8. A POST body that contains bytes >= 0x80 is decoded with | |
| `TextDecoder('big5')` before the HTTPS request; JSON/SSE replies (`\uXXXX` escapes) are rewritten to Big5 bytes before DOS sees them (characters Big5 lacks -> `?`). | |
| * **Allow-list in `relay()`**: host must match `*.hf.space`, path must start with `/gradio_api/`, GET/POST only, request body <= 8 KB, no credentials added, 150 s timeout. | |
| Other ports are refused. It is not an open proxy; keep it that way. | |
| ## 3. Layout and how to build | |
| See `README.md` for the tree. `tools/build_all.sh` (a few seconds once deps are cached): | |
| 1. `tools/fetch_deps.sh`: v86 0.5.472 (npm), SeaBIOS/VGABIOS, FreeDOS 1.4 floppy, Crynwr NE2000.COM, mTCP 2026-10-10 (bin + src), Open Watcom **1.9** into `deps/`, | |
| each checked against a pinned SHA-256 (change the hashes on purpose when upgrading). `DEPS=<dir>` reuses a cache. | |
| 2. `src/build.sh` -> `build/G.EXE` (16-bit DOS compact model, `-0 -mc -oh -os -s -zp2 -zpw -we`). mTCP sources are mirrored in lowercase first (Linux is case sensitive, mTCP is not). | |
| 3. `src/make_image.sh` -> `build/dosllm.img` (FreeDOS boot floppy: installer files removed, boot files + NE2000.COM + DHCP.EXE + G.EXE; ~1 MB free). | |
| 3b. `src/build.sh` also builds **`XIAOMAO.EXE`** from `src/xiaomao/` (patched for v86, needs `nasm`; built twice to compute the resident size) and `make_image.sh` adds it with | |
| `FONT.DAT`, `IME.DAT`, `HIMEMX.EXE` from `third_party/xiaomao/` (taken unmodified from the owner's c_dos build). `FDCONFIG.SYS` loads HimemX (XMS), `FDAUTO.BAT` runs XIAOMAO first. | |
| 4. `src/build_assets.py` -> `dist/assets.b64.js` (`window.DOSLLM_ASSETS` = v86.wasm, BIOSes, floppy as **gzip+base64 text**) + `dist/libv86.js`. | |
| Reason: HF static Spaces answer binary files with a CORS-less redirect; text files are served directly. | |
| 5. `build_all.sh` copies `site/*` and `src/` into `dist/`. **`dist/` is exactly what gets uploaded to the Space** (plus `dev/`). | |
| `src/g.cpp` outline: join argv into the question (printable ASCII; `"` and `\` removed) -> `netInit` -> `resolveHost(GRADIO_HOST)` -> | |
| exchange 1: `POST /gradio_api/call/chat` -> read `"event_id":"..."` -> exchange 2: `GET /gradio_api/call/chat/<id>` read as a line stream | |
| (`event: generating` prints a `.`; the `data:` line after `event: complete` is kept) -> decode the first JSON string after `[["` | |
| (`\n` -> space, `\uXXXX` -> ASCII or `?`) -> word-wrap at 78 columns as `A> ...`. Exit codes: 1 usage, 2 stack, 3 DNS, 4 exchange 1, 5 no event_id, 6 exchange 2, 7 no answer. | |
| `site/app.js`: unpack `DOSLLM_ASSETS` (`DecompressionStream`) -> start `V86` -> poll the DOS text screen until `TCP/IP is up` (printed by `FDAUTO.BAT`) -> enable buttons. | |
| `ask()` types the three commands (25 ms per key), waits until the last screen line is the prompt `X:\>` again (deadline 190 s), then `finish()` compares screen vs bridge answer. | |
| Buttons: main ask, 4 example chips, reboot (`vm.restart()`). `window.dosVm` exposes the v86 instance for tests. | |
| ## 4. Gradio API facts (verified on `tencent/Hy3`, Gradio 6.12, endpoint `/chat`) | |
| * `GET /gradio_api/info` lists endpoints/parameters. `/chat` has 9 inputs: `message, system_prompt, history, think_level, temperature, max_tokens, top_p, preserved_thinking, functions_json_str`. | |
| * **All 9 parameters are required on the raw `/call` API.** With only `message` the stream answers `event: error / data: null` (the official `gradio_client` fills the defaults). | |
| G.EXE sends `[question, "Answer in plain ASCII English in at most 40 words.", [], "no_think", null, 0, 0, null, ""]`. | |
| * Two steps: `POST /gradio_api/call/chat {"data":[...]}` -> `{"event_id":"..."}`, then `GET /gradio_api/call/chat/<event_id>` -> SSE: N x `event: generating` | |
| (cumulative `[[content, reasoning, tool_calls, history]]`), then `event: complete` with the same shape. A whole reply is ~3 KB, the complete frame ~0.5 KB, pure ASCII (JSON `\u` escapes). | |
| * The system prompt keeps replies short (fits the 80x25 DOS screen) and ASCII (FreeDOS has no CJK font). `no_think` avoids long chain-of-thought latency (~2 s model time vs several seconds). | |
| * The Space is only a front end: it needs its own secrets (`HY_API_KEY` etc.) to reach the upstream model. If the owner removes them or the Space sleeps, calls fail | |
| (symptom: `event: error`). Quick health check without DOS: `python3 -c "from gradio_client import Client; print(Client('tencent/Hy3').predict(message='hi', api_name='/chat'))"`. | |
| * Any other Space works only if its `/chat` has the same parameters; otherwise adapt `body` in `src/g.cpp` using `/gradio_api/info`. | |
| ## 5. Hard-won facts (do not rediscover these) | |
| * **NE2000 is NOT at 0x300.** SeaBIOS re-maps the PCI card: I/O **0xC000, IRQ 10**. Load with `NE2000 0x60 10 0xC000` | |
| (from `emu.v86.cpu.devices.pci.device_spaces`: BAR0 `c001`, dword 0x3c `10a`). Packet interrupt 0x60 must match `PACKETINT 0x60` in `MTCP.CFG`. | |
| * DHCP.EXE rewrites `A:\MTCP.CFG` on the floppy (writable). `MTCPCFG=A:\MTCP.CFG` is set in `FDAUTO.BAT`. | |
| * **v86's fake TCP never retransmits and ignores the guest's receive window.** It sends 1460-byte segments back to back, waiting only for the ACK of the previous one. | |
| If the guest has no room it drops the segment, never ACKs, and the connection hangs for good (visible in `dosVm.network_adapter.tcp_conn[...]`: `pending: true`, `sendBuf` > 0; | |
| the screen shows the `.` dots but no `A>` line). With a 2 KB receive buffer G.EXE hung about 1 time in 9 questions (once for 167 s until its own timeout); | |
| with `setRecvBuffer(8192)` (mTCP allows 512..16384) and 1 KB reads: 0 hangs in 50 consecutive questions (30 in the sibling project, 20 here). | |
| Any DOS program receiving more than a few hundred bytes must use a big buffer and drain it fast. `tests/stress.py` exists to catch this again. | |
| * **Use Open Watcom 1.9, not 2.0 beta.** EXEs from 2.0 beta end with `End: heap is corrupted!` (mTCP's `_heapchk` returns 3). `-we` bites on struct padding (W880). | |
| * mTCP needs `#include "global.cfg"` lowercase in `g.cfg` on Linux; `g.cfg` is mTCP's `SAMPLE.CFG` with `TCP_MAX_SOCKETS 2`. Rebuilt EXEs differ only by mTCP's embedded build date string. | |
| * **Typing into DOS:** `emu.keyboard_send_text(text, 25)`; without the 25 ms delay FreeCOM drops characters. Give each command >= 250 ms before the next. | |
| The question is sanitised twice (page and G.EXE): printable ASCII only; `" \ % & | < > ^ !` removed because FreeCOM would interpret them. DOS command line limit is 127 characters. | |
| * **Screen scaling / HiDPI (a real bug I shipped once).** In *graphics* mode v86 sets its canvas CSS size from `devicePixelRatio`. My first version scaled that canvas with a CSS | |
| `transform` computed from `canvas.width` (640), so on DPR 2 screens (retina laptops, most phones) the screen came out 2x too large inside `overflow:hidden`: only the top-left quarter was | |
| visible and long answers were cut off. At DPR 1 it showed fine but at 1.34x (uneven stroke widths). **Do not style v86's canvas.** `present()` in `app.js` hides it and draws its pixels | |
| onto our own `#view` canvas in *device* pixels (see section 10). In text mode (before XiaoMao loads) v86 draws an 80x25 DOM text layer, which `fitText()` scales with a CSS transform. | |
| `screen_set_scale()` and `use_graphical_text` produced a black/cropped screen: not used. | |
| * G.EXE uses a different local TCP port per connection (`pseudoPort() + counter`). DNS answers rotate between several A records (normal). DOS clock is **UTC**. | |
| * The page reads results from the DOS screen: lines `A> ...` plus 3-space-indented continuation lines. If you change G.EXE's output format, change `dosAnswer()` too. | |
| ## 6. Testing (Playwright) | |
| `pip install playwright && playwright install chromium`; Chromium flags `--no-sandbox --disable-gpu --disable-dev-shm-usage`. | |
| * `python3 -m http.server 8805 --directory dist` then | |
| `python3 tests/ask.py http://127.0.0.1:8805/ "question"` (also works on the live URL): waits for DOS networking, clicks the button, prints the log, the DOS-vs-LLM check, and saves a screenshot. | |
| * `python3 tests/stress.py 20 http://127.0.0.1:8805/`: asks N times and, if one question has not finished after 45 s, prints the DOS screen and v86's TCP connection state. | |
| * `python3 tests/display.py <url> <prefix>`: presenter geometry + screenshots at DPR 1/2/3 (look at the PNGs). | |
| * `python3 tests/multi.py <url> [questions...]`: a list of Chinese/English/edge-case questions on one page; `python3 tests/ime_typing.py <url>`: typing 你好 in DOS with the Zhuyin IME. | |
| * Results when this was written (local, English only, before the Chinese work): main button, example chip, reboot button all fine; **20/20 questions answered, every one `DOS = LLM`**; `17 times 23` -> DOS printed `391`. | |
| * Use `kill <pid>`; `pkill -f <name>` kills your own shell. | |
| ## 7. Deploy rules (the owner was angry about breakage before) | |
| 1. **Only write to `HelloSun/dosllm`** (and never to `dosmqtt`, `c_dos`, `flare-gamet2` or any other Space from this project). | |
| 2. **Never delete remote files.** Upload with `HfApi.create_commit` + `CommitOperationAdd` only; compare `list_repo_files` before/after and report `deleted: []`. | |
| 3. Account name is `HelloSun` (case matters: `hellosun/...` gave 403 "no rights to create a space"). Auth: read the token from `/root/.cache/huggingface/token`. | |
| Env `HF_TOKEN` is invalid. A token was once pasted into chat; it is exposed and lacks create rights: advise revoking it. **Never write a token into any file.** | |
| The `hf` CLI is broken in this sandbox (typer/click mismatch): use the Python API. | |
| 4. HF needs ~1 min to publish after upload; verify with `curl` for a new symbol, then run `tests/ask.py` on the live URL. | |
| ```python | |
| from huggingface_hub import HfApi, CommitOperationAdd | |
| from pathlib import Path | |
| api = HfApi(token=Path("/root/.cache/huggingface/token").read_text().strip()); repo = "HelloSun/dosllm" | |
| before = set(api.list_repo_files(repo, repo_type="space")) | |
| ops = [CommitOperationAdd(path_in_repo=str(p.relative_to(root)), path_or_fileobj=str(p)) for p in files] | |
| api.create_commit(repo_id=repo, repo_type="space", operations=ops, commit_message="...") | |
| assert not (before - set(api.list_repo_files(repo, repo_type="space"))) | |
| ``` | |
| Space file set: `index.html app.js bridge.js style.css assets.b64.js libv86.js README.md LICENSE LICENSE-GPL-3.txt NOTICE.md`, `src/` (sources and scripts), `dev/` (`dosllm-dev.zip`, `HANDOFF.md`). | |
| ## 8. Licensing | |
| G.EXE links mTCP (GPL-3.0-or-later), so our DOS sources, scripts and web code are GPL-3.0-or-later and the mTCP source archive is shipped in `third_party/`. | |
| XiaoMao Chinese is by **HelloSun** — `Copyright (C) 2026 HelloSun`, v1.0, first version 2026-10-25 — **GPL-3.0-only plus a GPLv3 §7(b) additional term: the author attribution must be preserved** in all copies and modified versions, | |
| including the notice XIAOMAO.EXE prints when it loads and the footer of a page that hosts it (`third_party/xiaomao/LICENSE-xiaomao`; `TYPE AUTHOR.TXT` in DOS). **Never remove or alter it.** Our copy is a *modified* version | |
| (marked in the header of `src/xiaomao/xiaomao.c`; keep that mark). It is a separate program on the floppy ( our patched copy of its source is in `src/xiaomao/`, upstream package in `third_party/xiaomao/`); `FONT.DAT` derives from GNU Unifont | |
| (GPL-2.0-or-later with the font embedding exception), `IME.DAT` from pypinyin/jieba/OpenCC data (MIT/MIT/Apache-2.0), `HIMEMX.EXE` is FDOS HimemX (GPL / Artistic). | |
| FreeDOS (GPL-2.0), v86 (BSD-2-Clause), SeaBIOS (LGPL-3.0) and the Crynwr packet driver (source must stay available: Crynwr 11.x sources are at | |
| https://archive.org/details/PKTD11A_ZIP) are listed in `site/NOTICE.md`. Open Watcom is only a build tool and is not redistributed. | |
| The Gradio Space (`tencent/Hy3`) is a third-party service with its own terms. | |
| ## 9. Known limits / ideas | |
| * Reply is delivered when the stream ends (DOS shows dots while it streams); one question per run, no conversation history (G.EXE sends `history=[]`). | |
| Multi-turn would need G.EXE to keep and resend `history` (the stream's last element), within DOS memory limits. | |
| * Big5 only (see section 10); replies capped by the system prompt at ~40 words / ~80 Chinese characters; `G.EXE` truncates a stream data line at 2000 bytes and an answer at ~1200 characters. | |
| * Only HTTP on port 80 and only `*.hf.space/gradio_api/` are bridged. No TLS in DOS, no auth/token support by design. | |
| * Needs WebAssembly, `DecompressionStream` and internet (DoH + Space). No offline mode. | |
| * Ideas: show the streamed text live (bridge could forward SSE chunks as they arrive), conversation history, a private-Space mode with a user-supplied token kept only in memory, | |
| sharing the WISP framing code with `dosmqtt`'s `bridge.js`. | |