| # HANDOFF — DOS MQTT LAB |
|
|
| Read this first. Everything below was checked against the real running page, not assumed. |
| Project owner language: Traditional Chinese. Replies: short, factual, no emojis, say limitations honestly. |
|
|
| ## 1. What it is (and what it is not) |
|
|
| * Space: `HelloSun/dosmqtt` (static, public). Live: https://hellosun-dosmqtt.static.hf.space/ |
| * Two pages: `index.html` (MQTT, two machines) and `gradio.html` (DOS asks a Gradio Space a question, one machine, section 9). |
| * Two **FreeDOS 1.4** machines ("A" listener, "B" sender) run side by side in the page on the **v86** x86 emulator. |
| * Inside each DOS: Crynwr **NE2000 packet driver** -> mTCP **DHCP.EXE** (gets 192.168.86.100) -> |
| **A.EXE / B.EXE**, real DOS programs (Open Watcom + mTCP library) doing DNS, TCP and **MQTT 3.1.1** (QoS 0) themselves. |
| * `A.EXE`: connect, subscribe, print `[hh:mm:ss] A rx: <msg>`, 1 s later print `[hh:mm:ss] A echo: <msg>`. |
| * `B.EXE`: run once: connect, publish `Hello from DOS B at hh:mm:ss`, disconnect, exit. The page types `B` every 5 s |
| (DOS has no scheduler). The page types `A` once. |
| * **Honest limits**: only the last hop is not TCP: the browser cannot open raw TCP, and a static Space has no server, |
| so `site/bridge.js` relays the guest's TCP streams on **port 1883** to a public broker's **MQTT-over-WSS** endpoint. |
| MQTT is identical on both transports, so the bytes DOS writes are exactly what the broker receives (verified with an |
| independent raw-TCP subscriber, see section 5). "Background" A is a second VM because DOS is single-tasking. |
| js-dos 6.22 (an earlier version of this Space) has no TCP/IP at all, which is why v86 is used now. |
|
|
| ## 2. Data path |
|
|
| ``` |
| A.EXE/B.EXE (mTCP TCP/IP stack, DOS) |
| -> packet driver NE2000.COM (INT 0x60, I/O 0xC000, IRQ 10) |
| -> v86 emulated NE2000 (PCI) DHCP + DNS answered by v86 itself |
| -> v86 "wisp" network adapter (terminates guest TCP) DNS = DoH to cloudflare-dns.com (needs internet) |
| -> WebSocket('wisp://mqttbridge.invalid/?m=A|B') <- monkeypatched in bridge.js (Proxy on window.WebSocket) |
| -> FakeWispServer (bridge.js): per TCP stream opens |
| new WebSocket('wss://<broker>', 'mqtt') (port must be 1883, else stream refused) |
| -> public broker |
| ``` |
| * WISP frames used: 1 CONNECT, 2 DATA, 3 CONTINUE (flow-control credit, we hand out 128 on stream 0 and per DATA), 4 CLOSE. |
| * Data the guest sends before the WSS is open is queued; broker->guest data is cut into 1 KB frames. |
| * Both VMs use the same IP; they are isolated (wisp backend isolates guests). |
|
|
| ## 3. Layout and how to build |
|
|
| See `README.md` for the tree. Build chain (`tools/build_all.sh`, a few seconds once deps are cached): |
|
|
| 1. `tools/fetch_deps.sh` downloads v86 0.5.472 (npm), SeaBIOS/VGABIOS, FreeDOS 1.4 floppy, Crynwr NE2000.COM, |
| mTCP 2026-10-10 (bin + src), Open Watcom **1.9** into `deps/`, and checks pinned SHA-256s (edit the hashes on purpose when upgrading). |
| 2. `src/build.sh` -> `build/A.EXE`, `build/B.EXE` (16-bit DOS compact model, `-0 -mc -oh -os -s -zp2 -zpw -we`). |
| mTCP sources are mirrored in lowercase first (Linux is case sensitive, mTCP is not). |
| 3. `src/make_image.sh` -> `build/dosmqtt.img` (FreeDOS boot floppy, strips installer files, adds boot files + NE2000.COM + DHCP.EXE + A/B.EXE; ~1 MB stays free). |
| 4. `src/build_assets.py` -> `dist/assets.b64.js` (v86.wasm, BIOSes, floppy as **gzip+base64 text**) and `dist/libv86.js`. |
| Reason: HF static Spaces answer binary files with a CORS-less redirect; text files are served directly. |
| 5. `build_all.sh` also copies `site/*` and `src/` into `dist/`. **`dist/` is exactly what gets uploaded to the Space.** |
|
|
| Rebuilt EXEs equal the deployed ones except 1 byte (mTCP embeds the build date `Oct 24 2026` vs `Oct 25 2026`). |
|
|
| `site/app.js` flow: unpack assets (`DecompressionStream`) -> start two `V86` instances -> poll the DOS text screens until |
| `TCP/IP is up` appears (printed by `FDAUTO.BAT`) -> enable the start button -> on start: type `SET MQTT_HOST/PORT/TOPIC` |
| in both machines, type `A` in machine A, wait for `subscribed to` on A's screen, then every 5 s type `B` in machine B |
| (skipped and logged if B's prompt `X:\>` is not the last screen line). Results are read **from the DOS screens** |
| (`emu.screen_adapter.get_text_screen()`), regex on `A rx:` / `A echo:` / `B sent:`; the log also shows bridge events. |
| A and B read host/port/topic from the DOS environment (`MQTT_HOST`, `MQTT_PORT` default 1883, `MQTT_TOPIC`) or from the command line `A <broker> <port> <topic>`. |
|
|
| ## 4. Hard-won facts (do not rediscover these) |
|
|
| * **NE2000 is NOT at 0x300.** SeaBIOS re-maps the PCI card: I/O **0xC000, IRQ 10**. Load with `NE2000 0x60 10 0xC000`. |
| (Read from `emu.v86.cpu.devices.pci.device_spaces`: BAR0 `c001`, dword 0x3c `10a`.) Packet interrupt 0x60 must match `PACKETINT 0x60` in `MTCP.CFG`. |
| * DHCP.EXE rewrites `A:\MTCP.CFG` on the floppy (writable). `MTCPCFG=A:\MTCP.CFG` is set in `FDAUTO.BAT`. |
| * **Use Open Watcom 1.9, not 2.0 beta.** EXEs from 2.0 beta end with `End: heap is corrupted!` (mTCP's `_heapchk`) even for an empty program |
| (`_heapchk` returned 3). With 1.9 it returns 0. `-we` (warnings are errors) bites on struct padding (W880): order fields so no padding is needed. |
| * mTCP needs `#include "global.cfg"` lowercase in `a.cfg/b.cfg` on Linux; the configs are copies of mTCP's `SAMPLE.CFG` with `TCP_MAX_SOCKETS 2`. |
| * **Typing into DOS:** `emu.keyboard_send_text(text, 25)` (25 ms per char). Without the delay FreeCOM drops characters |
| (`SET PATH=...` came out as `FRESET MTCPCFG=...`). Give each command >= 250 ms before the next. |
| * **Screen scaling:** in text mode v86 draws a DOM text layer (80x25, 14 px monospace), not the canvas. `screen_set_scale()` and |
| `use_graphical_text` turned the screen black/cropped. `fitScreen()` in `app.js` scales whichever layer is visible with a CSS `transform`. |
| * **v86's fake TCP never retransmits and ignores the guest's receive window.** It sends 1460-byte segments one after another |
| (waiting only for the ACK of the previous one). If the guest has no room for a segment it drops it, never ACKs, and the connection |
| hangs for good (seen in `emu.v86.network_adapter.tcp_conn[...]`: `pending: true`, `sendBuf` > 0). With a 2 KB receive buffer |
| G.EXE hung about 1 time in 9 answers (and once for 167 s); with `setRecvBuffer(8192)` and 1 KB reads: 30/30 answers, 0 hangs. |
| mTCP allows 512..16384 bytes. Any DOS program that receives more than a few hundred bytes must use a big buffer and drain it fast. |
| A.EXE/B.EXE still use 1024 (`RECV_BUFFER_SIZE` in `mqtt.h`): fine for their tiny MQTT packets, a risk if payloads ever grow. |
| * B uses a pseudo-random local TCP port per run; DNS answers rotate between several A records (that is normal). |
| * DOS clock is **UTC**; mTCP timer granularity is the 18.2 Hz BIOS tick (55 ms), so "1 s" echo reads 0.8-1.2 s from the host. |
| * **Broker quality differs a lot** (CONNECT -> CONNACK over WSS from headless Chromium in this sandbox; 12 attempts per broker, HiveMQ also 30): |
| `broker.emqx.io:8084` 12/12 fast (~560 ms); `test.mosquitto.org:8081` 12/12 fast (~390 ms); |
| `broker.hivemq.com:8884` only 7/12 (and 19/30 in another run) fast, the rest took ~10 s or never answered (an idle WS without CONNECT is closed at ~10 s); |
| `mqtt.eclipseprojects.io/mqtt` 0/12. Default is EMQX. B's 10 s CONNACK timeout then shows `MQTT: CONNECT refused or timed out`. |
| * A sends PINGREQ every 30 s (keepalive 60 s) and keeps <= 8 pending echoes. A has no reconnect: restart the VMs ("停止並重新開機"). |
| * The "stop" button can't send Ctrl-Break; it reboots both VMs (`vm.restart()`), then waits for `TCP/IP is up` again. |
|
|
| ## 5. Testing (Playwright + independent MQTT peer) |
|
|
| `pip install playwright && playwright install chromium`; Chromium flags `--no-sandbox --disable-gpu --disable-dev-shm-usage`. |
| * `python3 tests/ask.py <gradio.html url> "question" shot.png`: asks one question on the Gradio page and prints DOS/bridge/check results. |
| * `tests/serve_harness.sh`: bare v86 page on :8801 (verbose bridge logging) for manual DOS poking via `drive2.py` |
| (`session([(cmd, wait_s), ...])` prints the DOS screen + bridge log). |
| * `python3 -m http.server 8802 --directory dist`, then |
| `python3 tests/full.py http://127.0.0.1:8802/ <topic> <seconds> <shot.png>` runs the whole page (also works on the live URL). |
| * **Always cross-check with something that is not the page**: `python3 tests/sub.py <topic> <secs> broker.emqx.io` (raw TCP 1883 subscriber) |
| must print every `Hello from DOS B at ...`; `python3 tests/pub.py <topic> <delay> <n>` publishes to test A on its own. |
| Use the same topic in the page (`full.py` fills it) and the same broker host (default EMQX). |
| * Results when this was written: local 8/8 matched, 8 messages seen by the independent subscriber; **live URL** 9 B runs, 9 sent, A received 9 and echoed 8 |
| (the 9th echo was still within its 1 s delay when sampled); the independent subscriber saw all 9. |
| * Use `kill <pid>`; `pkill -f <name>` kills your own shell. |
|
|
| ## 6. Deploy rules (the owner was angry about breakage before) |
|
|
| 1. **Only write to `HelloSun/dosmqtt`.** Never touch `HelloSun/c_dos`, `flare-gamet2` or any other Space (reading `c_dos` for reference is fine). |
| 2. **Never delete remote files.** Upload with `HfApi.create_commit` + `CommitOperationAdd` only; compare `list_repo_files` before/after and report `deleted: []`. |
| 3. Account name is `HelloSun` (case matters: `hellosun/...` gave 403 "no rights to create a space"). Auth: read the token from |
| `/root/.cache/huggingface/token`. Env `HF_TOKEN` is invalid. A token was once pasted into chat; it is exposed and |
| lacks create rights: advise revoking it. **Never write a token into any file.** `hf` CLI is broken here (typer/click mismatch): use the Python API. |
| 4. After upload HF needs ~1 min to publish; verify with `curl` for a new symbol and then run `tests/full.py` on the live URL. |
|
|
| ```python |
| from huggingface_hub import HfApi, CommitOperationAdd |
| from pathlib import Path |
| api = HfApi(token=Path("/root/.cache/huggingface/token").read_text().strip()); repo = "HelloSun/dosmqtt" |
| before = set(api.list_repo_files(repo, repo_type="space")) |
| ops = [CommitOperationAdd(path_in_repo=str(p.relative_to(root)), path_or_fileobj=str(p)) for p in files] |
| api.create_commit(repo_id=repo, repo_type="space", operations=ops, commit_message="...") |
| assert not (before - set(api.list_repo_files(repo, repo_type="space"))) |
| ``` |
| Space file set: web files (both pages) + `assets.b64.js` + `gradio-img.b64.js` + `libv86.js` + `src/` + licenses, and `dev/` (this package, `dev/dosmqtt-dev.zip`, `dev/HANDOFF.md`). |
| Unused leftovers from the old js-dos version, **deliberately not deleted**: `js-dos.js`, `emu.b64.js`, `dos.b64.js`, `LICENSE-js-dos.txt`. |
|
|
| ## 9. DOS asks a Gradio Space (`gradio.html`) |
|
|
| Verified with `tencent/Hy3` (public, Gradio 6.12, endpoint `/chat`). Live page: https://hellosun-dosmqtt.static.hf.space/gradio.html |
|
|
| ``` |
| G.EXE (DOS, mTCP, plain HTTP/1.0 on port 80) |
| -> NE2000 -> v86 wisp adapter -> gradio-bridge.js (FakeWispServer) |
| -> fetch('https://<host>.hf.space/gradio_api/...') (CORS: Spaces echo the caller's Origin, incl. localhost) |
| ``` |
| * `G.EXE` (`src/g.cpp`): `G <question>`; host from env `GRADIO_HOST` (default `tencent-hy3.hf.space`). It does the documented Gradio calls itself: |
| `POST /gradio_api/call/chat` with `{"data":[question, system_prompt, [], "no_think", null, 0, 0, null, ""]}` -> `{"event_id":...}`, then |
| `GET /gradio_api/call/chat/<event_id>` (server-sent events: N x `generating`, then `complete`). It prints one `.` per `generating` frame, |
| decodes the first string of the `complete` frame (`[[content, reasoning, tool_calls, history]]`, `\uXXXX` -> ASCII, other chars -> `?`) and word-wraps it as `A> ...`. |
| * **All 9 parameters are required** on the raw `/call` API. With only `message` the stream answers `event: error / data: null` (the official `gradio_client` fills the defaults). |
| * The system prompt `Answer in plain ASCII English in at most 40 words.` keeps replies small (whole stream ~3 KB, complete frame ~0.5 KB) and ASCII, because FreeDOS has no CJK font. |
| * `site/gradio-bridge.js`: parses one HTTP request per TCP stream (port 80 only), relays it with `fetch` and answers `HTTP/1.0 <status>` with `Content-Length`, |
| then closes the stream. **Allow-list**: host must match `*.hf.space`, path must start with `/gradio_api/`, GET/POST only, body <= 8 KB, no credentials added, 150 s timeout. |
| The bridge also parses the SSE text to log `Gradio answer: ...` (captured for the self-check). |
| * `site/gradio.html` + `gradio.js`: types `SET GRADIO_HOST=...`, `CLS`, `G <question>` into DOS, waits for the prompt `A:\>`, then **compares the answer |
| printed on the DOS screen with the answer the bridge captured from Gradio** (tag `DOS = GRADIO`). `window.dosVm` exposes the v86 instance for tests. |
| * Assets: the page reuses `assets.b64.js` (wasm/BIOS) and adds `gradio-img.b64.js` (its own floppy, `build/gradio.img`, boot file `src/dos/FDAUTO_G.BAT`, prints `TCP/IP is up`). |
| `bridge.js`/`app.js`/`index.html` (MQTT page) were deliberately left untouched; `gradio-bridge.js` duplicates ~60 lines of WISP framing. A shared module would be a clean refactor. |
| * Limits: public Spaces only (no token is ever sent), non-ASCII becomes `?`, one reply is returned when the stream ends (DOS shows dots while it streams), |
| the question is sent to the chosen Space, so never type secrets. A Space whose `/chat` has other parameters needs `src/g.cpp` adapted (look at `/gradio_api/info`). |
| * Results: 30/30 consecutive questions answered with the 8 KB buffer, each `DOS = GRADIO`; `17 times 23` -> DOS printed `391`. |
|
|
| ## 7. Licensing |
|
|
| A.EXE/B.EXE link mTCP (GPL-3.0-or-later), so our DOS sources, scripts and web code are GPL-3.0-or-later and the mTCP source archive |
| is shipped in `third_party/`. FreeDOS (GPL-2.0), v86 (BSD-2-Clause), SeaBIOS (LGPL-3.0) and the Crynwr packet driver |
| (source must stay available: Crynwr 11.x sources are at https://archive.org/details/PKTD11A_ZIP) are listed in `site/NOTICE.md`. |
| Open Watcom is only a build tool and is not redistributed. |
| |
| ## 8. Known limits / ideas |
| |
| * QoS 0 only, no TLS and no auth in the DOS clients (TLS is done by the browser's WSS). Topic/payload are small (payload <= 95 chars in A). |
| * B is started by the page typing `B`; if the previous run is still going the tick is skipped (log shows it). A real in-DOS loop would need a `B.EXE -loop` mode. |
| * Only port 1883 is bridged. Other ports are refused on purpose. |
| * The page needs `DecompressionStream`, WebAssembly and internet (DoH + broker). No offline mode. |
| * Ideas: reconnect in A, in-DOS 5 s loop for B, MQTT QoS 1, mobile on-screen keyboard, a selectable second bridge (own WISP server) for brokers without WSS. |
| |
