dev/HANDOFF.md

HANDOFF — DOS MQTT LAB

Read this first. Everything below was checked against the real running page, not assumed. Project owner language: Traditional Chinese. Replies: short, factual, no emojis, say limitations honestly.

1. What it is (and what it is not)

• Space: HelloSun/dosmqtt (static, public). Live: https://hellosun-dosmqtt.static.hf.space/
• Two pages: index.html (MQTT, two machines) and gradio.html (DOS asks a Gradio Space a question, one machine, section 9).
• Two FreeDOS 1.4 machines ("A" listener, "B" sender) run side by side in the page on the v86 x86 emulator.
• Inside each DOS: Crynwr NE2000 packet driver -> mTCP DHCP.EXE (gets 192.168.86.100) -> A.EXE / B.EXE, real DOS programs (Open Watcom + mTCP library) doing DNS, TCP and MQTT 3.1.1 (QoS 0) themselves.
• A.EXE: connect, subscribe, print [hh:mm:ss] A rx: <msg>, 1 s later print [hh:mm:ss] A echo: <msg>.
• B.EXE: run once: connect, publish Hello from DOS B at hh:mm:ss, disconnect, exit. The page types B every 5 s (DOS has no scheduler). The page types A once.
• Honest limits: only the last hop is not TCP: the browser cannot open raw TCP, and a static Space has no server, so site/bridge.js relays the guest's TCP streams on port 1883 to a public broker's MQTT-over-WSS endpoint. MQTT is identical on both transports, so the bytes DOS writes are exactly what the broker receives (verified with an independent raw-TCP subscriber, see section 5). "Background" A is a second VM because DOS is single-tasking. js-dos 6.22 (an earlier version of this Space) has no TCP/IP at all, which is why v86 is used now.

2. Data path

A.EXE/B.EXE (mTCP TCP/IP stack, DOS)
   -> packet driver NE2000.COM (INT 0x60, I/O 0xC000, IRQ 10)
   -> v86 emulated NE2000 (PCI)                              DHCP + DNS answered by v86 itself
   -> v86 "wisp" network adapter (terminates guest TCP)      DNS = DoH to cloudflare-dns.com (needs internet)
   -> WebSocket('wisp://mqttbridge.invalid/?m=A|B')          <- monkeypatched in bridge.js (Proxy on window.WebSocket)
   -> FakeWispServer (bridge.js): per TCP stream opens
      new WebSocket('wss://<broker>', 'mqtt')  (port must be 1883, else stream refused)
   -> public broker

• WISP frames used: 1 CONNECT, 2 DATA, 3 CONTINUE (flow-control credit, we hand out 128 on stream 0 and per DATA), 4 CLOSE.
• Data the guest sends before the WSS is open is queued; broker->guest data is cut into 1 KB frames.
• Both VMs use the same IP; they are isolated (wisp backend isolates guests).

3. Layout and how to build

See README.md for the tree. Build chain (tools/build_all.sh, a few seconds once deps are cached):

1. tools/fetch_deps.sh downloads v86 0.5.472 (npm), SeaBIOS/VGABIOS, FreeDOS 1.4 floppy, Crynwr NE2000.COM, mTCP 2026-10-10 (bin + src), Open Watcom 1.9 into deps/, and checks pinned SHA-256s (edit the hashes on purpose when upgrading).
2. src/build.sh -> build/A.EXE, build/B.EXE (16-bit DOS compact model, -0 -mc -oh -os -s -zp2 -zpw -we). mTCP sources are mirrored in lowercase first (Linux is case sensitive, mTCP is not).
3. src/make_image.sh -> build/dosmqtt.img (FreeDOS boot floppy, strips installer files, adds boot files + NE2000.COM + DHCP.EXE + A/B.EXE; ~1 MB stays free).
4. src/build_assets.py -> dist/assets.b64.js (v86.wasm, BIOSes, floppy as gzip+base64 text) and dist/libv86.js. Reason: HF static Spaces answer binary files with a CORS-less redirect; text files are served directly.
5. build_all.sh also copies site/* and src/ into dist/. dist/ is exactly what gets uploaded to the Space.

Rebuilt EXEs equal the deployed ones except 1 byte (mTCP embeds the build date Oct 24 2026 vs Oct 25 2026).

site/app.js flow: unpack assets (DecompressionStream) -> start two V86 instances -> poll the DOS text screens until TCP/IP is up appears (printed by FDAUTO.BAT) -> enable the start button -> on start: type SET MQTT_HOST/PORT/TOPIC in both machines, type A in machine A, wait for subscribed to on A's screen, then every 5 s type B in machine B (skipped and logged if B's prompt X:\> is not the last screen line). Results are read from the DOS screens (emu.screen_adapter.get_text_screen()), regex on A rx: / A echo: / B sent:; the log also shows bridge events. A and B read host/port/topic from the DOS environment (MQTT_HOST, MQTT_PORT default 1883, MQTT_TOPIC) or from the command line A <broker> <port> <topic>.

4. Hard-won facts (do not rediscover these)

• NE2000 is NOT at 0x300. SeaBIOS re-maps the PCI card: I/O 0xC000, IRQ 10. Load with NE2000 0x60 10 0xC000. (Read from emu.v86.cpu.devices.pci.device_spaces: BAR0 c001, dword 0x3c 10a.) Packet interrupt 0x60 must match PACKETINT 0x60 in MTCP.CFG.
• DHCP.EXE rewrites A:\MTCP.CFG on the floppy (writable). MTCPCFG=A:\MTCP.CFG is set in FDAUTO.BAT.
• Use Open Watcom 1.9, not 2.0 beta. EXEs from 2.0 beta end with End: heap is corrupted! (mTCP's _heapchk) even for an empty program (_heapchk returned 3). With 1.9 it returns 0. -we (warnings are errors) bites on struct padding (W880): order fields so no padding is needed.
• mTCP needs #include "global.cfg" lowercase in a.cfg/b.cfg on Linux; the configs are copies of mTCP's SAMPLE.CFG with TCP_MAX_SOCKETS 2.
• Typing into DOS: emu.keyboard_send_text(text, 25) (25 ms per char). Without the delay FreeCOM drops characters (SET PATH=... came out as FRESET MTCPCFG=...). Give each command >= 250 ms before the next.
• Screen scaling: in text mode v86 draws a DOM text layer (80x25, 14 px monospace), not the canvas. screen_set_scale() and use_graphical_text turned the screen black/cropped. fitScreen() in app.js scales whichever layer is visible with a CSS transform.
• v86's fake TCP never retransmits and ignores the guest's receive window. It sends 1460-byte segments one after another (waiting only for the ACK of the previous one). If the guest has no room for a segment it drops it, never ACKs, and the connection hangs for good (seen in emu.v86.network_adapter.tcp_conn[...]: pending: true, sendBuf > 0). With a 2 KB receive buffer G.EXE hung about 1 time in 9 answers (and once for 167 s); with setRecvBuffer(8192) and 1 KB reads: 30/30 answers, 0 hangs. mTCP allows 512..16384 bytes. Any DOS program that receives more than a few hundred bytes must use a big buffer and drain it fast. A.EXE/B.EXE still use 1024 (RECV_BUFFER_SIZE in mqtt.h): fine for their tiny MQTT packets, a risk if payloads ever grow.
• B uses a pseudo-random local TCP port per run; DNS answers rotate between several A records (that is normal).
• DOS clock is UTC; mTCP timer granularity is the 18.2 Hz BIOS tick (55 ms), so "1 s" echo reads 0.8-1.2 s from the host.
• Broker quality differs a lot (CONNECT -> CONNACK over WSS from headless Chromium in this sandbox; 12 attempts per broker, HiveMQ also 30): broker.emqx.io:8084 12/12 fast (560 ms); test.mosquitto.org:8081 12/12 fast (390 ms); broker.hivemq.com:8884 only 7/12 (and 19/30 in another run) fast, the rest took ~10 s or never answered (an idle WS without CONNECT is closed at ~10 s); mqtt.eclipseprojects.io/mqtt 0/12. Default is EMQX. B's 10 s CONNACK timeout then shows MQTT: CONNECT refused or timed out.
• A sends PINGREQ every 30 s (keepalive 60 s) and keeps <= 8 pending echoes. A has no reconnect: restart the VMs ("停止並重新開機").
• The "stop" button can't send Ctrl-Break; it reboots both VMs (vm.restart()), then waits for TCP/IP is up again.

5. Testing (Playwright + independent MQTT peer)

pip install playwright && playwright install chromium; Chromium flags --no-sandbox --disable-gpu --disable-dev-shm-usage.

• python3 tests/ask.py <gradio.html url> "question" shot.png: asks one question on the Gradio page and prints DOS/bridge/check results.
• tests/serve_harness.sh: bare v86 page on :8801 (verbose bridge logging) for manual DOS poking via drive2.py (session([(cmd, wait_s), ...]) prints the DOS screen + bridge log).
• python3 -m http.server 8802 --directory dist, then python3 tests/full.py http://127.0.0.1:8802/ <topic> <seconds> <shot.png> runs the whole page (also works on the live URL).
• Always cross-check with something that is not the page: python3 tests/sub.py <topic> <secs> broker.emqx.io (raw TCP 1883 subscriber) must print every Hello from DOS B at ...; python3 tests/pub.py <topic> <delay> <n> publishes to test A on its own. Use the same topic in the page (full.py fills it) and the same broker host (default EMQX).
• Results when this was written: local 8/8 matched, 8 messages seen by the independent subscriber; live URL 9 B runs, 9 sent, A received 9 and echoed 8 (the 9th echo was still within its 1 s delay when sampled); the independent subscriber saw all 9.
• Use kill <pid>; pkill -f <name> kills your own shell.

6. Deploy rules (the owner was angry about breakage before)

1. Only write to HelloSun/dosmqtt. Never touch HelloSun/c_dos, flare-gamet2 or any other Space (reading c_dos for reference is fine).
2. Never delete remote files. Upload with HfApi.create_commit + CommitOperationAdd only; compare list_repo_files before/after and report deleted: [].
3. Account name is HelloSun (case matters: hellosun/... gave 403 "no rights to create a space"). Auth: read the token from /root/.cache/huggingface/token. Env HF_TOKEN is invalid. A token was once pasted into chat; it is exposed and lacks create rights: advise revoking it. Never write a token into any file. hf CLI is broken here (typer/click mismatch): use the Python API.
4. After upload HF needs ~1 min to publish; verify with curl for a new symbol and then run tests/full.py on the live URL.

from huggingface_hub import HfApi, CommitOperationAdd
from pathlib import Path
api = HfApi(token=Path("/root/.cache/huggingface/token").read_text().strip()); repo = "HelloSun/dosmqtt"
before = set(api.list_repo_files(repo, repo_type="space"))
ops = [CommitOperationAdd(path_in_repo=str(p.relative_to(root)), path_or_fileobj=str(p)) for p in files]
api.create_commit(repo_id=repo, repo_type="space", operations=ops, commit_message="...")
assert not (before - set(api.list_repo_files(repo, repo_type="space")))

Space file set: web files (both pages) + assets.b64.js + gradio-img.b64.js + libv86.js + src/ + licenses, and dev/ (this package, dev/dosmqtt-dev.zip, dev/HANDOFF.md). Unused leftovers from the old js-dos version, deliberately not deleted: js-dos.js, emu.b64.js, dos.b64.js, LICENSE-js-dos.txt.

9. DOS asks a Gradio Space (gradio.html)

Verified with tencent/Hy3 (public, Gradio 6.12, endpoint /chat). Live page: https://hellosun-dosmqtt.static.hf.space/gradio.html

G.EXE (DOS, mTCP, plain HTTP/1.0 on port 80)
   -> NE2000 -> v86 wisp adapter -> gradio-bridge.js (FakeWispServer)
   -> fetch('https://<host>.hf.space/gradio_api/...')      (CORS: Spaces echo the caller's Origin, incl. localhost)

• G.EXE (src/g.cpp): G <question>; host from env GRADIO_HOST (default tencent-hy3.hf.space). It does the documented Gradio calls itself: POST /gradio_api/call/chat with {"data":[question, system_prompt, [], "no_think", null, 0, 0, null, ""]} -> {"event_id":...}, then GET /gradio_api/call/chat/<event_id> (server-sent events: N x generating, then complete). It prints one . per generating frame, decodes the first string of the complete frame ([[content, reasoning, tool_calls, history]], \uXXXX -> ASCII, other chars -> ?) and word-wraps it as A> ....
• All 9 parameters are required on the raw /call API. With only message the stream answers event: error / data: null (the official gradio_client fills the defaults).
• The system prompt Answer in plain ASCII English in at most 40 words. keeps replies small (whole stream ~3 KB, complete frame ~0.5 KB) and ASCII, because FreeDOS has no CJK font.
• site/gradio-bridge.js: parses one HTTP request per TCP stream (port 80 only), relays it with fetch and answers HTTP/1.0 <status> with Content-Length, then closes the stream. Allow-list: host must match *.hf.space, path must start with /gradio_api/, GET/POST only, body <= 8 KB, no credentials added, 150 s timeout. The bridge also parses the SSE text to log Gradio answer: ... (captured for the self-check).
• site/gradio.html + gradio.js: types SET GRADIO_HOST=..., CLS, G <question> into DOS, waits for the prompt A:\>, then compares the answer printed on the DOS screen with the answer the bridge captured from Gradio (tag DOS = GRADIO). window.dosVm exposes the v86 instance for tests.
• Assets: the page reuses assets.b64.js (wasm/BIOS) and adds gradio-img.b64.js (its own floppy, build/gradio.img, boot file src/dos/FDAUTO_G.BAT, prints TCP/IP is up). bridge.js/app.js/index.html (MQTT page) were deliberately left untouched; gradio-bridge.js duplicates ~60 lines of WISP framing. A shared module would be a clean refactor.
• Limits: public Spaces only (no token is ever sent), non-ASCII becomes ?, one reply is returned when the stream ends (DOS shows dots while it streams), the question is sent to the chosen Space, so never type secrets. A Space whose /chat has other parameters needs src/g.cpp adapted (look at /gradio_api/info).
• Results: 30/30 consecutive questions answered with the 8 KB buffer, each DOS = GRADIO; 17 times 23 -> DOS printed 391.

7. Licensing

A.EXE/B.EXE link mTCP (GPL-3.0-or-later), so our DOS sources, scripts and web code are GPL-3.0-or-later and the mTCP source archive is shipped in third_party/. FreeDOS (GPL-2.0), v86 (BSD-2-Clause), SeaBIOS (LGPL-3.0) and the Crynwr packet driver (source must stay available: Crynwr 11.x sources are at https://archive.org/details/PKTD11A_ZIP) are listed in site/NOTICE.md. Open Watcom is only a build tool and is not redistributed.

8. Known limits / ideas

• QoS 0 only, no TLS and no auth in the DOS clients (TLS is done by the browser's WSS). Topic/payload are small (payload <= 95 chars in A).
• B is started by the page typing B; if the previous run is still going the tick is skipped (log shows it). A real in-DOS loop would need a B.EXE -loop mode.
• Only port 1883 is bridged. Other ports are refused on purpose.
• The page needs DecompressionStream, WebAssembly and internet (DoH + broker). No offline mode.
• Ideas: reconnect in A, in-DOS 5 s loop for B, MQTT QoS 1, mobile on-screen keyboard, a selectable second bridge (own WISP server) for brokers without WSS.