Neural DOOM gradio app (chromatic dark)
Browse files- README.md +15 -8
- WRITEUP.md +122 -0
- __pycache__/app.cpython-313.pyc +0 -0
- __pycache__/x86_core.cpython-313.pyc +0 -0
- __pycache__/x86_linux.cpython-313.pyc +0 -0
- __pycache__/x86_units.cpython-313.pyc +0 -0
- app.py +335 -0
- requirements.txt +5 -0
- x86_core.py +579 -0
- x86_linux.py +241 -0
- x86_units.py +342 -0
README.md
CHANGED
|
@@ -1,13 +1,20 @@
|
|
| 1 |
-
---
|
| 2 |
-
title: Neural
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
colorTo: purple
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version: 6.
|
| 8 |
-
python_version: '3.13'
|
| 9 |
app_file: app.py
|
| 10 |
-
pinned:
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: Neural DOOM
|
| 3 |
+
emoji: 💀
|
| 4 |
+
colorFrom: red
|
| 5 |
colorTo: purple
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 6.15.2
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
+
pinned: true
|
| 10 |
+
license: gpl-2.0
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# Neural DOOM
|
| 14 |
+
|
| 15 |
+
id Software's 1993 DOOM running on an i386 core whose every datapath unit is a
|
| 16 |
+
neural network verified bit-exact over its complete input domain. A full frame
|
| 17 |
+
(5,952,699 instructions) replayed fully neurally was bit-identical to the
|
| 18 |
+
golden run — framebuffer and all 128 MB of machine state. Weights, binary, WAD
|
| 19 |
+
and the title-frame snapshot are pulled from a private repo via the HF_TOKEN
|
| 20 |
+
Space secret. ZeroGPU re-verifies all 13 units exhaustively on demand.
|
WRITEUP.md
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Neural GB — a Game Boy whose logic is neural networks, verified bit-exact
|
| 2 |
+
|
| 3 |
+
**The claim:** a real, unmodified Game Boy game (*Usurper of the Ghoul Throne*,
|
| 4 |
+
a 512KB MBC5 cartridge from itch.io) boots and renders on an emulator in which
|
| 5 |
+
**every functional unit — instruction decode, the entire ALU, every rotate and
|
| 6 |
+
bit operation, tile decoding, palette mapping, sprite priority — is a trained
|
| 7 |
+
neural network**, and the output is **bit-identical** to a conventional
|
| 8 |
+
reference implementation: every pixel of the framebuffer and every byte of
|
| 9 |
+
machine state.
|
| 10 |
+
|
| 11 |
+

|
| 12 |
+
|
| 13 |
+
That title screen was not produced by ordinary emulator code. The frame's 4,178
|
| 14 |
+
CPU instructions were each decoded by a neural net and executed through neural
|
| 15 |
+
ALU nets, and each of its 23,040 pixels passed through neural tile-decode,
|
| 16 |
+
palette, and sprite-mux nets. `images/ghoul_golden.png` is the same frame from
|
| 17 |
+
the conventional implementation — they are identical to the bit
|
| 18 |
+
(`proof/4_neural_game_frame.txt`).
|
| 19 |
+
|
| 20 |
+
## The methodology
|
| 21 |
+
|
| 22 |
+
Neural networks are approximators; emulation needs exactness — 99.9% correct is
|
| 23 |
+
0% playable, because one wrong flag bit desyncs everything downstream. The
|
| 24 |
+
project's answer is a discipline applied uniformly to every unit:
|
| 25 |
+
|
| 26 |
+
1. **Decompose** the machine the way hardware engineers do: into small units
|
| 27 |
+
whose input spaces are completely enumerable (an 8-bit adder has 65,536
|
| 28 |
+
inputs; the opcode decoder has 256; a palette lookup has 1,024).
|
| 29 |
+
2. **Enumerate** the unit's full input domain and its golden outputs.
|
| 30 |
+
3. **Train** a small bit-level MLP on the *entire* domain.
|
| 31 |
+
4. **Verify exhaustively**: the unit passes only at N/N. 65,535/65,536 is a fail.
|
| 32 |
+
5. **Compose** verified units with plain wiring (routing, scheduling, address
|
| 33 |
+
arithmetic). Composition of exact units is exact — so programs run forever
|
| 34 |
+
with zero desync.
|
| 35 |
+
|
| 36 |
+
The rule that makes this scale: **anything too wide to enumerate is never
|
| 37 |
+
trained as one net — it is composed from verified narrow units**, the way
|
| 38 |
+
hardware ripples a carry. 16-bit arithmetic (2³² cases, untrainable) is two
|
| 39 |
+
passes through the verified 8-bit ADC/SBC nets (each 131,072 cases with the
|
| 40 |
+
carry-in as an explicit input). The 26 units and their verified domains are in
|
| 41 |
+
`proof/1_unit_verification.txt`; trained weights ship in `code/models/`.
|
| 42 |
+
|
| 43 |
+
## The build, brick by brick
|
| 44 |
+
|
| 45 |
+
| Brick | What it proved | Artifact |
|
| 46 |
+
|---|---|---|
|
| 47 |
+
| Single tile through neural PPU units | the unit method works for graphics | `images/neural_tile.png` |
|
| 48 |
+
| Shared-clock CPU+PPU machine | a mid-frame register write lands on the right **scanline** (82) | `images/neural_frame.png` |
|
| 49 |
+
| Per-dot orchestrator | a mid-scanline write lands on the right **pixel** (line 82, px 64) — a frame a scanline renderer cannot represent | `images/neural_frame_dot.png` |
|
| 50 |
+
| Complete SM83 CPU core | all 512 opcodes, stack, interrupts, m-cycle bus timing | `code/gb_sm83.py` |
|
| 51 |
+
| Full DMG console | MBC1/MBC5 banking, full PPU (BG+window+sprites, STAT/LYC, DMA), timer, joypad | `code/gb_console.py` |
|
| 52 |
+
| **A real game** | the whole stack holds on commercial-style code | `images/ghoul_neural.png` |
|
| 53 |
+
|
| 54 |
+
## The proof chain
|
| 55 |
+
|
| 56 |
+
Self-consistency isn't enough, so the core was held to the standards real
|
| 57 |
+
emulators are judged by:
|
| 58 |
+
|
| 59 |
+
- **SingleStepTests/sm83**: 512 opcodes × 1,000 randomized cases, each checking
|
| 60 |
+
full register/flag/RAM state and cycle counts — **all pass**
|
| 61 |
+
(`proof/2_singlesteptests.txt`).
|
| 62 |
+
- **Blargg's cpu_instrs**: the classic hardware-validated instruction exerciser,
|
| 63 |
+
reporting over the emulated serial port — **11/11 "Passed all tests"**
|
| 64 |
+
(`proof/3_blargg_cpu_instrs.txt`).
|
| 65 |
+
- **Golden vs neural bit-identity** on the game frame: framebuffer and complete
|
| 66 |
+
machine state identical (`proof/4_neural_game_frame.txt`). The comparison is
|
| 67 |
+
meaningful because both runs share one orchestrator — only the units differ.
|
| 68 |
+
- **Playability**: the turbo raster is pixel-identical to the per-dot reference
|
| 69 |
+
at the checked frame, runs ~25 fps, and the game responds to a Start press
|
| 70 |
+
within 2 frames (`proof/5_turbo_and_input.txt`, `images/ghoul_after_start.png`).
|
| 71 |
+
|
| 72 |
+
A story worth telling: across the entire build, the test suites found **exactly
|
| 73 |
+
one bug — in hand-written glue, not in any neural unit** (SWAP and SRL
|
| 74 |
+
transposed in the CB dispatch table). The verified units were flawless by
|
| 75 |
+
construction; the one unverified lookup table was where the bug hid. That is
|
| 76 |
+
the methodology's thesis in miniature.
|
| 77 |
+
|
| 78 |
+
## Run it yourself
|
| 79 |
+
|
| 80 |
+
```
|
| 81 |
+
cd code
|
| 82 |
+
pip install numpy torch pillow # torch only needed to load/train units
|
| 83 |
+
python gb_units.py # verify all 26 units (instant: cached weights)
|
| 84 |
+
python run_sst.py # 512-opcode suite (needs sm83/v1 JSONs)
|
| 85 |
+
python run_blargg.py # cpu_instrs 11/11 (needs cpu_instrs.gb)
|
| 86 |
+
python run_ghoul.py # the bit-exact neural game frame
|
| 87 |
+
python play_gb.py # PLAY IT (golden units, ~25 fps)
|
| 88 |
+
python play_gb.py --neural # every unit neural: slow-motion proof mode
|
| 89 |
+
```
|
| 90 |
+
|
| 91 |
+
Controls: arrows = d-pad, Z = A, X = B, Enter = Start, Shift = Select, Esc = quit.
|
| 92 |
+
|
| 93 |
+
## Honest limitations
|
| 94 |
+
|
| 95 |
+
Gameplay speed uses the golden (conventional) units — the fully neural machine
|
| 96 |
+
runs at ~5 s/frame, so `--neural` is a proof mode, not a way to play. The
|
| 97 |
+
orchestrator simplifies: OAM DMA is an instant copy, no sprite-fetch mode-3
|
| 98 |
+
stalls, no OAM/VRAM access blocking by PPU mode, and no audio. All
|
| 99 |
+
simplifications are identical in golden and neural runs, so every bit-identity
|
| 100 |
+
claim is unaffected by them.
|
| 101 |
+
|
| 102 |
+
## Epilogue: DOOM on a neural x86
|
| 103 |
+
|
| 104 |
+
The methodology then scaled to a 1993 PC. The same discipline — enumerate,
|
| 105 |
+
train, verify N/N, compose — produced 13 verified x86 datapath units (8-bit
|
| 106 |
+
ADC/SBB slices carrying all five x86 flags, logic ops, 1-bit shift slices, the
|
| 107 |
+
MASK8 multiplier partial-product slice, and the prefix/ModRM/SIB decode
|
| 108 |
+
tables). Everything wider is composition: 32-bit arithmetic ripples carries
|
| 109 |
+
through the 8-bit slices, multiplication is masked shifted adds, division is
|
| 110 |
+
restoring subtraction — silicon's own recipes.
|
| 111 |
+
|
| 112 |
+
The i386 core built from these units matches QEMU on 194/194 instruction
|
| 113 |
+
templates (2,560 randomized full-state cases). A minimal Linux shell (ELF
|
| 114 |
+
loader + ~30 syscalls, all clocks deterministic) runs real static binaries:
|
| 115 |
+
id Software's actual DOOM, cross-compiled to i386, boots through its full init
|
| 116 |
+
chain and renders the title screen in 20M instructions.
|
| 117 |
+
|
| 118 |
+
The proof: the machine was frozen at the title frame, and the next frame
|
| 119 |
+
(5,952,699 instructions) was run twice — once golden, once with every decode,
|
| 120 |
+
ALU, shift, multiply, and divide a neural network. Framebuffer: bit-identical.
|
| 121 |
+
All 128 MB of memory plus registers and flags: identical
|
| 122 |
+
(`proof/6_neural_doom_frame.txt`, `images/doom_neural_frame.png`).
|
__pycache__/app.cpython-313.pyc
ADDED
|
Binary file (23.7 kB). View file
|
|
|
__pycache__/x86_core.cpython-313.pyc
ADDED
|
Binary file (38.4 kB). View file
|
|
|
__pycache__/x86_linux.cpython-313.pyc
ADDED
|
Binary file (14.3 kB). View file
|
|
|
__pycache__/x86_units.cpython-313.pyc
ADDED
|
Binary file (32.6 kB). View file
|
|
|
app.py
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Neural DOOM -- ZeroGPU Gradio Space, chromatic dark UI.
|
| 3 |
+
|
| 4 |
+
DOOM runs on an i386 core whose every datapath unit (decode, ALU slices,
|
| 5 |
+
shifts, multiplier slices) is a neural network verified bit-exact over its
|
| 6 |
+
COMPLETE input domain. A full frame (5,952,699 instructions) was replayed
|
| 7 |
+
fully neurally and came out bit-identical to the golden run -- framebuffer
|
| 8 |
+
and all 128 MB of machine state.
|
| 9 |
+
|
| 10 |
+
Interactives:
|
| 11 |
+
* ZeroGPU: re-verify ALL 13 units exhaustively on the H200, live
|
| 12 |
+
* run a live neural execution segment from the title-frame snapshot,
|
| 13 |
+
bit-checked against the golden reference hash
|
| 14 |
+
* instruction microscope: single-step real DOOM code and watch every
|
| 15 |
+
neural unit fire, slice by slice
|
| 16 |
+
|
| 17 |
+
Assets (unit weights, DOOM binary, WAD, snapshot) come from a private HF
|
| 18 |
+
repo via the HF_TOKEN Space secret.
|
| 19 |
+
"""
|
| 20 |
+
import os, json, time, hashlib
|
| 21 |
+
import numpy as np
|
| 22 |
+
import gradio as gr
|
| 23 |
+
import spaces
|
| 24 |
+
import torch
|
| 25 |
+
from huggingface_hub import snapshot_download
|
| 26 |
+
|
| 27 |
+
ASSETS_REPO = "Quazim0t0/neural-x86-doom"
|
| 28 |
+
ASSETS = snapshot_download(ASSETS_REPO, token=os.environ.get("HF_TOKEN"))
|
| 29 |
+
|
| 30 |
+
import x86_units
|
| 31 |
+
x86_units.CACHE = os.path.join(ASSETS, "models")
|
| 32 |
+
from x86_units import build_all, NeuralUnits, GoldenUnits, ALU, BitMLP
|
| 33 |
+
from x86_units import (u_ADC8, u_SBB8, u_logic, u_SHL1, u_SHR1, u_MASK8,
|
| 34 |
+
u_NOT8, u_FLAGS8, u_prefix, u_modrm, u_sib)
|
| 35 |
+
from x86_linux import Linux386
|
| 36 |
+
|
| 37 |
+
SNAP = np.load(os.path.join(ASSETS, "doom_snapshot.npz"))
|
| 38 |
+
GOLDEN_REFS = json.load(open(os.path.join(ASSETS, "doom_golden_refs.json")))
|
| 39 |
+
ELF = open(os.path.join(ASSETS, "doom_i386"), "rb").read()
|
| 40 |
+
WAD = open(os.path.join(ASSETS, "doom1.wad"), "rb").read()
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def title_frame_image():
|
| 44 |
+
from PIL import Image
|
| 45 |
+
fb = SNAP["title_fb"].reshape(400, 640, 4)
|
| 46 |
+
return Image.fromarray(fb[:, :, [2, 1, 0]])
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def fresh_machine():
|
| 50 |
+
m = Linux386(ELF, argv=("doom", "-iwad", "doom1.wad"),
|
| 51 |
+
fs={"doom1.wad": WAD, "./doom1.wad": WAD})
|
| 52 |
+
m.mem[:] = SNAP["mem"].tobytes()
|
| 53 |
+
r = SNAP["regs"]
|
| 54 |
+
m.cpu.r = [int(x) for x in r[:8]]
|
| 55 |
+
(m.cpu.eip, m.cpu.gs_base, m.cpu.instr_count,
|
| 56 |
+
m.clock_ns, m.brk, m.mmap_ptr, m.next_fd) = (int(x) for x in r[8:15])
|
| 57 |
+
m.cpu.f = {str(k): int(v) for k, v in zip(SNAP["fkeys"], SNAP["fvals"])}
|
| 58 |
+
m.frames = []
|
| 59 |
+
return m
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
def state_hash(m):
|
| 63 |
+
h = hashlib.sha256()
|
| 64 |
+
h.update(m.mem)
|
| 65 |
+
h.update(repr((m.cpu.r, m.cpu.eip, sorted(m.cpu.f.items()), m.clock_ns)).encode())
|
| 66 |
+
return h.hexdigest()
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
# ================= ZeroGPU: exhaustive unit re-verification =================
|
| 70 |
+
UNIT_SPECS = [
|
| 71 |
+
("ADC8", u_ADC8, "8-bit add w/ carry + CF·OF·SF·ZF·AF"),
|
| 72 |
+
("SBB8", u_SBB8, "8-bit subtract w/ borrow + flags"),
|
| 73 |
+
("AND8", lambda: u_logic(lambda a, b: a & b), "bitwise AND + SF·ZF"),
|
| 74 |
+
("OR8", lambda: u_logic(lambda a, b: a | b), "bitwise OR + SF·ZF"),
|
| 75 |
+
("XOR8", lambda: u_logic(lambda a, b: a ^ b), "bitwise XOR + SF·ZF"),
|
| 76 |
+
("SHL1", u_SHL1, "shift-left-1 slice w/ carry chain"),
|
| 77 |
+
("SHR1", u_SHR1, "shift-right-1 slice w/ carry chain"),
|
| 78 |
+
("MASK8", u_MASK8, "multiplier partial-product slice"),
|
| 79 |
+
("NOT8", u_NOT8, "bitwise NOT"),
|
| 80 |
+
("FLAGS8", u_FLAGS8, "SF·ZF·parity extractor"),
|
| 81 |
+
("PREFIX", u_prefix, "instruction prefix classifier"),
|
| 82 |
+
("MODRM", u_modrm, "ModRM field decoder"),
|
| 83 |
+
("SIB", u_sib, "SIB field decoder"),
|
| 84 |
+
]
|
| 85 |
+
|
| 86 |
+
@spaces.GPU(duration=60)
|
| 87 |
+
def verify_all_units_gpu():
|
| 88 |
+
dev = "cuda" if torch.cuda.is_available() else "cpu"
|
| 89 |
+
gpu = torch.cuda.get_device_name(0) if dev == "cuda" else "CPU fallback"
|
| 90 |
+
rows = []
|
| 91 |
+
total_cases = 0
|
| 92 |
+
t_all = time.time()
|
| 93 |
+
for name, builder, desc in UNIT_SPECS:
|
| 94 |
+
X, Y = builder()
|
| 95 |
+
X = torch.tensor(X, device=dev); Y = torch.tensor(Y, device=dev)
|
| 96 |
+
net = BitMLP(X.shape[1], Y.shape[1]).to(dev)
|
| 97 |
+
net.load_state_dict(torch.load(
|
| 98 |
+
os.path.join(ASSETS, "models", f"{name}.pt"),
|
| 99 |
+
weights_only=True, map_location=dev))
|
| 100 |
+
t0 = time.time()
|
| 101 |
+
with torch.no_grad():
|
| 102 |
+
ok = int((((net(X) > 0).float() == Y).all(1)).sum())
|
| 103 |
+
total_cases += X.shape[0]
|
| 104 |
+
verdict = "✅ EXACT" if ok == X.shape[0] else "❌ FAIL"
|
| 105 |
+
rows.append(f"| `{name}` | {desc} | {ok:,} / {X.shape[0]:,} | {verdict} | "
|
| 106 |
+
f"{(time.time()-t0)*1000:.0f} ms |")
|
| 107 |
+
dt = time.time() - t_all
|
| 108 |
+
return (f"### {total_cases:,} cases — every possible input of every unit — "
|
| 109 |
+
f"re-verified on **{gpu}** in **{dt:.1f}s**\n\n"
|
| 110 |
+
"| unit | role | cases exact | verdict | GPU time |\n"
|
| 111 |
+
"|---|---|---|---|---|\n" + "\n".join(rows) +
|
| 112 |
+
"\n\n*This is the entire mathematical foundation of the machine, "
|
| 113 |
+
"re-proven from the shipped weights, live, just now.*")
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
# ================= live neural execution segment =================
|
| 117 |
+
_neural = None
|
| 118 |
+
def neural_units():
|
| 119 |
+
global _neural
|
| 120 |
+
if _neural is None:
|
| 121 |
+
_neural = NeuralUnits(build_all())
|
| 122 |
+
return _neural
|
| 123 |
+
|
| 124 |
+
def run_neural_segment(seg_choice, progress=gr.Progress()):
|
| 125 |
+
seg = int(seg_choice.split()[0].replace(",", ""))
|
| 126 |
+
ref = GOLDEN_REFS.get(str(seg))
|
| 127 |
+
units = neural_units()
|
| 128 |
+
m = fresh_machine()
|
| 129 |
+
m.cpu.u = units
|
| 130 |
+
m.cpu.alu = ALU(units)
|
| 131 |
+
t0 = time.time()
|
| 132 |
+
for i in range(seg):
|
| 133 |
+
m.cpu.step()
|
| 134 |
+
if i % 250 == 0:
|
| 135 |
+
progress(i / seg, desc=f"⚡ {i:,} / {seg:,} instructions through the neural nets")
|
| 136 |
+
dt = time.time() - t0
|
| 137 |
+
h = state_hash(m)
|
| 138 |
+
same = (h == ref)
|
| 139 |
+
badge = ("<div class='verdict-pass'>BIT-IDENTICAL ✅</div>" if same
|
| 140 |
+
else "<div class='verdict-fail'>MISMATCH ❌</div>")
|
| 141 |
+
return (f"{badge}\n\n**{seg:,} instructions of id Software's actual machine code** "
|
| 142 |
+
f"executed with every decode, ALU op, shift and multiply a neural "
|
| 143 |
+
f"network — {dt:.0f}s ({seg/dt:,.0f} instr/s).\n\n"
|
| 144 |
+
f"Resulting machine state (128 MB memory + registers + flags) hashed and "
|
| 145 |
+
f"compared against the golden reference:\n\n"
|
| 146 |
+
f"`neural {h[:40]}…`\n`golden {(ref or '?')[:40]}…`")
|
| 147 |
+
|
| 148 |
+
|
| 149 |
+
# ================= instruction microscope =================
|
| 150 |
+
class TracingUnits:
|
| 151 |
+
"""Proxy around NeuralUnits that records every unit invocation."""
|
| 152 |
+
def __init__(self, inner):
|
| 153 |
+
self._inner = inner
|
| 154 |
+
self.calls = []
|
| 155 |
+
def __getattr__(self, name):
|
| 156 |
+
fn = getattr(self._inner, name)
|
| 157 |
+
def wrap(*a):
|
| 158 |
+
r = fn(*a)
|
| 159 |
+
self.calls.append((name, a, r))
|
| 160 |
+
return r
|
| 161 |
+
return wrap
|
| 162 |
+
|
| 163 |
+
MICRO = {"m": None, "tr": None}
|
| 164 |
+
|
| 165 |
+
def micro_reset():
|
| 166 |
+
MICRO["m"] = None
|
| 167 |
+
return micro_step()
|
| 168 |
+
|
| 169 |
+
def micro_step():
|
| 170 |
+
if MICRO["m"] is None:
|
| 171 |
+
m = fresh_machine()
|
| 172 |
+
tr = TracingUnits(neural_units())
|
| 173 |
+
m.cpu.u = tr
|
| 174 |
+
m.cpu.alu = ALU(tr)
|
| 175 |
+
MICRO["m"], MICRO["tr"] = m, tr
|
| 176 |
+
m, tr = MICRO["m"], MICRO["tr"]
|
| 177 |
+
tr.calls = []
|
| 178 |
+
eip0 = m.cpu.eip
|
| 179 |
+
code = bytes(m.mem[eip0:eip0 + 12]).hex(" ")
|
| 180 |
+
m.cpu.step()
|
| 181 |
+
nbytes = (m.cpu.eip - eip0) if 0 < m.cpu.eip - eip0 <= 12 else 12
|
| 182 |
+
shown = code[:nbytes * 3 - 1] if nbytes else code
|
| 183 |
+
rows = []
|
| 184 |
+
for name, args, res in tr.calls[:40]:
|
| 185 |
+
a = ", ".join(f"{x:02x}" if isinstance(x, int) else str(x) for x in args)
|
| 186 |
+
if isinstance(res, tuple):
|
| 187 |
+
r = " ".join(f"{x:02x}" if isinstance(x, int) else str(x) for x in res)
|
| 188 |
+
elif isinstance(res, int):
|
| 189 |
+
r = f"{res:02x}"
|
| 190 |
+
else:
|
| 191 |
+
r = str(res)
|
| 192 |
+
rows.append(f"| `{name}` | `{a}` | `{r}` |")
|
| 193 |
+
more = f"\n*…plus {len(tr.calls)-40} more unit calls*" if len(tr.calls) > 40 else ""
|
| 194 |
+
regs = m.cpu.r
|
| 195 |
+
f = m.cpu.f
|
| 196 |
+
return (f"### instruction @ `eip {eip0:08x}` — bytes `{shown}`\n"
|
| 197 |
+
f"**{len(tr.calls)} neural unit calls** computed this instruction:\n\n"
|
| 198 |
+
"| unit | inputs | outputs |\n|---|---|---|\n" + "\n".join(rows) + more +
|
| 199 |
+
f"\n\n**after:** `eax {regs[0]:08x}` `ecx {regs[1]:08x}` `edx {regs[2]:08x}` "
|
| 200 |
+
f"`ebx {regs[3]:08x}` `esp {regs[4]:08x}` `ebp {regs[5]:08x}` "
|
| 201 |
+
f"`esi {regs[6]:08x}` `edi {regs[7]:08x}` → `eip {m.cpu.eip:08x}`\n\n"
|
| 202 |
+
f"flags: CF={f['CF']} ZF={f['ZF']} SF={f['SF']} OF={f['OF']} "
|
| 203 |
+
f"PF={f['PF']} AF={f['AF']}")
|
| 204 |
+
|
| 205 |
+
|
| 206 |
+
# ================= UI =================
|
| 207 |
+
WRITEUP = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), "WRITEUP.md"),
|
| 208 |
+
encoding="utf-8").read()
|
| 209 |
+
|
| 210 |
+
CSS = """
|
| 211 |
+
:root { color-scheme: dark; }
|
| 212 |
+
body, .gradio-container {
|
| 213 |
+
background: radial-gradient(ellipse at 20% -10%, #1a0a2e 0%, #0a0a14 45%, #050508 100%) !important;
|
| 214 |
+
}
|
| 215 |
+
#hero h1 {
|
| 216 |
+
font-size: 2.6em; font-weight: 900; letter-spacing: -0.02em;
|
| 217 |
+
background: linear-gradient(90deg, #ff3b3b, #ff9d00, #ffe600, #3bff8f, #00e0ff, #b03bff, #ff3b8f);
|
| 218 |
+
background-size: 300% 100%;
|
| 219 |
+
-webkit-background-clip: text; background-clip: text; color: transparent;
|
| 220 |
+
animation: chroma 8s linear infinite;
|
| 221 |
+
}
|
| 222 |
+
@keyframes chroma { 0% {background-position: 0% 50%;} 100% {background-position: 300% 50%;} }
|
| 223 |
+
#hero p, #hero li { color: #c8c8de; }
|
| 224 |
+
.panel {
|
| 225 |
+
border: 1px solid transparent; border-radius: 14px; padding: 4px;
|
| 226 |
+
background:
|
| 227 |
+
linear-gradient(#0e0e1a, #0e0e1a) padding-box,
|
| 228 |
+
linear-gradient(135deg, #ff3b3b66, #00e0ff66, #b03bff66) border-box;
|
| 229 |
+
box-shadow: 0 0 24px #00e0ff14, 0 0 48px #b03bff0f;
|
| 230 |
+
}
|
| 231 |
+
#doomframe { position: relative; border-radius: 10px; overflow: hidden;
|
| 232 |
+
box-shadow: 0 0 30px #ff3b3b33, 0 0 60px #ff3b3b1a; }
|
| 233 |
+
#doomframe::after {
|
| 234 |
+
content: ""; position: absolute; inset: 0; pointer-events: none;
|
| 235 |
+
background: repeating-linear-gradient(0deg, transparent 0 2px, #00000022 2px 4px);
|
| 236 |
+
}
|
| 237 |
+
button.primary, .primary {
|
| 238 |
+
background: linear-gradient(90deg, #ff3b3b, #b03bff) !important;
|
| 239 |
+
border: none !important; color: white !important; font-weight: 700 !important;
|
| 240 |
+
box-shadow: 0 0 18px #b03bff55 !important; transition: box-shadow .2s !important;
|
| 241 |
+
}
|
| 242 |
+
button.primary:hover { box-shadow: 0 0 30px #ff3b3b88 !important; }
|
| 243 |
+
.verdict-pass {
|
| 244 |
+
display: inline-block; padding: 10px 22px; border-radius: 10px; font-size: 1.3em;
|
| 245 |
+
font-weight: 900; color: #06140a;
|
| 246 |
+
background: linear-gradient(90deg, #3bff8f, #00e0ff);
|
| 247 |
+
box-shadow: 0 0 28px #3bff8f66; letter-spacing: .04em;
|
| 248 |
+
}
|
| 249 |
+
.verdict-fail {
|
| 250 |
+
display: inline-block; padding: 10px 22px; border-radius: 10px; font-size: 1.3em;
|
| 251 |
+
font-weight: 900; color: #fff; background: linear-gradient(90deg, #ff3b3b, #ff9d00);
|
| 252 |
+
}
|
| 253 |
+
.prooflog {
|
| 254 |
+
font-family: ui-monospace, monospace; color: #3bff8f; background: #050a07;
|
| 255 |
+
border: 1px solid #3bff8f44; border-radius: 10px; padding: 14px;
|
| 256 |
+
box-shadow: inset 0 0 24px #3bff8f11;
|
| 257 |
+
}
|
| 258 |
+
table { border-color: #2a2a44 !important; }
|
| 259 |
+
"""
|
| 260 |
+
|
| 261 |
+
FORCE_DARK = """
|
| 262 |
+
function refresh() {
|
| 263 |
+
const url = new URL(window.location);
|
| 264 |
+
if (url.searchParams.get('__theme') !== 'dark') {
|
| 265 |
+
url.searchParams.set('__theme', 'dark');
|
| 266 |
+
window.location.href = url.href;
|
| 267 |
+
}
|
| 268 |
+
}
|
| 269 |
+
"""
|
| 270 |
+
|
| 271 |
+
theme = gr.themes.Base(
|
| 272 |
+
primary_hue="purple", neutral_hue="slate",
|
| 273 |
+
font=[gr.themes.GoogleFont("Inter"), "system-ui", "sans-serif"],
|
| 274 |
+
font_mono=[gr.themes.GoogleFont("JetBrains Mono"), "ui-monospace", "monospace"],
|
| 275 |
+
)
|
| 276 |
+
|
| 277 |
+
with gr.Blocks(title="Neural DOOM") as demo:
|
| 278 |
+
with gr.Column(elem_id="hero"):
|
| 279 |
+
gr.Markdown(
|
| 280 |
+
"# NEURAL DOOM\n"
|
| 281 |
+
"### id Software's 1993 code, executed by neural networks — bit-exact\n"
|
| 282 |
+
"Every datapath unit of this i386 — instruction decode, every ALU slice, "
|
| 283 |
+
"every shift, every multiply — is a neural network verified over its "
|
| 284 |
+
"**complete input domain**. A full frame (**5,952,699 instructions**) was "
|
| 285 |
+
"replayed with every unit neural and came out **bit-identical** to the "
|
| 286 |
+
"conventional run: every pixel, every byte of all 128 MB of machine state.")
|
| 287 |
+
with gr.Row():
|
| 288 |
+
with gr.Column(scale=3):
|
| 289 |
+
gr.Image(value=title_frame_image(), elem_id="doomframe", show_label=False,
|
| 290 |
+
interactive=False)
|
| 291 |
+
with gr.Column(scale=2):
|
| 292 |
+
gr.HTML("<div class='prooflog'>instructions golden = 5,952,699<br>"
|
| 293 |
+
"instructions neural = 5,952,699<br>"
|
| 294 |
+
"framebuffer 640×400 ........ BIT-IDENTICAL<br>"
|
| 295 |
+
"machine state (128 MB) ..... IDENTICAL<br><br>"
|
| 296 |
+
">>> DOOM FRAME EXACT THROUGH THE FULLY NEURAL x86</div>")
|
| 297 |
+
gr.Markdown("_102 minutes of compute at ~970 instructions/second — every "
|
| 298 |
+
"one decoded and computed by neural nets, one verified 8-bit "
|
| 299 |
+
"slice at a time. This page lets you re-run the proofs yourself._")
|
| 300 |
+
gr.Markdown("---")
|
| 301 |
+
with gr.Row(equal_height=False):
|
| 302 |
+
with gr.Column(elem_classes="panel"):
|
| 303 |
+
gr.Markdown("## 🔬 Re-prove the foundation · ZeroGPU\n"
|
| 304 |
+
"One click re-verifies **all 13 units exhaustively** — every "
|
| 305 |
+
"possible input of every unit, 525k+ cases — on an H200 slice.")
|
| 306 |
+
v_btn = gr.Button("⚡ Re-verify all 13 units on GPU", variant="primary")
|
| 307 |
+
v_out = gr.Markdown()
|
| 308 |
+
with gr.Column(elem_classes="panel"):
|
| 309 |
+
gr.Markdown("## ⚙️ Run DOOM neurally, live\n"
|
| 310 |
+
"Executes real DOOM machine code from the title-frame snapshot "
|
| 311 |
+
"with **every unit neural**, then bit-checks the resulting "
|
| 312 |
+
"machine state against the golden reference hash.")
|
| 313 |
+
seg = gr.Dropdown(["10,000 instructions", "25,000 instructions",
|
| 314 |
+
"50,000 instructions"], value="10,000 instructions",
|
| 315 |
+
label="segment length", filterable=False)
|
| 316 |
+
r_btn = gr.Button("⚡ Execute neurally + verify", variant="primary")
|
| 317 |
+
r_out = gr.Markdown()
|
| 318 |
+
with gr.Column(elem_classes="panel"):
|
| 319 |
+
gr.Markdown("## 🧬 Instruction microscope\n"
|
| 320 |
+
"Single-step DOOM and watch the neural units fire. Each row is one "
|
| 321 |
+
"net computing one verified slice of the instruction — carry chains, "
|
| 322 |
+
"flag wiring, decode fields, all of it.")
|
| 323 |
+
with gr.Row():
|
| 324 |
+
s_btn = gr.Button("⏭ Step one instruction", variant="primary")
|
| 325 |
+
x_btn = gr.Button("↺ Reset to title frame")
|
| 326 |
+
s_out = gr.Markdown()
|
| 327 |
+
v_btn.click(verify_all_units_gpu, None, v_out)
|
| 328 |
+
r_btn.click(run_neural_segment, seg, r_out)
|
| 329 |
+
s_btn.click(micro_step, None, s_out)
|
| 330 |
+
x_btn.click(micro_reset, None, s_out)
|
| 331 |
+
gr.Markdown("---")
|
| 332 |
+
with gr.Accordion("📜 The full write-up — from a Game Boy smiley tile to DOOM", open=False):
|
| 333 |
+
gr.Markdown(WRITEUP)
|
| 334 |
+
|
| 335 |
+
demo.launch(theme=theme, css=CSS, js=FORCE_DARK)
|
requirements.txt
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
numpy
|
| 2 |
+
torch
|
| 3 |
+
pillow
|
| 4 |
+
huggingface_hub
|
| 5 |
+
spaces
|
x86_core.py
ADDED
|
@@ -0,0 +1,579 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
x86_core.py -- i386/i486 user-mode integer core (flat 32-bit protected mode),
|
| 3 |
+
every data transformation routed through the x86_units API (golden or neural).
|
| 4 |
+
|
| 5 |
+
Scope: the instruction set a -march=i486 -O1 musl/doomgeneric binary uses.
|
| 6 |
+
No FPU (DOOM is fixed-point), no paging (flat), no real mode. Segments are
|
| 7 |
+
flat except %gs, whose base is honored for i386 TLS.
|
| 8 |
+
|
| 9 |
+
Composition stances (consistent with the GB bricks):
|
| 10 |
+
- all arithmetic/logic/shift/mul/div through verified slice units (ALU class)
|
| 11 |
+
- effective-address arithmetic, immediate assembly, and dispatch are wiring
|
| 12 |
+
- one-operand IMUL/MUL produce edx:eax via the 8x8 partial-product tree;
|
| 13 |
+
IDIV/DIV are restoring division over verified subtract slices
|
| 14 |
+
"""
|
| 15 |
+
from x86_units import GoldenUnits, ALU
|
| 16 |
+
|
| 17 |
+
EAX, ECX, EDX, EBX, ESP, EBP, ESI, EDI = range(8)
|
| 18 |
+
CF, PF, AF, ZF, SF, OF, DF = "CF", "PF", "AF", "ZF", "SF", "OF", "DF"
|
| 19 |
+
M32 = 0xFFFFFFFF
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def sx8(v): return v - 0x100 if v & 0x80 else v
|
| 23 |
+
def sx16(v): return v - 0x10000 if v & 0x8000 else v
|
| 24 |
+
def sx32(v): return v - 0x100000000 if v & 0x80000000 else v
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
class CPUError(Exception):
|
| 28 |
+
pass
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
class X86:
|
| 32 |
+
def __init__(self, mem, units=None):
|
| 33 |
+
self.mem = mem # bytearray, flat
|
| 34 |
+
self.u = units or GoldenUnits()
|
| 35 |
+
self.alu = ALU(self.u)
|
| 36 |
+
self.r = [0] * 8
|
| 37 |
+
self.eip = 0
|
| 38 |
+
self.f = {CF: 0, PF: 0, AF: 0, ZF: 0, SF: 0, OF: 0, DF: 0}
|
| 39 |
+
self.gs_base = 0
|
| 40 |
+
self.exited = None # set to status on exit syscall
|
| 41 |
+
self.syscall = None # callback(cpu) for int 0x80
|
| 42 |
+
self.instr_count = 0
|
| 43 |
+
|
| 44 |
+
# ---------------- memory (wiring) ----------------
|
| 45 |
+
def rd(self, a, n):
|
| 46 |
+
a &= M32
|
| 47 |
+
return int.from_bytes(self.mem[a:a + n], "little")
|
| 48 |
+
def wr(self, a, v, n):
|
| 49 |
+
a &= M32
|
| 50 |
+
self.mem[a:a + n] = int(v & ((1 << (8 * n)) - 1)).to_bytes(n, "little")
|
| 51 |
+
|
| 52 |
+
def fetch(self, n=1):
|
| 53 |
+
v = self.rd(self.eip, n)
|
| 54 |
+
self.eip = (self.eip + n) & M32
|
| 55 |
+
return v
|
| 56 |
+
|
| 57 |
+
# ---------------- register access (wiring) ----------------
|
| 58 |
+
def get_r(self, i, size):
|
| 59 |
+
if size == 32: return self.r[i]
|
| 60 |
+
if size == 16: return self.r[i] & 0xFFFF
|
| 61 |
+
if i < 4: return self.r[i] & 0xFF # AL CL DL BL
|
| 62 |
+
return (self.r[i - 4] >> 8) & 0xFF # AH CH DH BH
|
| 63 |
+
def set_r(self, i, v, size):
|
| 64 |
+
if size == 32: self.r[i] = v & M32
|
| 65 |
+
elif size == 16: self.r[i] = (self.r[i] & 0xFFFF0000) | (v & 0xFFFF)
|
| 66 |
+
elif i < 4: self.r[i] = (self.r[i] & ~0xFF) | (v & 0xFF)
|
| 67 |
+
else: self.r[i - 4] = (self.r[i - 4] & ~0xFF00) | ((v & 0xFF) << 8)
|
| 68 |
+
|
| 69 |
+
# ---------------- flags ----------------
|
| 70 |
+
def setf(self, fl):
|
| 71 |
+
for k, v in fl.items():
|
| 72 |
+
self.f[k] = int(v)
|
| 73 |
+
def eflags(self):
|
| 74 |
+
f = self.f
|
| 75 |
+
return (f[CF] | (f[PF] << 2) | (f[AF] << 4) | (f[ZF] << 6) | (f[SF] << 7)
|
| 76 |
+
| (f[DF] << 10) | (f[OF] << 11) | 0x2)
|
| 77 |
+
def set_eflags(self, v):
|
| 78 |
+
self.f[CF] = v & 1; self.f[PF] = (v >> 2) & 1; self.f[AF] = (v >> 4) & 1
|
| 79 |
+
self.f[ZF] = (v >> 6) & 1; self.f[SF] = (v >> 7) & 1
|
| 80 |
+
self.f[DF] = (v >> 10) & 1; self.f[OF] = (v >> 11) & 1
|
| 81 |
+
|
| 82 |
+
def cond(self, cc):
|
| 83 |
+
f = self.f
|
| 84 |
+
v = [f[OF], f[CF], f[ZF], f[CF] | f[ZF], f[SF], f[PF],
|
| 85 |
+
f[SF] ^ f[OF], (f[SF] ^ f[OF]) | f[ZF]][cc >> 1]
|
| 86 |
+
return v ^ (cc & 1)
|
| 87 |
+
|
| 88 |
+
# ---------------- ModRM / SIB (decode units + EA wiring) ----------------
|
| 89 |
+
def modrm(self, size, seg_base=0):
|
| 90 |
+
mod, reg, rm = self.u.modrm(self.fetch())
|
| 91 |
+
if mod == 3:
|
| 92 |
+
return reg, ("r", rm)
|
| 93 |
+
if rm == 4:
|
| 94 |
+
ss, idx, base = self.u.sib(self.fetch())
|
| 95 |
+
ea = 0 if (base == 5 and mod == 0) else self.r[base]
|
| 96 |
+
if base == 5 and mod == 0:
|
| 97 |
+
ea = self.fetch(4)
|
| 98 |
+
if idx != 4:
|
| 99 |
+
ea = (ea + (self.r[idx] << ss)) & M32
|
| 100 |
+
elif rm == 5 and mod == 0:
|
| 101 |
+
ea = self.fetch(4)
|
| 102 |
+
else:
|
| 103 |
+
ea = self.r[rm]
|
| 104 |
+
if mod == 1: ea = (ea + sx8(self.fetch())) & M32
|
| 105 |
+
elif mod == 2: ea = (ea + self.fetch(4)) & M32
|
| 106 |
+
return reg, ("m", (ea + seg_base) & M32)
|
| 107 |
+
|
| 108 |
+
def get_rm(self, loc, size):
|
| 109 |
+
k, x = loc
|
| 110 |
+
return self.get_r(x, size) if k == "r" else self.rd(x, size // 8)
|
| 111 |
+
def set_rm(self, loc, v, size):
|
| 112 |
+
k, x = loc
|
| 113 |
+
if k == "r": self.set_r(x, v, size)
|
| 114 |
+
else: self.wr(x, v, size // 8)
|
| 115 |
+
|
| 116 |
+
# ---------------- ALU dispatch through units ----------------
|
| 117 |
+
def alu_op(self, op, a, b, size):
|
| 118 |
+
"""op: 0 ADD,1 OR,2 ADC,3 SBB,4 AND,5 SUB,6 XOR,7 CMP. Returns result or None."""
|
| 119 |
+
if op == 0: r, fl = self.alu.addsub(0, a, b, size)
|
| 120 |
+
elif op == 2: r, fl = self.alu.addsub(0, a, b, size, self.f[CF])
|
| 121 |
+
elif op in (5, 7): r, fl = self.alu.addsub(1, a, b, size)
|
| 122 |
+
elif op == 3: r, fl = self.alu.addsub(1, a, b, size, self.f[CF])
|
| 123 |
+
elif op == 1: r, fl = self.alu.logic("OR", a, b, size)
|
| 124 |
+
elif op == 4: r, fl = self.alu.logic("AND", a, b, size)
|
| 125 |
+
else: r, fl = self.alu.logic("XOR", a, b, size)
|
| 126 |
+
self.setf(fl)
|
| 127 |
+
return None if op == 7 else r
|
| 128 |
+
|
| 129 |
+
def incdec(self, v, size, dec):
|
| 130 |
+
r, fl = self.alu.addsub(dec, v, 1, size)
|
| 131 |
+
fl.pop("CF") # INC/DEC preserve CF
|
| 132 |
+
self.setf(fl)
|
| 133 |
+
return r
|
| 134 |
+
|
| 135 |
+
# ---------------- shifts/rotates: 1-bit verified slices, k times ----------------
|
| 136 |
+
def shift(self, kind, v, count, size):
|
| 137 |
+
count &= 31
|
| 138 |
+
if count == 0:
|
| 139 |
+
return v
|
| 140 |
+
msb = 1 << (size - 1)
|
| 141 |
+
orig_msb = 1 if v & msb else 0
|
| 142 |
+
for i in range(count):
|
| 143 |
+
if kind == 4: # SHL
|
| 144 |
+
v, c = self.alu.shl1(v, size)
|
| 145 |
+
elif kind == 5: # SHR
|
| 146 |
+
v, c = self.alu.shr1(v, size)
|
| 147 |
+
elif kind == 7: # SAR
|
| 148 |
+
v, c = self.alu.shr1(v, size, top=1 if v & msb else 0)
|
| 149 |
+
elif kind == 0: # ROL
|
| 150 |
+
v, c = self.alu.shl1(v, size)
|
| 151 |
+
if c: v |= 1
|
| 152 |
+
elif kind == 1: # ROR
|
| 153 |
+
v, c = self.alu.shr1(v, size, top=v & 1)
|
| 154 |
+
elif kind == 2: # RCL
|
| 155 |
+
v, nc = self.alu.shl1(v, size)
|
| 156 |
+
if self.f[CF]: v |= 1
|
| 157 |
+
c = nc
|
| 158 |
+
else: # RCR
|
| 159 |
+
v, c = self.alu.shr1(v, size, top=self.f[CF])
|
| 160 |
+
self.f[CF] = c
|
| 161 |
+
if kind in (4, 5, 7):
|
| 162 |
+
SF_, ZF_, PF_ = self.alu.flags_of(v, size)
|
| 163 |
+
self.f[SF], self.f[ZF], self.f[PF] = SF_, ZF_, PF_
|
| 164 |
+
if count == 1:
|
| 165 |
+
if kind == 4: self.f[OF] = ((v >> (size - 1)) & 1) ^ self.f[CF]
|
| 166 |
+
elif kind == 5: self.f[OF] = orig_msb
|
| 167 |
+
elif kind == 7: self.f[OF] = 0
|
| 168 |
+
elif kind == 0: self.f[OF] = ((v >> (size - 1)) & 1) ^ self.f[CF]
|
| 169 |
+
elif kind == 1: self.f[OF] = ((v >> (size - 1)) ^ (v >> (size - 2))) & 1
|
| 170 |
+
return v
|
| 171 |
+
|
| 172 |
+
def shd(self, left, v, fill, count, size):
|
| 173 |
+
"""SHLD/SHRD: double shift through verified 1-bit slices over 2*size bits."""
|
| 174 |
+
count &= 31
|
| 175 |
+
if count == 0:
|
| 176 |
+
return v
|
| 177 |
+
if left:
|
| 178 |
+
wide = (v << size) | fill
|
| 179 |
+
for _ in range(count):
|
| 180 |
+
wide, c = ALU.shl1(self.alu, wide, 2 * size) if False else self.alu.shl1(wide, 2 * size)
|
| 181 |
+
self.f[CF] = c
|
| 182 |
+
v = (wide >> size) & ((1 << size) - 1)
|
| 183 |
+
else:
|
| 184 |
+
wide = (fill << size) | v
|
| 185 |
+
for _ in range(count):
|
| 186 |
+
wide, c = self.alu.shr1(wide, 2 * size)
|
| 187 |
+
self.f[CF] = c
|
| 188 |
+
v = wide & ((1 << size) - 1)
|
| 189 |
+
SF_, ZF_, PF_ = self.alu.flags_of(v, size)
|
| 190 |
+
self.f[SF], self.f[ZF], self.f[PF] = SF_, ZF_, PF_
|
| 191 |
+
return v
|
| 192 |
+
|
| 193 |
+
# ---------------- stack (wiring over mem) ----------------
|
| 194 |
+
def push(self, v):
|
| 195 |
+
self.r[ESP] = (self.r[ESP] - 4) & M32
|
| 196 |
+
self.wr(self.r[ESP], v, 4)
|
| 197 |
+
def pop(self):
|
| 198 |
+
v = self.rd(self.r[ESP], 4)
|
| 199 |
+
self.r[ESP] = (self.r[ESP] + 4) & M32
|
| 200 |
+
return v
|
| 201 |
+
|
| 202 |
+
# ---------------- one instruction ----------------
|
| 203 |
+
def step(self):
|
| 204 |
+
self.instr_count += 1
|
| 205 |
+
start = self.eip
|
| 206 |
+
osize = 32; seg = 0
|
| 207 |
+
# prefixes (decode unit)
|
| 208 |
+
rep = None
|
| 209 |
+
while True:
|
| 210 |
+
b = self.rd(self.eip, 1)
|
| 211 |
+
isp, cls = self.u.prefix(b)
|
| 212 |
+
if not isp:
|
| 213 |
+
break
|
| 214 |
+
self.eip = (self.eip + 1) & M32
|
| 215 |
+
if cls == 1: osize = 16
|
| 216 |
+
elif cls == 4: rep = "ne"
|
| 217 |
+
elif cls == 5: rep = "e"
|
| 218 |
+
elif cls == 6 and b == 0x65: seg = self.gs_base
|
| 219 |
+
op = self.fetch()
|
| 220 |
+
try:
|
| 221 |
+
self.exec_op(op, osize, seg, rep)
|
| 222 |
+
except CPUError:
|
| 223 |
+
self.eip = start
|
| 224 |
+
raise
|
| 225 |
+
|
| 226 |
+
def exec_op(self, op, osize, seg, rep):
|
| 227 |
+
S = osize
|
| 228 |
+
# ---- ALU group 00-3D ----
|
| 229 |
+
if op < 0x40 and (op & 7) < 6:
|
| 230 |
+
kind = op >> 3
|
| 231 |
+
form = op & 7
|
| 232 |
+
if form in (0, 1): # r/m, r
|
| 233 |
+
size = 8 if form == 0 else S
|
| 234 |
+
reg, loc = self.modrm(size, seg)
|
| 235 |
+
r = self.alu_op(kind, self.get_rm(loc, size), self.get_r(reg, size), size)
|
| 236 |
+
if r is not None: self.set_rm(loc, r, size)
|
| 237 |
+
elif form in (2, 3): # r, r/m
|
| 238 |
+
size = 8 if form == 2 else S
|
| 239 |
+
reg, loc = self.modrm(size, seg)
|
| 240 |
+
r = self.alu_op(kind, self.get_r(reg, size), self.get_rm(loc, size), size)
|
| 241 |
+
if r is not None: self.set_r(reg, r, size)
|
| 242 |
+
else: # AL/eAX, imm
|
| 243 |
+
size = 8 if form == 4 else S
|
| 244 |
+
imm = self.fetch(size // 8)
|
| 245 |
+
r = self.alu_op(kind, self.get_r(EAX, size), imm, size)
|
| 246 |
+
if r is not None: self.set_r(EAX, r, size)
|
| 247 |
+
return
|
| 248 |
+
# ---- INC/DEC r32 40-4F ----
|
| 249 |
+
if 0x40 <= op <= 0x4F:
|
| 250 |
+
i = op & 7
|
| 251 |
+
self.set_r(i, self.incdec(self.get_r(i, S), S, (op >> 3) & 1), S)
|
| 252 |
+
return
|
| 253 |
+
# ---- PUSH/POP r32 50-5F ----
|
| 254 |
+
if 0x50 <= op <= 0x57: self.push(self.get_r(op & 7, 32)); return
|
| 255 |
+
if 0x58 <= op <= 0x5F: self.set_r(op & 7, self.pop(), 32); return
|
| 256 |
+
if op == 0x68: self.push(self.fetch(4)); return
|
| 257 |
+
if op == 0x6A: self.push(sx8(self.fetch()) & M32); return
|
| 258 |
+
if op == 0x8F:
|
| 259 |
+
_, loc = self.modrm(S, seg); self.set_rm(loc, self.pop(), 32); return
|
| 260 |
+
# ---- IMUL r,r/m,imm (69/6B) ----
|
| 261 |
+
if op in (0x69, 0x6B):
|
| 262 |
+
reg, loc = self.modrm(S, seg)
|
| 263 |
+
a = sx32(self.get_rm(loc, S)) if S == 32 else sx16(self.get_rm(loc, S))
|
| 264 |
+
b = sx8(self.fetch()) if op == 0x6B else (
|
| 265 |
+
sx32(self.fetch(4)) if S == 32 else sx16(self.fetch(2)))
|
| 266 |
+
full = self.alu.mul(abs(a), abs(b), S)
|
| 267 |
+
if (a < 0) != (b < 0): full = (-full) & ((1 << (2 * S)) - 1)
|
| 268 |
+
res = full & ((1 << S) - 1)
|
| 269 |
+
sres = sx32(res) if S == 32 else sx16(res)
|
| 270 |
+
ovf = int(sx32(full if S == 32 else 0) != a * b) if S == 32 else 0
|
| 271 |
+
ovf = int(((full ^ ((res >> (S-1)) * ((1 << (2*S)) - (1 << S)))) >> S) != 0)
|
| 272 |
+
self.f[CF] = self.f[OF] = ovf
|
| 273 |
+
self.set_r(reg, res, S)
|
| 274 |
+
return
|
| 275 |
+
# ---- Jcc short 70-7F ----
|
| 276 |
+
if 0x70 <= op <= 0x7F:
|
| 277 |
+
d = sx8(self.fetch())
|
| 278 |
+
if self.cond(op & 0xF): self.eip = (self.eip + d) & M32
|
| 279 |
+
return
|
| 280 |
+
# ---- group 80/81/83 ----
|
| 281 |
+
if op in (0x80, 0x81, 0x83):
|
| 282 |
+
size = 8 if op == 0x80 else S
|
| 283 |
+
reg, loc = self.modrm(size, seg)
|
| 284 |
+
imm = (self.fetch() if op == 0x80 else
|
| 285 |
+
sx8(self.fetch()) & ((1 << size) - 1) if op == 0x83 else
|
| 286 |
+
self.fetch(size // 8))
|
| 287 |
+
r = self.alu_op(reg, self.get_rm(loc, size), imm, size)
|
| 288 |
+
if r is not None: self.set_rm(loc, r, size)
|
| 289 |
+
return
|
| 290 |
+
# ---- TEST 84/85 ----
|
| 291 |
+
if op in (0x84, 0x85):
|
| 292 |
+
size = 8 if op == 0x84 else S
|
| 293 |
+
reg, loc = self.modrm(size, seg)
|
| 294 |
+
self.alu_op(4, self.get_rm(loc, size), self.get_r(reg, size), size)
|
| 295 |
+
return
|
| 296 |
+
# ---- XCHG 86/87 ----
|
| 297 |
+
if op in (0x86, 0x87):
|
| 298 |
+
size = 8 if op == 0x86 else S
|
| 299 |
+
reg, loc = self.modrm(size, seg)
|
| 300 |
+
a, b = self.get_rm(loc, size), self.get_r(reg, size)
|
| 301 |
+
self.set_rm(loc, b, size); self.set_r(reg, a, size)
|
| 302 |
+
return
|
| 303 |
+
# ---- MOV 88-8B ----
|
| 304 |
+
if 0x88 <= op <= 0x8B:
|
| 305 |
+
size = 8 if op in (0x88, 0x8A) else S
|
| 306 |
+
reg, loc = self.modrm(size, seg)
|
| 307 |
+
if op <= 0x89: self.set_rm(loc, self.get_r(reg, size), size)
|
| 308 |
+
else: self.set_r(reg, self.get_rm(loc, size), size)
|
| 309 |
+
return
|
| 310 |
+
if op in (0x8C, 0x8E): # MOV sreg (flat segments: no-op)
|
| 311 |
+
_, loc = self.modrm(16, seg)
|
| 312 |
+
if op == 0x8C: self.set_rm(loc, 0, 16)
|
| 313 |
+
return
|
| 314 |
+
if op == 0x8D: # LEA
|
| 315 |
+
reg, loc = self.modrm(S, seg)
|
| 316 |
+
assert loc[0] == "m"
|
| 317 |
+
self.set_r(reg, loc[1], S)
|
| 318 |
+
return
|
| 319 |
+
if op == 0x90: return # NOP
|
| 320 |
+
if 0x91 <= op <= 0x97: # XCHG eAX, r
|
| 321 |
+
i = op & 7
|
| 322 |
+
a = self.get_r(EAX, S); self.set_r(EAX, self.get_r(i, S), S); self.set_r(i, a, S)
|
| 323 |
+
return
|
| 324 |
+
if op == 0x98: # CWDE
|
| 325 |
+
self.set_r(EAX, sx16(self.get_r(EAX, 16)) & M32, 32); return
|
| 326 |
+
if op == 0x99: # CDQ
|
| 327 |
+
self.set_r(EDX, (M32 if self.r[EAX] & 0x80000000 else 0), 32); return
|
| 328 |
+
if op == 0x9C: self.push(self.eflags()); return # PUSHFD
|
| 329 |
+
if op == 0x9D: self.set_eflags(self.pop()); return
|
| 330 |
+
# ---- MOV moffs A0-A3 ----
|
| 331 |
+
if op in (0xA0, 0xA1):
|
| 332 |
+
a = self.fetch(4) + seg
|
| 333 |
+
self.set_r(EAX, self.rd(a, 1 if op == 0xA0 else S // 8), 8 if op == 0xA0 else S)
|
| 334 |
+
return
|
| 335 |
+
if op in (0xA2, 0xA3):
|
| 336 |
+
a = self.fetch(4) + seg
|
| 337 |
+
self.wr(a, self.get_r(EAX, 8 if op == 0xA2 else S), 1 if op == 0xA2 else S // 8)
|
| 338 |
+
return
|
| 339 |
+
# ---- string ops ----
|
| 340 |
+
if op in (0xA4, 0xA5, 0xAA, 0xAB, 0xA6, 0xA7, 0xAE, 0xAF, 0xAC, 0xAD):
|
| 341 |
+
self.string_op(op, S, rep)
|
| 342 |
+
return
|
| 343 |
+
if op in (0xA8, 0xA9): # TEST AL/eAX, imm
|
| 344 |
+
size = 8 if op == 0xA8 else S
|
| 345 |
+
self.alu_op(4, self.get_r(EAX, size), self.fetch(size // 8), size)
|
| 346 |
+
return
|
| 347 |
+
# ---- MOV r, imm B0-BF ----
|
| 348 |
+
if 0xB0 <= op <= 0xB7: self.set_r(op & 7, self.fetch(), 8); return
|
| 349 |
+
if 0xB8 <= op <= 0xBF: self.set_r(op & 7, self.fetch(S // 8), S); return
|
| 350 |
+
# ---- shift groups C0/C1/D0/D1/D3 ----
|
| 351 |
+
if op in (0xC0, 0xC1, 0xD0, 0xD1, 0xD2, 0xD3):
|
| 352 |
+
size = 8 if op in (0xC0, 0xD0, 0xD2) else S
|
| 353 |
+
kind, loc = self.modrm(size, seg)
|
| 354 |
+
cnt = (self.fetch() if op in (0xC0, 0xC1) else
|
| 355 |
+
1 if op in (0xD0, 0xD1) else self.get_r(ECX, 8))
|
| 356 |
+
self.set_rm(loc, self.shift(kind, self.get_rm(loc, size), cnt, size), size)
|
| 357 |
+
return
|
| 358 |
+
if op == 0xC2:
|
| 359 |
+
n = self.fetch(2); self.eip = self.pop()
|
| 360 |
+
self.r[ESP] = (self.r[ESP] + n) & M32; return
|
| 361 |
+
if op == 0xC3: self.eip = self.pop(); return
|
| 362 |
+
if op in (0xC6, 0xC7): # MOV r/m, imm
|
| 363 |
+
size = 8 if op == 0xC6 else S
|
| 364 |
+
_, loc = self.modrm(size, seg)
|
| 365 |
+
self.set_rm(loc, self.fetch(size // 8), size)
|
| 366 |
+
return
|
| 367 |
+
if op == 0xC9: # LEAVE
|
| 368 |
+
self.r[ESP] = self.r[EBP]; self.r[EBP] = self.pop(); return
|
| 369 |
+
if op == 0xCD: # INT imm
|
| 370 |
+
n = self.fetch()
|
| 371 |
+
if n == 0x80 and self.syscall:
|
| 372 |
+
self.syscall(self)
|
| 373 |
+
return
|
| 374 |
+
raise CPUError(f"INT {n:#x}")
|
| 375 |
+
# ---- group F6/F7: TEST/NOT/NEG/MUL/IMUL/DIV/IDIV ----
|
| 376 |
+
if op in (0xF6, 0xF7):
|
| 377 |
+
size = 8 if op == 0xF6 else S
|
| 378 |
+
kind, loc = self.modrm(size, seg)
|
| 379 |
+
v = self.get_rm(loc, size)
|
| 380 |
+
if kind in (0, 1):
|
| 381 |
+
self.alu_op(4, v, self.fetch(size // 8), size)
|
| 382 |
+
elif kind == 2:
|
| 383 |
+
self.set_rm(loc, self.alu.not_(v, size), size)
|
| 384 |
+
elif kind == 3: # NEG
|
| 385 |
+
r, fl = self.alu.addsub(1, 0, v, size)
|
| 386 |
+
self.setf(fl); self.set_rm(loc, r, size)
|
| 387 |
+
else:
|
| 388 |
+
self.muldiv(kind, v, size)
|
| 389 |
+
return
|
| 390 |
+
if op == 0xF8: self.f[CF] = 0; return # CLC
|
| 391 |
+
if op == 0xF9: self.f[CF] = 1; return # STC
|
| 392 |
+
if op == 0xFC: self.f[DF] = 0; return # CLD
|
| 393 |
+
if op == 0xFD: self.f[DF] = 1; return # STD
|
| 394 |
+
# ---- group FE/FF ----
|
| 395 |
+
if op in (0xFE, 0xFF):
|
| 396 |
+
size = 8 if op == 0xFE else S
|
| 397 |
+
kind, loc = self.modrm(size, seg)
|
| 398 |
+
if kind in (0, 1):
|
| 399 |
+
self.set_rm(loc, self.incdec(self.get_rm(loc, size), size, kind), size)
|
| 400 |
+
elif kind == 2: self.push(self.eip); self.eip = self.get_rm(loc, 32)
|
| 401 |
+
elif kind == 4: self.eip = self.get_rm(loc, 32)
|
| 402 |
+
elif kind == 6: self.push(self.get_rm(loc, 32))
|
| 403 |
+
else: raise CPUError(f"FF/{kind}")
|
| 404 |
+
return
|
| 405 |
+
if op == 0xE8: # CALL rel32
|
| 406 |
+
d = sx32(self.fetch(4)); self.push(self.eip)
|
| 407 |
+
self.eip = (self.eip + d) & M32; return
|
| 408 |
+
if op == 0xE9:
|
| 409 |
+
d = sx32(self.fetch(4)); self.eip = (self.eip + d) & M32; return
|
| 410 |
+
if op == 0xEB:
|
| 411 |
+
d = sx8(self.fetch()); self.eip = (self.eip + d) & M32; return
|
| 412 |
+
# ---- 0F escape ----
|
| 413 |
+
if op == 0x0F:
|
| 414 |
+
self.exec_0f(self.fetch(), S, seg)
|
| 415 |
+
return
|
| 416 |
+
raise CPUError(f"opcode {op:#04x}")
|
| 417 |
+
|
| 418 |
+
def exec_0f(self, op, S, seg):
|
| 419 |
+
if 0x80 <= op <= 0x8F: # Jcc rel32
|
| 420 |
+
d = sx32(self.fetch(4))
|
| 421 |
+
if self.cond(op & 0xF): self.eip = (self.eip + d) & M32
|
| 422 |
+
return
|
| 423 |
+
if 0x90 <= op <= 0x9F: # SETcc
|
| 424 |
+
_, loc = self.modrm(8, seg)
|
| 425 |
+
self.set_rm(loc, 1 if self.cond(op & 0xF) else 0, 8)
|
| 426 |
+
return
|
| 427 |
+
if op in (0xB6, 0xB7, 0xBE, 0xBF): # MOVZX/MOVSX
|
| 428 |
+
ssize = 8 if op in (0xB6, 0xBE) else 16
|
| 429 |
+
reg, loc = self.modrm(ssize, seg)
|
| 430 |
+
v = self.get_rm(loc, ssize)
|
| 431 |
+
if op >= 0xBE:
|
| 432 |
+
v = (sx8(v) if ssize == 8 else sx16(v)) & M32
|
| 433 |
+
self.set_r(reg, v, S)
|
| 434 |
+
return
|
| 435 |
+
if op == 0xAF: # IMUL r, r/m
|
| 436 |
+
reg, loc = self.modrm(S, seg)
|
| 437 |
+
a = sx32(self.get_r(reg, S)); b = sx32(self.get_rm(loc, S))
|
| 438 |
+
full = self.alu.mul(abs(a), abs(b), S)
|
| 439 |
+
if (a < 0) != (b < 0): full = (-full) & ((1 << (2 * S)) - 1)
|
| 440 |
+
res = full & M32
|
| 441 |
+
self.f[CF] = self.f[OF] = int(sx32(res) != a * b)
|
| 442 |
+
self.set_r(reg, res, S)
|
| 443 |
+
return
|
| 444 |
+
if op in (0xA4, 0xA5): # SHLD
|
| 445 |
+
reg, loc = self.modrm(S, seg)
|
| 446 |
+
cnt = self.fetch() if op == 0xA4 else self.get_r(ECX, 8)
|
| 447 |
+
self.set_rm(loc, self.shd(1, self.get_rm(loc, S), self.get_r(reg, S), cnt, S), S)
|
| 448 |
+
return
|
| 449 |
+
if op in (0xAC, 0xAD): # SHRD
|
| 450 |
+
reg, loc = self.modrm(S, seg)
|
| 451 |
+
cnt = self.fetch() if op == 0xAC else self.get_r(ECX, 8)
|
| 452 |
+
self.set_rm(loc, self.shd(0, self.get_rm(loc, S), self.get_r(reg, S), cnt, S), S)
|
| 453 |
+
return
|
| 454 |
+
if op == 0xBC or op == 0xBD: # BSF/BSR
|
| 455 |
+
reg, loc = self.modrm(S, seg)
|
| 456 |
+
v = self.get_rm(loc, S)
|
| 457 |
+
self.f[ZF] = int(v == 0)
|
| 458 |
+
if v:
|
| 459 |
+
idx = (v & -v).bit_length() - 1 if op == 0xBC else v.bit_length() - 1
|
| 460 |
+
self.set_r(reg, idx, S)
|
| 461 |
+
return
|
| 462 |
+
if op == 0xA3 or op == 0xAB or op == 0xB3 or op == 0xBA: # BT/BTS/BTR (+grp8)
|
| 463 |
+
if op == 0xBA:
|
| 464 |
+
kind, loc = self.modrm(S, seg)
|
| 465 |
+
bit = self.fetch() & (S - 1)
|
| 466 |
+
else:
|
| 467 |
+
reg, loc = self.modrm(S, seg)
|
| 468 |
+
bit = self.get_r(reg, S) & (S - 1)
|
| 469 |
+
kind = {0xA3: 4, 0xAB: 5, 0xB3: 6}[op]
|
| 470 |
+
v = self.get_rm(loc, S)
|
| 471 |
+
self.f[CF] = (v >> bit) & 1
|
| 472 |
+
if kind == 5: self.set_rm(loc, v | (1 << bit), S)
|
| 473 |
+
elif kind == 6: self.set_rm(loc, v & ~(1 << bit), S)
|
| 474 |
+
return
|
| 475 |
+
if op in (0xB0, 0xB1): # CMPXCHG
|
| 476 |
+
size = 8 if op == 0xB0 else S
|
| 477 |
+
reg, loc = self.modrm(size, seg)
|
| 478 |
+
dst = self.get_rm(loc, size); acc = self.get_r(EAX, size)
|
| 479 |
+
self.alu_op(7, acc, dst, size) # CMP semantics for flags
|
| 480 |
+
if self.f[ZF]:
|
| 481 |
+
self.set_rm(loc, self.get_r(reg, size), size)
|
| 482 |
+
else:
|
| 483 |
+
self.set_r(EAX, dst, size)
|
| 484 |
+
return
|
| 485 |
+
if op in (0xC0, 0xC1): # XADD
|
| 486 |
+
size = 8 if op == 0xC0 else S
|
| 487 |
+
reg, loc = self.modrm(size, seg)
|
| 488 |
+
a, b = self.get_rm(loc, size), self.get_r(reg, size)
|
| 489 |
+
r = self.alu_op(0, a, b, size)
|
| 490 |
+
self.set_r(reg, a, size); self.set_rm(loc, r, size)
|
| 491 |
+
return
|
| 492 |
+
if 0xC8 <= op <= 0xCF: # BSWAP
|
| 493 |
+
i = op & 7; v = self.r[i]
|
| 494 |
+
self.r[i] = int.from_bytes(v.to_bytes(4, "little"), "big")
|
| 495 |
+
return
|
| 496 |
+
if op == 0xA2: # CPUID
|
| 497 |
+
self.r[EAX] = self.r[EBX] = self.r[ECX] = self.r[EDX] = 0
|
| 498 |
+
return
|
| 499 |
+
if 0x40 <= op <= 0x4F: # CMOVcc
|
| 500 |
+
reg, loc = self.modrm(S, seg)
|
| 501 |
+
v = self.get_rm(loc, S)
|
| 502 |
+
if self.cond(op & 0xF): self.set_r(reg, v, S)
|
| 503 |
+
return
|
| 504 |
+
raise CPUError(f"0F {op:#04x}")
|
| 505 |
+
|
| 506 |
+
# ---------------- MUL/IMUL/DIV/IDIV (edx:eax forms) ----------------
|
| 507 |
+
def muldiv(self, kind, v, size):
|
| 508 |
+
a = self.get_r(EAX, size)
|
| 509 |
+
if kind == 4: # MUL
|
| 510 |
+
full = self.alu.mul(a, v, size)
|
| 511 |
+
lo = full & ((1 << size) - 1); hi = full >> size
|
| 512 |
+
self.set_r(EAX, lo, size)
|
| 513 |
+
if size == 8: self.set_r(EAX, full, 16)
|
| 514 |
+
else: self.set_r(EDX, hi, size)
|
| 515 |
+
self.f[CF] = self.f[OF] = int(hi != 0)
|
| 516 |
+
elif kind == 5: # IMUL
|
| 517 |
+
sa = sx8(a) if size == 8 else sx16(a) if size == 16 else sx32(a)
|
| 518 |
+
sv = sx8(v) if size == 8 else sx16(v) if size == 16 else sx32(v)
|
| 519 |
+
full = self.alu.mul(abs(sa), abs(sv), size)
|
| 520 |
+
if (sa < 0) != (sv < 0): full = (-full) & ((1 << (2 * size)) - 1)
|
| 521 |
+
lo = full & ((1 << size) - 1); hi = full >> size
|
| 522 |
+
if size == 8: self.set_r(EAX, full, 16)
|
| 523 |
+
else: self.set_r(EAX, lo, size); self.set_r(EDX, hi, size)
|
| 524 |
+
sign_ext = ((1 << size) - 1) if lo >> (size - 1) else 0
|
| 525 |
+
self.f[CF] = self.f[OF] = int(hi != sign_ext)
|
| 526 |
+
else: # DIV/IDIV
|
| 527 |
+
if size == 8:
|
| 528 |
+
num = self.get_r(EAX, 16)
|
| 529 |
+
else:
|
| 530 |
+
num = (self.get_r(EDX, size) << size) | a
|
| 531 |
+
if kind == 6: # DIV
|
| 532 |
+
q, r = self.alu.divmod_(num, v, 2 * size)
|
| 533 |
+
if q >= (1 << size): raise CPUError("#DE overflow")
|
| 534 |
+
else: # IDIV
|
| 535 |
+
bits2 = 2 * size
|
| 536 |
+
snum = num - (1 << bits2) if num >> (bits2 - 1) else num
|
| 537 |
+
sden = sx8(v) if size == 8 else sx16(v) if size == 16 else sx32(v)
|
| 538 |
+
q0, r0 = self.alu.divmod_(abs(snum), abs(sden), bits2)
|
| 539 |
+
q = -q0 if (snum < 0) != (sden < 0) else q0
|
| 540 |
+
r = -r0 if snum < 0 else r0
|
| 541 |
+
if not (-(1 << (size - 1)) <= q < (1 << (size - 1))):
|
| 542 |
+
raise CPUError("#DE overflow")
|
| 543 |
+
q &= (1 << size) - 1; r &= (1 << size) - 1
|
| 544 |
+
if size == 8:
|
| 545 |
+
self.set_r(EAX, q, 8); self.set_r(4, r, 8) # AH
|
| 546 |
+
else:
|
| 547 |
+
self.set_r(EAX, q, size); self.set_r(EDX, r, size)
|
| 548 |
+
|
| 549 |
+
# ---------------- string ops ----------------
|
| 550 |
+
def string_op(self, op, S, rep):
|
| 551 |
+
size = 8 if op in (0xA4, 0xAA, 0xA6, 0xAE, 0xAC) else S
|
| 552 |
+
n = size // 8
|
| 553 |
+
step = -n if self.f[DF] else n
|
| 554 |
+
def one():
|
| 555 |
+
si, di = self.r[ESI], self.r[EDI]
|
| 556 |
+
if op in (0xA4, 0xA5): # MOVS
|
| 557 |
+
self.wr(di, self.rd(si, n), n)
|
| 558 |
+
self.r[ESI] = (si + step) & M32; self.r[EDI] = (di + step) & M32
|
| 559 |
+
elif op in (0xAA, 0xAB): # STOS
|
| 560 |
+
self.wr(di, self.get_r(EAX, size), n)
|
| 561 |
+
self.r[EDI] = (di + step) & M32
|
| 562 |
+
elif op in (0xAC, 0xAD): # LODS
|
| 563 |
+
self.set_r(EAX, self.rd(si, n), size)
|
| 564 |
+
self.r[ESI] = (si + step) & M32
|
| 565 |
+
elif op in (0xA6, 0xA7): # CMPS
|
| 566 |
+
self.alu_op(7, self.rd(si, n), self.rd(di, n), size)
|
| 567 |
+
self.r[ESI] = (si + step) & M32; self.r[EDI] = (di + step) & M32
|
| 568 |
+
else: # SCAS
|
| 569 |
+
self.alu_op(7, self.get_r(EAX, size), self.rd(di, n), size)
|
| 570 |
+
self.r[EDI] = (di + step) & M32
|
| 571 |
+
if rep is None:
|
| 572 |
+
one(); return
|
| 573 |
+
cmp_op = op in (0xA6, 0xA7, 0xAE, 0xAF)
|
| 574 |
+
while self.r[ECX]:
|
| 575 |
+
one()
|
| 576 |
+
self.r[ECX] = (self.r[ECX] - 1) & M32
|
| 577 |
+
if cmp_op:
|
| 578 |
+
if rep == "e" and not self.f[ZF]: break
|
| 579 |
+
if rep == "ne" and self.f[ZF]: break
|
x86_linux.py
ADDED
|
@@ -0,0 +1,241 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
x86_linux.py -- minimal Linux userland around the x86 core: static ELF loader,
|
| 3 |
+
stack/auxv setup, and the i386 int-0x80 syscall shim. This is the orchestrator
|
| 4 |
+
shell (all wiring) -- the analogue of the GB console around the SM83.
|
| 5 |
+
|
| 6 |
+
Just enough POSIX for a static musl binary: write/writev for output, read/open/
|
| 7 |
+
close/lseek over an in-memory FS, brk + anonymous mmap2 for malloc, the i386
|
| 8 |
+
TLS dance (set_thread_area + %gs), clock_gettime, and exit_group.
|
| 9 |
+
|
| 10 |
+
Special fds let a host front-end talk to the program without a kernel:
|
| 11 |
+
fd 100: DG_DrawFrame writes the framebuffer here (host captures frames)
|
| 12 |
+
fd 101: key events are read from here (host-provided queue)
|
| 13 |
+
"""
|
| 14 |
+
import struct, time
|
| 15 |
+
from x86_core import X86, CPUError, EAX, EBX, ECX, EDX, ESI, EDI, EBP, ESP, M32
|
| 16 |
+
|
| 17 |
+
MEM_SIZE = 0x08000000 # 128 MB flat
|
| 18 |
+
STACK_TOP = 0x07FF0000
|
| 19 |
+
MMAP_BASE = 0x04000000
|
| 20 |
+
|
| 21 |
+
ENOSYS, EBADF, ENOENT, ENOTTY, EINVAL = 38, 9, 2, 25, 22
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
class Linux386:
|
| 25 |
+
def __init__(self, elf_bytes, argv=("prog",), fs=None, trace_sys=False):
|
| 26 |
+
self.mem = bytearray(MEM_SIZE)
|
| 27 |
+
self.cpu = X86(self.mem)
|
| 28 |
+
self.cpu.syscall = self.do_syscall
|
| 29 |
+
self.fs = dict(fs or {}) # path -> bytes
|
| 30 |
+
self.fds = {0: None, 1: None, 2: None}
|
| 31 |
+
self.next_fd = 3
|
| 32 |
+
self.stdout = bytearray()
|
| 33 |
+
self.frames = [] # fd 100 writes land here
|
| 34 |
+
self.keys = [] # fd 101 reads consume this
|
| 35 |
+
self.brk = 0
|
| 36 |
+
self.mmap_ptr = MMAP_BASE
|
| 37 |
+
self.trace = trace_sys
|
| 38 |
+
self.tls_base = 0
|
| 39 |
+
self.clock_ns = 0
|
| 40 |
+
self.load_elf(elf_bytes, argv)
|
| 41 |
+
|
| 42 |
+
# ---------------- ELF ----------------
|
| 43 |
+
def load_elf(self, b, argv):
|
| 44 |
+
assert b[:4] == b"\x7fELF" and b[4] == 1, "need ELF32"
|
| 45 |
+
e_entry, e_phoff = struct.unpack_from("<II", b, 24)
|
| 46 |
+
e_phentsize, e_phnum = struct.unpack_from("<HH", b, 42)
|
| 47 |
+
phdr_vaddr = 0
|
| 48 |
+
top = 0
|
| 49 |
+
for i in range(e_phnum):
|
| 50 |
+
off = e_phoff + i * e_phentsize
|
| 51 |
+
p_type, p_offset, p_vaddr, _, p_filesz, p_memsz, p_flags, _ = \
|
| 52 |
+
struct.unpack_from("<8I", b, off)
|
| 53 |
+
if p_type == 1: # PT_LOAD
|
| 54 |
+
self.mem[p_vaddr:p_vaddr + p_filesz] = b[p_offset:p_offset + p_filesz]
|
| 55 |
+
top = max(top, p_vaddr + p_memsz)
|
| 56 |
+
if p_offset <= e_phoff < p_offset + p_filesz:
|
| 57 |
+
phdr_vaddr = p_vaddr + (e_phoff - p_offset)
|
| 58 |
+
elif p_type == 6: # PT_PHDR
|
| 59 |
+
phdr_vaddr = p_vaddr
|
| 60 |
+
self.brk = (top + 0xFFF) & ~0xFFF
|
| 61 |
+
|
| 62 |
+
# ---- stack: argc argv envp auxv ----
|
| 63 |
+
sp = STACK_TOP
|
| 64 |
+
def push_bytes(data):
|
| 65 |
+
nonlocal sp
|
| 66 |
+
sp -= len(data); self.mem[sp:sp + len(data)] = data
|
| 67 |
+
return sp
|
| 68 |
+
arg_ptrs = [push_bytes(a.encode() + b"\0") for a in argv]
|
| 69 |
+
rnd = push_bytes(bytes(range(16)))
|
| 70 |
+
sp &= ~0xF
|
| 71 |
+
aux = [(3, phdr_vaddr), (4, e_phentsize), (5, e_phnum), (6, 4096),
|
| 72 |
+
(9, e_entry), (11, 1000), (12, 1000), (13, 1000), (14, 1000),
|
| 73 |
+
(16, 0), (17, 100), (23, 0), (25, rnd), (0, 0)]
|
| 74 |
+
blob = b""
|
| 75 |
+
for k, v in aux:
|
| 76 |
+
blob += struct.pack("<II", k, v)
|
| 77 |
+
blob = struct.pack("<I", len(argv)) \
|
| 78 |
+
+ b"".join(struct.pack("<I", p) for p in arg_ptrs) + b"\0\0\0\0" \
|
| 79 |
+
+ b"\0\0\0\0" + blob # empty envp
|
| 80 |
+
sp -= len(blob); sp &= ~0xF
|
| 81 |
+
self.mem[sp:sp + len(blob)] = blob
|
| 82 |
+
self.cpu.r[ESP] = sp
|
| 83 |
+
self.cpu.eip = e_entry
|
| 84 |
+
|
| 85 |
+
# ---------------- syscalls ----------------
|
| 86 |
+
def cstr(self, a):
|
| 87 |
+
e = self.mem.index(b"\0", a)
|
| 88 |
+
return self.mem[a:e].decode("latin1")
|
| 89 |
+
|
| 90 |
+
def do_syscall(self, cpu):
|
| 91 |
+
n = cpu.r[EAX]
|
| 92 |
+
a1, a2, a3 = cpu.r[EBX], cpu.r[ECX], cpu.r[EDX]
|
| 93 |
+
r = self.sys(n, a1, a2, a3, cpu)
|
| 94 |
+
if self.trace:
|
| 95 |
+
print(f" sys{n}({a1:#x},{a2:#x},{a3:#x}) = {r:#x}" if r >= 0
|
| 96 |
+
else f" sys{n} = -{-r}")
|
| 97 |
+
cpu.r[EAX] = r & M32
|
| 98 |
+
|
| 99 |
+
def sys(self, n, a1, a2, a3, cpu):
|
| 100 |
+
if n in (1, 252): # exit / exit_group
|
| 101 |
+
cpu.exited = a1
|
| 102 |
+
return 0
|
| 103 |
+
if n == 3 or n == 145: # read / readv
|
| 104 |
+
return self.do_read(n, a1, a2, a3)
|
| 105 |
+
if n == 4 or n == 146: # write / writev
|
| 106 |
+
return self.do_write(n, a1, a2, a3)
|
| 107 |
+
if n == 5: # open
|
| 108 |
+
path = self.cstr(a1)
|
| 109 |
+
if path not in self.fs:
|
| 110 |
+
if a2 & 0x40: # O_CREAT
|
| 111 |
+
self.fs[path] = b""
|
| 112 |
+
else:
|
| 113 |
+
return -ENOENT
|
| 114 |
+
if a2 & 0x200: # O_TRUNC
|
| 115 |
+
self.fs[path] = b""
|
| 116 |
+
self.fds[self.next_fd] = [path, 0]
|
| 117 |
+
self.next_fd += 1
|
| 118 |
+
return self.next_fd - 1
|
| 119 |
+
if n == 6: self.fds.pop(a1, None); return 0 # close
|
| 120 |
+
if n == 19 or n == 140: # lseek / _llseek
|
| 121 |
+
f = self.fds.get(a1)
|
| 122 |
+
if not f: return -EBADF
|
| 123 |
+
if n == 19:
|
| 124 |
+
off, whence = (a2 if a2 < 0x80000000 else a2 - (1 << 32)), a3
|
| 125 |
+
else:
|
| 126 |
+
off = (a2 << 32) | a3
|
| 127 |
+
whence = self.cpu.r[EDI]
|
| 128 |
+
size = len(self.fs[f[0]])
|
| 129 |
+
f[1] = off if whence == 0 else f[1] + off if whence == 1 else size + off
|
| 130 |
+
if n == 140:
|
| 131 |
+
self.cpu.wr(self.cpu.r[ESI], f[1], 8)
|
| 132 |
+
return 0
|
| 133 |
+
return f[1]
|
| 134 |
+
if n == 45: # brk
|
| 135 |
+
if a1: self.brk = a1
|
| 136 |
+
return self.brk
|
| 137 |
+
if n == 54: return -ENOTTY # ioctl
|
| 138 |
+
if n == 90 or n == 192: # mmap / mmap2
|
| 139 |
+
length = a2
|
| 140 |
+
ptr = self.mmap_ptr
|
| 141 |
+
self.mmap_ptr = (self.mmap_ptr + length + 0xFFF) & ~0xFFF
|
| 142 |
+
return ptr
|
| 143 |
+
if n == 91: return 0 # munmap
|
| 144 |
+
if n == 125: return 0 # mprotect
|
| 145 |
+
if n in (174, 175, 126): return 0 # signals: ignore
|
| 146 |
+
if n == 197 or n == 195: # fstat64/stat64 -> zeros
|
| 147 |
+
self.mem[a2:a2 + 96] = bytes(96)
|
| 148 |
+
return 0
|
| 149 |
+
if n == 243: # set_thread_area
|
| 150 |
+
entry = self.cpu.rd(a1, 4)
|
| 151 |
+
base = self.cpu.rd(a1 + 4, 4)
|
| 152 |
+
self.cpu.gs_base = base
|
| 153 |
+
if entry == M32:
|
| 154 |
+
self.cpu.wr(a1, 6, 4) # assign entry 6
|
| 155 |
+
return 0
|
| 156 |
+
if n == 258: return 1 # set_tid_address -> tid
|
| 157 |
+
if n == 224: return 1 # gettid
|
| 158 |
+
if n == 20: return 1 # getpid
|
| 159 |
+
if n in (199, 200, 201, 202): return 1000 # get*id32
|
| 160 |
+
if n == 78: # gettimeofday (deterministic)
|
| 161 |
+
self.clock_ns += 1_000_000
|
| 162 |
+
self.cpu.wr(a1, self.clock_ns // 10**9, 4)
|
| 163 |
+
self.cpu.wr(a1 + 4, self.clock_ns % 10**9 // 1000, 4)
|
| 164 |
+
return 0
|
| 165 |
+
if n == 265 or n == 407: # clock_gettime(64), deterministic
|
| 166 |
+
self.clock_ns += 1_000_000
|
| 167 |
+
s, ns = self.clock_ns // 10**9, self.clock_ns % 10**9
|
| 168 |
+
if n == 265:
|
| 169 |
+
self.cpu.wr(a2, s, 4); self.cpu.wr(a2 + 4, ns, 4)
|
| 170 |
+
else:
|
| 171 |
+
self.cpu.wr(a2, s, 8); self.cpu.wr(a2 + 8, ns, 4)
|
| 172 |
+
return 0
|
| 173 |
+
if n == 162 or n == 158: # nanosleep / yield
|
| 174 |
+
return 0
|
| 175 |
+
if n == 122: # uname
|
| 176 |
+
self.mem[a1:a1 + 65 * 6] = bytes(65 * 6)
|
| 177 |
+
for i, s in enumerate([b"Linux", b"neural", b"5.0.0", b"#1", b"i686", b""]):
|
| 178 |
+
self.mem[a1 + 65 * i:a1 + 65 * i + len(s)] = s
|
| 179 |
+
return 0
|
| 180 |
+
if n == 33: return -ENOENT # access
|
| 181 |
+
if n == 221: return 0 # fcntl64
|
| 182 |
+
if n == 240: return 0 # futex
|
| 183 |
+
if n == 270: return 0 # tgkill
|
| 184 |
+
if n == 39: return 0 # mkdir
|
| 185 |
+
if n in (10, 38, 12): return 0 # unlink/rename/chdir
|
| 186 |
+
if n == 13: # time (deterministic)
|
| 187 |
+
self.clock_ns += 10**9
|
| 188 |
+
t = self.clock_ns // 10**9
|
| 189 |
+
if a1: self.cpu.wr(a1, t, 4)
|
| 190 |
+
return t
|
| 191 |
+
if n == 183: # getcwd
|
| 192 |
+
self.mem[a1:a1+2] = b"/" + bytes(1)
|
| 193 |
+
return 2
|
| 194 |
+
if n == 85: return -EINVAL # readlink
|
| 195 |
+
raise CPUError(f"unimplemented syscall {n}")
|
| 196 |
+
|
| 197 |
+
def do_read(self, n, fd, buf, cnt):
|
| 198 |
+
if fd == 101: # key queue
|
| 199 |
+
if not self.keys: return 0
|
| 200 |
+
data = bytes(self.keys[:cnt]); del self.keys[:cnt]
|
| 201 |
+
self.mem[buf:buf + len(data)] = data
|
| 202 |
+
return len(data)
|
| 203 |
+
f = self.fds.get(fd)
|
| 204 |
+
if not f: return -EBADF
|
| 205 |
+
if n == 145: # readv
|
| 206 |
+
total = 0
|
| 207 |
+
for i in range(cnt):
|
| 208 |
+
p = self.cpu.rd(buf + 8 * i, 4); l = self.cpu.rd(buf + 8 * i + 4, 4)
|
| 209 |
+
total += self.do_read(3, fd, p, l)
|
| 210 |
+
return total
|
| 211 |
+
data = self.fs[f[0]][f[1]:f[1] + cnt]
|
| 212 |
+
self.mem[buf:buf + len(data)] = data
|
| 213 |
+
f[1] += len(data)
|
| 214 |
+
return len(data)
|
| 215 |
+
|
| 216 |
+
def do_write(self, n, fd, buf, cnt):
|
| 217 |
+
if n == 146: # writev
|
| 218 |
+
total = 0
|
| 219 |
+
for i in range(cnt):
|
| 220 |
+
p = self.cpu.rd(buf + 8 * i, 4); l = self.cpu.rd(buf + 8 * i + 4, 4)
|
| 221 |
+
total += self.do_write(4, fd, p, l)
|
| 222 |
+
return total
|
| 223 |
+
data = bytes(self.mem[buf:buf + cnt])
|
| 224 |
+
if fd in (1, 2): self.stdout += data
|
| 225 |
+
elif fd == 100: self.frames.append(data)
|
| 226 |
+
elif fd in self.fds and self.fds[fd]:
|
| 227 |
+
path, off = self.fds[fd]
|
| 228 |
+
cur = self.fs[path]
|
| 229 |
+
if off > len(cur): cur = cur + bytes(off - len(cur))
|
| 230 |
+
self.fs[path] = cur[:off] + data + cur[off + len(data):]
|
| 231 |
+
self.fds[fd][1] = off + len(data)
|
| 232 |
+
return cnt
|
| 233 |
+
|
| 234 |
+
# ---------------- run ----------------
|
| 235 |
+
def run(self, max_instr=200_000_000):
|
| 236 |
+
cpu = self.cpu
|
| 237 |
+
for _ in range(max_instr):
|
| 238 |
+
cpu.step()
|
| 239 |
+
if cpu.exited is not None:
|
| 240 |
+
return cpu.exited
|
| 241 |
+
raise RuntimeError("instruction budget exhausted")
|
x86_units.py
ADDED
|
@@ -0,0 +1,342 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
x86_units.py -- the i386 integer datapath as verified-exact neural units.
|
| 3 |
+
|
| 4 |
+
Same discipline as the Game Boy bricks: every unit's golden semantics are
|
| 5 |
+
enumerated over its COMPLETE input domain, a bit-level MLP is trained on the
|
| 6 |
+
whole space, and a pass is N/N. Anything wider than a unit's domain is COMPOSED
|
| 7 |
+
from verified slices the way the silicon does it:
|
| 8 |
+
|
| 9 |
+
32-bit add/sub = four ADC8/SBB8 slices, carry rippled
|
| 10 |
+
(CF chains; OF/SF read from the top slice; AF/PF from
|
| 11 |
+
the low slice; ZF = AND of slice ZFs)
|
| 12 |
+
shifts/rotates by k = k applications of 1-bit slice units (domain 512 each)
|
| 13 |
+
through the byte lanes
|
| 14 |
+
32x32 multiply = 8x8->16 partial products (domain 65536) + ADC trees
|
| 15 |
+
divide = restoring division: 32 iterations of verified
|
| 16 |
+
shift/subtract/compare slices
|
| 17 |
+
|
| 18 |
+
x86 flag semantics per slice:
|
| 19 |
+
CF carry/borrow out of the slice (chained)
|
| 20 |
+
OF signed overflow of the slice (meaningful for the top slice)
|
| 21 |
+
SF bit 7 of the slice result (meaningful for the top slice)
|
| 22 |
+
ZF slice result == 0 (composed by AND across slices)
|
| 23 |
+
AF carry out of bit 3 (meaningful for the low slice)
|
| 24 |
+
PF even parity of the slice (x86 PF is parity of the LOW byte only)
|
| 25 |
+
"""
|
| 26 |
+
import os, time
|
| 27 |
+
import numpy as np, torch, torch.nn as nn
|
| 28 |
+
|
| 29 |
+
torch.manual_seed(0); np.random.seed(0); torch.set_num_threads(1)
|
| 30 |
+
CACHE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "models")
|
| 31 |
+
os.makedirs(CACHE, exist_ok=True)
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def bits(x, n):
|
| 35 |
+
x = np.asarray(x)
|
| 36 |
+
return ((x[:, None] >> np.arange(n)[None]) & 1).astype(np.float32)
|
| 37 |
+
|
| 38 |
+
PARITY = np.array([bin(i).count("1") % 2 == 0 for i in range(256)], np.int64)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
class BitMLP(nn.Module):
|
| 42 |
+
def __init__(self, nin, nout, h=256):
|
| 43 |
+
super().__init__()
|
| 44 |
+
self.net = nn.Sequential(nn.Linear(nin, h), nn.ReLU(),
|
| 45 |
+
nn.Linear(h, h), nn.ReLU(), nn.Linear(h, nout))
|
| 46 |
+
def forward(self, x): return self.net(x)
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def _ok(net, X, Y):
|
| 50 |
+
with torch.no_grad():
|
| 51 |
+
outs = []
|
| 52 |
+
for i in range(0, X.shape[0], 65536):
|
| 53 |
+
outs.append((net(X[i:i+65536]) > 0).float() == Y[i:i+65536])
|
| 54 |
+
return bool(torch.cat(outs).all())
|
| 55 |
+
|
| 56 |
+
def train_verify(name, X, Y, lr=3e-3, bs=4096, max_steps=60000, cap=600):
|
| 57 |
+
X = torch.tensor(X); Y = torch.tensor(Y)
|
| 58 |
+
net = BitMLP(X.shape[1], Y.shape[1])
|
| 59 |
+
path = os.path.join(CACHE, f"{name}.pt")
|
| 60 |
+
if os.path.exists(path):
|
| 61 |
+
net.load_state_dict(torch.load(path, weights_only=True))
|
| 62 |
+
if _ok(net, X, Y):
|
| 63 |
+
print(f" {name:9s} {X.shape[0]:6d}/{X.shape[0]:<6d} EXACT (cached)", flush=True)
|
| 64 |
+
return net
|
| 65 |
+
opt = torch.optim.Adam(net.parameters(), lr)
|
| 66 |
+
lossf = nn.BCEWithLogitsLoss(); N = X.shape[0]; t0 = time.time()
|
| 67 |
+
for s in range(1, max_steps + 1):
|
| 68 |
+
idx = torch.randint(0, N, (min(bs, N),))
|
| 69 |
+
opt.zero_grad(); lossf(net(X[idx]), Y[idx]).backward(); opt.step()
|
| 70 |
+
if s % 500 == 0 and (_ok(net, X, Y) or time.time() - t0 > cap):
|
| 71 |
+
break
|
| 72 |
+
ok = _ok(net, X, Y)
|
| 73 |
+
print(f" {name:9s} {X.shape[0]:6d}/{X.shape[0]:<6d} "
|
| 74 |
+
f"{'EXACT' if ok else 'NOT EXACT'} ({time.time()-t0:.0f}s)", flush=True)
|
| 75 |
+
if not ok:
|
| 76 |
+
raise RuntimeError(f"unit {name} failed exact verification")
|
| 77 |
+
torch.save(net.state_dict(), path)
|
| 78 |
+
return net
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
# ===================== golden slice semantics, full domains =====================
|
| 82 |
+
def dom_abc():
|
| 83 |
+
i = np.arange(131072); return i >> 9, (i >> 1) & 0xFF, i & 1
|
| 84 |
+
|
| 85 |
+
def u_ADC8():
|
| 86 |
+
a, b, c = dom_abc(); t = a + b + c; res = t & 0xFF
|
| 87 |
+
CF = t > 0xFF
|
| 88 |
+
OF = ((~(a ^ b)) & (a ^ res) & 0x80) != 0
|
| 89 |
+
SF = res >> 7; ZF = res == 0
|
| 90 |
+
AF = ((a & 0xF) + (b & 0xF) + c) > 0xF
|
| 91 |
+
X = np.concatenate([bits(a, 8), bits(b, 8), c[:, None].astype(np.float32)], 1)
|
| 92 |
+
Y = np.stack([CF, OF, SF, ZF, AF], 1).astype(np.float32)
|
| 93 |
+
return X, np.concatenate([bits(res, 8), Y], 1)
|
| 94 |
+
|
| 95 |
+
def u_SBB8():
|
| 96 |
+
a, b, c = dom_abc(); t = a - b - c; res = t & 0xFF
|
| 97 |
+
CF = t < 0
|
| 98 |
+
OF = (((a ^ b)) & (a ^ res) & 0x80) != 0
|
| 99 |
+
SF = res >> 7; ZF = res == 0
|
| 100 |
+
AF = ((a & 0xF) - (b & 0xF) - c) < 0
|
| 101 |
+
X = np.concatenate([bits(a, 8), bits(b, 8), c[:, None].astype(np.float32)], 1)
|
| 102 |
+
Y = np.stack([CF, OF, SF, ZF, AF], 1).astype(np.float32)
|
| 103 |
+
return X, np.concatenate([bits(res, 8), Y], 1)
|
| 104 |
+
|
| 105 |
+
def u_logic(f):
|
| 106 |
+
a = np.repeat(np.arange(256), 256); b = np.tile(np.arange(256), 256)
|
| 107 |
+
res = f(a, b) & 0xFF
|
| 108 |
+
SF = res >> 7; ZF = res == 0
|
| 109 |
+
X = np.concatenate([bits(a, 8), bits(b, 8)], 1)
|
| 110 |
+
Y = np.concatenate([bits(res, 8),
|
| 111 |
+
np.stack([SF, ZF], 1).astype(np.float32)], 1)
|
| 112 |
+
return X, Y
|
| 113 |
+
|
| 114 |
+
def u_SHL1(): # (byte, carry_in_bit) -> (byte<<1 | cin, carry_out = old bit7)
|
| 115 |
+
v = np.repeat(np.arange(256), 2); c = np.tile(np.arange(2), 256)
|
| 116 |
+
res = ((v << 1) | c) & 0xFF; co = v >> 7
|
| 117 |
+
X = np.concatenate([bits(v, 8), c[:, None].astype(np.float32)], 1)
|
| 118 |
+
return X, np.concatenate([bits(res, 8), co[:, None].astype(np.float32)], 1)
|
| 119 |
+
|
| 120 |
+
def u_SHR1(): # (byte, carry_in_bit_into_top) -> (cin<<7 | byte>>1, carry_out = old bit0)
|
| 121 |
+
v = np.repeat(np.arange(256), 2); c = np.tile(np.arange(2), 256)
|
| 122 |
+
res = (v >> 1) | (c << 7); co = v & 1
|
| 123 |
+
X = np.concatenate([bits(v, 8), c[:, None].astype(np.float32)], 1)
|
| 124 |
+
return X, np.concatenate([bits(res, 8), co[:, None].astype(np.float32)], 1)
|
| 125 |
+
|
| 126 |
+
def u_MASK8(): # (byte, bit) -> byte*bit : the multiplier partial-product slice
|
| 127 |
+
v = np.repeat(np.arange(256), 2); c = np.tile(np.arange(2), 256)
|
| 128 |
+
res = v * c
|
| 129 |
+
X = np.concatenate([bits(v, 8), c[:, None].astype(np.float32)], 1)
|
| 130 |
+
return X, bits(res, 8)
|
| 131 |
+
|
| 132 |
+
def u_NOT8():
|
| 133 |
+
v = np.arange(256); return bits(v, 8), bits(~v & 0xFF, 8)
|
| 134 |
+
|
| 135 |
+
def u_FLAGS8(): # res byte -> SF, ZF8, PF (post-hoc flag extraction for shifts etc.)
|
| 136 |
+
v = np.arange(256)
|
| 137 |
+
return bits(v, 8), np.stack([v >> 7, v == 0, PARITY[v]], 1).astype(np.float32)
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
# ===================== decode tables (enumerable lookups) =====================
|
| 141 |
+
# prefix classification of a byte in 32-bit code
|
| 142 |
+
PFX = {0x66: 1, 0x67: 2, 0xF0: 3, 0xF2: 4, 0xF3: 5,
|
| 143 |
+
0x26: 6, 0x2E: 6, 0x36: 6, 0x3E: 6, 0x64: 6, 0x65: 6}
|
| 144 |
+
|
| 145 |
+
def u_prefix(): # byte -> is_prefix(1) + class(3 bits)
|
| 146 |
+
v = np.arange(256)
|
| 147 |
+
isp = np.array([1 if b in PFX else 0 for b in v], np.float32)
|
| 148 |
+
cls = np.array([PFX.get(b, 0) for b in v])
|
| 149 |
+
return bits(v, 8), np.concatenate([isp[:, None], bits(cls, 3)], 1)
|
| 150 |
+
|
| 151 |
+
def u_modrm(): # modrm byte -> mod(2) reg(3) rm(3)
|
| 152 |
+
v = np.arange(256)
|
| 153 |
+
return bits(v, 8), np.concatenate([bits(v >> 6, 2), bits((v >> 3) & 7, 3),
|
| 154 |
+
bits(v & 7, 3)], 1)
|
| 155 |
+
|
| 156 |
+
def u_sib(): # sib byte -> scale(2) index(3) base(3)
|
| 157 |
+
v = np.arange(256)
|
| 158 |
+
return bits(v, 8), np.concatenate([bits(v >> 6, 2), bits((v >> 3) & 7, 3),
|
| 159 |
+
bits(v & 7, 3)], 1)
|
| 160 |
+
|
| 161 |
+
|
| 162 |
+
# ===================== build / load all =====================
|
| 163 |
+
def build_all():
|
| 164 |
+
print("Training/loading + exhaustively verifying x86 datapath units:", flush=True)
|
| 165 |
+
u = {}
|
| 166 |
+
u["ADC8"] = train_verify("ADC8", *u_ADC8())
|
| 167 |
+
u["SBB8"] = train_verify("SBB8", *u_SBB8())
|
| 168 |
+
u["AND8"] = train_verify("AND8", *u_logic(lambda a, b: a & b))
|
| 169 |
+
u["OR8"] = train_verify("OR8", *u_logic(lambda a, b: a | b))
|
| 170 |
+
u["XOR8"] = train_verify("XOR8", *u_logic(lambda a, b: a ^ b))
|
| 171 |
+
u["SHL1"] = train_verify("SHL1", *u_SHL1(), bs=512, cap=60)
|
| 172 |
+
u["SHR1"] = train_verify("SHR1", *u_SHR1(), bs=512, cap=60)
|
| 173 |
+
u["MASK8"] = train_verify("MASK8", *u_MASK8(), bs=512, cap=60)
|
| 174 |
+
u["NOT8"] = train_verify("NOT8", *u_NOT8(), bs=256, cap=40)
|
| 175 |
+
u["FLAGS8"] = train_verify("FLAGS8", *u_FLAGS8(), bs=256, cap=40)
|
| 176 |
+
u["PREFIX"] = train_verify("PREFIX", *u_prefix(), bs=256, cap=60)
|
| 177 |
+
u["MODRM"] = train_verify("MODRM", *u_modrm(), bs=256, cap=60)
|
| 178 |
+
u["SIB"] = train_verify("SIB", *u_sib(), bs=256, cap=60)
|
| 179 |
+
return u
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
# ===================== unit APIs: golden vs neural, one signature =====================
|
| 183 |
+
class GoldenUnits:
|
| 184 |
+
def adc8(self, a, b, c):
|
| 185 |
+
t = a + b + c; r = t & 0xFF
|
| 186 |
+
return (r, int(t > 0xFF), int(((~(a ^ b)) & (a ^ r) & 0x80) != 0), r >> 7,
|
| 187 |
+
int(r == 0), int(((a & 0xF) + (b & 0xF) + c) > 0xF))
|
| 188 |
+
def sbb8(self, a, b, c):
|
| 189 |
+
t = a - b - c; r = t & 0xFF
|
| 190 |
+
return (r, int(t < 0), int((((a ^ b)) & (a ^ r) & 0x80) != 0), r >> 7,
|
| 191 |
+
int(r == 0), int(((a & 0xF) - (b & 0xF) - c) < 0))
|
| 192 |
+
def logic8(self, kind, a, b):
|
| 193 |
+
r = {"AND8": a & b, "OR8": a | b, "XOR8": a ^ b}[kind] & 0xFF
|
| 194 |
+
return r, r >> 7, int(r == 0)
|
| 195 |
+
def shl1(self, v, c): return ((v << 1) | c) & 0xFF, v >> 7
|
| 196 |
+
def shr1(self, v, c): return (v >> 1) | (c << 7), v & 1
|
| 197 |
+
def mask8(self, v, bit): return v * bit
|
| 198 |
+
def not8(self, v): return ~v & 0xFF
|
| 199 |
+
def flags8(self, v): return v >> 7, int(v == 0), int(PARITY[v])
|
| 200 |
+
def prefix(self, b): return (1 if b in PFX else 0), PFX.get(b, 0)
|
| 201 |
+
def modrm(self, b): return b >> 6, (b >> 3) & 7, b & 7
|
| 202 |
+
def sib(self, b): return b >> 6, (b >> 3) & 7, b & 7
|
| 203 |
+
|
| 204 |
+
|
| 205 |
+
class NeuralUnits:
|
| 206 |
+
def __init__(self, nets): self.n = nets
|
| 207 |
+
def _run(self, name, ib):
|
| 208 |
+
with torch.no_grad():
|
| 209 |
+
o = self.n[name](torch.tensor(np.array([ib], np.float32)))[0]
|
| 210 |
+
return (o > 0).long().numpy()
|
| 211 |
+
@staticmethod
|
| 212 |
+
def _i(o, a, n): return int(sum(int(o[a + i]) << i for i in range(n)))
|
| 213 |
+
|
| 214 |
+
def adc8(self, a, b, c):
|
| 215 |
+
o = self._run("ADC8", list(bits([a], 8)[0]) + list(bits([b], 8)[0]) + [c])
|
| 216 |
+
return (self._i(o, 0, 8),) + tuple(int(o[8 + i]) for i in range(5))
|
| 217 |
+
def sbb8(self, a, b, c):
|
| 218 |
+
o = self._run("SBB8", list(bits([a], 8)[0]) + list(bits([b], 8)[0]) + [c])
|
| 219 |
+
return (self._i(o, 0, 8),) + tuple(int(o[8 + i]) for i in range(5))
|
| 220 |
+
def logic8(self, kind, a, b):
|
| 221 |
+
o = self._run(kind, list(bits([a], 8)[0]) + list(bits([b], 8)[0]))
|
| 222 |
+
return self._i(o, 0, 8), int(o[8]), int(o[9])
|
| 223 |
+
def shl1(self, v, c):
|
| 224 |
+
o = self._run("SHL1", list(bits([v], 8)[0]) + [c])
|
| 225 |
+
return self._i(o, 0, 8), int(o[8])
|
| 226 |
+
def shr1(self, v, c):
|
| 227 |
+
o = self._run("SHR1", list(bits([v], 8)[0]) + [c])
|
| 228 |
+
return self._i(o, 0, 8), int(o[8])
|
| 229 |
+
def mask8(self, v, bit):
|
| 230 |
+
o = self._run("MASK8", list(bits([v], 8)[0]) + [bit])
|
| 231 |
+
return self._i(o, 0, 8)
|
| 232 |
+
def not8(self, v): return self._i(self._run("NOT8", list(bits([v], 8)[0])), 0, 8)
|
| 233 |
+
def flags8(self, v):
|
| 234 |
+
o = self._run("FLAGS8", list(bits([v], 8)[0]))
|
| 235 |
+
return int(o[0]), int(o[1]), int(o[2])
|
| 236 |
+
def prefix(self, b):
|
| 237 |
+
o = self._run("PREFIX", list(bits([b], 8)[0]))
|
| 238 |
+
return int(o[0]), self._i(o, 1, 3)
|
| 239 |
+
def modrm(self, b):
|
| 240 |
+
o = self._run("MODRM", list(bits([b], 8)[0]))
|
| 241 |
+
return self._i(o, 0, 2), self._i(o, 2, 3), self._i(o, 5, 3)
|
| 242 |
+
def sib(self, b):
|
| 243 |
+
o = self._run("SIB", list(bits([b], 8)[0]))
|
| 244 |
+
return self._i(o, 0, 2), self._i(o, 2, 3), self._i(o, 5, 3)
|
| 245 |
+
|
| 246 |
+
|
| 247 |
+
# ===================== composed wide ops (wiring over verified slices) =====================
|
| 248 |
+
class ALU:
|
| 249 |
+
"""Composed 8/16/32-bit x86 ALU over the slice units. Returns (result, flags-dict).
|
| 250 |
+
Flag wiring: CF chained; OF/SF from top slice; AF/PF from low slice; ZF = AND."""
|
| 251 |
+
def __init__(self, u): self.u = u
|
| 252 |
+
|
| 253 |
+
def _split(self, v, size): return [(v >> (8 * i)) & 0xFF for i in range(size // 8)]
|
| 254 |
+
def _join(self, b): return sum(x << (8 * i) for i, x in enumerate(b))
|
| 255 |
+
|
| 256 |
+
def addsub(self, sub, a, b, size, cin=0):
|
| 257 |
+
u = self.u; outs = []; c = cin
|
| 258 |
+
fl = {}
|
| 259 |
+
for i, (ab, bb) in enumerate(zip(self._split(a, size), self._split(b, size))):
|
| 260 |
+
r, c, OF, SF, ZF, AF = (u.sbb8 if sub else u.adc8)(ab, bb, c)
|
| 261 |
+
outs.append(r)
|
| 262 |
+
if i == 0: fl["AF"], fl["PF"] = AF, u.flags8(r)[2]
|
| 263 |
+
fl["OF"], fl["SF"] = OF, SF
|
| 264 |
+
fl.setdefault("ZF", 1); fl["ZF"] &= ZF
|
| 265 |
+
fl["CF"] = c
|
| 266 |
+
return self._join(outs), fl
|
| 267 |
+
|
| 268 |
+
def logic(self, kind, a, b, size):
|
| 269 |
+
u = self.u; outs = []; fl = {"CF": 0, "OF": 0, "AF": 0}
|
| 270 |
+
for i, (ab, bb) in enumerate(zip(self._split(a, size), self._split(b, size))):
|
| 271 |
+
r, SF, ZF = u.logic8(kind + "8", ab, bb)
|
| 272 |
+
outs.append(r)
|
| 273 |
+
if i == 0: fl["PF"] = u.flags8(r)[2]
|
| 274 |
+
fl["SF"] = SF
|
| 275 |
+
fl.setdefault("ZF", 1); fl["ZF"] &= ZF
|
| 276 |
+
return self._join(outs), fl
|
| 277 |
+
|
| 278 |
+
def not_(self, a, size):
|
| 279 |
+
return self._join([self.u.not8(b) for b in self._split(a, size)])
|
| 280 |
+
|
| 281 |
+
def shl1(self, a, size):
|
| 282 |
+
c = 0; outs = []
|
| 283 |
+
for b in self._split(a, size):
|
| 284 |
+
r, c = self.u.shl1(b, c); outs.append(r)
|
| 285 |
+
return self._join(outs), c
|
| 286 |
+
def shr1(self, a, size, top=0):
|
| 287 |
+
outs = [0] * (size // 8); c = top
|
| 288 |
+
for i in reversed(range(size // 8)):
|
| 289 |
+
r, c = self.u.shr1((a >> (8 * i)) & 0xFF, c); outs[i] = r
|
| 290 |
+
return self._join(outs), c
|
| 291 |
+
|
| 292 |
+
def flags_of(self, v, size):
|
| 293 |
+
"""SF/ZF/PF wiring for shift/rotate results."""
|
| 294 |
+
bs = self._split(v, size)
|
| 295 |
+
SF = ZF = PF = None
|
| 296 |
+
ZF = 1
|
| 297 |
+
for i, b in enumerate(bs):
|
| 298 |
+
s, z, p = self.u.flags8(b)
|
| 299 |
+
if i == 0: PF = p
|
| 300 |
+
SF = s; ZF &= z
|
| 301 |
+
return SF, ZF, PF
|
| 302 |
+
|
| 303 |
+
def mul(self, a, b, size):
|
| 304 |
+
"""unsigned multiply: binary long multiplication -- for each bit j of b,
|
| 305 |
+
a MASK8-sliced partial product of a is shifted (wiring) and ADC-summed."""
|
| 306 |
+
ab = self._split(a, size)
|
| 307 |
+
acc = 0
|
| 308 |
+
for j in range(size):
|
| 309 |
+
bit = (b >> j) & 1
|
| 310 |
+
masked = self._join([self.u.mask8(x, bit) for x in ab])
|
| 311 |
+
acc = self.add_wide(acc, masked << j, 2 * size)
|
| 312 |
+
return acc
|
| 313 |
+
def add_wide(self, a, b, bits_):
|
| 314 |
+
"""carry-rippled add of arbitrary width (wiring over adc8)."""
|
| 315 |
+
c = 0; out = 0
|
| 316 |
+
for i in range(bits_ // 8):
|
| 317 |
+
r, c, *_ = self.u.adc8((a >> (8*i)) & 0xFF, (b >> (8*i)) & 0xFF, c)
|
| 318 |
+
out |= r << (8 * i)
|
| 319 |
+
return out
|
| 320 |
+
def sub_wide(self, a, b, bits_):
|
| 321 |
+
c = 0; out = 0
|
| 322 |
+
for i in range(bits_ // 8):
|
| 323 |
+
r, c, *_ = self.u.sbb8((a >> (8*i)) & 0xFF, (b >> (8*i)) & 0xFF, c)
|
| 324 |
+
out |= r << (8 * i)
|
| 325 |
+
return out, c # c = borrow
|
| 326 |
+
|
| 327 |
+
def divmod_(self, num, den, nbits):
|
| 328 |
+
"""restoring division: nbits iterations of verified shift/sub/restore."""
|
| 329 |
+
if den == 0:
|
| 330 |
+
raise ZeroDivisionError("#DE")
|
| 331 |
+
q = 0; rem = 0
|
| 332 |
+
for i in reversed(range(nbits)):
|
| 333 |
+
rem = (rem << 1) | ((num >> i) & 1) # wiring (bit select)
|
| 334 |
+
diff, borrow = self.sub_wide(rem, den, ((nbits + 7) // 8) * 8 + 8)
|
| 335 |
+
if not borrow:
|
| 336 |
+
rem = diff; q |= (1 << i)
|
| 337 |
+
return q, rem
|
| 338 |
+
|
| 339 |
+
|
| 340 |
+
if __name__ == "__main__":
|
| 341 |
+
build_all()
|
| 342 |
+
print("\nALL x86 DATAPATH UNITS VERIFIED EXACT.")
|