| --- |
| license: cc-by-4.0 |
| language: |
| - en |
| tags: |
| - robotics |
| - connectomics |
| - malecns |
| - mlx |
| - simulation |
| - experimental |
| model-index: |
| - name: fly-brain-codex |
| results: [] |
| --- |
| # Fly Brain Codex |
|
|
| An experimental controller for a native macOS robot-arm simulator, built using a derived MaleCNS connectome. **MaleCNS is an anatomical wiring map, not a pretrained robot policy.** This release includes our learned parameters, derived sparse graph assets, frozen training features/targets, and historical evaluation records. |
|
|
| [Source, setup and demo](https://github.com/monomyth/fly-brain-codex) · [Simulator branch](https://github.com/monomyth/rebot-motion-lab/tree/fly-brain-codex) |
|
|
| ## Choose a profile |
|
|
| | Artifact | What it contains | Measured status | |
| |---|---|---| |
| | `stable.tar.gz` | Retained `retain-grasp-20260911` UI controller | 19/20 pickup-and-hold trials in X343–357, Y−10–8 mm; separate real UI pickup/hold/drop checks passed | |
| | `experimental.tar.gz` | Newer `touch-direct-20260912`, using bilateral finger contact to select pickup/holding parameter banks | 12/15 full tasks completed; 20 planned, 5 unrun. Not qualified for default UI deployment | |
| | `cuda-continuation.tar.gz` | Latest 12,000-iteration RTX 4090 continuation fit, parameters and fit metrics | Not imported into the deployed controller; no native closed-loop evaluation | |
| | `runtime-assets.tar.gz` | Shared prepared graph, sensory/motor/feedback circuits and brain-overlay geometry | Shared once by both profiles | |
|
|
| “Stable” identifies the retained UI baseline, not production-grade reliability. The validated task uses a **20 mm cube and a folded start**. Broad arbitrary placement and other cube sizes are not established. Historical results do not guarantee the same rate on another machine. |
|
|
| ## Run |
|
|
| Clone the [source repository](https://github.com/monomyth/fly-brain-codex), follow its macOS simulator build instructions, then run: |
|
|
| ```sh |
| python3 scripts/download_model.py |
| .venv/bin/python scripts/run_mlx_ui.py --check-deployment |
| ``` |
|
|
| The downloader pins this Hub release by commit and SHA-256 in the source repository, verifies the archives, preserves differing existing assets, and writes only project-local `data/` and a local runtime configuration. No Hub login is needed for a public download. `--profile experimental` downloads the research controller without activating it. |
|
|
| `training-bundle.tar.gz` additionally contains the compact, versioned frozen-feature continuation experiment and its matching trainer; see the source repository training guide. It does not contain the full historical RGB dataset. |
|
|
| ## What controls the arm |
|
|
| Front and wrist-mounted RGB renders are converted to luminance and mapped to annotated retinal neurons. Joint, aperture and contact feedback drive an engineered sensory encoding. A fixed persistent sensory graph runs on MLX/Metal; a constrained learned motor head runs on CPU and produces six joint targets plus gripper aperture at a nominal 2 Hz. Cube coordinates and inverse kinematics are used by teachers/evaluation, not by the actor's action selection. |
|
|
| Training uses demonstration supervision and task-space objectives, with PyTorch MPS/CUDA optimization of selected existing signed connections and response offsets. It is not a validated biological learning simulation. A fixed dopamine multiplier in offline training is not demonstrated reward-driven fly learning. The touch-bank selector is engineered. |
|
|
| After a qualifying hold (at least 100 mm clearance, at most 5° tilt for five seconds), a **scripted completion routine opens the gripper and verifies the floor landing**. Release is not a learned neural action. RealityKit, native actuators, floor limiting, camera projection, and the fly-to-robot mapping are engineered components. |
|
|
| ## Reuse and provenance |
|
|
| Numerical arrays in the packaged checkpoints are byte-identical to their tested source versions. JSON provenance paths were normalized; `relocation.json` records before/after hashes. The installer preserves original historical evidence and rebases checkpoint addresses and metadata hashes in separate local copies. This is relocation verification, not a new simulator or hardware qualification. Backend parity was measured on Apple M2 Max with MLX 0.32.2. |
|
|
| The bundles contain frozen features/targets, not the entire historical RGB observation corpus. Raw MaleCNS download tables are not duplicated here. Download/rebuild commands and research scripts live in the source repository. |
|
|
| ## Attribution and license |
|
|
| Derived assets and published model artifacts: **CC BY 4.0**. Source code: MIT in the GitHub repository. MaleCNS work is by FlyEM at HHMI Janelia, University of Cambridge, MRC Laboratory of Molecular Biology, Google Research, and collaborators. Preserve the bundled `ATTRIBUTION.md` files. |
|
|
| Source: [MaleCNS downloads and attribution](https://male-cns.janelia.org/download/). This independent experiment is not an official MaleCNS, Google, Janelia, Orbbec, or ReBot release, and does not establish biological fidelity or physical-robot safety. |
|
|
| Publication metadata is portable: local account paths and machine names are removed, and archive ownership fields are anonymized. Numerical arrays are unchanged. |
|
|
