diw22/DWinters_Humanoid-models

Kraftwerk Bot: trained models

Trained weights and results for github.com/diw22/DWinters_Humanoid: an audio-conditioned OpenVLA-OFT that hears a drum loop with one part missing and plays that part on a simulated Unitree G1. Download into the code's runs/ folder:

hf download diw22/DWinters_Humanoid-models --local-dir runs

Folder	Model
three_part/	OpenVLA-OFT + MERT audio, trained on claps, rims and shakers (LoRA adapters, audio and state projectors, action head)
rims_only/	three_part fine-tuned on rims alone
rims_student/	a 1.8M-parameter student distilled from rims_only (runs on a CPU)
jaco_v1/	results of version 1 (claps on two simulated Jaco arms)

Each folder also has config.json (normalisation and training settings), metrics.jsonl (training curve) and evaluation_test.json (closed-loop results on the held-out takes, MuJoCo 3.3.0); the 7B models have test videos. Results and how to reproduce them are in the GitHub README.

The 7B models are LoRA adapters and heads for openvla/openvla-7b, which builds on Llama 2: use them under OpenVLA's and Llama 2's terms. The audio encoder, MERT-v1-95M, is not included; the code downloads it.