Oasis Action World Model Integration
ADAM treats Oasis as a first-class trainer named Oasis Action World Model.
It uses the existing Oasis-Game-Trainer/roblox_action_flow_app.py worker and
keeps training and playable inference in separate processes.
Choose temporal_latent for new recordings. It learns a timed transition from several recent frames and the inputs that happened between them. That lets it distinguish, for example, a stationary car from a car still coasting after the accelerator was released, or a player standing still from one falling with no keys held. The player has a Temporal: advance during no-input switch: turn it on to see learned idle dynamics, or turn it off to advance only when you provide an input.
Choose temporal_pixel_flow when you want the same timed frame and input history while predicting full RGB frames directly. It does not use a VAE, so it avoids the latent codec's spatial compression and uses more GPU memory. It is a separate checkpoint format; it cannot continue a Temporal Latent or Pixel Flow checkpoint. Start with batch size 1 if full RGB training runs out of memory.
Dataset Layout
Use one or more recorder folders. Multiple folders can be supplied with semicolon separators.
My_Oasis_Dataset/
dataset_info.json
actions.jsonl
frames/
frame_00000000.png
frame_00000001.png
Each actions.jsonl row must identify the frame and its labels:
{"session_id":"run1","frame_index":1,"filename":"frame_00000001.png","w":1,"a":0,"s":0,"d":0,"jump":0,"mouse_dx":0.0,"mouse_dy":0.0,"zoom":0.0}
Rows describe the action for the transition into that frame. ADAM validates missing frames, invalid labels, duplicate or gapped frame indexes, broken images, inconsistent resolutions, empty sequences, and missing action columns. It does not guess missing labels.
For temporal models, use a current v5 recorder dataset. It contains timestamps, input intervals, episode IDs, and discontinuity markers so ADAM never trains across a reset, teleport, pause screen, or missing frame. Keep no-input frames: they are the evidence needed to learn coasting, gravity, and other motion that continues after an action ends.
Minecraft Beta-Style Recording Notes
Record your own gameplay. Start with 256x144 at 8-12 FPS for a first pipeline
test, then increase data before long runs. Keep every clip as its own session or
recording folder so training never learns fake transitions between worlds.
Capture a mix of walking forward, strafing, turning, jumping, climbing, terrain changes, interiors, caves, water, sky, and short idle moments. Avoid long standing-still stretches; idle is useful, but a dataset dominated by idle frames will make the model ignore controls.
A good smoke dataset is 300-1,000 labelled frames. A more meaningful first Minecraft experiment is 5,000-20,000 labelled frames with clean W/A/S/D/Space coverage and synchronized frame/action timestamps.
RTX 3060 12 GB Starting Point
Use temporal_latent, 256x144, batch size 2, mixed precision no, history
frames 4, future frames 3, VAE warm-up 5, gradient accumulation 1,
loader workers 2, save every 5 epochs, preview every 5 epochs, and 5-10
epochs for a pipeline test. Try fp16 only after full precision is stable.