Add MPS fast path for Apple Silicon (bucketed static shapes + compiled LM)

#3

Adds an opt-in MPS fast path for Apple Silicon (decision path, batch=1).

  • What: compile backbone.language_model with precomputed inputs_embeds (eager embed+vision+M-RoPE), static buckets, dynamic=False (Metal). Vision stays eager. CUDA/packed/FLA unchanged.
  • Code: thx02/mps_fast.py, minimal wiring in thx02/model.py (source branch: https://huggingface.co/pylotlight/THX-02/tree/mps-fast-path).
  • Measured (M5 Pro, MPS, 13 req): ~1.31x mean (314->245 ms), 18/18 top-1, max |Δlogit| 0.125, ~63 s warmup.

Details/charts in discussion #2.

HAL-X org

This looks great, 1.31x with identical top-1 answers is exactly what we hoped for. Please push the files when ready and we'll review and merge.

pylotlight changed pull request status to open
pylotlight changed pull request status to closed
pylotlight changed pull request status to open
•
This comment has been hidden
doofz changed pull request status to merged
HAL-X org

Merged.

Sign up or log in to comment