Add MPS fast path for Apple Silicon (bucketed static shapes + compiled LM)
#3
by pylotlight - opened
Adds an opt-in MPS fast path for Apple Silicon (decision path, batch=1).
- What: compile
backbone.language_modelwith precomputedinputs_embeds(eager embed+vision+M-RoPE), static buckets,dynamic=False(Metal). Vision stays eager. CUDA/packed/FLA unchanged. - Code:
thx02/mps_fast.py, minimal wiring inthx02/model.py(source branch: https://huggingface.co/pylotlight/THX-02/tree/mps-fast-path). - Measured (M5 Pro, MPS, 13 req): ~1.31x mean (314->245 ms), 18/18 top-1, max |Δlogit| 0.125, ~63 s warmup.
Details/charts in discussion #2.
This looks great, 1.31x with identical top-1 answers is exactly what we hoped for. Please push the files when ready and we'll review and merge.
pylotlight changed pull request status to open
pylotlight changed pull request status to closed
pylotlight changed pull request status to open
doofz changed pull request status to merged
Merged.