lace to list and track work on adding support to new ops for the MPS backend. Most requested ops (extracted from comments to this issue and #77764) PyTorch MPS Ops Project : Project to track all the ops for MPS backend. There are a very large number of operators in pytorch and...

ibution needs confirmation (it is by failing-since alignment, not root-caused). Same #188251 bump has produced other confirmed regressions: #188492 (swin inductor accuracy) and #188841 (compile + float8 training numerics). Filed by pt2-oncall triage from HUD signal. cc @ptrblc...

failing-since alignment, not root-caused). Same #188251 bump has produced other confirmed regressions: #188492 (swin inductor accuracy) and #188841 (compile + float8 training numerics). Filed by pt2-oncall triage from HUD signal. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @...

09 notes flip is the only remaining MPSGraph user in Indexing.mm, but other files still lower gather/scatter-like ops to MPSGraph. Related: #84520 (MPS 32-bit limits umbrella — but here nothing asserts, it corrupts silently), #144824 (OOB indexing returns 0), #154235 (missing...

slicing — the device copy is intact; only the gather is broken. All indices involved are valid; this is not the OOB-index case of #154235 / #144824, though the failure surface is the same "OOB reads return 0" behavior. Root cause (as far as I can tell) In 2.12.0, index_select_...

Summary kernel_information.json (#160540, extended in #183952) and the profiler-timeline provenance (#186230, #187328) map each generated kernel to a set of pre/post-grad node names, stack traces, and now extern I/O shapes. What is still missing is the phys

/_inductor/fx_passes/pad_mm.py#L636 https://github.com/pytorch/pytorch/blob/main/torch/_dynamo/convert_frame.py#L172 (Logic refers to issue #97894) Minified repro import torch @torch.compile def some_fn(x): return torch.pow(x, 0.1) / 2 with torch.device("cpu"): x = torch.ones(...

s; OpInfo / common_methods_invocations.py remains the broad semantic owner. Revisit LayerNorm backward after the active reduce/loss stacks. #182109 already has feedback asking for clearer rationale, benchmark script, and before/after results. If revived, split it into smaller...

tern used by the leaves. No user-visible behavior change; this is the cross-op infrastructure reviewers should see first. Scalar read leaf: #182731. Rebase on the UMA base helper. Scope: _local_scalar_dense / .item() / scalar truthiness direct CPU read. MPS->CPU copy leaf: #18...

leaf: #182731. Rebase on the UMA base helper. Scope: _local_scalar_dense / .item() / scalar truthiness direct CPU read. MPS->CPU copy leaf: #182736. Rebase on the UMA base helper. Scope: contiguous same-dtype copy_from_mps_ / .to("cpu") / .numpy() direct read. CPU->MPS copy le...

182736. Rebase on the UMA base helper. Scope: contiguous same-dtype copy_from_mps_ / .to("cpu") / .numpy() direct read. CPU->MPS copy leaf: #182749. Rebase on the UMA base helper. Scope: contiguous same-dtype CPU->MPS direct write, with non-blocking/non-contiguous/dtype-cast f...

elper. Scope: contiguous same-dtype CPU->MPS direct write, with non-blocking/non-contiguous/dtype-cast fallbacks preserved. Fill/zero leaf: #182791. Rebase after the write-side copy leaf. Scope: dense zero_/fill_ via memset/std::fill_n, with negative-zero and unsupported dtype...

182791 draft/pending rework Rebase onto the UMA base/write-side direction; do not review as a standalone final PR. MPS capture/replay stack #183688 is much larger than a reviewable PyTorch PR and should be treated as a reference branch to split, not a final PR. Order: Capture...

lit at the last-dim vs non-last-dim boundary and stacked as cumulative-leaf forks on main (external contributor, no ghstack): softmax core (#187456, 52a3223): native last-dim _softmax/_softmax_backward (single-row + 8-wide-half + looped online-softmax); non-last-dim and huge-r...

-dim-only gate. Residual dim=0-backward tail is documented (MPSGraph's fused reduce still wins there; escape hatch covers it). log_softmax (#187457, 3d6f962): native log_softmax stacked on the softmax kernels. cross_entropy (#187458, ee60969): fused 2D cross-entropy Metal kern...

antiations each), not chained on each other. Independent loss-style work: #187559 (nll_loss, review comments addressed at fc26ad2e55e3) and #187458 (cross_entropy) do not fit the same elementwise fused-loss template. Softmax / log_softmax / cross_entropy: refreshed and opened...

into the existing TestMetalLibrary.test_reduction_utils owner surface instead of adding a standalone regression. Land the loss-stack base: #187553 (mse_loss). Scope: shared fused-loss infrastructure (FusedLossParams, fused_loss_pass1, fused_loss_bwd, fused_loss_reduce, TensorI...

ases); OpInfo consistency (including the f16 grad test) remains the broad semantic owner. Stack: rebuild #187557 (binary_cross_entropy) and #187556 (smooth_l1_loss/huber_loss) as sibling leaves on the final #187553 (small op functor + instantiations each), not chained on each...

d, mixed dtypes, out-variant edge cases); OpInfo consistency (including the f16 grad test) remains the broad semantic owner. Stack: rebuild #187557 (binary_cross_entropy) and #187556 (smooth_l1_loss/huber_loss) as sibling leaves on the final #187553 (small op functor + instant...

s) as sibling leaves on the final #187553 (small op functor + instantiations each), not chained on each other. Independent loss-style work: #187559 (nll_loss, review comments addressed at fc26ad2e55e3) and #187458 (cross_entropy) do not fit the same elementwise fused-loss temp...

guard; prod perf: outer/inner shape specializations, benchmark, perf table. Review var/std after the reduce base/prod direction is settled: #187787. Current state: rebuilt and pushed at cd01523561027; ready as the Welford follow-up, but should be read after #188156/#188333 bec...

anup: complex enablement, integer rejection, duplicate-dim behavior, and large-output errors if those remain separate review concerns. Keep #187976 independent. #187976 fixes an int64 amax/amin partial-simdgroup bug in the existing value-reduction helper path. It is a sibling...

A standalone test_mps.py case needs a clear reason why no existing surface owns the route. Order of operations Land the reduce-family base: #188156. Scope: collapse generic sum/mean/nansum/count_nonzero onto the shared value_reduction<Op, Load, TA> path already used by min/max...

route for non-contiguous multi-dim sum-family reductions and load policies, not just a private helper. Land prod on top of the reduce base: #188333. Scope: migrate prod to value_reduction<ProdOp> and remove its shape-dependent MPSGraph route. This supersedes the closed standal...

ned MPSGraph fallback, so it is correct standalone and strictly reduces the softmax graph-cache for the common last-dim case. softmax perf (#189146, 8fa0937): adds the huge-row two-pass and non-last-dim (coalesced/tiled/blocked) native kernels on top of core and removes the la...

to it unconditionally. There are several recently filed issues that highlight the TMA path for pointwise/reduction is not yet fully mature: #186241 #185513 #185222 #184714 #184563 Phase 2 — Once we are confident the TMA path is correct and mature: Enabling autotuning alone (wi...

Hi @chunhuanMeng , please help handle this issue and #187811

is replaced by _exposes_streams(device), which checks whether iface.Stream is not DeviceInterface.Stream. Please refer to the dedicated RFC #189135 for detailed design elaboration. Dynamo Registry Splicing Dynamo answers two questions for every value/call during symbolic execu...

e-specialized tensor types — is handled by one optional slot dynamo_tensor_classes (default empty, safe). Please refer to the dedicated RFC #189136 for detailed design elaboration. Inductor Codegen Registration Interfaces Inductor's codegen path has four things that lack regis...

E_TO_ATEN fallback. No-c-shim backends will still fail AOTI at link time. This needs a separate proposal. Please refer to the dedicated RFC #189137 for detailed design elaboration. For a more comprehensive and detailed understanding of each component's design, we highly encour...

der some specific conditions, raise a RuntimeError with message: "Global alloc not supported yet.". I think this is linked to an old issue: #69078, however, I managed to consistently reproduce this error. The code that reproduces the bug is quite long and needs some explanatio...

not collect Expected behavior Compile succeeds, or the unsupported/unsafe option is ignored with a warning — not a native SIGSEGV. Related #180397 proposes a capture/replay API for MPS and notes MPS currently has no CUDA-Graphs equivalent — possibly useful context, since trito...

Opened a draft PR with the scoped post-grad rewrite: #189127 The patch extends the existing addmm bias-unfuse pattern to baddbmm: it only applies on GPU when the input is bias-like/broadcast and all b

Opened a draft PR with the focused cache-gate patch: #189124 The patch keeps the ordinary single-config fast path unchanged, but lets DSR-eligible single-config reductions create/load an autotune cach

used to optimize the binary. The current plan is that this needs the following high level steps: Integrate LLVM-BOLT into the build system #186245 Add a profile staleness metric script which guides profile updates Enable profile collection in PyTorch CI cc @malfet @ptrblck @ms...

add this so we can keep track of how various changes impact build times. The main motivation for this comes from the desire to ensure that #186245 does not introduce major build time overhead but metrics should be helpful in general. Alternatives No response Additional context...

o Kineto could utilize the export_using_protocol method in order to access them in Pytorch. A draft PR of the changes has been opened here: #189097 Alternatives No response Additional context No response cc @robieta @chaekit @guotuofeng @guyang3532 @dzhulgakov @davidberard98 @...

d-environment linux-jammy-cuda13.0-py3.12-gcc11-sm100, single unsharded (1, 1) job on one Blackwell GPU. This is the NVIDIA/b200 sibling of #188557 (which tracks the same failure mode on ROCm gfx950). Same root cause and fix levers; different infra/owners, so filing separately...

never completes. Genuine per-test failures in the same job (e.g. the test_backward_prod_cuda_float32 mem-leak-check, tracked separately in #140556) are a distinct problem from this timeout. Fix levers (CI-side) Increase the shard count (currently 8) so each shard's slice fits...

0 runner pool: investigate GPU state on linux.dgx.b200.8 (leaked processes, compute mode, or reimage). Nothing to fix in test code. Related #154297 - Hangs and timeouts on dist.reduce_scatter on B200 GPU (user-reported; possibly shared underlying B200 cause). #162178 - [CUDA]...

test code. Related #154297 - Hangs and timeouts on dist.reduce_scatter on B200 GPU (user-reported; possibly shared underlying B200 cause). #162178 - [CUDA] Umbrella Issues/Failures on B200 Runner (stale umbrella). cc @ezyang @gchanan @kadeng @msaroufim @awgu @wanchaol @fegin @...

al MLA / attention changes that alter KV-cache layout/strides and Sm90 backend selection for the MLA path -- most notably: vllm-project/vllm#45111 - "Re-enable cross-layer KV cache layout for MLA via stride-aware kernels" (changes MLA KV-cache strides/layout) -- primary suspec...

iPy to a NumPy-2-compatible build (SciPy >=1.13 supports NumPy 2). Whoever owns the H100 image / bumped numpy 2.2.6 should confirm. Related #173059 -- same class (NumPy binary/ABI incompatibility breaking test imports), different instance (pandas + test_dataloader/test_datapip...

s-is, so only the slice at index 0 along the broadcast dimension gets values and the rest stay zero. This looks like the same root cause as #158861, where `@nikitaved` noted that explicit broadcasting via `sparse_broadcast_to` followed by coalesce is missing. Proposed fix: bef...

each worker do a serial 1-element torch.sqrt first; this drops the rate to 0 (5,200 first-calls), which is the basis of the fix proposed in #188791. mkl_vml_race_repro.py """Minimal repro: MKL VML first-call dispatch race in torch.sqrt. Spawns waves of worker processes on an o...

Here the build is fine, the GPU is found, tests run and pass, and the job is killed purely by the 270-min wall-clock. (Other gfx1100 issues #152595 / #165141 / #169857 / #177834 are unrelated.) Suggested fix Unblock now (CI-side): increase shard count for this job (it's num_sh...

s confirmed for the current runs but not individually proven as the culprit for all 388 red runs. This is NOT covered by the existing issue #165040 "DISABLED rocm / linux-jammy-rocm-py3_10-gfx1100 / test (default)" (OPEN; module: rocm, module: ci) is adjacent but distinct: it'...

uild is fine, the GPU is found, tests run and pass, and the job is killed purely by the 270-min wall-clock. (Other gfx1100 issues #152595 / #165141 / #169857 / #177834 are unrelated.) Suggested fix Unblock now (CI-side): increase shard count for this job (it's num_shards=2 tod...

ne, the GPU is found, tests run and pass, and the job is killed purely by the 270-min wall-clock. (Other gfx1100 issues #152595 / #165141 / #169857 / #177834 are unrelated.) Suggested fix Unblock now (CI-side): increase shard count for this job (it's num_shards=2 today) so eac...

U is found, tests run and pass, and the job is killed purely by the 270-min wall-clock. (Other gfx1100 issues #152595 / #165141 / #169857 / #177834 are unrelated.) Suggested fix Unblock now (CI-side): increase shard count for this job (it's num_shards=2 today) so each Test ste...

e returns [nan, nan, nan], but Inductor returns [nan, 0., nan]. This changes the isnan result for the +inf element. This appears related to #187332 , but it affects torch.special.bessel_y0. Repro import torch def test_bessel_y0_mismatch(): x = torch.tensor([float('nan'), float...

ing unwinding → std::terminate. This is a pre-existing latent bug (the per-segment unmap loop had the same defect); it is not introduced by #187955, but that PR's unmapHandles rework touches the same code, so filing as follow-up. Mechanism map() / fromShared() set handles_[beg...

Being fixed in #187955: mapAndSetAccess is now strongly exception-safe — on a partial map/setAccess failure it unmaps the mapped prefix and releases the range's h

y implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on #77764. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op....

xpu: GPU detected, data transfer works, but any non-trivial computation (matmul) fails with could not create an engine. Related to existing #179891 (get_device_properties segfault on BMG). Intel compute-runtime issue cross-reference: intel/compute-runtime#947 Environment Compo...

Thanks @Sahil170595. The PR is #188719

itional nodes, switch can do the same in CUDA >= 12.8. Alternatives No response Additional context This has previously been requested here: #62530 cc @mcarilli @ezyang @eellison @penguinwu @BoyuanFeng @chauhang @ydwu4 @bdhirsh @bobrenjc93 @aorenste

aise a clean Python exception such as OverflowError, ValueError, or a normal CUDA OOM RuntimeError, not SystemError. This may be related to #177827, but it does not seem to be the same issue. #177827 is about negative allocation sizes, while this issue is triggered by a positi...

MetalShaderLibrary& MetalShaderLibrary::getBundledLibrary() { static BundledShaderLibrary* l = new BundledShaderLibrary(); return *l; } PR: #188801 (will be scoped down to just this leak). cc @malfet @aditvenk @kulinseth @DenisVieriu97 @jhavukainen @Isalia20 Versions PyTorch v...

Implemented the core pipeline changes in #188951, covers matrix generation, Makefile, Dockerfile, and workflow env vars. Would appreciate any feedback on the approach.

shaders, fixed for torch.mm/torch.bmm via tiling in PR #117549 but not all code paths), #122045 (F.linear wrong results for large inputs), #117826 (incorrect einsum gradient on MPS). Minimal reproduction The model is Embedding → 2 residual Linear blocks → output Linear, traine...

~1e12 in backward (not NaN). This is a silent gradient corruption — users see finite gradients when they should see NaN. Submitted test PR: #188066 Detected via NeuralDBG — causal diagnostic engine for PyTorch training.

1-D weight correctly, which is why eager rank >= 3 works. Context Found while adding 1-D-weight OpInfo samples to sample_inputs_linear for #187988 / PR #187989 (an unrelated MPS backward crash with 1-D weights): a sample with a scalar bias turns 14 functorch OpInfo tests red o...

orrectly, which is why eager rank >= 3 works. Context Found while adding 1-D-weight OpInfo samples to sample_inputs_linear for #187988 / PR #187989 (an unrelated MPS backward crash with 1-D weights): a sample with a scalar bias turns 14 functorch OpInfo tests red on all backen...

nce / ROCm follow-up #182028 ROCm MI200 — flex vs SDPA assert_close tolerances #186889 XPU — relaxed assertEqual vs eager (atomic ordering) #169694 Transformer tests — golden precision / reference choice #183691 Test infra — tolerance overrides (helps symptoms; root cause is s...

ose tolerances #186889 XPU — relaxed assertEqual vs eager (atomic ordering) #169694 Transformer tests — golden precision / reference choice #183691 Test infra — tolerance overrides (helps symptoms; root cause is still design in individual tests) 4. Conclusion & proposed next s...

oesn't solve the root of the issue. Tight same() / eager-fp16 equality is stricter than the bug needs and varies by hardware (e.g. #179958, #184531) — false failures even when the grid fix is sound. 3. Other examples (same pattern family) Link Note #183630 test_bmm_large_batch...

Here is a PR that implements tests for bmm issue in a way that I think is better than current implementation: #187050 It shouldn't be used to close the issue, just serve as an example of the direction that I propose to move towards to.

ike GraphPP and Eager Torchtitan PP. We are changing PP to no longer handle microbatch splitting and let the user supply microbatches here: #188500 Torchtitan will now produce microbatches right at source i.e. dataloader: pytorch/torchtitan#3856 So no changes are required from...

tem.py (which is brittle) and implement their own workarounds for asynchronous saving over fsspec. Additional context This is drafted in PR #188698 and discussed offline with @LucasLLC and @meetv18. Opening this issue to take it further with the process cc @awgu @wanchaol @feg...

fast_accum_True_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia

m_True_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia

ast_accum_False_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia

_False_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia

ast_accum_False_scaling_block_sizes0_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia

_False_scaling_block_sizes0_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia

_.from needs to be renamed" 'test_advancedindex_mixed_cpu_devices' - "FIXME" 'test_advancedindex_mixed_devices_error' - "FIXME" #108181 and #108798 'test_typed_storage_deprecation_warning' - "FIXME" 'test_pin_memory' - "pin_memory isn't yet supported in TorchInductor" 'test_me...

: module 'cutlass.cute.arch' has no attribute 'ProxyKind' (unpinned cutlass_api vs pinned nvidia-cutlass-dsl==4.5.2). Root cause tracked in #188477. Unstable until the cutlass_api / cutlass-dsl version skew is resolved. cc @malfet @pytorch/pytorch-dev-infra

fast_accum_True_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-...

m_True_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @Xia...

fast_accum_True_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @we...

m_True_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv...

ast_accum_False_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-...

_False_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @Xia...

fast_accum_True_scaling_block_sizes0_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @we...

m_True_scaling_block_sizes0_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv...

ast_accum_False_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @mruberry @jianyuh @nikitaved @walterddr @lezcano @chauhang @penguinwu @v...

_False_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @ptrblck @msaroufim @eqy @tinglvv @nWEIdia @mruberry @jianyuh @nikitaved @walterddr @lezcano @chauhang @penguinwu @voznesensk...

ast_accum_False_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @we...

_False_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv...

ast_accum_False_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @we...

_False_scaling_block_sizes2_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv...

ast_accum_False_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @we...

_False_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv...

fast_accum_True_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @we...

m_True_scaling_block_sizes1_cuda One of ~14 failing test_main_loop_scaling parametrizations (w = BlockWise1x128 recipes). See also #188675, #188704. cc @mruberry @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv...

he system, it downloads its own python-six sources files from the web (pypi to be exact) This makes pytorch not buildable from sources only #188263 shows the changes needed for allowing air-gapped builds To reproduce the bug simply try to build pytorch on a machine without int...

@Blooming-Tree I noticed that #188420, #188430, and #188432 appear to have the same methodology issue. They compare eager FP64 against compiled FP32 rather than eager FP32 again

@Blooming-Tree I noticed that #188420, #188430, and #188432 appear to have the same methodology issue. They compare eager FP64 against compiled FP32 rather than eager FP32 against compil

@Blooming-Tree I noticed that #188420, #188430, and #188432 appear to have the same methodology issue. They compare eager FP64 against compiled FP32 rather than eager FP32 against compiled FP32, so t

ue, please check the box on this page and assign the issue to yourself. Thanks! C10D NCCL #131203 #112815 #94003 #58856 GLOO #115740 #16295 #15963 Tcpstore #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC #125365 #74180 #70753 #705...

an issue, please check the box on this page and assign the issue to yourself. Thanks! C10D NCCL #131203 #112815 #94003 #58856 GLOO #115740 #16295 #15963 Tcpstore #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC #125365 #74180 #7075...

1 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC #125365 #74180 #70753 #70546 #69179 #69178 #64093 #51062 DeviceMesh and DTensor DeviceMesh #132833 #123906 #122525 #122664 #121951 #121776 #121372 #120706 #120692 #110026 #98875 #98059 #9...

u would like to take an issue, please check the box on this page and assign the issue to yourself. Thanks! C10D NCCL #131203 #112815 #94003 #58856 GLOO #115740 #16295 #15963 Tcpstore #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC...

e #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC #125365 #74180 #70753 #70546 #69179 #69178 #64093 #51062 DeviceMesh and DTensor DeviceMesh #132833 #123906 #122525 #122664 #121951 #121776 #121372 #120706 #120692 #110026 #98875 #9...

Tcpstore #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC #125365 #74180 #70753 #70546 #69179 #69178 #64093 #51062 DeviceMesh and DTensor DeviceMesh #132833 #123906 #122525 #122664 #121951 #121776 #121372 #120706 #120692 #110026 #9...

#15963 Tcpstore #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100448 RPC #125365 #74180 #70753 #70546 #69179 #69178 #64093 #51062 DeviceMesh and DTensor DeviceMesh #132833 #123906 #122525 #122664 #121951 #121776 #121372 #120706 #120692 #11...

. If you would like to take an issue, please check the box on this page and assign the issue to yourself. Thanks! C10D NCCL #131203 #112815 #94003 #58856 GLOO #115740 #16295 #15963 Tcpstore #61671 #50701 MultiProcessing #65559 MultiThreadedTestCase #107425 #107424 #100459 #100...

14 #132447 #131446 #127373 #126868 #126852 #126493 #123294 DDP, FSDP, PiPPy FSDP1 #123726 #105024 FSDP2 #120961 DDP #106361 #104328 #104011 #103625 #77342 #77317 pipeline #121234 Other DCP #113937 #113936 Functional Collectives #107278 ShardedTensor #78068 cc @XilunWu @H-Huang...

in #132114 #132447 #131446 #127373 #126868 #126852 #126493 #123294 DDP, FSDP, PiPPy FSDP1 #123726 #105024 FSDP2 #120961 DDP #106361 #104328 #104011 #103625 #77342 #77317 pipeline #121234 Other DCP #113937 #113936 Functional Collectives #107278 ShardedTensor #78068 cc @XilunWu...

ributed 500/shm_broadcast hang) and 4 (CPU-MM 3 timeout) are silent on build #72925 (2026-06-18) — both passed; never filed upstream. Build #73764 status (2026-06-23) Still reproducing (fixes in pytorch main, not yet in test-channel wheel): #187484 (Inductor FxGraphCache guard...

eam. #187484 — [Inductor] w8a8 block-fp8 BackendCompilerFailed: inductor (DeepGEMM H100, FusedMoE H100/B200, Kernels MoE Test 1–4) · Cause: #184193 · Fix: revert #187666 (release/2.13, bisected — all 75 failures pass reverted) #187735 — [CPU multimodal] qwen2_vl multi-image to...

ture, motivation and pitch These enhancements are to have a better UX when using foreach_map, suggested in a few places, but most recently, #158371 (comment) These enhancements should allow easier compiler-first custom optimizer implementations. Alternatives No response Additi...

lace to list and track work on adding support to new ops for the MPS backend. Most requested ops (extracted from comments to this issue and #141287) PyTorch MPS Ops Project : Project to track all the ops for MPS backend. There are a very large number of operators in pytorch an...

speeding up pickling itself, it would be great to have core support for arrays of strings represented as tensors (and simple data frames): #101699 If there is a million-sized list of python strings, pickle probably would be slower compared to a single tensor of concatenated ut...

ses manual attention. It works on both AOTriton v0.11.2 and CK: a, w = attn(x, x, x, need_weights=True) # OK on both backends Updated PR at #188114 with all four files.

uted.device_mesh import init_device_mesh from torch.distributed.tensor.parallel import ColwiseParallel, parallelize_module # Workaround for #175467 (DTensorSpec not registered as pytree constant). torch.utils._pytree.register_constant( torch.distributed.tensor._dtensor_spec.DT...

There is an AI generated fix here: #184508 Which is waiting for review.

recompile under torch.compile on every new input shape, while the functional variants specialize only once. Reported originally as part of #135859 for linalg.norm and cholesky; the max / min / topk cases (structured ops) are fixed in #187952, but the composite ops need a diffe...

ze only once. Reported originally as part of #135859 for linalg.norm and cholesky; the max / min / topk cases (structured ops) are fixed in #187952, but the composite ops need a different fix. Minimal repro: import torch torch._logging.set_logs(recompiles=True) def f(x): out =...