Silent Failures Beyond the 32-Bit Index Range: A Differential Characterization of Large-Tensor Matrix Multiplication in PyTorch's MPS Backend
Apple Silicon machines with large unified memory make it possible to hold large tensors on a desktop GPU, but PyTorch's Metal Performance Shaders backend silently returns wrong results for batched matrix multiplication with more than $2^{32}$ elements is found.