PyTorch solve_ex hits 10x 512 cliff

My read is this is almost certainly a backend-selection or kernel-tuning cliff, not a mathematical property of the solve itself. GPU library performance is piecewise, not smooth, so a one-element shape change can matter more than a larger algorithmic change.

• –Exact matrix sizes can flip PyTorch/cuSOLVER/MAGMA onto different kernels, tile sizes, or workspace paths
• –Batched linalg is especially sensitive because the “same” operation can hit very different implementation paths as dimensions cross a threshold
• –If you rely on GPU solves in training or inference, benchmark neighboring shapes, dtypes, and batch sizes before you lock architecture
• –Padding or chunking away from pathological sizes is often the pragmatic fix when you hit a cliff like this