The sparse module of the popular SciPy Python library is widely used across applications in scientific computing, data analysis, and machine learning. The standard implementation of SciPy is restricted to a single CPU and cannot take advantage of modern distributed and accelerated computing resources. We introduce Legate Sparse, a system that transparently distributes and accelerates unmodified sparse matrix-based SciPy programs across clusters of CPUs and GPUs, and composes with cuNumeric, a distributed NumPy library. Legate Sparse uses a combination of static and dynamic techniques to performantly compose independently written sparse and dense array programming libraries, providing a unified Python interface for distributed sparse and dense array computations. We show that Legate Sparse is competitive with single-GPU libraries like CuPy and the industry-standard PETSc library on up to 1280 CPU cores and 192 GPUs of the Summit supercomputer, while offering the productivity benefits of idiomatic SciPy and NumPy.


  title={Legate Sparse: Distributed Sparse Computing in Python},
  author={Rohan Yadav and Wonchan Lee and Melih Elibol and Manolis Papadakis and Taylor Lee-Patti and Michael Garland and Alex Aiken and Fredrik Kjolstad and Michael Bauer},
  journal={ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis},