08-29, 11:30–11:50 (Europe/Berlin), Room 6
The array API standard is unifying the ecosystem of Python array computing, facilitating greater interoperability between array libraries, including NumPy, CuPy, PyTorch, JAX, and Dask. Find out how we are using it in SciPy to bring support for hardware-accelerated (e.g. GPU) and distributed arrays to our users, and how you can do the same in your library.
SciPy has had "support for distributed and GPU arrays" on its roadmap for over five years now. For a library built around NumPy arrays, that is easier said than done. There are other array libraries, such as CuPy, PyTorch, JAX, and Dask, which can help address these user wishes.
Supporting multiple array libraries is not simple, since their APIs differ. Lengthy if-else statements in every function won't cut it - what we want is to be able to write 'array-agnostic' code which will work with multiple array libraries, without having to special-case on the input array type.
The Python array API standard aims to standardise functionality that exists in most array libraries. It specifies an API which 'array-consumer' libraries can use to write array-agnostic code. In this talk, I give a brief introduction to the standard, before explaining how we are implementing support for it in SciPy, and the progress which we have made so far.
Rough talk outline:
- 5 mins - what is the array API standard, and why should you care?
- 7 mins - what work is needed for a consumer library to adopt the array API standard, and what does that look like in SciPy? What tools are available to help?
- 3 mins - current progress in SciPy & looking to the future.
Find out how we are using the array API standard in SciPy to bring support for hardware-accelerated and distributed arrays to our users, and how you can do the same in your library.
Category [High Performance Computing] –Vector and Array Manipulation
Expected audience expertise: Domain –none
Expected audience expertise: Python –some
Public link to supporting material – Project Homepage / Git –
SciPy Maintainer | Computer Science & Philosophy Undergraduate Student at Christ Church, University of Oxford | Based in Newcastle upon Tyne, UK