So far, most nonlinear optimization modelers and solvers have primarily targeted CPU architectures. However, with the emergence of heterogeneous computing architectures, leveraging massively parallel accelerators in nonlinear optimization has become crucial for performance. As part of the Exascale Computing Project ExaSGD, we are studying how to efficiently run nonlinear optimization algorithms at exascale using GPU accelerators.