We introduce ExaTron.jl which is a scalable GPU-MPI-based batch solver for many small nonlinear programming problems. Its algorithm is based on a trust-region Newton algorithm for solving bound constrained nonlinear nonconvex problems. In contrast to existing work in the literature, it completely works on GPUs without requiring data transfers between CPU and GPU during its procedure. This enables us to eliminate one of the main performance bottlenecks under memory-bound situation. We present ExaTron.jl's architecture, its kernel design principles, and implementation details with experimental results comparing different design choices. We have implemented an ADMM algorithm for solving alternating current optimal power flow, where tens of thousands of small nonlinear nonconvex problems are solved by ExaTron.jl. We demonstrate a linear scaling of parallel computational performance of ExaTron.jl on Summit at Oak Ridge National Laboratory.