In this talk, we give an introduction to the AlphaZero algorithm and discuss some research challenges of using it beyond board games. In an effort to make this algorithm widely accessible to students and researchers, we introduce AlphaZero.jl. We show how this package leverages Julia's strengths to provide an implementation that is simple and flexible, while being up to two orders of magnitude faster than comparable Python implementations.