Astronomy is experiencing an explosion in the number of observations of celestial objects due to large-scale imaging surveys aimed to understand the formation & evolution of many classes of objects. Our team of data scientists and astronomers is involved in two such surveys: KiDS and Euclid. Our objective is to extract samples of various rare types of objects with high completeness/recall and purity/precision from an ocean of up to billions of source detections. This often requires a laborious effort combining automated feature extraction followed by human inspection. We have encountered the limit of this approach for KiDS, while Euclid will increase the challenge by an order of magnitude. In this paper we report on experiments to simultaneously identify 3 classes of rare objects (galaxy mergers, strong gravitational lenses and asteroid streaks) via a tiered cascade of Convolutional Neural Networks.