Radio observations serve as a powerful means of exploring the universe. Tianlai telescope observes the dark energy of the universe through radio radiation. The Tianlai telescope is a radio array composed of multiple radio telescopes. It obtains the distribution of matter in the universe by observing the 21cm spectral line of neutral hydrogen, and indirectly realizes the observation of dark energy. However, the 21cm spectral line of radio observation is very weak, and it is easily interfered by other weak celestial sources, which is a problem that must be solved for normal operation. In order to detect these weak radio signals, we propose a celestial source fringes detection method based on the Transformer self-attention mechanism. To prove the effectiveness of our proposed method, we performed some experiments on the method and compared with other deep learning methods in terms of mean of Average Precision. We also verified the impact of data quality on the model, and fringe signals can be detected at low signal-to-noise ratios. Experimental results show that the method achieves comparable detection performance for radio celestial source fringes in terms of accuracy and location regression. This study is important for the detection of dark energy in the universe through radio observation of the neutral hydrogen 21cm signal.