As downstream testers, we used to hit a few regression bugs when testing downstream. Normally they would get fixed upstream first, then downstream, which causes some pain points for both testers and developers. For testers, they need to run git bisect and repeat tests to find out the culprit. It takes much time and effort especially when there are a lot of patches and the testing is complex. For developers, they need to revisit the patches to find out the root cause. It might be hard to recall all the details as the patches could be worked on quite a long time ago. Besides, we have a strict schedule for each release, and if a bug is discovered in a late phase, it would be risky to get it fixed and tested in limited time, thus compromising our product to some extent.
Based on all the reasons, we brought some testing from downstream to upstream. Not only test upstream code regularly, but also run some tests against patches under review, engage with developers actively, and provide test results before the patches get merged into master. A lot of benefits for upstream and downstream products and also individuals have been gained. In this talk, we will introduce our experience of upstream testing, the effort we put, and the benefits and achievements we get. Also some tips about undertaking upstream testing and insights regarding how to cooperate with developers upstream better will be shared. We intend to call for more participation from both developers and testers for upstream testing to help improve the upstream code quality together.