We created the wiki pages The process of review, test, merge and deploy and Testing process to agree on a way how we deal with pull requests. They are missing some details about the staging and merging which we should add.
Based on the experiences in Australia, I would like to suggest the following:
- Stage a merge commit of master and the pull request: Whenever a pull request is created, Github creates a merge commit to find out if there are any merge conflicts. Our continuous integration (CI: Travis and Semaphore) uses that merge commit to test the merged code which can be different to the feature branch. We should stage the same version to make sure that merging the code doesn’t introduce another bug.
- Re-trigger CI: When the master branch is updated, our CI services don’t re-run automatically, because that would result in too many test runs. So when we stage a branch that is not based on or merged with master, we should trigger CI to run on the merged code again. In the past, we created branches called
merged-pullxxxxfor this purpose, but they were “polluting” our repository and were never deleted. - Automate these tasks and checks in some way.
- Enable testers to stage pull requests themselves.
The questions are:
- Do you think the above are good ideas?
- If yes, how should we implement them?
Some ideas include:
- Create a special branch for each staging server. That enables us to track what is staged and triggers Travis again.
- Or create a fork for each staging server so that we have the full Travis capacity for each of them. It also means that we don’t have random branches on the main repository. The downside is that not everything is in one place.
- Our use of Buildkite enabled us to stage via a friendly user interface and it automates all the merging and checking that tests pass. We can re-activate that. The current scripts need some updates according to the new process.
- We build our own little interface for staging servers to make sure that we do the right thing when staging.