Matt’s original implementation didn’t merge master into the PR. He changed his pull request so that it’s now using a merge commit, but I’m not sure if he updated UK staging.
Preproduction is “same as production” which is not the case when we test on staging, they have staging data not production data. Preproduction is you copy the production and test on it so you have the real data and real problems… I learnt that in the discussions with GLP IT architect when they were thinking about using OFN.
https://www.ibm.com/support/knowledgecenter/en/SSWT7D_1.0.0/com.ibm.commercecloud.overview.doc/concepts/cov_overview_environment_preprod.htm
From what I understood any reasonable size project has that, @sauloperez you didn’t have that at Redhat ?
I don’t know if we would need that if we go toward continuous integration… maybe not.
I think I was confused between release and upgrade. In my mind on continuous integration as soon as a PR is merged a new release happen and a new upgrade happen, but probably not, we still have a periodical upgrade process from what I understand now.
Anyway it’s a bit clearer to me and all what you said makes sense, it seems you have a plan that you are all happy with so let’s action it !
Thank you for the clarification @MyriamBoure. Pre-production could be useful, but it’s not as simple as copying the data. We need to make sure that we are not emailing real customers, charging real cards etc. Automating the database changes could be a large project, maybe only medium.
@MyriamBoure @maikel we are actually doing both. UK staging for example has production data. I agree that concerning people’s data it’s not at all a good practice right now (it is not anonymized), but it really helped me on Bulk invoice printing to have real live cases on a large database.
So maybe the first step here is to ensure our staging data is getting better/larger (we already started that) and see in a second one if we need to set-up production clones?
I’ve been thinking lately about continous delivery and continous deployment and I wanted to study a bit more the topic. From our beloved Martin Fowler, I came across https://www.continuousdelivery.com/ and I think it’s one of the best sources on the matter. Make sure you check it out!
Also, I remember thinking there might better things we can do other than investing in pre-production environments but I don’t remember what was the alternative I thought of or why :trollface: I’ll try to remember.
Hello!
Re point 2 staging PR to master, I think we are doing it right now, is this correct? ping @maikel @luisramos0 @sauloperez @kristinalim @Matt-Yorkley
If we are doing this, tester can run a 5 min sanity check on each PR. This current sanity check involves:
1- You can login
2- You can create a product and it appears correctly in the shop as supposed
3- You can checkout
4- You see the order on the order view
More than that on each PR and testing will take too much time. If we do this and we remove release testing it would mean that features like BOM, Inventory, Subs, won’t be manually tested regularly unless there is a PR touching them.
I’m ok with that because I would prefer us to work on improving our automated tests (including an acceptance test framework Seed data [development] [provisioning] [deployment] or visual testing Automated visual testing ).
Any thoughts? Pinging @lin_d_hop as well
Those four tests seem very reasonable. And we can automate them as well. Let’s have a session about Selenium at some point. @Rachel
In addition, we should always look beyond the scope of the current pull request and think about other areas that could be affected. That may be a guess. Or if we can’t think of anything, we can just choose something we haven’t tested for a while. The goal is that we do test features like BOM or inventories from time to time, just not for every pull request. But if we cycle through them, we could still have a really good test coverage at any given point in time. And I don’t think these tests should be standardised. They have to be creative to increase our chance of catching uncommon bugs.