We’ve been doing some thinking about how to proceed with a number of big sysadmin tasks that are currently on our plate:
- Canada Production needs to be upgraded from Ubuntu 14 and provisioned with ofn-install.
- We need to undertake a managed rollout of v2
- We’re ready to start testing and upgrading all servers to Ubuntu 18
If we try to do all of these things at once it will be absolute chaos, so we have a proposed roadmap for accomplishing all of these jobs (in order):
- Upgrade Canada Production to Ubuntu 16 (with ofn-install) ASAP.
- Rollout v2. See: OFN v2 rollout plan
- Postpone testing and upgrading servers to Ubuntu 18 until Q3/Q4. This will not be a quick process, and we shouldn’t do it until the v2 rollout is finished and the dust has settled. The plan here would also include keeping half our staging servers on 16 and half on 18 so we can compare/diagnose builds on both versions whilst the upgrades are still in progress.
So, the top priority now is the upgrade and migration of Canada Production, which we can plan to do over the next month. This will need some co-ordination between the instance and sysadmins. The plan for the upgrade looks like this:
- We’ll need a second server for Canada so we can do: Current Server -> New Server. This will need some co-ordination in terms of web hosting accounts and access etc.
- Start setting up the new server with ofn-install (making sure to deploy with the same release), and make sure it’s working. Copy any custom files like logos, TOC’s, etc.
- Plan a time for the switchover and announce scheduled downtime to Canada’s users (we should be able to use our timezone differences to do it while Canada is asleep)
- At the scheduled time we shut down the current server, export the database, import it into the new server, then switch the DNS so the domain points to the new server’s address. This shouldn’t take more than 10 minutes in theory, but we can announce an hour downtime so we leave some room for the unexpected.
- After it’s all done and the new setup is working, the old server can then be cancelled.
How does that sound?