How to upgrade your Scaleway server

You have a Scaleway Virtual Private Server (VPS) and you are considering upgrading your installed Linux distribution. Perhaps you have been notified by Scaleway to upgrade an old Linux version. The email asks you to upgrade but does not give you the necessary information on how to upgrade or how to avoid certain pitfalls.

What could go wrong when you try to upgrade?

A few things can go wrong.

Watch out for Bootscripts

The most important thing that can go wrong, is if your VPS is using a bootscript. A bootscript is a fixed way for booting your Linux server and this included a generic Scaleway-provided Linux kernel. You would be running Ubuntu but the Linux kernel would be a common Scaleway Linux kernel among all Linux distributions. The config options would be set in stone and that would cause some issues. That situation changed and Scaleway now uses the distro Linux kernels. But since Scaleway sent an email about old Linux versions, then you need to check this one out.

To verify, go into the Advanced Settings, in Boot Mode. If it looks as follows, then you are using a bootscript. When you upgrade the Linux version, then the Linux kernel will stay the same as instructed by the bootscript. The proper Boot Mode should be “Use local boot” so that your VPS is using your distro’s Linux kernel. Fun fact #39192: If you offer Ubuntu to your users but you do not use the Ubuntu kernel, then Canonical does not grant you a (free) right to advertise that you are offering “Ubuntu” because it’s not really real Ubuntu (the Linux kernel is not a stock Ubuntu Linux kernel). Since around 2019 Scaleway would default with the Use local boot Boot Mode. In my case it was indeed Use local boot, therefore I did not have to deal with bootscripts. I just clicked on Use bootscript for the purposes for this post; I did not apply this change.

Verify that the console works (Serial console, recovery)

You normally connect to your Linux server using SSH. But what happens if something goes wrong and you lose access? Specifically, if you are upgrading your Linux installation? You need a separate way, a backup option, to connect back to the server. This is achieved with the Console. It opens up a browser window that gives you access to the Linux Console of the server, over the web. It’s separate from SSH, therefore if SSH access is not available but the server is still running, you can still access here. Note that when you upgrade Debian or Ubuntu over SSH with do-release-upgrade, the upgrader creates a screen session that you can detach and attach at will. If you lose SSH access, connect to the Console and attach there.

Note two things.

1. The Console in Scaleway does not work on Firefox. Anything that is based on chromium should work fine. It is not clear why it does not work. If you try to place your mouse cursor on the button, it shows Firefox is not currently compatible with the serial console.
2. Make sure that you know the username and password of your non-root account on your Scaleway server. No, really. You would normally connect with SSH and Public-Key Authentication. For what is worth, the account could be locked. Try it out now and get a shell.

Beware of firewalls and security policies and security groups

When you are upgrading the distribution with Debian and Ubuntu, and you do so over SSH, the installer/upgrader will tell you that it will open a backup SSH server on a different port, like 1022. It will also tell you to open that port, if you use a Linux firewall on your server. If you plan to keep that as a backup option, note that Scaleway has a facility called Security Groups that works like a global firewall of your Scaleway servers. That is, you can block access to certain ports if you specify them in the Security Group, and you have assigned those Scaleway servers in that Security Group.

Therefore, if you plan to rely on access to port 1022, make sure that the Security Group does not block it.

How to avoid having things go wrong?

When you upgrade a Linux distribution, you are asked all sort of questions along the way. Most likely, the upgrader will ask if you want to keep back a certain configuration file, or if you want to have it replaced by the newer version.

If you are upgrading your Ubuntu server, you would install the ubuntu-release-upgrader-core package and then run do-release-upgrade.

$ sudo apt install ubuntu-release-upgrader-core
...
$ sudo do-release-upgrade
...

To avoid making a mistake here, you can launch a new Scaleway server with that old Linux distro version and perform an upgrade there. By doing so, you will note that you will be asked

1. whether to keep your old SSH configuration or install a new one. Install the new one and make a note to apply later any changes from the old configuration.
2. whether to be asked specifically which services to restart or let the system do these automatically. You would consider this if the server deals with a lot of traffic.
3. whether to keep or install the new configuration for the Web server. Most likely you keep the old configuration. Or your Web servers will not start automatically and you need to fix the configuration files manually.
4. whether you want to keep or update grub. AFAIK, grub is not used here, so the answer does not matter.
5. whether you want to upgrade to the snap package of LXD. If you use LXD, you should have switched already to the snap package of LXD so that you are not asked here. If you do not use LXD, then before the upgrade you should uninstall LXD (the DEB version) so that the upgrade does not install the snap package of LXD. If the installer decides that you must upgrade LXD, you cannot select to skip it; you will get the snap package of LXD.

Here are some relevant screenshots.

Troubleshooting

You have upgraded your server but your WordPress site does not start. Why? Here’s a screenshot.

A WordPress website requires PHP and normally the PHP package should update automatically. It actually does update automatically. The problem is with the Unix socket for PHP. The Web server (NGINX in our case) needs access to the Unix socket of PHP. In Ubuntu the Unix socket looks like /run/php/php7.4-fpm.sock.

# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ .php$ {
     include snippets/fastcgi-php.conf;
     #
     # # With php7.0-cgi alone:
     # fastcgi_pass 127.0.0.1:9000;
     # With php7.0-fpm:
     fastcgi_pass unix:/run/php/php7.4-fpm.sock;
}

A request

Scaleway, if you are reading this, please have a look at this feature request.

Simos Xenitellis