Booting up an AMD EPYC server at packet.net

This is the second post in the series. The first was about selecting the appropriate settings so that we can deploy a server.

  1. A closer look at AMD EPYC baremetal servers at packet.net
  2. Booting up the AMD EPYC baremetal server at packet.net (this post)
  3. Configuring LXD on the AMD EPYC baremetal server at packet.net
  4. Benchmarking LXD on an AMD EPYC server at packet.net

In the following, we press the Deploy Servers button and the baremetal server gets provisioned. Before the server gets ready to use, it has to boot twice. During the first boot, the disks are configured and the operating system is installed. In the second boot, our server boots up and is ready to use.

Pressing Deploy Servers and the first boot for installation

This is the button for Deploy Servers. Press it.

Button to deploy the server(s).

The baremetal server gets provisioned and the boot process is started. The Web management interface now looks like this.

The server gets deployed. Notice the last column “View Progress”. It takes some time for a server to get deploy and become fully operational.

Under the View Progress column, there are icons of a clock and a lifebuoy. The clock shows the progress in easily-readable text. Here is how it looks. In the screenshot, it shows the the installation (first boot) has completed and it is about to boot again for the final time.

The lifebuoy shows the following popup window, which has information on how to connect to the Out-of-Band Console. Out-of-Band means that the network connection is independent from the main network connection to the server. It makes sense anyway, because the server has not even started yet. And the Console means that we get to view the console of the server, seeing all boot-up messages, and Linux kernel messages. This SSH connection requires that you have uploaded earlier your SSH public key to packet.net. When you ssh from your computer, SSH has access to the corresponding private key.

Let’s SSH to that address and start perusing the boot-up messages! We get a read-only session, and we can perform some actions.

$ ssh f82319e2-13fe-4478-b30b-22afd7fcd74a@sos.ams1.packet.net
[Read-only SOS Session. Use ~? for help.]
~?
Supported escape sequences:
~. - terminate connection (and any multiplexed sessions)
~B - send a BREAK to the remote system
~C - open a command line
~R - request rekey
~V/v - decrease/increase verbosity (LogLevel)
~^Z - suspend ssh
~# - list forwarded connections
~& - background ssh (when waiting for connections to terminate)
~? - this message
~~ - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

It takes some time for the server to actually boot for the first time, therefore, we have a good chance to view the full boot messages. Now, it boots for the first time. The very first messages are the following. I have not tried pressing any of these.

KEY MAPPING FOR CONSOLE REDIRECTION:

Use the <ESC><1> key sequence for <F1>
Use the <ESC><2> key sequence for <F2>
Use the <ESC><3> key sequence for <F3>
Use the <ESC><0> key sequence for <F4>
Use the <ESC><!> key sequence for <F5>
Use the <ESC><@> key sequence for <F5>

Use the <ESC><Ctrl><M> key sequence for <Ctrl><M>
Use the <ESC><Ctrl><M> key sequence for <Ctrl><H>
Use the <ESC><Ctrl><M> key sequence for <Ctrl><I>
Use the <ESC><Ctrl><M> key sequence for <Ctrl><J>

Use the <ESC><X><X> key sequence for <Alt><x>, where x is any letter
key, and X is the upper case of that key

Use the <ESC><R><ESC><r><ESC><R> key sequence for <Ctrl><Alt><Del>

Press the spacebar to pause…

Then, the server actually boots.

Initializing PCIe, USB, and Video… Done
PowerEdge R6415
BIOS Version: 1.4.3
PXE Boot Requested by iDRAC
F2     = System Setup
F10 = Lifecycle Controller (Config
iDRAC, Update FW, Install OS)
F11 = Boot Manager
F12 = PXE Boot
iDRAC IP: 10.248.46.33
Initializing Firmware Interfaces…
Enumerating Boot options…
Enumerating Boot options… Done
Loading Lifecycle Controller Drivers…
Loading Lifecycle Controller Drivers…Done
Lifecycle Controller: Collecting System Inventory…
iDRAC IP:  10.248.46.33

It says PowerEdge R6415, which means it’s a Dell rack server. The default configuration costs $2,129.00. If you customize the server and just select the correct CPU and 64GB RAM, you get to $4,046.69. If you add on top of that the 960GB in disk space (four disks), the server is probably well over $5000.

The Dell PowerEdge R6415.

Here are the full boot messages for the first boot. Specifically, it uses iPXE to boot for the first time over the network. It boots into an installer that is based on Alpine Linux and then performs the initialization of the disks for the Linux distribution of our choice (here, Ubuntu). It gets the configuration from several other servers over HTTP. It even clones a Git repository with configuration about the supported Linux distributions as a way to receive their configuration files. There are also steps where the server phones home to to register that it is starting and what stage it is in. And finally it reboots so that the server boots into the requested Linux distribution.

Booting the Linux distribution (Ubuntu 18.04)

The booting process continues with the second reboot and Ubuntu boots up. It is all unattended and we continue to view the process through the out-of-band console. Here is an image of the Grub menu that quickly flashes by.

Grub menu on packet.net baremetal server.

Here are the full kernel messages of the server up to the point of spawning the console login. Some interesting information include:

  1. The kernel version is Linux version 4.15.0-20-generic (buildd@lgw01-amd64-039) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 (Ubuntu 4.15.0-20.21-generic 4.15.17). This looks like a Linux kernel that was built by Canonical.
  2. The exact CPU is CPU0: AMD EPYC 7401P 24-Core Processor (family: 0x17, model: 0x1, stepping: 0x2). There are two threads in each core, total 48.
  3. There are a lot of BogoMIPS, smpboot: Total of 48 processors activated (191,638.46 BogoMIPS)
  4. It currently applies the following microcode update, microcode: CPU0: patch_level=0x08001227
  5. There are two 480 GB/447 GiB SSD disks (Intel).
  6. There are also two 120 GB/112 GiB M.2 SATA SSD disks.
  7. There are two ConnectX®-4 Lx EN Cards (bonded). These are 10Gbps Ethernet network cards.

Now, the server is ready to connect to. In the following we see what options we have to connect to the server.

Connecting to the server

First of all, let’s see the status page of the running server at packet.net. We get a good overview. What is highlighted, is the SSH root password. I obviously changed the value for the purposes of the screenshot. Thankfully, the default SSH configuration of Ubuntu does not allow password authentication, therefore you cannot SSH with a password because Permission denied (publickey). So, what use does this password have? This password is used for the out-of-band console. You get there a login prompt, you put root and this password. But to connect to the out-of-band console, you need to connect first with SSH using public-key authentication.

Status page of server.

Let’s get a shell through the out-of-band console.

$ ssh f82319e2-13fe-4478-b30b-22afd7fcd74a@sos.ams1.packet.net
[SOS Session Ready. Use ~? for help.]
[Note: You may need to press RETURN or Ctrl+L to get a prompt.]

Ubuntu 18.04 LTS myserver ttyS1

myserver login: root
Password: *********
Welcome to Ubuntu 18.04 LTS (GNU/Linux 4.15.0-20-generic x86_64)

* Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

 * Security certifications for Ubuntu!
We now have FIPS, STIG, CC and a CIS Benchmark.
- http://bit.ly/Security_Certification

* Want to make a highly secure kiosk, smart display or touchscreen?
Here's a step-by-step tutorial for a rainy weekend, or a startup.
- https://bit.ly/secure-kiosk

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@myserver:~#

And here we connect using SSH and public-key authentication, directly to the server.

$ ssh root@147.75.83.19
Welcome to Ubuntu 18.04 LTS (GNU/Linux 4.15.0-20-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage * Security certifications for Ubuntu!
We now have FIPS, STIG, CC and a CIS Benchmark.
- http://bit.ly/Security_Certification
* Want to make a highly secure kiosk, smart display or touchscreen?
Here's a step-by-step tutorial for a rainy weekend, or a startup.

- https://bit.ly/secure-kiosk
Last login: Mon Oct 1 22:22:22 2018
root@myserver:~#

Summary

We have managed to connect to the server, either the out-of-band console or directly with SSH. We are ready to move to the next step and actually use the server.

  1. A closer look at AMD EPYC baremetal servers at packet.net
  2. Booting up the AMD EPYC baremetal server at packet.net (this post)
  3. Configuring LXD on the AMD EPYC baremetal server at packet.net
  4. Benchmarking LXD on an AMD EPYC server at packet.net

Permanent link to this article: https://blog.simos.info/booting-up-an-amd-epyc-server-at-packet-net/

1 ping

  1. […] Booting up an AMD EPYC server at packet.net […]

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.