Comparison between LXC and LXD

More, but on the same hardware

Traditionally, we would have a physical computer and expect to run a single operating system on it. One way to go over this limitation, is to use virtualization, which allows us to run multiple operating systems (virtual machines) on a computer. For virtualization to work efficiently, we would need special virtualization support from the CPU (Intel CPUs: VT-x, AMD CPUs: AMD V). Relevant virtualization software include KVM, Xen, VirtualBox, Hyper-V and VMWare.

Virtualization is good, but takes lots of system resources because you boot up a full operating system for each virtual machine. Can we have an additional option for lightweight virtual machines that do not require to boot their own Linux kernel but can reusing the running Linux kernel of the host? Well, we can, and these are the Linux Containers.

Linux Containers

To be able to support Linux Containers, the Linux kernel had to be improved with some sort of container support. However, instead of getting actual dedicated container support, the decision was to add kernel features (primitives) that collectively could be used to implement Linux Containers. Such kernel features are control groups (cgroups) and namespaces.

There are several implementations of Linux containers, two of which are LXC and LXD. These two are system containers, because the containers look and behave like systems (or virtual machines). There are other containers, the application containers (such as Docker) with the purpose to run a distinct application instead of behaving as a virtual machine.

LXC and LXD

LXC came first, at around 2008. LXC is quite hands-on and the learning curve is somewhat steep. LXD was launched in 2015, using the same building components as LXC, with the aim to make system containers more usable and user-friendly.

Which to use

Currently, both LXC and LXD are supported and are being developed. If you are a new user without concrete needs for LXC, you should use LXD. If you are an existing LXC user, you can consider migrating to LXD. There is even a tool to migrate existing LXC containers to LXD containers. There are some cases where you would opt for LXC. For example, LXC is written in C and can be compiled to many systems. In contrast, LXD is written in the Go language, and some architectures may not have yet a Go compiler.

Terminology

  • host, the computer that runs either LXC or LXD.
  • container image, an image file for a Linux distribution that will be used by the host to start a container.
  • container, an instance of a container image.
  • remote, a repository with container images.

Command line

LXC provides many commands such as lxc-create, lxc-ls, and lxc-destroy. That is, each command has a “lxc-” prefix.

LXD provides two commands, lxd and lxc. lxd is the hypervisor (the LXD service), while lxc is the default CLI client that communicates with the running lxd hypervisor. Normally, you would run “sudo lxd init” after you have just installed LXD, where init is the subcommand to guide you through the initialization. After the initialization, you would run lxc with different subcommands to manage your containers.

Here is an example of creating a container for LXC. We install LXC (package lxc), create the container, then start it, next get a shell into it, and finally delete it.

$ sudo apt-get update
$ sudo apt install lxc
$ sudo lxc-create -t download -n mycontainer -- --dist ubuntu --release bionic --arch amd64
Using image from local cache
Unpacking the rootfs

---
You just created an Ubuntu bionic amd64 (20191208_07:42) container.

To enable SSH, run: apt install openssh-server
No default root or user password are set by LXC.
$ sudo lxc-ls --fancy
NAME        STATE   AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED 
mycontainer STOPPED 0         -      -    -    false        
$ sudo lxc-start -n mycontainer
$ sudo lxc-ls --fancy
NAME        STATE   AUTOSTART GROUPS IPV4       IPV6 UNPRIVILEGED 
mycontainer RUNNING 0         -      10.0.3.242 -    false        
$ sudo lxc-attach -n mycontainer
root@mycontainer:~# exit
$ sudo lxc-stop -n mycontainer
$ sudo lxc-destroy -n mycontainer
lxc-destroy: mycontainer: tools/lxc_destroy.c: main: 271 Destroyed container mycontainer
$ 

Here is an example of creating a container for LXD. LXD is pre-installed in Ubuntu, therefore we do not install here. We perform the initialization, then launch a container, execute a shell into it, subsequently exit the container, stop it and finally delete it.

$ sudo lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Would you like to connect to a MAAS server? (yes/no) [default=no]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=lxdbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
Would you like LXD to be available over the network? (yes/no) [default=no]: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes] 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: 

$ lxc launch ubuntu:18.04 mycontainer
Creating mycontainer
Starting mycontainer
$ lxc exec mycontainer -- /bin/bash
root@mycontainer:~# exit
$ lxc stop mycontainer
$ lxc delete mycontainer
$ 

Packaging

Both LXC and LXD are available as snap packages, which allows them to run on many different Linux distributions.

Both LXC and LXD are also available as native packages for several Linux distributions. In the case of Debian/Ubuntu, the LXC package is lxc, while the LXD package is lxd.

Linux DistributionLXDLXC
Any (snap package)lxdlxc
Ubuntu (deb package)lxdlxc

Privileged and unprivileged containers

When you run a privileged container, the UIDs of the files and processes in the container match those on the host. A root account in the container has the same UID as the root account on the host.

When you run an unprivileged container, the UIDs in the container are shifted to a numerical range that is separate from that on the host. On a Unix system, the UIDs typically follow the range of 0 to 65535 (root, and nobody). In an unprivileged container, the accounts are mapped (for example) to the range 100000 to 165535 on the host (therefore, the container root will be UID 100000 on the host). But from within the container, you will be seeing the UID range of 0 to 65535. This is good in terms of security.

The LXC instructions that were shown above, create a privileged container. You need to perform extra work to enable unprivileged containers in LXC.

LXD creates by default unprivileged containers. You need to be explicit in LXD to create privileged containers (-c security.privileged=true).

Discussion forum and support

The discussion forum for both LXC and LXD is

https://discuss.linuxcontainers.org/

Permanent link to this article: https://blog.simos.info/comparison-between-lxc-and-lxd/

6 comments

Skip to comment form

    • Jim on December 27, 2019 at 15:13
    • Reply

    Thanks. Clear and concise

    1. Thanks!

    • Nompy on December 30, 2019 at 14:50
    • Reply

    I’m still confused. So LXD is a daemon that is managing containers and LXC is it’s command-line interface, and together they make a single package called LXD, right? And there is another containter management daemon, also called LXD?

    1. LXC is not the command-line interface of LXD.

      LXD is a daemon (service) that manages Linux containers and is based on liblxc as does LXC.
      LXD offers a REST API that is accessible that the default LXD command-line utility lxc.
      But you can use other clients as well, even Web-based.

      LXC is also based on liblxc, but includes a set of many separate command-line utilities that each manage directly the containers. There is no REST API in between.

    • Bernd Bausch on August 4, 2021 at 03:53
    • Reply

    What is confusing to beginners like me: LXD has a daemon named “lxd”, which exposes an API, and two CLIs named “lxd” and “lxc”. The “lxd” CLI is not the same as the daemon, and the “lxc” CLI is not the same as LXC. Unfortunate naming, but hard to change now after so many years.

    More details, if I understand it well (caveat: most likely not 100% correct):

    The LXD CLI “lxc” uses the public LXD API and has the purpose of managing containers. It can access the API of a remote LXD daemon, thereby managing containers on remote systems.

    The LXD CLI “lxd” accesses the locally running LXD daemon (via a UNIX socket?) and has the purpose of configuring it.

    LXC doesn’t have a daemon and doesn’t expose an API on the network. The LXC CLI consists of commands with a dash, “lxc-create”, “lxc-destroy” and so on, which use the liblxc1 libraries to do their work. With the LXC CLI, you can only manage containers on the current host.

  1. excellent article. makes very clear about the question.
    it is so bad that LXD creators has used such misleading names for lxc and lxd utilities.

Leave a Reply to Simos XenitellisCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.