How to run a Windows virtual machine on LXD on Linux

Update 27th April 2021: The post has been updated and rechecked. Now, the stable version of distrobuilder is able to repack-windows and you do not need use the version from the edge channel.

LXD is a hypervisor to run both system containers (a la LXC) and virtual machines (a la QEMU) on Linux distributions. System containers are lightweight because they are based solely on the Linux kernel for their virtualization features, and support Linux guests only. However, virtual machines can run other operating systems. In this post, we see how to run Windows in a LXD virtual machine.

The benefit with running Windows through LXD is that you are using the familiar LXD workflow and takes away some of the the complexity from the other ways of running a VM (like virt-manager).

The content of this tutorial came from https://discuss.linuxcontainers.org/t/running-virtual-machines-with-lxd-4-0/7519 Look towards the end of the thread where Stéphane Graber describes how to simplify the process compared to the instructions at the top of that thread.

The prerequisite is that you have LXD configured and running.

In the following, we

  1. Download a Windows 10 ISO from Microsoft.
  2. Prepare the ISO using distrobuilder (we do this once per ISO).
  3. Start the virtual machine from that prepared ISO and go through the installation.

Download a Windows 10 ISO

You can download a Windows 10 ISO from Microsoft, through the following URL.

https://www.microsoft.com/en-us/software-download/windows10ISO

You will be given the option to select Windows 10. Then, select a language for the Windows 10 ISO, and finally to select whether to download a 32-bit or 64-bit version. Select your preferred language, and then choose the 64-bit version. At the time of writing, the ISO filename is Win10_20H2_v2_EnglishInternational_x64.iso (I selected the English International version).

Once the ISO file has been downloaded, move to the next section.

Prepare the ISO using distrobuilder

Install the distrobuilder package. It is available as a snap package, using the classic type of confinement. Currently, the repack-windows option is available in the very latest version of distrobuilder. Therefore, we have to add for now the --edge parameter. This means that distrobuilderas a snap package will be updated frequently, being the edge channel. Once you perform the repack, you may switch distrobuilder to the stable channel with sudo snap refresh distrobuilder --stable for the typical use of the tool. In addition, distrobuilder, although it is a snap package, it has the classic confinement, which means that it uses packages from the host. It uses libguestfs-tools which provides the utility hivexregedit. In addition, it uses executables from the wimtools package, where WIM stands for Windows IMage.. We install those packages as well.

sudo snap install distrobuilder --classic
sudo apt install -y libguestfs-tools wimtools

Then, run distrobuilder over the Windows 10 ISO as follows. Obviously, the filename of the ISO file might be different in your case. Adapt and overcome.

$ sudo distrobuilder repack-windows Win10_20H2_v2_EnglishInternational_x64.iso Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso 
INFO	Mounting Windows ISO
INFO	Mounting driver ISO
INFO	Modifying WIM file	{"file": "boot.wim", "index": 2}
INFO	Modifying WIM file	{"file": "install.wim", "index": 1}
INFO	Modifying WIM file	{"file": "install.wim", "index": 2}
INFO	Modifying WIM file	{"file": "install.wim", "index": 3}
INFO	Modifying WIM file	{"file": "install.wim", "index": 4}
INFO	Modifying WIM file	{"file": "install.wim", "index": 5}
INFO	Modifying WIM file	{"file": "install.wim", "index": 6}
INFO	Modifying WIM file	{"file": "install.wim", "index": 7}
INFO	Modifying WIM file	{"file": "install.wim", "index": 8}
INFO	Modifying WIM file	{"file": "install.wim", "index": 9}
INFO	Modifying WIM file	{"file": "install.wim", "index": 10}
INFO	Modifying WIM file	{"file": "install.wim", "index": 11}
INFO	Generating new ISO
$ 

In my case, it generated a new ISO, the one with a name Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso. This process takes time, and it took five minutes on a Linode VPS. We have the prepared ISO, we can now start.

Start the Windows 10 installation

Run the following commands to initialize the VM, to configure (=increase) the allocated disk space and finally attach the full path of your prepared ISO file. Note that the installation of Windows 10 takes about 10GB (before updates), therefore a 30GB disk gives you about 20GB of free space.

lxc init win10 --empty --vm -c security.secureboot=false
lxc config device override win10 root size=30GiB
lxc config device add win10 iso disk source=/home/myusername/Downloads/Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso boot.priority=10

Now, the VM win10 has been configured and it is ready to be started. The following command starts the virtual machine and opens up a VGA console so that we go through the graphical installation of Windows.

lxc start win10 --console=vga
Windows 10 is booting up in a LXD virtual machine. The LXD logo is shown above.

Note that the installation of Windows 10 does a few restarts before you end up on the desktop. Those will appear as errors in your terminal that look like the following.

(remote-viewer:13539): dbind-WARNING **: 15:51:45.744: Couldn't register with accessibility bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

(remote-viewer:13539): GLib-GIO-CRITICAL **: 15:51:45.765: g_dbus_proxy_new_sync: assertion 'G_IS_DBUS_CONNECTION (connection)' failed

You just need to launch again the console as follows. To start the VM we run lxc start but to re-attach to the running VM (since it was restarted by itself), we run lxc console.

lxc console win10 --type=vga

It will take you two lxc consoles until you get to the Windows desktop.

Windows 10 booted up in a LXD virtual machine.

Notes

  1. When installing Windows, you will be prompted to use your personal information. If this is an expendable virtual machine, you can get away with creating a new outlook.com account and adding some telephone number (it does not verify the telephone number). You will be prompted for a PIN as well, and it will be asking for that PIN when you use this installation.
  2. To shutdown the VM, click to Shutdown from the Windows desktop. To start it again at a later time, run the following (as we did before).
    lxc start win10 --console=vga
  3. When the VM is stopped, you may remove the Windows 10 ISO with the following. iso was the name of the ISO device we setup earlier.
    lxc config device remove win10 iso
  4. This VM setup does not have sound. It is probably easy to add that using a QEMU option. There are no virtio drivers yet for a virtualized audio card. But it is not an issue if you go for Audio Passthrough, using a tool like Remmina. For this to work, you need to enable Remote Desktop in Windows, and also should have installed Windows Professional (instead of Windows Home).
  5. By default, the VM gets a private IP address, one from the lxdbr0 network range. I could not get to ping that IP address from the host or from a container. It is likely a VM issue. But of couse, the VM has networking. edit It is a Windows Firewall issue. You can enable an inbound IPv4 ICMP rule, such as the following. We selected to enable some existing rule instead of composing it from scratch.

Summary

We have seen how to get to run Windows 10 on a LXD virtual machine. At the moment, these are the easiest instructions. Once more features are added, I’ll update this page.

Permanent link to this article: https://blog.simos.info/how-to-run-a-windows-virtual-machine-on-lxd-on-linux/

39 comments

Skip to comment form

  1. Hi Simos;

    Thanks for everyting about LXC/LXD. I wrote samething on container discuss. If anyone want to use only copy/paste…

    snap install distrobuilder --classic --edge
    apt install -y libguestfs-tools wimtools
    distrobuilder repack-windows /home/muslu/win10.iso /home/muslu/win10_distrobuilder.iso
    lxc init win10 --empty --vm -c security.secureboot=false
    lxc config device override win10 root size=50GiB
    lxc config device add win10 iso disk source=/home/muslu/win10_distrobuilder.iso boot.priority=10
    lxc start win10 --console=vga

    Note: remmina (for slice) or virt-viewer can connect vga.

    apt install -yqq remmina-plugin-spice virt-viewer

    1. Thanks Muslu!

      I notice that you install distrobuilder from the edge channel. However, since recently, the stable channel has been updated and there is no need any more to use the edge channel. Have a look.

      $ snap info distrobuilder
      name:      distrobuilder
      summary:   Image builder for LXC and LXD
      publisher: Stéphane Graber (stgraber)
      store-url: https://snapcraft.io/distrobuilder
      contact:   https://github.com/lxc/distrobuilder/issues
      license:   Apache-2.0
      description: |
        Distrobuilder is a tool which lets you create container images for LXC and
        LXD.
      commands:
        - distrobuilder
      snap-id:      pAkHI18sZCL3Pfd0r904v29gh2j9OX2k
      tracking:     latest/stable
      refresh-date: 2 days ago, at 19:49 GMT
      channels:
        latest/stable:    1.1         2021-01-20 (257) 12MB classic
        latest/candidate: 1.1         2020-10-16 (257) 12MB classic
        latest/beta:      ↑                                 
        latest/edge:      git-2b6b394 2021-02-23 (408)  8MB classic
      installed:          1.1                    (257) 12MB classic
      

      I notice that you install libguestfs-tools wimtools. What is the significance of these tools?

      How do you connect to the Windows VM through SPICE (with remmina)?

    • Mihail Igoshin on March 3, 2021 at 17:09
    • Reply

    For me distrobuilder does not recognize command repack-windows. I’ve installed it via snap as is show in this tutorial.

    “Error: unknown command “repack-windows” for “distrobuilder”

    “$ distrobuilder –version
    1.1”

    1. The Windows support in distrobuilder has been added recently, which means that the package should be recent enough.

      Here is the version I have installed,

      $ snap info distrobuilder
      name:      distrobuilder
      summary:   Image builder for LXC and LXD
      publisher: Stéphane Graber (stgraber)
      store-url: https://snapcraft.io/distrobuilder
      contact:   https://github.com/lxc/distrobuilder/issues
      license:   Apache-2.0
      description: |
        Distrobuilder is a tool which lets you create container images for LXC and LXD.
      commands:
        - distrobuilder
      snap-id:      pAkHI18sZCL3Pfd0r904v29gh2j9OX2k
      tracking:     latest/stable
      refresh-date: 4 days ago, at 19:49 EET
      channels:
        latest/stable:    1.1         2021-01-20 (257) 12MB classic
        latest/candidate: 1.1         2020-10-16 (257) 12MB classic
        latest/beta:      ↑                                 
        latest/edge:      git-2b6b394 2021-02-23 (408)  8MB classic
      installed:          1.1                    (257) 12MB classic
      

      I tried to repack the ISO, and I got as well

      Error: unknown command "repack-windows" for "distrobuilder"
      Run 'distrobuilder --help' for usage.
      

      Ok, this is embarrassing and thanks for reporting. Above it show that I have installed distrobuilder from the stable, published on the 20th of January 2021. But that is not recent enough. We need to refresh our distrobuilder so that it comes from the edge channel for now. Once the version from the stable channel gets updated, we can switch back to the stable channel.

      Here is how to switch from the stable channel to the edge channel.

      sudo snap refresh distrobuilder --edge
      

      I am updating the post as well on this.
      Thanks for reporting!

        • Mihail Igoshin on March 3, 2021 at 19:27
        • Reply

        Thanks. 🙂

    • Mihail Igoshin on March 3, 2021 at 20:25
    • Reply

    I’ve got error:

    “Error: Required tool “hivexregedit” is missing”

    After fast google solution seems to be:

    sudo apt-get install libguestfs-tools

    1. Thanks for this. Apparently, my test system is not vanilla any more 🙁 and I have to reinstall.
      I’ll update accordingly.

    • Tett on April 26, 2021 at 11:17
    • Reply

    root@lxd1:/# sudo distrobuilder repack-windows –windows-version=w10 path/to/iso/windows.iso path/to/iso/windows_distrobuilder.iso
    INFO Mounting Windows ISO
    INFO Mounting driver ISO
    INFO Modifying WIM file {“file”: “boot.wim”, “index”: 2}
    Error: Failed to retrieve wim file information: Failed to run: wimlib-imagex info /var/cache/distrobuilder.271484116/overlay/sources/install.wim: [ERROR] Can’t open “/var/cache/distrobuilder.271484116/overlay/sources/install.wim” read-only: No such file or directory
    ERROR: Exiting with error code 47:
    Failed to open a file

    1. Hi!

      Most likely you have a corrupted virtio-win.iso file.

      When you run distrobuilder to repack the Windows ISO, it downloads the virtio-win.iso from https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/virtio-win.iso and puts it into /tmp/distrobuilder/.

      This download is about 500MB and if you interrupt it, distrobuilder keeps that incomplete cached version and does not erase it.

      Then, if you run again distrobuilder to repack the Windows ISO, distrobuilder will not check whether the virtio-win.iso is intact, and will try to use it. Obviously it will fail.

      The workaround is to remove the incomplete file so that `distrobuilder downloads it again.

      sudo rm /tmp/distrobuilder/virtio-win.iso
      

      If you run a slow internet connection (like mobile internet), then you can also do the following. The -c flag will Continue the download from where it was interrupted.

      cd /tmp/distrobuilder/
      sudo wget -c https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/virtio-win.iso
      
    • aamcatamney on April 26, 2021 at 16:30
    • Reply

    I’m also seeing this error 🙁

    1. Hi!

      See above reply.

    • aamcatamney on April 27, 2021 at 00:46
    • Reply

    Hey Simos,
    Thanks for your replay. I have tried your steps and I am still getting the same error.
    I have also tried chaning the cache dir and running in debug mode to try and get more info:

    2021-04-26T23:44:32Z DEBUG distrobuilder/main_repack-windows.go:566 Updating Windows registry {“hivefile”: “DRIVERS”}
    2021-04-26T23:44:33Z DEBUG distrobuilder/main_repack-windows.go:573 Updating Windows registry {“hivefile”: “SYSTEM”}
    2021-04-26T23:44:33Z DEBUG distrobuilder/main_repack-windows.go:580 Updating Windows registry {“hivefile”: “SOFTWARE”}
    Error: Failed to retrieve wim file information: Failed to run: wimlib-imagex info /home/anthony/cache/overlay/sources/install.wim: [ERROR] Can’t open “/home/anthony/cache/overlay/sources/install.wim” read-only: No such file or directory
    ERROR: Exiting with error code 47:
    Failed to open a file.

    I have also tried manually downloading the files.
    Do you think there has been a breaking change with virtio-win.iso or distrobuilder?

    1. I get the same errors and cannot figure out what configuration remains on the system that causes the errors. I even purged distrobuilder before installing again. Still, the same error.

      So I decided to start from a clean system, and launched a VPS with Ubuntu 20.04. I downloaded the Windows ISO through that VPS and run the command. It worked fine, and finally I downloaded the generated .iso.

      See my comment below for the full list of commands. In addition, I updated this post.

    • Tett on April 27, 2021 at 05:36
    • Reply

    Could u provide a fresh command list ? coz I dont know after puts virtio-win.iso file into tmp location then what will the command procedure . Plz give us a command list

    1. See my comment below that shows how to use a VPS to do the distrobuilder work there. By doing so, you are guaranteed to have a clean system.

      Can you tell me which version of Ubuntu you have? The distrowatch snap, having the classic confinement, depends on the host’s packages. Therefore, it might be an issue relating to particular host packages.

  2. I have tried the whole process from scratch and have updated the whole post accordingly.
    To make sure I have a clean Ubuntu 20.04 LTS system, I created a $5 VPS (Linode, in my case) and run the following exact commands.

    Note the Windows ISO URL. I attempted a download from my desktop just to let it download a bit and then click on the Firefox download manager to Copy Download Link. Then, I pasted that command in the wget command below. It got downloaded at 35MB/s :-). Therefore, replace with your download link from Firefox, and cancel the download from your desktop. The next command to the wget is to rename (mv) the file so that the filename is cleaned up.

    snap install distrobuilder --classic
    wget 'https://software-download.microsoft.com/pr/Win10_20H2_v2_EnglishInternational_x64.iso?t=...'
    mv Win10_20H2_v2_EnglishInternational_x64.iso\?t\=...   Win10_20H2_v2_EnglishInternational_x64.iso
    sudo apt install -y libguestfs-tools wimtools
    distrobuilder repack-windows Win10_20H2_v2_EnglishInternational_x64.iso Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso
    

    Here is the generated file,

    root@localhost:~# ls -l
    total 12392636
    -rw-r--r-- 1 root root 6475390976 Apr 27 16:55 Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso
    -rw-r--r-- 1 root root 6214653952 Nov 20 08:19 Win10_20H2_v2_EnglishInternational_x64.iso
    drwxr-xr-x 4 root root       4096 Apr 27 16:35 snap
    root@localhost:~# 
    

    You can use either SSH to download the ISO, or install a Web server like nginx, and place the ISO there. Normally, you would use SSH for integrity purposes. We show how to use nginx here.

    root@localhost:~# sudo apt install -y nginx
    root@localhost:~# sudo mv Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso /var/www/html/
    root@localhost:~# 
    

    And finally, from your desktop computer, download the ISO. Use the -c flag to continue the download in case it gets interrupted.

    wget -c _IPADDRESS_OF_MY_VPS_/Win10_20H2_v2_EnglishInternational_x64-distrobuilder.iso
    

    And that’s it! Do not forget to destroy the VPS once you are done.

    I do not know what went wrong with the previous instructions. I do too get those errors when I run the repack-windows command on my desktop. I have tested the above instructions with distrobuilder from the edge channel and it still worked.

    • Tett on May 2, 2021 at 09:05
    • Reply
    root@lxd1:/path/to/iso# sudo snap install distrobuilder --classic
    snap "distrobuilder" is already installed, see 'snap help refresh'
    root@lxd1:/path/to/iso# sudo apt install -y libguestfs-tools wimtools
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    libguestfs-tools is already the newest version (1:1.40.2-7ubuntu5).
    wimtools is already the newest version (1.13.1-1).
    0 upgraded, 0 newly installed, 0 to remove and 52 not upgraded.
    root@lxd1:/path/to/iso# sudo distrobuilder repack-windows Win10_20H2_English_x64.iso Win10_20H2_English_x64_distrobuilder.iso
    INFO    Mounting Windows ISO
    INFO    Mounting driver ISO
    INFO    Modifying WIM file      {"file": "boot.wim", "index": 2}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 1}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 2}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 3}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 4}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 5}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 6}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 7}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 8}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 9}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 10}
    INFO    Modifying WIM file      {"file": "install.wim", "index": 11}
    INFO    Generating new ISO
    root@lxd1:/path/to/iso# lxc init win10 --empty --vm -c security.secureboot=false
    Creating win10
    root@lxd1:/path/to/iso# lxc config device override win10 root size=30GiB
    Device root overridden for win10
    root@lxd1:/path/to/iso# lxc config device add win10 iso disk source=/path/to/iso/windows.iso boot.priority=10                                           windows.iso
    root@lxd1:/path/to/iso# lxc config device add win10 iso disk source=/path/to/iso/Win10_20H2_English_x64_distrobuilder.iso boot.priority=10              Device iso added to win10
    root@lxd1:/path/to/iso# lxc start win10 --console=vga
    X11 connection rejected because of wrong authentication.
    Unable to init server: Could not connect: Connection refused
    
    (remote-viewer:4176671): Gtk-WARNING **: 08:02:40.776: cannot open display: localhost:10.0
    

    stack in this state. I have ubuntu 20.04 and lxd 4.13. Where is fault?

    1. Hi!

      You are running the LXD commands as root and I suppose that you are connected to your host as a non-root user.
      You should log out from root and go back to your non-root user account (such as user or ubuntu).
      To verify that you can run X11 apps from within an account from the terminal, try to run xclock. If it works, you are good to go.

      Also, the message says localhost:10.0. Normally, if you run echo $DISPLAY, it should say something like :0 or :1.
      You typically get localhost:10.0 when you connect to the system through SSH. If that is the case, you should use ssh -X (i.e. add the -X parameter) in order to enable X11 forwarding. If it still does not work, use -Y instead (trusted X11 forwarding). If it still fails, describe your setup so that I cat try to replicate.

    • Tett on May 5, 2021 at 10:54
    • Reply

    I am trying it to by non root user and it shows bellow error

    tech@lxd1:~$ lxc start win10 –console=vga
    Unable to init server: Could not connect: Connection refused

    (remote-viewer:50182): Gtk-WARNING **: 09:51:01.766: cannot open display:

    I have comment out

    #AllowAgentForwarding yes
    #AllowTcpForwarding yes
    #GatewayPorts no
    X11Forwarding yes
    #ForwardX11 yes
    #ForwardX11Trusted yes
    #X11DisplayOffset 10
    #X11UseLocalhost yes
    #PermitTTY yes
    #PrintMotd no
    #PrintLastLog yes
    #TCPKeepAlive yes
    #PermitUserEnvironment no
    #Compression delayed
    #ClientAliveInterval 0
    #ClientAliveCountMax 3
    #UseDNS no
    #PidFile /var/run/sshd.pid
    #MaxStartups 10:30:100
    #PermitTunnel no
    #ChrootDirectory none
    #VersionAddendum none

    1. I asked about the localhost:10.0 part, and why you get 10.0.

      I used SSH to connect from my system to my system (i.e. ssh localhost), and I got

      $ echo $DISPLAY
      localhost:10.0
      

      I also tried to start the win10 VM through the SSH session and I also got the same error,

      $ lxc start win10 --console=vga
      X11 connection rejected because of wrong authentication.
      Unable to init server: Could not connect: Connection refused
      
      (remote-viewer:19881): Gtk-WARNING **: 12:00:17.968: cannot open display: localhost:10.0
      ^C^C^C^\SIGQUIT: quit
      

      Therefore, you are connecting with SSH to the LXD host and want to run the GUI of Win10 through X11 forwarding.

      Get a shell to the LXD host and install x11-apps and mesa-utils. These will give you the xlock application, and the glxgears GL application.
      You can see that both run fine from the remote LXD host. Below, I ssh to localhost for demonstration purposes.

      $ ssh -X localhost
      $ echo $DISPLAY
      localhost:10.0
      $ xclock                 # you see the clock. This is a pure X11 application.
      ^C
      $ glxgears               # you see the gears. This is a GL application.
      ^C
      $ lxc start win10 --console=vga
      X11 connection rejected because of wrong authentication.
      Unable to init server: Could not connect: Connection refused
      
      (remote-viewer:19881): Gtk-WARNING **: 12:00:17.968: cannot open display: localhost:10.0
      ^C^C^C^\SIGQUIT: quit
      

      Therefore, something is happening in LXD and most likely it is a bug in LXD.
      As long as you can run other GUI applications, you should be able to run the remote viewer.

      Visit https://github.com/lxc/lxd/issues and file a bug report.
      Mention that you connect to the LXD host with SSH. And while you can run X11 and GL applications
      through X11 forwarding, you cannot get the remote viewer for the Win10 VM to work.

    • Tett on May 5, 2021 at 10:59
    • Reply

    root user shows

    root@lxd1:~# echo $DISPLAY
    localhost:10.0

    sudo user shows no output

    tech@lxd1:~$ echo $DISPLAY

    tech@lxd1:~$

    • Jim McCulloch on May 8, 2021 at 21:36
    • Reply

    This is a massive pain! I started 7 hours ago, I got to and saw the Windows install screen once 2 hours ago and I can’t get back to it after 3 hours of trying to get the install started. On the positive side this site got me further along than any other.

    1. Do you get a specific error or is it just general slowness?

      Things to do to get better performance:
      1. Use ZFS or btrfs for the storage pool in LXD. dir is likely bad for performance.
      2. A VM needs much more RAM than a system container. If you have less than 8GB of total RAM, you should look into this.

      An LXD VM uses qemu behind the scenes and does not implement virtualization from scratch. If you install virt-manager, you will be able to manage the same VM and perhaps fine-tune it.

    • Jim McCulloch on May 11, 2021 at 05:22
    • Reply

    Thanks for the reply Simos.

    I spent the majority of the mostly sleepless weekend working on it. What I finally found was that the drive that it was being installed on was way to small and didn’t resize. I think I found that with ‘lxc info –show-log win10’. It showed that the drive had 0 bytes left. I really think this is the issue. How do I select which drive it is installed on?

    I am not a Linux expert in any sense of the word, I’ve dabbled in Linux since the mid to late 90’s when it was a massive pain to get a Desktop up and running until now. Those times were spaced out generously.

    PC: HP ProLiant DL360p Gen8 – Xeon 2640 x2 at 2.5GHz – 64GB RAM – 8 600GB SAS 10K Drives in a couple of various RAID configurations. Performance shouldn’t be an issue.

    OS: Ubuntu 20.04

    Windows 10 will not install for the above reason. My lack of current knowledge of Linux probably.
    However I’ve read that you can change the number of CPUs and amount of memory for container.

    Any help would be very much appreciated and thanks again for the reply.

    1. Hi Jim!

      When you setup LXD, you allocate space that will be used for the containers and virtual machines. This space has to be big enough so that when you instruct Win10 to use, let’s say, 30GB, then LXD should have that space available somewhere.

      Let’s look into these in detail.

      With the following, you can see the available storage pools in LXD; where LXD can store containers and virtual machines. Here you can see that my storage pool is called lxd, uses ZFS in some way to organize the space, and I have 34 instances (either containers or virtual machines!). Obviously, when I do not use an instance, I stop it so save precious memory.

      $ lxc storage list
      +------+--------+--------+-------------+---------+
      | NAME | DRIVER | SOURCE | DESCRIPTION | USED BY |
      +------+--------+--------+-------------+---------+
      | lxd  | zfs    | lxd    |             | 68      |
      +------+--------+--------+-------------+---------+
      

      But, what about the space?
      You can run the following, where lxd is the name of the storage pool in my case. In my case, I use a disk partition for ZFS.

      $ lxc storage show lxd
      config:
        source: lxd
        volatile.initial_source: /dev/sdd8
        zfs.pool_name: lxd
      description: ""
      name: lxd
      driver: zfs
      ...
      

      Now, I am able to run the following and will show me the total size and the free space available.

      zpool list
      

      Go through these steps and report back the free space available. Most likely you use a storage pool with a loop file, and the space in that file has been exhausted. There are ways around the issue, and we will explore them once we have an idea about the diskspace picture.

    • James McCulloch on May 12, 2021 at 01:42
    • Reply

    Simos,

    I missed the entire lxd step. I started the whole process on this page. Here is the output from your reply, I new I was missing something, just didn’t know what it was. Do you have a page on the lxd configuration? Thanks for the generous help,

    $ lxc info --show-log win10
    Name: win10
    Location: none
    Remote: unix://
    Architecture: x86_64
    Created: 2021/05/09 14:55 UTC
    Status: Stopped
    Type: virtual-machine
    Profiles: default
    Pid: -1
    Resources:
      Processes: 0
      Disk usage:
        root: 52.18MB
    Error: open /var/snap/lxd/common/lxd/logs/win10/qemu.log: no such file or directory
    
    $ lxc storage list
    +---------+--------+---------+-------------+---------+
    |  NAME   | DRIVER | SOURCE  | DESCRIPTION | USED BY |
    +---------+--------+---------+-------------+---------+
    | default | zfs    | default |             | 2       |
    +---------+--------+---------+-------------+---------+
    
    $ lxc storage show lxd
    Error: not found
    

    Jim

    1. James, from what I can see, you have used the defaults when you set up LXD. Your storage pool uses ZFS, and most likely over a loop file. This means that LXD pre-allocated the space for you, and this is a fixed number of GBs.

      The last command that you tried, should be lxc storage show default, because default is the name of the storage pool from the previous command. The lxc storage show default will definitely show whether you are using a loop file (a big file that is formatted as ZFS), or a partition (like /dev/sdb4).

      At this stage you can run zpool list, which will show you the size of the storage pool, how much is allocated and how much is free.

      So, what do you do if you are stuck with a small storage pool? Being Linux and all, there are too ways around it. The obvious one is to tear everything apart and reinstall from scratch. When you run sudo lxd init this time, you would allocate more space. But that would be too easy.

      First, if you have a spare partition, with 80GB or more, then it would be great to put there the storage pool. ZFS on a block partition (instead of a loop file) is even more performant. If that is the case, then tell me to show you the lxc storage commands to create a second storage pool, and move your existing containers/VMs over to the second storage pool. Finally, you make the second storage pool the new default storage pool (and can get rid of the first one).

      Second, if you have lots of free space on your root partition (i.e. the / partition), then you can create an additional second storage pool. Now, this can be tricky if you have created separate / and /home/ partitions, and opted to allocate just 30-40GB for / as most installation guides say.

    • Don Chai on June 4, 2021 at 15:35
    • Reply

    Having these errors while installing it on LXD v4.8, ARM64 RaspberryPi4B8GB, Ubuntu 21.04 (Node #2/3 cluster)

    user@rpi2:~$ lxc info –show-log win10
    Name: win10
    Location: rpi2
    Remote: unix://
    Architecture: aarch64
    Created: 2021/06/04 14:02 UTC
    Status: Stopped
    Type: virtual-machine
    Profiles: default

    Log:

    Error screenshot: https://tinyurl.com/2cnz5s6w

    Any pointer to move forward from the errors?

    1. Hi!

      The first paragraph in the screenshot says that it could not find a disk.
      And the next paragraphs show the attempt to use PXE, which is a network booting protocol.

      Since you are running this on the RPi4, I suppose you are already familiar with the special instructions at https://www.worproject.ml/
      There should be a few more steps to get from https://www.worproject.ml/ to running distrobuilder which I have not tried.

  3. Thanks Simos,

    Progress thus far,

    Managed to convert to ISO particularly for ARM64 via uup by scripts from worproject (thanks for pointing out), and done distrobuilder (about 6GB+) in rpi#2. Boot into CD/DVD… on Ubuntu desktop (new cluster node#4/4 on x86_64 platform) with spice installed and got this “Guest disabled display.” right after after boot progress bar complete, argh…

    Still need more luck, shall keep trying.

    Progress screenshots:

    https://tinyurl.com/stz585kw

    https://tinyurl.com/44m457yp

    1. The message Guest disabled display is a Qemu error message. Something is wrong with the piping of the display of the VM to a window on the host. spice is a bit more advanced from the default VNC way.

      In the snap package of LXD for ARM, do you get libspice-server.so somewhere in /snap/lxd/current/?

  4. Yeah Simos,

    Somehow it’s located under aarch64-linux-gnu folder as below.

    /snap/lxd/current/lib/aarch64-linux-gnu/libspice-server.so

    Screenshot: https://tinyurl.com/2p65nk2v

  5. Tried on Windows with remote viewer instead, not able to key anything on remote viewer screen and powershell thrown some critical spice errors.

    Error screenshot: https://tinyurl.com/yth8npte

    • Mihail Igoshin on July 5, 2021 at 20:10
    • Reply

    I have some issues, I’m unable to start properly the vm.
    $ lxd –version
    4.15

    $ lxc start win10 –console=vga
    LXD automatically uses either spicy or remote-viewer when present.
    As neither could be found, the raw SPICE socket can be found at:
    spice+unix:///home/igoshin/snap/lxd/common/config/sockets/776543779.spice

    $ lxc info –show-log win10
    Name: win10
    Location: none
    Remote: unix://
    Architecture: x86_64
    Created: 2021/07/05 19:01 UTC
    Status: Running
    Type: virtual-machine
    Profiles: default
    Pid: 418943
    Resources:
    Processes: -1
    Network usage:
    eth0:
    Bytes received: 5.34kB
    Bytes sent: 4.91kB
    Packets received: 20
    Packets sent: 22

    Log:

    warning: tap: open vhost char device failed: Permission denied
    warning: tap: open vhost char device failed: Permission denied

    • daalocni on July 15, 2021 at 07:34
    • Reply

    sorry for bad english
    hello, is it possible to do something like this in lxd windows vm https://blog.tmm.cx/2020/05/15/passing-an-intel-gpu-to-a-linux-kvm-virtual-machine/

    i tried adding the i915-GVTg_V5_4 profile as an mdev gpu to the instance, and it sort of worked, it was detected in windows and auto installed the driver, device manager says device ok and in task manager it also appears, but i noticed i was still only seeing display feed from virtual gpu, tried disabling that in device manager but after i no longer had display (black screen) in spicy and had to restore snapshot. have any ideas?

    • Stefan on August 23, 2021 at 18:23
    • Reply

    Just a note for users with a Ryzen based system who want to install WSL on their LXC Win10: Don’t do it. It will break your Win10 beyond repair, constant BSOD’s. Nested virtualization is not working yet. Took me 2 days to find this out. I Hope it will safe others some time.

  6. Hello Simos,

    thanks your explanations. I have been trying to setup a Windows 10 VM under ArchLinux and LXD 4.18 – but w/e I tried when trying to initially boot I get thrown into a BSOD with a “System Thread Exception Not Handled” error message. I injected the necessary drivers beforehand using distrobuilder and am using that .iso to boot up.

    Using that very same .iso and creating a VM under libvirt works perfectly fine and boots into the Windows installer which I don’t even see when trying to use LXD.

    My google-fu is failing me it seems – any thoughts on this?

    Thanks!

  7. Hello Simos,

    thank you for this great tutorial eveything is working like a charm.
    I notice only one problem, i cant set a proper display resolution, inside windows settings is set to 800×600 and the menu is greyed out.
    I have tried to update gpu driver from win10 lxc without success, any thoughts on this? thank you in advance

    • GreenLepra on November 13, 2021 at 10:15
    • Reply

    Is there any way to use shiftfs with this ?

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: