How to run a Windows virtual machine on Incus on Linux

Incus is a manager for virtual machines and system containers.

A virtual machine (VM) is an instance of an operating system that runs on a computer, along with the main operating system. A virtual machine uses hardware virtualization features for the separation from the main operating system. With virtual machines, the full operating system boots up in them. Therefore, it is possible to run Windows in a virtual machine, even if your host is Linux.

A system container is an instance of an operating system that also runs on a computer, along with the main operating system. A system container, instead, uses security primitives of the Linux kernel for the separation from the main operating system. You can think of system containers as software virtual machines. The system containers reuse the running Linux kernel of the host, therefore you can only have Linux system containers, any Linux distribution.

In this post we are going to see how to create an Incus virtual machine with Windows.

Table of Contents

• Updates
• Prerequisites
• Cheat sheet
• Downloading a Windows ISO file
• Preparing the Windows ISO file
• Creating the Windows VM on Incus
• Booting the Windows virtual machine
• Reconnecting to the Windows virtual machine
• Removing the virtual Windows DVD ISO
• Screenshots
• Summary
• Bonus material #1 – Incus profile for Windows
• Bonus material #2 – Windows ISO as a volume in Incus
• Bonus material #3 – CPU Pinning for QEMU
  • CPU Pinning
  • Using -cpu host
• Bonus material #4 – How to mount a directory from the host to the Windows VM
  • Installing WinFSP in the Windows VM
  • Installing the VirtIO Windows Guest Tools in the Windows VM
  • Enabling the VirtioFsSvc service in the Windows VM
  • Adding the disk device
  • Removing the disk device
• Troubleshooting
  • Error: This revision of snap “distrobuilder” was published using classic confinement…
  • Error: You must be root to run this tool
  • Error: Failed to check dependencies: Required tool “hivexregedit” is missing
  • Error: Failed to check dependencies: Required tool “wimlib-imagex” is missing
  • Error: Failed to check dependencies: Required tool “rsync” is missing
  • Error: Failed add validation for device “install”: Missing source path for disk “install”
  • Error: The instance is already running
  • Error: Instance is not running
  • Error: Failed to detect Windows version. Please provide the version using the –windows-version flag
  • Error: Windows crashes during the first moments of booting up
  • Error: When I enable audio in the Windows VM, I lose audio from the host
  • How much space is available in Incus to use for my VM?
  • Which version of Windows should I use in the Incus VM?
  • How to get logs of the virtual machine if something goes wrong?
  • I am using Fedora. What are the dependencies for distrobuilder?

Updates

Update #4: In Bonus Material #4. How to mount a directory from the host to the Windows VM.

Update #3: In Bonus Material #3. If the Windows VM crashes, there are two workarounds.

Update #2: In Bonus Material #2. Instead of keeping the Windows ISO file on your host, you can place it in the Incus storage pool.

Update #1: In Bonus Material #1. You can create an Incus profile that you can easily use when you want to create a Windows VM.

Prerequisites

1. You should have a system that runs Incus.
2. A Windows ISO file or enough bandwidth to download one.
3. A system with support for hardware virtualization so that it can run virtual machines.

Cheat sheet

This is a summary of the commands. You are supposed to consult this section after you thoroughly read the post.

sudo snap install distrobuilder --classic
sudo distrobuilder repack-windows Win11_23H2_English_x64v2.iso Win11_23H2_English_x64v2.incus.iso
incus init win11vm --empty --vm
incus config device override win11vm root size=55GiB
incus config set win11vm limits.cpu=4 limits.memory=6GiB
incus config device add win11vm vtpm tpm path=/dev/tpm0
incus config device add win11vm install disk source=/home/myusername/Downloads/Win11_23H2_English_x64v2.incus.iso boot.priority=10
incus start win11vm --console=vga
incus console win11vm --type=vga
incus config device remove win11vm install

Downloading a Windows ISO file

You can download Windows ISO files from Microsoft, for example, at https://www.microsoft.com/en-us/software-download/ In my case I am selecting Windows 11. This site offers ISO images for Windows 8.1, Windows 10 and Windows 11. However, currently, the distrobuilder companion for Incus can only prepare ISO images for Windows 11, Windows 10, Windows Server 2019, Windows Server 2012, Windows server 2016, Windows Server 2019 and Windows Server 2022.

When you select the version of Windows, you are asked the type of file to download. Select the Windows Disk Image (ISO). Then, select your preferred product language. Finally, click to download the ISO file. The Windows 11 ISO should be about 6GB in size.

The file has been downloaded. In my case, the filename was Win11_23H2_English_x64v2.iso.

Preparing the Windows ISO file

The Windows ISO file needs to be prepared before it is used by Incus as a virtual DVD image to boot the virtual machine. When Incus starts up the empty virtual machine for the first time, it will find the ISO file attached as a virtual DVD disk, and boot it. By booting the virtual DVD disk, the installation process of Windows will start.

To prepare the original Windows ISO file, we use distrobuilder. First, we install it. Then, we install two packages that are required by distrobuilder but are not supplied by its snap package. Since the snap package is available with the classic confinement, it does has access to the host’s packages. If you do not install these packages, you will get errors while running distrobuilder.

$ sudo snap install distrobuilder --classic
distrobuilder 3.0 from Stéphane Graber (stgraber) installed
$ sudo apt install -y libguestfs-tools wimtools rsync
...
$

Then, we can run distrobuilder to repack the ISO file in a special way that Windows ISO files require. The following procedure took about 2 minutes on my computer. The new, prepared, ISO file has been named Win11_23H2_English_x64v2.incus.iso and it is for use with Incus only.

$ sudo distrobuilder repack-windows Win11_23H2_English_x64v2.iso Win11_23H2_English_x64v2.incus.iso 
[sudo] password for myusername:  
INFO   [2024-01-30T02:23:01+02:00] Mounting Windows ISO                         
mount: /var/cache/distrobuilder.3138255837/source: WARNING: source write-protected, mounted read-only.
INFO   [2024-01-30T02:23:01+02:00] Downloading drivers ISO                      
INFO   [2024-01-30T02:24:45+02:00] Mounting driver ISO                          
...
$ 

The ISO file is prepared, we can now work on the Incus VM for Windows. The ISO preparation takes about 5 minutes.

Creating the Windows VM on Incus

We are going to create the VM through special steps.

1. We initialize the VM. It is a VM because --vm, it is called win11vm in our case, and it is --empty. It is important to use --empty because we are not using some container or virtual machine image from the repositories. The virtual machine is really empty.
2. We override the default size of the disk of the VM to 60GB. The default size for the disk in Incus 10GB which is not enough for the installation of Windows. At the end of the installation, Windows use about 12GB of space. See the Troubleshooting section to figure out how much space you can afford to use or how to choose an older version of Windows because you do not have enough space for the VM.
3. We allocate 4 CPU cores for the VM, and 8GB of RAM memory. Specify accordingly for your system. The minimum requirements for Windows 11 is 2 CPU cores and 4GB RAM.
4. We enable Trusted Platform Module 2.0 emulation (tpm) so that Windows 11 are able to boot.
5. We enable audio support through QEMU. (Note: since writing this post, Incus was updated to use version 8.1. It now requires to append -audio spice to the audio configuration.)

$ incus init win11vm --empty --vm
Creating win11vm
$ incus config device override win11vm root size=55GiB
Device root overridden for win11vm
$ incus config set win11vm limits.cpu=4 limits.memory=6GiB
$ incus config device add win11vm vtpm tpm path=/dev/tpm0
Device vtpm added to win11vm
$ incus config set win11vm raw.qemu -- "-device intel-hda -device hda-duplex -audio spice"
$ incus config device add win11vm install disk source=/home/myusername/Downloads/Win11_23H2_English_x64v2.incus.iso boot.priority=10
Device install added to win11vm
$ 

Now we are ready to boot the VM with the attached ISO image that will appear as a DVD disk. From there on, the installation of Windows will commence.

Booting the Windows virtual machine

Initially, the VM is in a stopped state. We are going to start it and at the same time request to open a window that shows the console of the VM. This helps us to perform the initial installation. If you get a message that the VM is already running, then see the next section on how to reconnect to the VM.

$ incus start win11vm --console=vga

Reconnecting to the Windows virtual machine

If you close the Windows virtual machine window (pun not intended), then the window closes but the VM continues to run. In that case, you can reconnect to the running Windows VM by running the incus console command with the parameter --type=vga. If you get a message that the VM is stopped, then see the previous section on how to start the VM.

$ incus console win11vm --type=vga

Removing the virtual Windows DVD ISO

The virtual Windows DVD ISO is attached to the Incus instance. When you boot the Windows Incus instance, you are asked to press any key to boot from the DVD and there’s a small delay before continuing. If you have already installed Windows, you can remove the virtual Windows DVD ISO from the Incus instance with the following command. We are using the incus config command, subcommand device remove, on the instance win11vm, for the device called install.

$ incus config device remove win11vm install
Device install removed from win11vm
$ 

Screenshots

When you first launch the Incus instance with Windows, you are booting from the attached virtual Windows DVD. As it is common with the booting process, you are asked to press any key in order to boot from the DVD. Otherwise, the system will boot from the VM hard disk, which is empty. Therefore, you need to click inside the window and press any key in order to get the attached virtual Windows DVD to boot. If you do not make it on time as in the screenshot, click on the menu Send Key, then Ctrl+Alt+Del to send the signal to reboot the VM.

When the VM is booting, the operating system shows an image that is provided by the UEFI environment. On your desktop when you start Windows, it would show the logo of the manufacturer of the motherboard.

In this case, Incus is showing the name of the company of the packager of Incus that I am using.

Some other random screenshot.

Summary

We show how to

1. how to download the ISO file from Microsoft,
2. how to prepare the ISO file so that it can be used by Incus,
3. setup an Incus VM to install Windows,
4. how to start or reconnect to the VM.
5. how to create an Incus profile with the configuration
6. and below some troubleshooting issues.

Bonus material #1 – Incus profile for Windows

When you run many Windows VMs, you might want to simplify the process of setting up the Incus instance. For example, the allocated amount of disk space, how many CPU cores and amount of RAM memory.

You can create Incus profiles to describe sets of configuration. When you create the Incus instance for Windows, specify the profile and you avoid repeating the same configuration steps.

To do so, we make a copy of the default profile into a new name, let’s call it windows. Then, we add configuration to that profile.

$ incus profile copy default windows
$ incus profile show windows
config: {}
description: Default Incus profile
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: windows
used_by: []
$ incus profile set windows limits.cpu=4
$ incus profile set windows limits.memory=6GiB
$ incus profile set windows raw.qemu="-device intel-hda -device hda-duplex -audio spice"
$ incus profile device add windows vtpm tpm path=/dev/tpm0
$ incus profile device set windows root size=55GB
$ incus profile set windows --property description="Windows profile 4CPU, 6GB RAM, 55GB space"
$ incus profile show windows
config:
  limits.cpu: "4"
  limits.memory: 6GiB
  raw.qemu: -device intel-hda -device hda-duplex -audio spice
description: Windows profile 4CPU, 6GB RAM, 55GB space
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    size: 55GB
    type: disk
  vtpm:
    path: /dev/tpm0
    type: tpm
name: windows
used_by: []
$ incus profile list
+---------+--------------------------------------------------------------------+---------+
|  NAME   |                            DESCRIPTION                             | USED BY |
+---------+--------------------------------------------------------------------+---------+
| default | Default Incus profile                                              | 42      |
+---------+--------------------------------------------------------------------+---------+
| gui     | GUI Wayland and xWayland profile with pulseaudio, shifting enabled | 3       |
+---------+--------------------------------------------------------------------+---------+
| windows | Windows profile 4CPU, 6GB RAM, 55GB space                         | 0       |
+---------+--------------------------------------------------------------------+---------+
$ 

When you create the profile, then the commands to start the virtual machine become shorter.

$ incus init win11vm --empty --vm --profile windows
Creating win11vm
$ incus config device add win11vm install disk source=/home/myusername/Downloads/Win11_23H2_English_x64v2.incus.iso boot.priority=10
Device install added to win11vm
$ 

Note that if you use many versions of Windows, you can create several profiles for each version. Windows 11 requires that there are available 52GB of disk space but Windows 10 can fit in much less space.

Bonus material #2 – Windows ISO as a volume in Incus

This tutorial describes how to use the prepared ISO file of a version of Windows to boot the VM. This prepared ISO file is located somewhere on the filesystem of the host. However, there is the option to import the prepared ISO image into a volume of your storage pool. That is, the location where Incus stores the images of your containers and virtual machines, can also be a place where you store the prepared ISO image.

First, we find out the name of our storage pool. It is called default. We can list the volumes in the default storage pool. Then, we import the prepared Windows ISO into the default storage pool as a volume, called win11 and of type iso.

$ incus storage list
+---------+--------+---------+-------------+---------+---------+
|  NAME   | DRIVER | SOURCE  | DESCRIPTION | USED BY |  STATE  |
+---------+--------+---------+-------------+---------+---------+
| default | zfs    | default |             | 76      | CREATED |
+---------+--------+---------+-------------+---------+---------+
$ incus storage volume list default
...
$ incus storage volume import default Win11_23H2_English_x64v2.incus.iso win11 --type=iso
Importing custom volume: 100% (268.86MB/s) 
$ incus storage volume list default -c tnc
+-----------------+------------------------------------------------------------------+--------------+
|      TYPE       |                               NAME                               | CONTENT-TYPE |
+-----------------+------------------------------------------------------------------+--------------+
+-----------------+------------------------------------------------------------------+--------------+
| container       | wordpress                                                        | filesystem   |
+-----------------+------------------------------------------------------------------+--------------+
| custom          | win11                                                            | iso          |
+-----------------+------------------------------------------------------------------+--------------+
...
$ 

Then, you can add the win11 volume to the win11vm virtual machine as a device called install

$ incus config device add win11vm install disk pool=default source=win11 boot.priority=3 

Then continue with starting the virtual machine.

Bonus material #3 – CPU Pinning for QEMU

Incus is using QEMU for hardware virtualization. When you install Incus, you are also installing a separate version of QEMU for the purposes of Incus. In some cases QEMU may have a hard time to perform the virtualization. It appears that this is related to how we allocate CPU cores to QEMU. If we specify that QEMU should use any 4 cores of our system, then this might cause crashes to the virtual machine. The workaround is to specify explicitly specific CPU cores, or allow QEMU to give a more equal access to the CPU to the Windows VM.

Therefore, if the Windows VM is crashing when you start it, you can try with only one of the following workarounds. In this order.

CPU Pinning

CPU Pinning in Incus means that you specify which specific CPU cores should be allocated to the VM. If you do not specify, then QEMU may switch the CPU cores while the VM is running. Normally this is OK but on some hardware configurations this does not appear to work well.

Let’s say you have a 6-core CPU, with two threads per CPU. In the following you can see two of the cores (in blue) and their respective threads (in green). When you perform CPU Pinning, you specify the thread IDs. Therefore, it makes sense to select both threads of a core.

$ incus info --resources
...
  Cores:
    - Core 0
      Frequency: 3653Mhz
      Threads:
        - 0 (id: 0, online: true, NUMA node: 0)
        - 1 (id: 6, online: true, NUMA node: 0)
    - Core 1
      Frequency: 2683Mhz
      Threads:
        - 0 (id: 1, online: true, NUMA node: 0)
        - 1 (id: 7, online: true, NUMA node: 0)
...
$

Therefore, if we want to allocate the two full cores of the CPU (4 vCPUs) to Windows, we would need to specify the IDs 0, 6, 1, 7. Therefore, we would run the following, then start the virtual machine.

incus config set win11vm limits.cpu 0,1,6,7

Thanks to Mark in the comments for the tip.

Using -cpu host

When you run QEMU, you tend to give less access to the features of the CPU to the virtual machine. It’s good to do that. However, when the virtual machine does not work, you may have to give more access to the CPU features to the VM. In my case I had some issues with the Windows VM crashing as soon as it started running. I resolved this my recreating the VM. It that fails, and CPU pinning does not help, then the ultimate workaround is to give more access to the CPU to the VM.

Append the following to the raw.qemu line. It gives a fuller access to the features of the CPU to the virtual machine. This might have a small impact to the host’s speed.

-cpu host

Thanks to Stéphane for the hint.

Bonus material #4 – How to mount a directory from the host to the Windows VM

When you launch Windows VMs with Incus, the VM does not receive the Agent software that you normally expect to receive a Linux VM. There is no such Agent software for Windows yet. In addition, when you prepare the Windows ISO file for Incus using distrobuilder, the repack procedure of distrobuilder does not inject the packages to mount directories. Therefore, we need to install two packages in the Windows VM on demand, and following that, we can add a disk device using the Incus command incus config device add .... The first package is WinFSP and the second is the VirtIO Windows Guest Tools.

Installing WinFSP in the Windows VM

Visit https://winfsp.dev/, switch to the Download tab, and click the green button to download the WinFsp Installer. Once you download, install the package.

WinFSP stands for Windows File System Proxy, and it provides support for custom filesystems on Windows.

Installing the VirtIO Windows Guest Tools in the Windows VM

This package is available at https://github.com/virtio-win/virtio-win-pkg-scripts/blob/master/README.md The filename is virtio-win-guest-tools.exe(direct link). You would need to download this file from within the Windows VM and install it. Alternative, you can get the ISO file from the same website, attach to the VM and finally install the package.

By installing this package, your Windows VM will get a VirtioFsSvc service which is not enabled by default.

Verify that the Red Hat VirtIO SCSI pass-through storage controller has been installed on your Windows VM.

Enabling the VirtioFsSvc service in the Windows VM

Open the Computer Management utility and switch to the Services list. Double-click on the VirtIO-FS Service to open the settings.

Click to Start the service and make sure that the Startup type is set to Automatic.

Adding the disk device

Finally, you can add the disk device (i.e. a directory on the host) to the Windows VM.

Let’s assume there is a directory called SHARED on the host. Use the incus config device add command to add a directory from the host to the Windows VM. disk is the type of the device. source is the path for the directory on the host. path is the name of the share in the VM (it is not used in the case of Windows, as the share will get the name of the device).

$ cd
$ mkdir SHARED
$ incus config device add win11vm SHARED disk source=/home/myusername/SHARED path=SHARED
Device SHARED added to win11vm
$ 

Open Windows Explorer in the Windows VM and notice the new shared drive incus_SHARED (Z:). The contents of the Z: drive are the same with the contents of the directory /home/myusername/SHARED on the host.

Removing the disk device

To remove the disk device, shutdown the Windows VM and then run the following.

$ incus config device show win11vm
SHARED:
  path: SHARED
  source: /home/myusername/SHARED
  type: disk
$ incus config device remove win11vm SHARED
Device SHARED removed from win11vm
$ 

Troubleshooting

Error: This revision of snap "distrobuilder" was published using classic confinement...

You are trying to install the distrobuilder snap package. It does not install.

distrobuilder requires full access to your system because it uses loop devices, etc. Snap packages are confined. Therefore, by using the classic confinement, distrobuilder does not have any restrictions. To accept the classic confinement, you need to specify it when installing the package. Therefore, install distrobuilder by running the following command.

$ sudo snap install distrobuilder --classic
distrobuilder 3.0 from Stéphane Graber (stgraber) installed
$ 

Error: You must be root to run this tool

When you run the distrobuilder, you get the error message that you must be root to run the tool.

Indeed, you need elevated permissions to run distrobuilder. Run it either as root, or pre-pend sudo when running it.

Error: Failed to check dependencies: Required tool “hivexregedit” is missing

Error: Failed to check dependencies: Required tool “wimlib-imagex” is missing

Error: Failed to check dependencies: Required tool “rsync” is missing

You are trying to run distrobuilder and prepare a Windows ISO. You get this error message.

The snap package for Distrobuilder has package dependencies from the host but since it is a snap package, it is not able to pull them in automatically. Therefore if you get this error, you need to install the following packages first.

$ sudo apt install -y libguestfs-tools wimtools rsync

Error: Failed add validation for device "install": Missing source path for disk "install"

You are trying to attach the ISO image to an Incus VM and it fails.

Double-check the path to the ISO file. If you copy-pasted the command, you would need to update the path.

Error: The instance is already running

You are trying to start an Incus VM and you get the message that the instance is already running.

In that case, you should reconnect to the VM, using the following command.

$ incus console win11vm --type=vga

Error: Instance is not running

You are trying to reconnect to an Incus VM and you get the message that the instance is not running.

In that case, you should start the instance, using the following command.

$ incus start win11vm --console=vga

Error: Failed to detect Windows version. Please provide the version using the --windows-version flag

You are trying to prepare an ISO for a version of Windows such as Windows 7 or Windows 8.1 or Windows Server 2012.

distrobuilder tries to detect automatically the version of Windows in the ISO file. If it does not, you get the option to specify the version. The currently supported versions are

1. Windows 11, use w11
2. Windows 10, use w10
3. Windows Server 2012, use 2k12
4. Windows Server 2016, use 2k16
5. Windows Server 2019, use 2k19
6. Windows Server 2022, use 2k22

Error: Windows crashes during the first moments of booting up

When booting the virtual machine and the Windows ISO is starting up, it is possible for Windows to crash withing a few moments. And then keep crashing. While writing this tutorial and trying the instructions several times, I got into a case that Windows kept crashing. It was not clear why it was happening. I eventually removed the VM and started over again. Then it work.

If you get into such a situation, it is important to collect information in order to figure out why it happens. In any case, a workaround is to add the following configuration. We appended to the instructions in QEMU the -cpu host directive. This means that QEMU will emulate a CPU that is the same as the host.

incus profile set windows raw.qemu="-device intel-hda -device hda-duplex -cpu host"
or
incus config set win11vm raw.qemu -- "-device intel-hda -device hda-duplex cpu host"

Error: When I enable audio in the Windows VM, I lose audio from the host

This looks to be an issue with the virtualization. I do not know if there is a way to retain audio on the host while it works in the virtual machine. If you do not need audio in the Windows VM, then do not enable audio. You can do so using a separate Incus profile. If you know of a workaround that enables audio both on the host and in the Windows virtual machine, post it in the comments!

How much space is available in Incus to use for my VM?

First we find the name of the Incus storage pool. Then, we ask Incus to show the free space in that storage pool. In this case, 60GB is used by all instances and there’s still plenty of space to go.

$ incus storage list
+---------+--------+---------+-------------+---------+---------+
|  NAME   | DRIVER | SOURCE  | DESCRIPTION | USED BY |  STATE  |
+---------+--------+---------+-------------+---------+---------+
| default | zfs    | default |             | 65      | CREATED |
+---------+--------+---------+-------------+---------+---------+
$ incus storage info default
info:
  description: ""
  driver: zfs
  name: default
  space used: 98.65GiB
  total space: 512.22GiB
used by:
...
$ 

In this example, the total space is about 500GB and we have used only about 100GB. Plenty of free space for VMs.

Which version of Windows should I use in the Incus VM?

If you want a VM with Windows and you are not particularly interested in a specific version of Windows, I suggest to opt for Windows 10. Unlike Windows 10, Windows 11 23H2 require a disk partition of at least 52GB of space and they refuse to install if you do not provide enough space. In contrast, Windows 10 will happily install with less space, like 32GB of disk space.

When you install Windows 11, it takes up about 18GB of disk space. When you install Windows 10, it takes up about 12GB of disk space.

How to get logs of the virtual machine if something goes wrong?

You can use the following command. It shows info from the instance, including any available logs. In this example, there aren’t any logs.

$ incus info --show-log win11vm
Name: win11vm
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2024/02/01 00:00 GMT+0
Last Used: 2024/02/01 01:23 GMT+0

Log:

$ 

I am using Fedora. What are the dependencies for distrobuilder?

If you use Fedora and you want to run distrobuilder from there, you need to install the following packages.

sudo yum install -y genisoimage perl-hivex wimlib-utils