Windows-on-NixOS, part 1: Migrating bare-metal to a VM

This is part 1 of a series of blog posts explaining how we took an existing Windows installation on hardware and moved it into a VM running on top of NixOS.

Background

We have a decently-equipped desktop PC sitting in our office, which is designated for data experiments using TensorFlow and such. During off-hours, it’s also used for games, and for that purpose it has Windows installed on it. We decided to try moving Windows into a VM within NixOS so that we could run both operating systems in parallel. Advantages this would give us included:

• Avoiding the need to reboot to switch operating systems
• Being able to run NixOS services and manage the machine with NixOps
• Snapshotting the Windows installation so we could recover it if we break it

Requirements

• Run NixOS and Windows in parallel
• Use existing Windows installation, because nobody here felt like installing Windows
• Not break the existing bare-metal Windows installation, in case the whole thing doesn’t work
• Be able to play games :)

Step 1: get Windows running

We installed NixOS on a zfs pool on an additional (unused) hard drive. We enabled SSH in order to have remote access, and libvirtd as a virtual machine manager.

In the resulting NixOS installation, we created a copy-on-write image using the existing windows installation hard drive as a basis, using qemu-img create -f qcow2 -o backing_file=/dev/disk/by-id/wwn-0x5001bXXXXXXXXXXX /var/lib/libvirt/images/windows.qcow2. This allowed us to boot with the same data as on the original hard drive, while:

• having it available read-write to the guest
• but also not touching the original (to avoid breaking the Windows installation in case we wanted to boot it on hardware again)
• and without IO-intensively imaging the whole hard drive.

What this would presumably not have allowed is booting the hardware Windows installation and still having the VM working afterwards, since the qcow2 image only recorded the differences from the physical device. Changing the contents of the physical device would probably have broken the filesystem horribly in the qcow2 image. We avoided this by not trying it.

We then created the virtual machine itself, using virt-manager’s wizard. The main key options used here were “Import existing disk image” for the installation method and the UEFI firmware option (since the existing Windows installation used UEFI). This worked out of the box, to our great excitement.

Step 2: Migration to zvol

Once this was confirmed to work, we copied the data from the copy-on-write image to a ZFS volume, removing an unnecessary layer of copy-on-write. This took a while, since almost all the data had to be copied from the original SSD to the zfs pool. qemu-img again proved handy for this, preserving the changes from the qcow2 image:

# zfs create -V 478G -o compression=zle spuinix/windows
# qemu-img convert -O raw -p /var/lib/libvirt/images/windows.qcow2 /dev/zvol/spuinix/windows
# zfs snapshot spuinix/windows@original-migrated

We then changed the VM’s storage to point to the zvol, checked again that it boots (concerned that the slight size increase may have made windows unhappy)—and it did!

Summary

By moving Windows into a VM using a copy-on-write image, we fulfilled the first three of our requirements and made a NixOS installation available on the machine in addition to Windows. However, this wasn’t the end of the story—the VM was still grossly insufficient for gaming, primarily because the graphics hardware was entirely emulated; there were also some other oddities and tuning that helped improve performance. We’ll cover the remaining adjustments in upcoming blog posts!