KVM Server

Basic KVM server setup on ubuntu (18.04).

Files

Install Service

Ensure hardware can support virtualization.

egrep -c '(vmx|svm)' /proc/cpuinfo

Anything <= 0 means that hardware virtualization is disabled or not supported with hardware
AMD Processers: Check BIOS, ensure IMMOU and SVM is enabled.
Intel processors: Check BIOS, ensure IMMOU and VT-d is enabled.

Verify KVM accleration can be used.

apt install cpu-checker
kvm-ok

This should clearly state if accleration can be used.

Install KVM packages.

apt install qemu qemu-kvm qemu-efi qemu-utils libvirt-bin libvirt-clients libvirt-daemon-system virt-manager

Set user group permissions.

adduser {USER} libvirt
adduser {USER} libvirt-qemu

This is so a normal user can run virt-manager, instead of logging in as root.

Add/Update storage pools

By default a single location is used for VM’s and ISO images. Create/map additional locations for storage pools to mount ISO images to keep separate from VM images.

Launch VM manager with X11 forwarding enabled.

virt-manager

KVM Specific Issues

There seems to be an issue with Netplan bridging, KVM, and using the same bridged for host networking traffic as well as VM traffic. The workaround is to have a separate bridged adapter. This is a longstanding bug with KVM and can be fixed by modifying sysctl settings.

See Docker Bridged Adapters to resolve Docker issues.

Create a Network Bridge

This is so VM’s can get an IP on the host network, instead of using NAT.

Show all network adapters (including currently unassigned) for usage.

ip link show
lspci | grep ethernet

0644 root root /etc/netplan/01-netcfg.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: yes
    eno2:
      dhcp4: no

  bridges:
    br0:
      interfaces: [eno2]
      dhcp4: no
      macaddress: A0:B1:C2:D3:E4:F5

Note

MAC is randomly generated on boot if not specified for the bridge network.

Netplan seems funky in consistently applying changes. In most cases a reboot applies the config correctly. See Netplan documentation.

Apply the network configuration.

netplan --debug apply
networkctl status -a

ip a should also display corresponding information.

Add Bridge to KVM Menu Drop Down

By default if you add a bridge, you will have to select Specify Shared Device Name then entering your bridge (typically br0). This will add the bridge directly to the dropdown menu instead.

Create an XML configuration file with your settings:

0644 root root /etc/libvertd/qemu/networks/br0.xml

<network>
  <name>br0</name>
  <bridge name='br0'/>
  <forward mode='bridge'/>
</network>

See network XML file format.
See bug for creating bridged netplan interfaces for libvirtd.

Import the network into KVM, start it and set to autostart.

virsh net-define /etc/libvertd/qemu/networks/br0.xml
virsh net-start br0
virsh net-autostart br0

Show virtual networks, persistent should read yes for it to autostart.

virsh net-list --all

Remove Pre-made NAT Virtual Bridge

This network is not needed if using bridging.

Identify the NAT virtual network.

virsh net-list -all

Set network inactive, remove it and restart libvirtd.

virsh net-destroy br1
virsh net-undefine br1
service libvirtd restart

Confirm the network no longer exists.

virsh net-list --all

Allow SSH Connections Through UFW.

Creating New VM

Setup a standard VM to use the network bridge.

Launch VM manager with X11 forwarding enabled.

virt-manager

Important

Be sure to begin installation for VM to be created.

Install Guest OS Tools

These are only needed if you want to use a GUI in linux (required for windows).

Linux

apt install spice-vdagent xserver-org-video-qxl

Windows 10 requires signed virtio drivers. Drivers have been signed with the Red Hat vendor signature.

Install signed virtio Guest Tools.

Latest Windows Spice Guest Tools

Convert XenServer XVA to KVM Image

XenServer images cannot be directly imported, they must be converted first. VM’s should be exported 1 instance per XVA image export.

Install Build Tools

apt install cmake gcc build-essentials libssl-dev

Clone xva-img Tool & Build

git clone https://github.com/eriklax/xva-img.git
cd xva-img/
cmake .
sudo make install

Extract VM from XVA Image

mkdir my_vm
tar -xvf my_vm.xva -C my_vm
chmod -R 0755 my_vm
xva-img -p disk-export my_vm/Ref\:{XXX}/ my_vm/ref-{XXX}.raw

Disks have no permissions by default.
There will be one Ref:XXX directory per disk. Generally, keep this named as the reference number for sanity, until you know what they are.
Note: RAW is generally better for performance and long term performance.

(Optional): Convert Disk Image to qcow2

qemu-img convert -f raw -O qcow2 my_vm/ref-{XXX}.raw my_vm/ref-{XXX}.qcow2

QCOW images are generally slower but allow for deduplication and consolidation of unused space.

Grab Metadata From VM

VM metadata (such as # of CPU’s, memory, MAC) are not extracted by default. This should be extracted for correct VM import into KVM.

CPU

grep -o '.\{0,40\}CPU.\{0,40\}' my_vm/ova.xml

Memory

grep -o '.\{0,40\}memory.\{0,40\}' my_vm/ova.xml

MAC

grep -o '.\{0,40\}MAC.\{0,40\}' my_vm/ova.xml

Hostname

grep -o '.\{0,40\}hostname.\{0,40\}' my_vm/ova.xml

Export KVM Image

Useful for a configuration backup or moving to a new system.

Dump the Current VM configuration

virsh dumpxml {VM NAME} > {VM NAME}.xml

Copy the XML file and associated disks to new location.

Import VM

virsh create {VM NAME}.xml

Update disk location in XML file if location has changed.

Moving KVM Images

KVM images are stored in two locations, configuration and disk images.

Ensure VM is stopped.
Move VM disk images to new location.
Update location information in XML file /etc/libvirtd/qemu/{VM}.xml.
Restart service service libvirtd restart.

Moving KVM Storage Pool

The default image storage location makes sense for linux (/var), but not for servers centralizing data to storage pools.

By default, a single pool default is used for both VM images and ISO images. Service requires a restart on changes.

List all pools

virsh pool-list
virsh pool-info {POOL NAME}

Delete a pool

virsh pool-destroy {POOL}

This will only remove the pool in KVM, not delete the underlying data.
Alternatively, you can just delete the definition in /etc/libvirtd/storage and corresponding autostart file if existing /etc/libvirtd/storage/autostart.

Move pool storage location while running

virsh pool-edit {POOL}

Update location for storage.
Generally need to restart libvirtd for changes to apply.

Dump Disk Image Pool

virsh pool-dumpxml default > pool.xml

Assumes pool name is default.
Make sure disk images are moved to new location.
Update disk image locations in XML file.

Destroy existing pool, import new pool from XML dump

virsh pool-destory default
virsh pool-create pool.xml

Mount RAW Disk Image

This will enable mounting of a RAW disk image outside of the VM.

Ensure the RAW image is readable

fdisk -l /var/lib/libvirt/images/{IMAGE}.RAW

Determine Sector Size.
Determine Start Sector for partition to mount.

Important

The sector offset is:

Sector Start * Sector Size = Sector Offset

Mount the partition as a block loop device

losetup -r -o {SECTOR OFFSET} /dev/loop0 /var/lib/libvirt/{IMAGE}.RAW

losetup -d /dev/loop0 can be used to detach device later.
losetup -l will show looped devices current mounted.

Mount the Filesystem

mount /dev/loop0 /mnt/image

Threadripper BSOD Windows 10 1803+

Windows 10 versions 1803+ will BSOD on installation due to a unavaliable MSR registers in KVM.

A registers patch has been created and will be avaliable in the 4.20+ kernel release.

Temporary Workaround

Emulating a Opteron Generation 5 processer will prevent bluescreens from happening. This will be an emulated CPU instead of passthrough.

Create a VM as normal and shutdown. Edit the VM definition to force emulate an Opteron processor, and reload the definition.

0644 root root /etc/libvirt/qemu/threadripper-vm.xml

<cpu mode='custom' match='exact' check='partial'>
    <model fallback='allow'>Opteron_G5</model>
    <topology sockets='1' cores='8' threads='1'/>
    <feature policy='disable' name='xop'/>
    <feature policy='disable' name='fma4'/>
    <feature policy='disable' name='tbm'/>
</cpu>

Define the VM profile for KVM

virsh define /etc/libvirt/qemu/threadripper-vm.xml

References