Migrating Proxmox VE Virtual Machines to LINSTOR Storage

Proxmox VE allows users to migrate a virtual machine’s (VM) virtual hard drive between different storage back ends. You might want to migrate a virtual machine’s hard disk from “local directory” storage to LINSTOR®, for example. You can do this while the virtual machine is running (online migration), or when the virtual machine is stopped (offline migration).

When performing an online migration of a virtual drive from any storage back end to a LINSTOR storage back end, there is a storage option that you must be temporarily configure for online migration to succeed. Offline migrations of storage to the LINSTOR back end do not require any additional options to succeed.

This knowledge base article describes the storage option, exactsize yes, and how it is enabled and disabled for online migration of virtual drive storage to a LINSTOR storage backend.

When trying to move a virtual drive from any storage back end to a LINSTOR storage back end while the virtual machine is running, you will receive an error similar to the following:

create full clone of drive scsi0 (local:100/vm-100-disk-0.qcow2)

NOTICE
  Trying to create diskful resource (pm-8f31c976) on (proxmox-0).
drive mirror is starting for drive-scsi0
drive-scsi0: Cancelling block job
drive-scsi0: Done.
TASK ERROR: storage migration failed: block job (mirror) error: drive-scsi0: Source and target image have different sizes (io-status: ok)

Proxmox is very particular about the sizes of source and destination volumes. If they do not match exactly, Proxmox will refuse to migrate the storage. LINSTOR slightly rounds volume sizes (for example, to allocate space for DRBD internal metadata), which can result in a situation where storage cannot be moved because the LINSTOR disk is slightly larger than the source disk. DRBD® offers an option where the user can set a specific size, which is then presented as the visible size, even if the actual backing devices are larger (for example, 3G devices with a 2G user-visible size). If you enable the exactsize option in Proxmox storage configuration file, /etc/pve/storage.cfg, LINSTOR will use this setting. This will make the device appear to have the exact same size as the source volume, allowing Proxmox to permit the migration. For technical reasons, this should only be done temporarily, as it complicates resizing.

To successfully online migrate a virtual machine’s disk to a LINSTOR back end, temporarily enable the exactsize yes option in /etc/pve/storage.cfg.

On one node in your Proxmox VE cluster, open /etc/pve/storage.cfg in a text editor, and add the exactsize yes line to your LINSTOR storage configuration. It should look similar to this:

drbd: drbdstorage
    content images,rootdir
    controller 192.168.222.130
    resourcegroup thinpool-3-replica
    exactsize yes

This will allow the virtual disk to be online migrated to the LINSTOR storage back end.

Once you’ve migrated each of the virtual machine’s disk images, you should disable this setting in the configuration by opening /etc/pve/storage.cfg in a text editor and removing the exactsize yes line completely, or “commenting it out” by adding a # to the beginning of the line. Additionally, you should delete the ExactSize setting from the LINSTOR resource definitions for all online virtual machine disks by using the following command (requires jq be installed on the system) from any node with LINSTOR client access to the LINSTOR controller:

# linstor -m --output-version v1 rd l | \
  jq '.[][].name' | \
  xargs -I {} linstor rd sp {} DrbdOptions/ExactSize False

It is safe to enable and disable the exactsize yes option as often as needed.

These details can be found in the LINSTOR User Guide section for Proxmox VE integration.

Written by: MDK - 2024-08-29

Reviewed by: MAT - 2024-08-30