Disk Controller and Cache Settings

Proxmox Disk Controller and Cache

This page provides information on disk-related settings when configuring virtual machines, including key considerations and precautions.

About SCSI Controllers

When creating a new VM and selecting a SCSI controller, the default option is “VirtIO Single.” This means that if multiple virtual disks are created (as described in the next section), they will all appear to be connected to this single controller. This setup optimizes memory and device usage within the virtual machine.

Choosing “VirtIO SCSI” allows each virtual disk to be assigned to a separate SCSI bus, which requires additional driver space and memory. However, distributing I/O operations can improve performance under certain workloads.

Other physical SCSI card emulations are generally not recommended.

About Disk Controllers

IDE (also known as ATA) was established in 1988, ensuring compatibility with nearly all operating systems. For OS versions released before 2003, using IDE may be the best choice. While physical IDE had a maximum speed of 133MB/sec, virtualized IDE does not have this limitation unless the OS enforces it. However, IDE does not support hot-swapping, meaning devices cannot be removed while the VM is running.

SATA is suitable for OS versions released after 2003 and is the most compatible option for modern operating systems. Physically, it is an evolution of IDE with serial transmission, offering speeds up to 6Gbps (around 600MB/sec). However, in a virtualized environment, there are no inherent speed restrictions unless enforced by the guest OS.

VirtIO SCSI and VirtIO SCSI Single provide the fastest disk access. However, they require the guest OS to have para-virtualized drivers. If the system lacks any IDE/SATA disks, the OS installation process will fail unless the VirtIO SCSI driver is installed. Linux distributions have included this driver since around 2012, FreeBSD since 2014, and for Windows OS, an additional ISO file containing VirtIO drivers must be provided during installation.

About Disk Cache

Disk cache settings involve a trade-off between data integrity and performance.

Feature	writethrough	directsync	none	writeback	unsafe
Cache Usage	Uses page cache (beneficial for read operations)	Does not use cache	Does not use cache	Uses page cache	Uses page cache
fsync Issuance	Issued on each write	Issued on each write	Not issued	Issued with delay	Not issued
Write Process	Writes to cache, then flushes to disk immediately	Writes directly to disk and flushes immediately	Writes directly to disk without flushing	Writes to cache and flushes with delay	Writes to cache without flushing
Read Performance	High (cache utilization)	Low (disk-dependent)	Low (disk-dependent)	High (cache utilization)	High (cache utilization)
Write Performance	Moderate (cache-assisted)	Low (no cache)	Moderate (no cache)	High (delayed cache flush)	Very high (delayed cache flush)
Data Integrity	High	Very high	Moderate (disk-dependent)	Low (delayed writes)	Very low (high data loss risk)
Use Case	Systems prioritizing read performance and integrity	Systems requiring maximum data integrity	Performance-driven systems with moderate integrity needs	Write-intensive systems balancing performance and integrity	Performance-focused systems where data loss is acceptable

TRIM (Discard)

The Discard command informs the storage system that certain blocks are no longer needed.

Originally introduced for SSD wear leveling, some OS may require SSD emulation to support TRIM. Beyond SSDs, TRIM can also aid secure deletion by removing encryption keys instead of overwriting data.

In private cloud environments, TRIM allows thin-provisioned storage to reclaim space. Normally, once storage is used by a VM, even deleted data remains allocated at the storage level.

fstrim -v /mountpoint

For real-time TRIM operations, mount with the discard option:

mount -o discard,remount /

IO Thread

Enabling IO threads in VirtIO SCSI offloads disk write operations to a separate thread from the main OS thread.

This setup requires additional CPU resources but reduces latency, particularly in VMs with multiple virtual disks handling simultaneous read/write operations.