Migrating a database to cloud infrastructure is rarely the performance-neutral exercise the architecture diagram suggests. Teams that have spent years tuning on-premises storage, whether direct-attached NVMe or a well-understood SAN, land on generic cloud block volumes and immediately notice the gap: latency climbs, IOPS consistency degrades under load, and the tuning levers they relied on either behave differently or disappear entirely. The problem is not cloud storage in principle. It is that most cloud block storage advice is written for the median workload, not for the OLTP database running 50,000 transactions per minute with a 2 ms latency budget.
This post covers the levers that actually move the needle: storage protocol selection, Linux I/O scheduler configuration, and queue depth tuning. None of it is exotic. All of it is skipped in generic migration guides, and that gap is where most database storage performance problems live.
Why cloud block storage latency surprises database teams
On-premises database storage is usually tuned around predictable, low-latency paths. A locally attached NVMe drive completes I/O in tens of microseconds. Legacy SAN protocols like iSCSI or FCP add software and network overhead that can push each I/O into the hundreds of microseconds, but teams have learned to account for that. Cloud block volumes introduce a third category: network-attached storage with variable overhead that depends on the hypervisor, the storage backend, and what else is running on the shared fabric at that moment.
Database workloads are particularly sensitive to this because their requirements cluster around two things: IOPS consistency and latency predictability. A burst of 4K random reads at queue depth 1 reveals the true floor of a storage path. A sustained mixed read/write workload at realistic concurrency reveals whether that floor holds. Shared infrastructure can pass the first test and fail the second in ways that are genuinely hard to predict without workload-specific benchmarking before the migration.
The protocol layer: NVMe is not a marketing claim
The choice of storage protocol sets a hard ceiling on what tuning can achieve. NVMe’s latency profile, 20 to 100 microseconds for enterprise drives, is not a marginal improvement over SATA SSDs (80 to 200 µs) or even the best SAS SSDs (50 to 200 µs). At the queue depths and concurrency levels typical of OLTP databases, the gap compounds. Enterprise PCIe/NVMe SSDs deliver roughly 1,400,000 read IOPS in controlled benchmarks, approximately 16× more than enterprise SATA SSDs at 85,000 read IOPS. Gen4 NVMe reaches sequential read bandwidth exceeding 7 GB/s; Gen5 doubles that again.
The architectural reason matters as much as the headline figure. AHCI, the command interface underlying SATA, supports a single queue of 32 commands. NVMe supports up to 65,535 queues of 65,535 commands each. For a database with many concurrent sessions, this is not a theoretical advantage. It is the difference between serialised I/O that accumulates latency under load and parallel I/O that scales with concurrency.
If your storage protocol cannot keep pace with the database’s concurrency model, no amount of scheduler or queue depth tuning will compensate. The ceiling is set at the protocol layer.
When block storage is delivered over the network, the protocol choice matters again. NVMe over Fabrics (NVMe-oF) preserves much of the latency advantage of locally attached NVMe by extending the NVMe command set across a network fabric, rather than translating it into iSCSI or FCP semantics and paying the overhead of that translation on every I/O. Benchmarks consistently show NVMe-oF delivering significantly more IOPS per CPU core than iSCSI, which matters when storage throughput is competing with database CPU for the same host resources.
Linux I/O scheduler: the default is correct, until it is not
The Linux kernel selects an I/O scheduler based on device type. For NVMe devices, the default is none, meaning requests are passed directly to the driver without reordering. This is intentional. If the device already handles request optimisation in firmware, adding a software scheduler layer costs CPU cycles without improving latency. The same logic applies to guest VMs using virtio-blk: the host is already scheduling, and doing it twice wastes work.
The case for mq-deadline is narrower but real. It is appropriate for rotating storage, for workloads where preventing I/O starvation matters more than minimising average latency, and for mixed read/write patterns where write requests need a guaranteed service window. Under mq-deadline, read requests carry a 500 ms deadline and writes carry 5 seconds; requests approaching their deadline are promoted regardless of their position in the queue. Oracle explicitly recommends none for NVMe and SSDs, and mq-deadline for rotating storage, on Oracle Linux 8 and later, RHEL 8 and later, and SUSE Linux Enterprise Server 15 and later.
The practical error is leaving mq-deadline active on NVMe volumes because it was set for the on-premises HDD array and nobody changed it during migration. On NVMe, it adds latency without adding value.
Queue depth: the tuning lever with the sharpest tradeoffs
Queue depth controls how many I/O operations are in flight simultaneously. Setting it too low leaves device bandwidth unused; setting it too high introduces contention and amplifies latency jitter, particularly during microbursts. For database workloads, the tradeoff is asymmetric: a brief latency spike at high queue depth can cause transaction timeouts that a slightly lower IOPS ceiling would not.
Queue depths between 32 and 128, with block sizes between 4 KiB and 16 KiB, cover the range most database teams should benchmark first. The right value depends on the workload’s concurrency model and the storage backend’s internal parallelism. A good benchmark sequence runs the same workload at queue depths of 1, 4, 8, 16, 32, 64, and 128, then plots both IOPS and average latency. The inflection point where latency starts rising faster than IOPS is the practical ceiling for that workload on that storage path.
Excessive queue depth has a secondary cost: it raises CPU overhead per second and makes latency distribution wider. For a database with tight latency SLOs, a p99 latency that doubles at queue depth 128 compared to queue depth 32 is a real operational problem, even if average latency is acceptable.
Bare-metal-backed storage and the hypervisor overhead question
One variable that generic cloud advice consistently glosses over is hypervisor overhead on the storage path. Every layer of virtualisation between the database process and the physical NVMe drive adds latency and, more importantly, adds variance. For latency-sensitive OLTP workloads, the consistency of the storage path matters as much as its peak throughput.
This is where bare-metal-backed block storage changes the calculation. At MUC1, our Munich datacenter, we run bare-metal infrastructure specifically because the finance and industrial workloads concentrated in the DACH market have tight latency requirements that shared hypervisor layers cannot reliably meet. Block storage backed by locally attached NVMe on bare metal eliminates the hypervisor overhead variable entirely, which means the tuning work described above operates on a stable baseline rather than a moving target.
Benchmarking before you migrate, not after
The sequence matters. The common failure mode is migrating first, hitting a performance wall, then attempting to tune against a production system under pressure. The correct sequence is to establish the on-premises baseline with a tool like fio at realistic queue depths and block sizes, replicate that workload profile against the target cloud storage, and compare the results before any production traffic moves.
Specific numbers to capture: 4K random read IOPS at queue depth 1 (the latency floor), 4K random read IOPS at the expected concurrent session count (the throughput ceiling), p99 latency at both queue depths, and the latency distribution under a sustained mixed read/write load. If the cloud storage path cannot meet the on-premises baseline on these metrics in a controlled test, it will not meet it in production.
Block storage I/O tuning is not a post-migration cleanup task. It is a prerequisite for knowing whether the migration is viable at all.
