NVMe Hardware RAID vs NVMe Direct | xByte OnPrem Pros

In this episode of On-Prem Pros, your hosts Ryan Brown, Josh Moss, and Tyler Young (Solution Architects at Xbyte Technologies) dive deep into the heated debate between NVMe hardware RAID and software RAID.

They share real-world performance benchmarks, configuration tips, and use-case considerations to determine which solution truly offers better value and performance for today’s demanding workloads.

Key Points :

• Hardware RAID is often chosen out of habit or legacy familiarity, not necessarily because it performs better.
• NVMe Direct (software RAID) consistently outperformed hardware RAID in several benchmark tests.
• Tweaking RAID controller settings (like caching policies) significantly affects performance outcomes.
• Hardware RAID showed advantages in specific scenarios like large backup workloads.
• Tools like Dell Live Optics help analyze your existing workload to make smarter configuration choices.

Despite common assumptions, hardware RAID isn't always the better choice, as familiarity often drives its selection rather than performance benefits.

• VMware doesn’t support software RAID, but most other OSs like Windows and Linux do.
• Legacy users stick to hardware RAID due to past experiences with SAS-based controllers.
• Software RAID offers flexibility and better performance in many modern use cases.
• The discussion sets up the need to test assumptions with real data.

The team describes their benchmark setup using two identical Dell servers-one with hardware RAID and one with NVMe Direct-to create a level playing field for comparison.

• Testing was done using raw devices to eliminate file system variables.
• Systems used: Dell PowerEdge R7615 with AMD CPUs and Gen4 NVMe drives.
• All tests were run on Ubuntu with consistent hardware configurations.

The first benchmark using 4K block size in a database-style workload showed NVMe Direct far outperforming hardware RAID, even with caching tweaks.

• NVMe Direct achieved up to 398K read IOPS vs. 146K with default RAID settings.
• Tuning controller to write-through closed the performance gap but didn’t beat software RAID.
• Default settings from Dell are not optimized for NVMe workloads.
• Proper tuning nearly tripled IOPS over default configurations.

Increasing block size to 64K changed the game, but NVMe Direct still outperformed hardware RAID.

• Larger block size helped hardware RAID catch up in performance.
• Read and write IOPS became more balanced across both setups.
• Controller settings significantly influenced both read and write performance.
• Caching policy changes are non-destructive and can be done post-deployment.

Streaming workloads using 1MB sequential reads again saw NVMe Direct pulling far ahead of hardware RAID in both IOPS and throughput.

• NVMe Direct handled 3x more IOPS compared to hardware RAID.
• RAID controller bottleneck became apparent with larger read workloads.
• No performance improvement was seen from controller tuning.
• Throughput is more critical than IOPS in streaming scenarios.

In a backup-style workload with 1MB sequential writes, hardware RAID finally took the lead, showing its strengths in specific write-heavy use cases.

• Hardware RAID with default caching hit 9.5 GB/s write speeds.
• NVMe Direct lagged behind in sequential write performance.
• Controller write caching benefits these types of workloads significantly.
• RAID tuning (like stripe width) must be chosen at setup-it’s a destructive change later.

To help customers choose the right configuration, the team recommends using tools like Dell Live Optics to understand real workload characteristics before making hardware decisions.

• Many users don’t know their actual I/O sizes or read/write ratios.
• Live Optics provides a full performance profile for better planning.
• It helps justify hardware decisions to management with before-and-after comparisons.
• Regular assessments can catch workload changes that require re-tuning.