News
PCIe Gen4 - RAID0 NVMe SSDs
Following on from our PCIE GEN 4.0 - IMPACT FOR MEDIA SERVERS news article, we tested four different high performance 1TB PCIe Gen 4 NVMe SSDs setup in RAID 0 ( Stripped ) and the results are excellent! See the full article for more detail.
In our previous article in regards to PCIe Gen4 and the new performance levels we are now seeing in media servers, we decided to test what kind of performance we could see with multiple Gen4 NVMe SSDs in a RAID 0 ( stripped ) setup.
Striping distributes the contents of each file among all disks in the RAID set. The benefit of RAID 0 is that the throughput of read and write operations to any file is multiplied by the number of disks because, unlike spanned volumes, reads and writes are done concurrently.
For more information on RAID, please look at our What is RAID informational page.
Test Bench
| CPU | Threadripper PRO 3955WX |
|---|---|
| MOTHERBOARD | ASUS PRO WS WRX80E-SAGE SE WIFI |
| MEMORY | 8 x Hynix 16GB DDR4 3200 ECC - 128GB Total |
| GPU | nVidia A6000 |
| STORAGE | Samsung 980 PRO 250GB NVMe - BOOT |
Testing Software
For our test we used Crystal Disk Mark, a well known tool for measuring read and write speeds on hard drives, solid state drives and NVMe drives. For the test we used the crystal disk mark NVMe default configuration:
- Test 1 - Sequential Read & Write, 1MiB Block Size, 8 Queues, 1 Thread.
- Test 2 - Sequential Read & Write, 128KiB Block Size, 32 Queues, 1 Thread.
- Test 3 - Random Read & Write, 4KiB Block Size, 32 Queues, 16 Threads.
- Test 4 - Random Read & Write, 4KiB Block Size, 1 Queue, 1 Threads.
Each of these tests was set to run five times, with an 8GiB random data set with the final result shown as an average across the five runs.
A good way to imagine the different queue and thread settings is as follows: 4KiB Block Size, 32 Queues, 16 Threads - would be 16 people asking for 4KiB of data, 32 pieces at a time.
HighPoint SSD7505 PCIe Gen4 x16 4-Port M.2 NVMe RAID Controller
We decided to use the HighPoint SSD7505 as the RAID controller in this test. This controller supports up to four Gen4 NVMe M.2s, up to 32TB of storage per controller and it also supports RAID 0, 1 & 10. As well as single disk passthrough. This made it an ideal candidate for this test.
PCIe Gen4 NVMe M.2s
We decided to look at three different high performance products currently available:Each test configuration contained four of the same model drive, configured into RAID 0 giving a single volume of 4TB.
Results
Sabrent Rocket
| Capacity | Interface | Flash Memory Type | Max Sequential | |
|---|---|---|---|---|
| Read | Write | |||
| 1TB | 4x PCIe 4.0 | BiCS4 3D NAND TLC | 5000 MB/s | 4400 MB/s |
Sabrent Rocket 4 Plus
| Capacity | Interface | Flash Memory Type | Max Sequential | |
|---|---|---|---|---|
| Read | Write | |||
| 1TB | 4x PCIe 4.0 | TLC NAND | 7000 MB/s | 5300 MB/s |
Samsung 980 PRO
| Capacity | Interface | Flash Memory Type | Max Sequential | |
|---|---|---|---|---|
| Read | Write | |||
| 1TB | 4x PCIe 4.0 | V-NAND 3-bit MLC | 7000 MB/s | 5000 MB/s |
Results Comparison
8K UHD Uncompressed
As part of our testing, we setup and ran a PIXERA instance on our test bench to see what kind of uncompressed image sequence performance we could get from this configuration. From our 8K UHD Bit Rate Calculation, you can see the estimated bit rate for an 8K UHD, 60FPS image sequence is 5.56 GB/s. Although this is well within the sequential read figures, there is no guarantee the images are stored sequentially on the drive and does not account for other reads and write occurring at the same time, which can be quite likely during media server software operation.
In our test setup, the outputs were running at 4K UHD, and the image sequence was scaled down to fit on the physical outputs. However the read requirement will stay the same. In this setup we were able to get two 8K UHD, 60 FPS image sequences running concurrently on a single system.
This shows great promise for future high end media servers and high resolution uncompressed content playback.
Finally { }
Looking at the results between the three different NVMe SSDs, there does not seem to be huge performance different between them. The 980 PRO keeps pace on the read side, coming ahead in the random 4KiB Q1T1 test, which is arguably the toughest test for a storage device. As well as the 1MiB sequential test. The Sabrent Rocket 4 Plus performs between in general with write operations, however there was a marked difference with the random 4KiB Q32T16, with the Samsung 980 PRO jumping ahead. This may well be due to Samsungs 1GB of LPDDR4 cache listed for the 980 PRO, providing a decent temporary storage space when getting hit from multiple sources. I am sure the Sabrent drives include a DRAM cache, but do not list any specifications for it.
All in all, any of these drive setups are going to provide an excellent read and write performance in comparison to older generation hardware. The cost to get this level of performance is reducing massively, from the cost to get a similar performance level with previous hardware. This will mean higher resolution, higher framerate uncompressed content will become more and more accessible.