Disk perf – NVME vs SATA

오늘은 NVME 와 SATA 의 성능 비교 테스트 했던 내용 정리 해서 올려 봅니다.  최근에 발표된 Intel SSD DC P4600 1.6TB NVME (U.2 Interface) 와 1TB SATA3 6Gbps link 의 IOzone 성능 비교 결과 자료 입니다.

  1. CPU : Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz x 1CPU
  2. Memory : 64GB
  3. Network : Intel X722 for 10GBASE-T
  4. RAID : None
  5. File System : xfs
  6. NVME : Intel SSD DC P4600 1.6TB NVME (U.2 Interface)
  7. SATA : 1TB Toshiba 7200rpm SATA3 6Gbps link
  8. OS : CentOS7.4 x64 edition
  9. Iozone Version : v3.471

스토리지 구성 환경

[root@hci1 ~]# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 931.5G  0 disk 
└─sda1        8:1    0 931.5G  0 part /mnt
sdb           8:16   0 931.5G  0 disk 
└─sdb1        8:17   0 931.5G  0 part 
sdc           8:32   0 931.5G  0 disk 
└─sdc1        8:33   0 931.5G  0 part 
nvme0n1     259:0    0   1.5T  0 disk 
├─nvme0n1p1 259:1    0   200M  0 part /boot/efi
├─nvme0n1p2 259:2    0     1G  0 part /boot
├─nvme0n1p3 259:3    0     1G  0 part [SWAP]
├─nvme0n1p4 259:4    0   128G  0 part /
└─nvme0n1p5 259:5    0   1.3T  0 part /var/lib/libvirt/images

NVME IOzone Test 결과 (평균 쓰기 2GB/sec , 읽기 6GB/sec)을 보여 주고 있습니다. dstat 결과에서는 write 평균  2.5~2.8GB/sec 정도의 성능을 보이는것으로 나오네요. 초기 Write 성능 하고, 마지막에 Random Write 하고 크게 차이 없이 비슷한 성능을 보여주고 있습니다.

[root@hci1 images]# ./iozone -s 4g -t 8 -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.471 $
		Compiled for 64 bit mode.
		Build: linux-AMD64 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov.

	Run began: Thu Jan 11 22:35:35 2018

	File size set to 4194304 kB
	Command line used: ./iozone -s 4g -t 8 -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
	Throughput test with 8 processes
	Each process writes a 4194304 kByte file in 4 kByte records

	Children see throughput for  8 initial writers 	= 2007336.72 kB/sec
	Parent sees throughput for  8 initial writers 	= 1374282.37 kB/sec
	Min throughput per process 			=  249708.02 kB/sec 
	Max throughput per process 			=  252134.64 kB/sec
	Avg throughput per process 			=  250917.09 kB/sec
	Min xfer 					= 4155468.00 kB

	Children see throughput for  8 rewriters 	= 2061249.83 kB/sec
	Parent sees throughput for  8 rewriters 	= 1391764.66 kB/sec
	Min throughput per process 			=  254481.56 kB/sec 
	Max throughput per process 			=  268137.38 kB/sec
	Avg throughput per process 			=  257656.23 kB/sec
	Min xfer 					= 3980972.00 kB

	Children see throughput for  8 readers 		= 6314758.88 kB/sec
	Parent sees throughput for  8 readers 		= 6314221.01 kB/sec
	Min throughput per process 			=  755203.00 kB/sec 
	Max throughput per process 			=  857677.38 kB/sec
	Avg throughput per process 			=  789344.86 kB/sec
	Min xfer 					= 3693180.00 kB

	Children see throughput for 8 re-readers 	= 6337455.19 kB/sec
	Parent sees throughput for 8 re-readers 	= 6337047.09 kB/sec
	Min throughput per process 			=  697131.38 kB/sec 
	Max throughput per process 			=  877066.50 kB/sec
	Avg throughput per process 			=  792181.90 kB/sec
	Min xfer 					= 3333832.00 kB

	Children see throughput for 8 random readers 	= 6256650.12 kB/sec
	Parent sees throughput for 8 random readers 	= 6255758.48 kB/sec
	Min throughput per process 			=  767580.06 kB/sec 
	Max throughput per process 			=  804192.50 kB/sec
	Avg throughput per process 			=  782081.27 kB/sec
	Min xfer 					= 4003364.00 kB

	Children see throughput for 8 random writers 	= 1954045.70 kB/sec
	Parent sees throughput for 8 random writers 	= 1314693.28 kB/sec
	Min throughput per process 			=  240611.81 kB/sec 
	Max throughput per process 			=  248439.33 kB/sec
	Avg throughput per process 			=  244255.71 kB/sec
	Min xfer 					= 4062156.00 kB



iozone test complete.

1TB TOSHIBA SATA3 6Gbps Link 성능 결과 Write 250MB/sec, OS Cache 영향으로 Read 14~22GB/sec , dstat 결과에서는 write 성능이 150~180MB/sec 정도 보여주고 있습니다.

[root@hci1 mnt]# ./iozone -s 4g -t 8 -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.471 $
		Compiled for 64 bit mode.
		Build: linux-AMD64 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov.

	Run began: Thu Jan 11 22:51:12 2018

	File size set to 4194304 kB
	Command line used: ./iozone -s 4g -t 8 -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
	Throughput test with 8 processes
	Each process writes a 4194304 kByte file in 4 kByte records

	Children see throughput for  8 initial writers 	=  259753.54 kB/sec
	Parent sees throughput for  8 initial writers 	=  161080.31 kB/sec
	Min throughput per process 			=   31635.43 kB/sec 
	Max throughput per process 			=   32821.76 kB/sec
	Avg throughput per process 			=   32469.19 kB/sec
	Min xfer 					= 4042980.00 kB

	Children see throughput for  8 rewriters 	=  265830.41 kB/sec
	Parent sees throughput for  8 rewriters 	=  162069.36 kB/sec
	Min throughput per process 			=   32293.76 kB/sec 
	Max throughput per process 			=   34493.11 kB/sec
	Avg throughput per process 			=   33228.80 kB/sec
	Min xfer 					= 3927408.00 kB

	Children see throughput for  8 readers 		= 22174536.12 kB/sec
	Parent sees throughput for  8 readers 		= 21807914.80 kB/sec
	Min throughput per process 			= 2092213.38 kB/sec 
	Max throughput per process 			= 3015406.50 kB/sec
	Avg throughput per process 			= 2771817.02 kB/sec
	Min xfer 					= 2910208.00 kB

	Children see throughput for 8 re-readers 	= 18929034.00 kB/sec
	Parent sees throughput for 8 re-readers 	= 18407705.34 kB/sec
	Min throughput per process 			= 2107364.75 kB/sec 
	Max throughput per process 			= 3094943.25 kB/sec
	Avg throughput per process 			= 2366129.25 kB/sec
	Min xfer 					= 2855924.00 kB

	Children see throughput for 8 random readers 	= 14670952.25 kB/sec
	Parent sees throughput for 8 random readers 	= 14456079.19 kB/sec
	Min throughput per process 			= 1822894.12 kB/sec 
	Max throughput per process 			= 1845722.00 kB/sec
	Avg throughput per process 			= 1833869.03 kB/sec
	Min xfer 					= 4142436.00 kB

	Children see throughput for 8 random writers 	=   70237.77 kB/sec
	Parent sees throughput for 8 random writers 	=   27706.80 kB/sec
	Min throughput per process 			=    8365.54 kB/sec 
	Max throughput per process 			=    8932.87 kB/sec
	Avg throughput per process 			=    8779.72 kB/sec
	Min xfer 					= 3928196.00 kB



iozone test complete.

결과 내용을 보면 SATA 에서는 NVME보다 Read 부분이 아주 크게 나와 있는데, 생성되는 파일이 아래의 사이즈로 생성이 되고, 실 메모리 보다 작다보니, memory cache 의 영향으로 read 부문이 아주 크게 나오는거 같네요. 결국 Write 부문 정도만 참고 하면 될거 같습니다. Random IO Write 부문의 성능 차이가 확연하게 보이네요.

[root@hci1 images]# ls -lahs /mnt
total 33G
   0 drwxr-xr-x   2 root root  196 Jan 11 22:51 .
   0 dr-xr-xr-x. 17 root root  262 Jan 11 19:21 ..
348K -rwxr-xr-x   1 root root 348K Jan 11 21:13 iozone
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.0
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.1
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.2
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.3
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.4
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.5
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.6
4.0G -rw-r-----   1 root root 4.0G Jan 11 23:00 iozone.DUMMY.7

답글 남기기

이메일은 공개되지 않습니다. 필수 입력창은 * 로 표시되어 있습니다