Saturday, March 19, 2011

Calculate chunk size for RAID device

Chunk size is a term often used in Linux Software RAID , In hardware RAID, different vendor has different definition e.g EMC call it element size.
Chunk size is the minimum amount of data written to each member before moving to to the next. So it is only significant in Round-Robin Raid types: Raid 0/RAID5/RAID 6 ..etc. The purpose of tuning  Chunk size is to evenly distribute request to each member in RAID.
Chunk Size=avgrq-sz/ number of data disks
avgrq-sz:
The average size (in 512 Byte sectors) of the requests  that  were issued to the device.
number of data disks: Data disk only, excluding parity disk in RAID5/6
#Get avgrq-sz for a device since host is up
$iostat -x /dev/sdc
Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sdc               1.96     1.68    1.33    0.54   274.67   273.13   293.56     0.02    8.61   0.90   0.17
#Size in kb
$ echo 293.56*512/1024|bc -l
146.78
#RAID 0 with 2 disks
Chunk Size(KB)=146.78/2=73.39
#Chunk size should be 2^n
Chunk Size (KB)=73.39=~64
#Create RAID 0 with chunk size=64 
 
$mdadm -C /dev/md0 –l 0 -n 2 –chunk-size 64 /dev/sda1 /dev/sdb1
#Create file system with optimal stride
#stride and chunk has the same meaning but different unit.
#Stride is on file system level, it is the number of blocks written to disk before moving to the next.
stride=chunk size / block size
#If I choose block size =4096, then stride is 64/4=16
$mkfs.ext3 -b 4096 -E stride=16 /dev/md0

Stride is irrelevant for hardware RAID, which is presented to host as single harddisk.

2 comments:

  1. Just wanted to drop a comment and say I am new to your blog and really like what I am reading.RAID is a technology that is used to increase the performance and/or reliability of data storage.
    data recovery nashville

    ReplyDelete
  2. An accurate and clear how-to.

    A few notes:
    1) If you have a 4K-sectore drive, I think you should multiply avgrq by 4096. Basically, you should find that out the sector size and then use that:
    cat /sys/block/sda/queue/physical_block_size

    2) If you're on a recent mkfs.ext3 or mkfs.ext4, there's also the question of stripe-width. I found this calculator useful: http://busybox.net/~aldot/mkfs_stride.html

    Regards

    ReplyDelete