Benchmarking disk IO accurately by avoiding file cache

Linux tends to allocate memory to file cache as much as possible. It is good to boost file system performance in general, but it is not desired when measuring pure disk IO performance. The general practice for popular benchmarking tools e.g. Iozone/Bonnie is to create test file twice the size the memory. It can’t eliminate file cache effect completely and is not convenient if the server has huge memory.
The trick is to use ramfs to “pin” data into file cache to consume almost all free memory, so there will be no additional free memory for file cache during benchmarking. I don’t use tmpfs because it use virtual memory( physical memory + swap).
####Current free memory

[root@centos-5.2]$ free -m
             total       used       free     shared    buffers     cached
Mem:           249         30        218          0          2          4
-/+ buffers/cache:         23        226
Swap:          509          4        505

####mount as ramfs

$mkdir –p /mnt/ram
$mount -t ramfs ramfs /mnt/ram

####use –a option to show all mount.

$ df –ah | grep ramfs
ramfs                    0     0     0   -  /mnt/ram

####Create dummy file to consume almost all free memory, I will leave 20M free mem to run benchmarking program

$ dd if=/dev/zero of=/mnt/ram/ramfs.file bs=1024k count=195
195+0 records in
195+0 records out
204472320 bytes (204 MB) copied, 0.834913 seconds, 245 MB/s

####The195M data is pinned in memory as shown in cached column

$ free -m
             total       used       free     shared    buffers     cached
Mem:           249        226         23          0          2        200
-/+ buffers/cache:         23        226
Swap:          509          4        505