using an old celeron 466MHz machine with 2 new Maxtor 6Y080L0 disks in raid1 software array. DMA and 32bit-access is enabled (hdparm) for both disks.
# uptime
17:43:19 up 2:09, 3 users, load average: 1.29, 1.06, 1.06
# mdadm -D /dev/md0
/dev/md0:
Version : 00.90.01
Creation Time : Mon Jul 24 12:35:51 2006
Raid Level : raid1
Array Size : 72685952 (69.32 GiB 74.43 GB)
Device Size : 72685952 (69.32 GiB 74.43 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Tue Jul 25 15:34:25 2006
State : dirty, recovering
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Rebuild Status : 43% complete
Number Major Minor RaidDevice State
0 3 6 0 active sync /dev/hda6
1 22 6 1 active sync /dev/hdc6
UUID : 5937930e:dd70ac10:9bfff6fe:1421d511
Events : 0.5046
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 hdc6[1] hda6[0]
72685952 blocks [2/2] [UU]
[========>............] resync = 44.4% (32314240/72685952) finish=128.7min speed=5223K/sec
as you can see for 2 hours only 40% is done. I wander is it any faster to rebuild 63Gb array on faster processor?
add/edit 23/01/2007 :
looking at md.c from 2.4.34 kernel sources:
/*
* Current RAID-1,4,5 parallel reconstruction 'guaranteed speed limit'
* is 100 KB/sec, so the extra system load does not show up that much.
* Increase it if you want to have more _guaranteed_ speed. Note that
* the RAID driver will use the maximum available bandwith if the IO
* subsystem is idle. There is also an 'absolute maximum' reconstruction
* speed limit - in case reconstruction slows down your system despite
* idle IO detection.
*
* you can change it via /proc/sys/dev/raid/speed_limit_min and _max.
*/
So, increasing this 2 variables can dramatically reduce rebuild time, for example, rebuilding 1Tb raid5 with 3 sata (each 500gb) disks with default settings 100/10000 gives me aprox.rebuild end in 800 minutes. Increasing speed_limit_max to 50000 cut rebuild time to 300 minutes.
# mdadm -D /dev/md5
/dev/md5:
Version : 00.90.00
Creation Time : Fri Jan 19 15:06:16 2007
Raid Level : raid5
Array Size : 976767872 (931.52 GiB 1000.21 GB)
Device Size : 488383936 (465.76 GiB 500.11 GB)
Raid Devices : 3
Total Devices : 4
Preferred Minor : 5
Persistence : Superblock is persistent
Update Time : Tue Jan 23 11:40:34 2007
State : active, degraded, recovering
Active Devices : 2
Working Devices : 2
Failed Devices : 2
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 64K
Rebuild Status : 32% complete
UUID : d43db220:e08760aa:fc62ddd8:61d2e6ab
Events : 0.12
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 1 1 active sync /dev/sda1
2 0 0 2 removed
3 8 33 3 spare /dev/sdc1
# cat /proc/mdstat
md5 : active raid5 sdb1[0] sdc1[3] sda1[1]
976767872 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
[======>..............] recovery = 30.2% (147512916/488383936) finish=162.8min speed=34882K/secHope this helps.
PS: thanks to linux kernel developers and Neil Brown for the excellent mdadm tool.
Linux software raid rebuild
My PIII 600 with two 20 gig drives UDMA 33 are rebuilding at about 7,000K/sec which estimates about a 45minute raid 1 build. Yours is only going at 5,223K/sec, which doesn't seem all that great. If you can get your DMA speed up to 66, 100, or 133 you'll obviously have a much quicker disk IO speed. If your onboard IDE controler doesn't support higher speeds, but your drives do you could always through in a PCI controler card like those offered by Promise, with added support through your kernel. The drives will be relabeled to /dev/sdX though.
Since software raid does use a lot of IO and cpu while rebuilding, yes a faster processor would help. While I'm rebuilding an array my load is at ~1.10
re: rebuild time
and yet another example:
/dev/md2 is 2Gb raid1(mirror) over 2 partitions (sda6,sdb6), which is 2 sata disks in sun fire x2100 server (Dual Core AMD Opteron Processor 175).
Aug 30 18:28:09 doltan kernel: md: syncing RAID array md2
Aug 30 18:28:09 doltan kernel: md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc.
Aug 30 18:28:09 doltan kernel: md: using maximum available idle IO bandwith (but not more than 200000 KB/sec) for reconstruction.
Aug 30 18:28:09 doltan kernel: md: using 128k window, over a total of 1959808 blocks.
Aug 30 18:28:46 doltan kernel: md: md2: sync done.
as you can see 2Gb raid sync was done in 37 seconds, really nice! :-)
PS: i'm not used nvidia chipset bios raid functionality on this server MB, since i learn about dmraid utility only today.
hdparm
If you have this problem, turning on DMA for your disks will increase the speed greatly.
test speed:
hdparm -tT /dev/hda
enable dma for all array members:
hdparm -d1 /dev/hda
hdparm -d1 /dev/hdb
hdparm -d1 /dev/hdc
hdparm -d1 /dev/hdd
test speed:
hdparm -tT /dev/hda
see how fast it is?
hdparm
thanks for the comment, i have noted in the first post about this hdparm setting.
i'm always trying to enable DMA transfer mode for the hard disks. But only recent sata disks visible as a scsi devices to kernel (/dev/sd?) and hdparm gives error trying to operate on it.
Talking about hdparm, it's also usefull to enable 32 bit transfers with hdparm -c 1 /dev/hda.
Check your redhatish (fedora) /etc/sysconfig/harddisks file.