My filesyserver at home is a opensolaris box using raidz under zfs and right now it is having some problems. Recently I had to replace a drive that did not fail but it was making some clicking noises. After repplacing it the first resilver left me with an error about a snapshot that i didn't need being damaged. okay i can live with that, so i just need to delete the snapshot and bring the pools redundancy back up. well, it reslivered everything, but it didn't seem like the pool was running completely. i cleared the error and it started to resilver again. same thing happened again but now i am getting a warning for something else <0x5c>:<0x64d62> and now the resilver is going past 100% and copying more then the amount of data i have. some posts online suggest disabling snapshots because they are keeping it from completing. didnt work so i reboot the computer and when it comes back online it starts resilvering automatically. again it goes to 100% and keeps working but this time it takes two files with it, the one before and <0x2b3>:<0x64d62>. looking to another post i see to upgrade opensolaris and try again, i do that, but now i am starting to get worried about playing fast and loose with my data. i still hvent fixed the problem but i want to have some assurance i am not going to make it worse.
now i want to buy 4x more 1.5TB drives and copy everything over to some external drives, destroy the existing zpool, make a new zpool with all the 1.5TB drives i have, and then copy the stuff back from the external drives.
Unfortunally i think this is going to cost around 600$, 400$ for the 4 drives and the last 200$ for a cheap 4 port sata controller that works with solaris and 3x harddrive heatpipes that fit in the 5 1/4 slots. not very cheap, but i already bought all the hardware for the fileserver and this would give me a huge capacity increase when i needed it anyway.
now to just find a place to drop 2.2TBs of data off while i work on my computer, that shouldn't be too hard but it seems like it will take a bunch of time if i have to do it over ethernet or usb.
i did two things at once so i don't know which helped, but i deleted all my snapsnots on that pool and restarted after updating opensolaris. Now i am once again enjoying the security raidz offers while at the same thing realizing the benefit of raidz2.