12TB drives are coming!

http://www.hgst.com/products/hard-drives/ultrastar-he12

HGST, Helium, PMR, 8 platters. Start saving your $$$ 

In another year or two we'll have 20-24TB drives, bigger than entire servers.

These huge drives make parity solutions like Unraid very attractive. A RAID rebuild on these would cause a heart attack

It already takes about 30hrs to do a parity check on my server with dual 8TB Seagate archive drives, granted it might be a lot faster were I not using archive drives. I can't imagine how long a 12TB check would take?

It already takes about 30hrs to do a parity check on my server with dual 8TB Seagate archive drives, granted it might be a lot faster were I not using archive drives. I can't imagine how long a 12TB check would take?

Pretty much this!

4 TB parity check is running nicely over night and the server is ready for duty next day.

30hrs will reduce performance pretty much which is in general not acceptable!

So unRAID needs a feature to pause parity checks.

Even better and a solution I would prefer to have this days already is

to perform parity checks during idle times.

Instead of spinning down the drives run the parity check if scheduled.

Doesn't matter if it takes days to complete imo. All that matters is that each bit has been checked

once in a while.

It already takes about 30hrs to do a parity check on my server with dual 8TB Seagate archive drives, granted it might be a lot faster were I not using archive drives. I can't imagine how long a 12TB check would take?

That's a bit on the slow side... mine takes 19hrs; granted all my other drives as 4TB Reds though...

It already takes about 30hrs to do a parity check on my server with dual 8TB Seagate archive drives, granted it might be a lot faster were I not using archive drives. I can't imagine how long a 12TB check would take?

That's a bit on the slow side... mine takes 19hrs; granted all my other drives as 4TB Reds though...

Your parity check is faster because your drive are smaller (4TB compared to 8TB) - your parity check speed is potentially slower than his if you extrapulate the drive size.

It already takes about 30hrs to do a parity check on my server with dual 8TB Seagate archive drives, granted it might be a lot faster were I not using archive drives. I can't imagine how long a 12TB check would take?

That's a bit on the slow side... mine takes 19hrs; granted all my other drives as 4TB Reds though...

 

Your parity check is faster because your drive are smaller (4TB compared to 8TB) - your parity check speed is potentially slower than his if you extrapulate the drive size.

He meant he has 8TB and 4TB disks, parity check speed is influenced by a lot of factors, including controllers used, how many different sizes of disks are used, disk platter size, etc.

In the past I had a server with 8TB disks only and parity check took ~15Hours, now it has 8TB and 3TB disks, it takes ~18 and half hours.

 

4 TB parity check is running nicely over night and the server is ready for duty next day.

30hrs will reduce performance pretty much which is in general not acceptable!

So unRAID needs a feature to pause parity checks.

 

Even better and a solution I would prefer to have this days already is

to perform parity checks during idle times.

Instead of spinning down the drives run the parity check if scheduled.

Doesn't matter if it takes days to complete imo. All that matters is that each bit has been checked

once in a while.

Has this been put in the correct forum already?

Storing the point reached in last parity check, and continuing based on time schedule or usage pattern. 1 hour a day should usually cover the 30 hour prediction.

Has this been put in the correct forum already?

 

Storing the point reached in last parity check, and continuing based on time schedule or usage pattern. 1 hour a day should usually cover the 30 hour prediction.

Can't tell - not by me.

I remember discussions about pausing/throttling the parity check.

Can't remember if "parity checks during idle times" has been proposed yet?

Has this been put in the correct forum already?

 

Storing the point reached in last parity check, and continuing based on time schedule or usage pattern. 1 hour a day should usually cover the 30 hour prediction.

 

Can't tell - not by me.

I remember discussions about pausing/throttling the parity check.

 

Can't remember if "parity checks during idle times" has been proposed yet?

I believe in the last remarks about that from Tom were about throttling it:

http://lime-technology.com/forum/index.php?topic=53120.msg510930#msg510930

I believe in the last remarks about that from Tom were about throttling it:

 

http://lime-technology.com/forum/index.php?topic=53120.msg510930#msg510930

That thread is about providing pre and post parity check scripts. Which seem to be aimed at process control for things outside parity check.

Saving a checkpoint and restarting from the last checkpoint, seems completely different.

I believe in the last remarks about that from Tom were about throttling it:

 

http://lime-technology.com/forum/index.php?topic=53120.msg510930#msg510930

 

That thread is about providing pre and post parity check scripts. Which seem to be aimed at process control for things outside parity check.

 

Saving a checkpoint and restarting from the last checkpoint, seems completely different.

True, but it's where Tom last spoke about some sort of parity check control by using throttling, feel free to create a request for checkpoints if you'd prefer that.

It already takes about 30hrs to do a parity check on my server with dual 8TB Seagate archive drives, granted it might be a lot faster were I not using archive drives. I can't imagine how long a 12TB check would take?

That's a bit on the slow side... mine takes 19hrs; granted all my other drives as 4TB Reds though...

 

Your parity check is faster because your drive are smaller (4TB compared to 8TB) - your parity check speed is potentially slower than his if you extrapulate the drive size.

Actually having 4TB drives makes the parity checks SLOWER than they would be if all of the drives were the same.  Having mixed drive sizes results in the check slowing down as each different size drive is processing the much-slower inner cylinders.  If the system was all 8TB drives, the checks would be appreciably faster than a mixed 8TB/4TB system.

... granted it might be a lot faster were I not using archive drives.

No, that's not the reason.  Archive drives are just as fast as standard PMR drives for reads.  Your parity checks are significantly slowed down because of the mix of sizes you have -- 3TB, 4TB, 5TB, etc.    The parity check slows down appreciably as the 3TB drives process the inner cylinders;  then will repeat that process for the 4TB drives; then the 5TB drives; etc.    What would really speed up your parity checks is if all of your drives were 8TB 

If you want to see the impact, watch your parity check next time -- pay attention to the speed as it nears 3TB (say at the 2.8TB point);  then watch what happens as it crosses 3TB (it will speed up a good bit);  then repeat that as it nears 4TB; and again at 5TB.  Once it passes 5TB it should speed up a LOT, as the only drives remaining are your 8TB units.  Then it'll slow down again as those move to the inner cylinders.

I just want HGST 4TB drives for under $100

... granted it might be a lot faster were I not using archive drives.

 

No, that's not the reason.  Archive drives are just as fast as standard PMR drives for reads.  Your parity checks are significantly slowed down because of the mix of sizes you have -- 3TB, 4TB, 5TB, etc.    The parity check slows down appreciably as the 3TB drives process the inner cylinders;  then will repeat that process for the 4TB drives; then the 5TB drives; etc.    What would really speed up your parity checks is if all of your drives were 8TB 

 

If you want to see the impact, watch your parity check next time -- pay attention to the speed as it nears 3TB (say at the 2.8TB point);  then watch what happens as it crosses 3TB (it will speed up a good bit);  then repeat that as it nears 4TB; and again at 5TB.  Once it passes 5TB it should speed up a LOT, as the only drives remaining are your 8TB units.  Then it'll slow down again as those move to the inner cylinders.

So true... I had a mix of 8TB, 4TB and 3TB and that took like 22+hrs to run. and yes, the speed starts dropping as you reach the end of one of the drive sizes (@3TB mark, then @4TB mark) getting rid of all the 3TB disks made everything way faster (My 3TB disks were first gen WD reds with sub 1TB platters, boy were they relatively slow.

A very comfortable way to check the "brakes" during parity check is jbartlett's

drive performance testing aka diskspeed script.

The slowest drive in the array will dictate the check speed.