Background clearing of new drives?

So, went to stick in a new drive, and for some bizarre reason UnRAID now wants to clear the drive.  Yeah, so 10 minutes later it's at 1%.  Which means my server is going to be basically offline for the guts of 12 hours.  Err...no.

 

I also don't want to reset the array to add the drive.

 

So, what crap is this?  Why isn't the clearing done in the background?  Who thought it would be a good idea to force people to clear drives and have their server basically offline for hours upon hours at a time?

 

Looks like I'm back to the good old preclear.sh script...

Correct me if I'm wrong (which does happen), but doesn't this plugin allow this function?

 

http://lime-technology.com/forum/index.php?topic=39985.0

This is a request for official limetech support, not community support.

Since you've already added the drive and clearing has started, you're really out of luck unless you want to abort the process ... and I'm not sure what state that would leave the system in (it may require a new parity sync).

 

This is exactly why Joe L wrote the pre-clear script many years ago -- to allow drives to be "pre-cleared" BEFORE you add them to the array ... and it was done in coordination with Tom, so that UnRAID recognizes the pre-clear "signature" and will skip the clearing process for those drives.

 

There is, as smdion noted above, a pre-clear Plug-In that invokes the pre-clear script via the GUI, but this is still just using Joe's pre-clear script BEFORE the disk is added to the array.

 

What you did was add a disk that hadn't been cleared, so UnRAID has to clear it -- and as you've noted the array isn't available during this time.

 

If you want to abort and do the clearing off-line, there are a couple of options ...

 

(1)  Do a New Config and assign all of your current drives plus the parity drive, checking the "Parity is already valid" box.    This will cause a parity check to start when you Start the array ... but you can abort this if you're CERTAIN it's really valid.

 

(2)  Do a New Config and assign all of your current drives AND the new drive -- but don't check the "Parity is already valid" box => this will result in a new Parity Sync and will use the new drive without clearing it (no need, since parity will reflect its actual state).  You'll still need to format the new drive, but that's very quick.

 

Or you can simply wait for the clear to finish -- which, at this stage, is probably the best choice.

 

As for doing the clearing in the background => probably a good idea ... and likely would have been implemented at some point in the last few years, except Joe L's script has been available, and in addition to simply clearing the drives it does a fairly thorough test to confirm a new drive doesn't have infant mortality issues.  It's become so widely used that the demand for modifying the "native" way of doing it has been pretty low.    ... and the Pre-Clear Plug-In that makes it easy to do this through the GUI makes this an even lower priority.

 

In 6.2, when you add a new drive to an existing array and start the array, a clear will happen automatically but the array will start and be usable while its running.  When the clear completes, the device(s) will be automatically attached / mounted to the system for you.  This isn't the same as preclear because its happening as you add the devices.  To preclear, you can still use the script and gfjardims plugin if necessary.

In 6.2, when you add a new drive to an existing array and start the array, a clear will happen automatically but the array will start and be usable while its running.  When the clear completes, the device(s) will be automatically attached / mounted to the system for you.  This isn't the same as preclear because its happening as you add the devices.  To preclear, you can still use the script and gfjardims plugin if necessary.

 

Will this have an option to disable? What if I want to add a disk that was previously part of the array and has data on it?

 

In 6.2, when you add a new drive to an existing array and start the array, a clear will happen automatically but the array will start and be usable while its running.  When the clear completes, the device(s) will be automatically attached / mounted to the system for you.  This isn't the same as preclear because its happening as you add the devices.  To preclear, you can still use the script and gfjardims plugin if necessary.

 

Will this have an option to disable? What if I want to add a disk that was previously part of the array and has data on it?

Good question. I'll double check the behavior for that but I think we covered that scenario already.

... Will this have an option to disable? What if I want to add a disk that was previously part of the array and has data on it?

 

Assuming you mean you want to keep the data, the only way to do this is via a New Config ... which will leave the data intact and do a new parity sync.

 

... Will this have an option to disable? What if I want to add a disk that was previously part of the array and has data on it?

 

Assuming you mean you want to keep the data, the only way to do this is via a New Config ... which will leave the data intact and do a new parity sync.

So both parity disks have to be rebuilt? That doesn't sound right. I figured this new dual parity stuff would eliminate many of the currently accepted risks.

... Will this have an option to disable? What if I want to add a disk that was previously part of the array and has data on it?

 

Assuming you mean you want to keep the data, the only way to do this is via a New Config ... which will leave the data intact and do a new parity sync.

So both parity disks have to be rebuilt? That doesn't sound right. I figured this new dual parity stuff would eliminate many of the currently accepted risks.

Adding a new data disk to an existing array will require it to be cleared (thus losing the data) regardless of whether you have single or dual parity.  in this case parity is unaffected because the disk has been cleared so that it automatically matches existing parity so parity remains unchanged.

 

The New Config approach to adding a disk and keeping its data is also not affected by whether you have single or dual parity - in both cases parity will need rebuilding from the data disks.

The New Config approach to adding a disk and keeping its data is also not affected by whether you have single or dual parity - in both cases parity will need rebuilding from the data disks.

Since one drive can be removed and a second drive still reconstructed, could you not add one drive with arbitrary data to a dual parity protected array and still maintain the ability to rebuild a single drive? Since I haven't slogged through the math on dual parity, I'm just guessing.

The New Config approach to adding a disk and keeping its data is also not affected by whether you have single or dual parity - in both cases parity will need rebuilding from the data disks.

Since one drive can be removed and a second drive still reconstructed, could you not add one drive with arbitrary data to a dual parity protected array and still maintain the ability to rebuild a single drive? Since I haven't slogged through the math on dual parity, I'm just guessing.

I very much doubt it as adding a drive that already has data on it without clearing it first would invalidate both parity drives.

The New Config approach to adding a disk and keeping its data is also not affected by whether you have single or dual parity - in both cases parity will need rebuilding from the data disks.

Since one drive can be removed and a second drive still reconstructed, could you not add one drive with arbitrary data to a dual parity protected array and still maintain the ability to rebuild a single drive? Since I haven't slogged through the math on dual parity, I'm just guessing.

 

No, you can't add a disk that contains data without recomputing the parity information ... regardless of the level of fault tolerance you have (single parity, dual parity, triple parity, etc.).

 

Note that the change JonP referenced in the forthcoming v6.2 is a VERY nice change => the system will clear a drive before it's actually added to the array, thus eliminating the "array not available" time while clearing.    This is a very welcome change.