madshi

how to repair array with a "too large" new disk?

23 posts in this topic Last Reply

Recommended Posts

Posted (edited)

Current situation:

- Parity 2: 4TB

- Disk 1: not installed (4TB disk died)

- Disk 2: 4TB

- Disk 3: 4TB

- Disk 4: 1TB

 

Bought 2 replacement drives:

- 10TB

- 10TB

 

Wanted end result:

- Parity: 10TB

- Disk 1: 4TB

- Disk 2: 4TB

- Disk 3: 4TB

- Disk 4: 10TB

 

Questions:

1) I once had 2 parity disks, but "Parity 1" died and I didn't replace it, so the remaining parity disk is named "Parity 2". But it shouldn't matter that it's not "Parity 1", right? Feels weird to have only "Parity 2" but not "Parity 1". But it's only a naming issue, no need to worry, right?

 

2) What is the best way (order of steps) to achieve my "Wanted end result"?

 

Thanks for your help!  😀

 

(P.S: Both new 10TB drives are almost done with their 3 cycle Preclear.)

Edited by madshi

Share this post


Link to post

Is there data on disk1 (I.e. is it being emulated by the other disks plus parity).    The answer will determine ways forward.   If it does contain data then you will want to use the ‘parity swap’ procedure that is used when you want to replace parity with a larger drive and use the old parity drive to replace the failed drive.

Share this post


Link to post

Yes, there is data on disk1 (emulated by the other disks plus parity).

 

So I would do a "parity swap" as the first step, which would replace the current 4TB parity disk with a new 10TB parity disk, correct? And such a parity swap is possible without data loss while the array has an "emulated" disk in it? I don't suppose there's a way to "repair" the array first? I would feel safer if putting the array back into a fully protected state would be my first step, if there's any way to do that?

 

Thank you!

Share this post


Link to post

Not sure if this is "best way", but instinctively I would suggest adding one of the 10TB drives as Parity1 and let it build and check parity afterwards, then remove Parity2 and use it as the replacement for Disk1 and do a parity check, then replace Disk4 with the other 10TB and do a parity check afterwards.

 

There are several gotcha's along the way with this that may require new configs or other gyrations, @johnnie.black would probably know off the top of his head whether such manipulations are necessary.

 

A parity swap procedure will work, but won't change your parity drive type, it will remain type2 instead of rebuilding as type1.

 

This whole procedure ASSumes the rest of your drives are in perfect health, which isn't a guarantee, especially since you just had a drive failure. I suggest posting the diagnostics zip file here to help analyze the level of risk you are currently facing.

 

Do you have current backups of ALL data that you value? If not, I suggest re-purposing one of your new 10TB drives as a backup drive FIRST, before mucking about with the array drives. 

 

Share this post


Link to post
1 minute ago, madshi said:

I would feel safer if putting the array back into a fully protected state would be my first step, if there's any way to do that?

Yes, replace the failed disk with a drive the same size as your current parity drive. You could do that by resizing one of the 10TB drives to 4TB temporarily using a HPA. However, you would be once again putting the array at the same amount of risk when you go through the procedure to reclaim the other 6TB.

 

Backup, backup, backup.

Share this post


Link to post
4 minutes ago, jonathanm said:

There are several gotcha's along the way with this that may require new configs or other gyrations, @johnnie.black would probably know off the top of his head whether such manipulations are necessary.

New config wouldn't be needed, only to unassign parity2 and start the array without it before using to replace disk1.

 

 

Share this post


Link to post

Good thinking on the backup. Unfortunately, I "lied" about the number of drives to keep my first post short and simple. It's actually 8 disks + parity, and most of the drives are almost full. Which means I don't have enough space (by a mile) on the 10TB drive to backup all data, unfortunately.

 

I guess I could backup the emulated "Disk 1" on one of the 10TB drives, but doing that would require a full read of all drives, anyway, so if that succeeds, then an arrray rebuild should succeed, as well, so trying to back up "Disk 1" on a 10TB drive wouldn't actually help, I think?

 

FWIW, unRaid insisted on doing a parity check with the simulated drive (after a power out), and it "succeeded". Not sure what it was good for (probably nothing), but at least reading all drives didn't make any further drives fail so far.

 

I suppose I can't simply tell unRAID to replace the broken 4TB disk with a 10TB disk, treating the 10TB as a 4TB disk, can I? Doing that sounds like your HPA suggestion, but I'm not sure how I would do that HPA stuff.

 

Thanks again for your help, guys!

Share this post


Link to post

Either do a parity swap or what jonathanm suggested, IMO both are good options.

Share this post


Link to post
14 minutes ago, madshi said:

I suppose I can't simply tell unRAID to replace the broken 4TB disk with a 10TB disk, treating the 10TB as a 4TB disk, can I? Doing that sounds like your HPA suggestion, but I'm not sure how I would do that HPA stuff.

HPA is exactly how you would do that, but honestly, you aren't going to save any reads doing that unless you are willing to permanently leave it that way. Rebuilding it from the 4TB to 10TB after you get parity straightened out puts you in the same situation you are in right now, except you would be trusting your new 10TB parity drive to do the rebuild instead of the current 4TB, which is still going to be in the array anyway.

 

I think my initial gut reaction of how to proceed is probably the most straight forward way to accomplish your end goal.

Share this post


Link to post

After thinking this through, I like the idea of adding a 2nd Parity disk, because doing that would directly restore array protection right away. So I would do:

 

1) Add a 10TB disk as "Parity 1".

2) Replace the dead 4TB disk with the other 10TB disk.

3) Remove the 4TB "Parity 2" disk.

4) Replace the 1TB disk with the 4TB disk (which was "Parity 2").

 

Correct? Two more questions, if you don't mind:

 

A) Just to be safe, can you guys confirm that adding a 2nd Parity disk will work as expected, and fully restore array protection, even though "Disk 1" is still missing/dead/emulated?

B) Is there a simple way to remove "Parity 2" in step 3) above, without losing array protection?

 

Thanks!!  😍

Share this post


Link to post
1 minute ago, johnnie.black said:

Either do a parity swap or what jonathanm suggested, IMO both are good options.

 

32 minutes ago, jonathanm said:

A parity swap procedure will work, but won't change your parity drive type, it will remain type2 instead of rebuilding as type1.

 

In one sense type 1 and type 2 parity are equivalent, they both allow a single drive failure each. However, parity 2 is rigid as to drive slot assignments, parity 1 is slot agnostic for data drive.

 

Parity swap is a single operation, my suggestion requires extra time at risk with 2 rebuild operations to accomplish the same task, but changes to type 1 parity.

 

Quickest way forward is stick with type 2 parity and do the swap.

 

Only you can make the call of how long to be at risk. Safest would probably be to do the swap, then add the 2nd 10TB as type 1 parity. Downside is no additional free space right now, upside is extra protection from drive failure as you start swapping out your small drives for 10TB units.

Share this post


Link to post
1 minute ago, madshi said:

2) Replace the dead 4TB disk with the other 10TB disk.

 

You can't do that while the 4TB is assigned as parity 2. No data disks larger than parity. Period.

Share this post


Link to post
Posted (edited)

Haha, you're right, of course, I missed that.  <sigh>

 

Hmmmm... I'm trying to come up with a solution that gets me protection back in the first step, and if possible I'd not lose protection again. How about this?

 

1) Add a 10TB disk as "Parity 1".

2) Do a "Parity 2" swap 4TB -> 10TB. -> So I have 2 10TB Parity disks now!

3) Replace the missing "Disk 1" with the old 4TB Parity disk.

4) Remove the 10TB "Parity 2" disk.

5) Replace the 1TB disk with the 10TB disk.

 

Would that work? Can I remove the "Parity 2" disk in step 4) without losing protection?

 

Thanks!

Edited by madshi

Share this post


Link to post
5 minutes ago, madshi said:

2) Do a "Parity 2" swap 4TB -> 10TB. -> So I have 2 10TB Parity disks now!

3) Replace the missing "Disk 1" with the old 4TB Parity disk.

This is a single step

 

5 minutes ago, madshi said:

Would that work? Can I remove the "Parity 2" disk in step 4) without losing protection?

Yes.

Share this post


Link to post
2 minutes ago, johnnie.black said:
8 minutes ago, madshi said:

Would that work? Can I remove the "Parity 2" disk in step 4) without losing protection?

Yes.

But obviously you won't be protected during the rebuild if another disk fails.

Share this post


Link to post
1 minute ago, johnnie.black said:

But obviously you won't be protected during the rebuild if another disk fails.

 

Are you saying I'll lose data if there's a disk failure while processing the single/combined step 2) + 3)?

Share this post


Link to post
Just now, madshi said:

 

Are you saying I'll lose data if there's a disk failure while processing the single/combined step 2) + 3)?

No, on step five, since you'll be down to one parity.

Share this post


Link to post

Ok, I understand. But can't I protect against that by saving the unRAID config (flash drive contents) before doing step 5? If then another disk fails during the rebuild, I can restore the saved config and unRAID doesn't have to know that I tried to do a rebuild.

Share this post


Link to post

I won't get involved in the main discussion since those other guys have it well in hand. I did want to comment on this bit though:

 

1 hour ago, jonathanm said:

Do you have current backups of ALL data that you value?

1 hour ago, madshi said:

I don't have enough space (by a mile) on the 10TB drive to backup all data

 

Even dual parity is no substitute for backups. There are plenty of ways to lose data that don't involve disk failure.

 

You don't have to backup everything, but you absolutely must have a good backup plan for any data that is important and irreplaceable. You get to decide what that means.

  • Upvote 1

Share this post


Link to post
10 minutes ago, johnnie.black said:

Yes as long as data in unchanged.

 

I'll probably mount the array in maintenance mode to make sure no data is changed.

 

Thanks again for all your help, everyone!

 

FYI, I've created a new feature request here to make it possible to replace (healthy) harddisks without any risk:

 

 

11 minutes ago, trurl said:

Even dual parity is no substitute for backups. There are plenty of ways to lose data that don't involve disk failure.

 

You don't have to backup everything, but you absolutely must have a good backup plan for any data that is important and irreplaceable. You get to decide what that means.

 

Yes. The most important data is stored on another PC in addition to the unRAID array.

Share this post


Link to post

See my comments on your feature request.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Copyright © 2005-2018 Lime Technology, Inc.
unRAID® is a registered trademark of Lime Technology, Inc.