Best Practice to move data from crashed unRAID to a new config

Hi there

As this is my first post after using unRAID for three years, kudos to the community for the massive amount of knowledge and work put into this forum and all the scripts, add-ons, plugins etc. While unRAID is good, it is the community which makes it a great solution.

My name is Ralf, and I`m an unRAID user.

I`m looking for your input on how to resolve a situation in a reasonable way. That´s what happened:

- unRAID pro box v5.0.5 containing parity, cache and 15 data disks, all SATA HDD ranging from 1 to 3 Tb.

- replaced parity with a 6 Tb drive, rebuild parity.

- moved 3 Tb (ex-)parity into another slot to replace an old data disk which was running to hot for my taste.

- while reconstructing the data onto the "new" disk, one of my additional SATA controllers became faulty

- as a consequence all 5 disks attached to it delivered faulty information, the reconstruction was not successful.

- After array stop and restart (during initial troubleshooting) a couple of data disks redballed.

- Shut down now!

Ok - deep breath.

1st question:

Can there be data loss on other drives than the one I started to reconstruct?

2nd question:

As I still have the old data drive intact and assuming #1 has to be accepted whatever the answer is, what would be the best way to bring the roughly 30 Tb of data into a new system?

I do see a couple of possibilities here.

Currently I`m waiting for new controllers to be delivered while the system is quite busy preclearing new disks, thus not only checking the new disks but the original board as well.

What is your opinion on how to proceed assuming everything checks out well?

sol.#1 - Attach disks the way they were before the crash (yes, I´ve backed up my flash regularly and took screenies of the assignments before and after any change) - hit initconfig - rebuild parity - off you go,

sol.#2 - Start a new unRAID, set-up shares and split level and everything else better than with the 1st one. Putting the original data disk one by into a linux box I have at my desk and moving the files over the network onto the new unRAID or

sol.#3 - As #2, but using the nice bays I have to swap drive by drive directly into the refurbished system, mounting them outside the array or

sol.#4...?

I tend to #3, albeit I haven´t figured out how to mount the ReiserFS disks outside the array. Basically, that should be like this (for I=1 to 15)

  - stopping the array, shutting down the box (I´m not risking hot-swapping right now)

  - physically attaching a data disk, firing up the system, not assigning this disk to the array

  - creating a mount point 

      $  mkdir/mnt/whatever

  - mounting the disk

      $ mount -r -t reiserfs /dev/sdx /mnt/whatever -o umask=111,dmask=000 -v

    replacing the x with the current assignment of said data disk

  - copying the data onto the new shares (cp oder rsync?)

  - unmounting the disk, stopping the array, shutting down the box (Next I)

Any input from your side on what to do best? I can read, type and click, among other things. You should not assume that the CPU coordinating this is up to the task. So any pointers on how to avoid more problems are greatly appreciated.

Thanks

R

Okay, from your description, it seems that NO disk has actually failed at this point.

i.e. you were reconstructing the data for a disk that was running hot -- but hadn't failed;  and the reconstruction failed, which means you had bad READS from the disks attached to the failing controller.

Assuming that's the case, then after you have a new controller I would just do this:

(a)  Do a New Config and assign the 6TB drive as parity, and ALL of your original data disks to the array.  Then let it do a parity sync;  and after that do a parity check to confirm all went well.

(b)  Now shut down; remove the disk you wanted to replace and put your old parity drive in its place; Start the array (it will note the missing disk);  Stop it and assign the old parity drive in its place; and then Start it and let it do the reconstruction.

©  After the reconstruction finishes, run a non-correcting parity check to confirm all went well.  (This is the only time I ever recommend non-correcting checks)

Done 

Your description is what I intended with solution #1. Should be the easy way out.

Thx you for looking over this. 

R

Hi there

 

As this is my first post after using unRAID for three years, kudos to the community for the massive amount of knowledge and work put into this forum and all the scripts, add-ons, plugins etc. While unRAID is good, it is the community which makes it a great solution.

My name is Ralf, and I`m an unRAID user.

Hi Ralf!  Wish your first post was under better circumstances, but you've come to the right place for help.

1st question:

Can there be data loss on other drives than the one I started to reconstruct?

No. When reconstructing a disk, you are building it from parity and your data disks.  Those disks are just in a read state during the reconstruct.

I want to reply to your other questions but I'm actually out at a dinner party right now and can't review all the information. Hopefully someone else can take this the rest of the way for you tonight or I will have to look at this later.

EDIT:  Damnit I need to read the whole thread before replying in the future. Thanks guys for helping him out.

Solution #1. Make sure that a parity check is performed both before and after and disk changes.

Solution #1. Make sure that a parity check is performed both before and after and disk changes.

As I noted earlier, that's definitely the way to go ... HOWEVER, you can't do a parity check, since you have 2 red-balled disks.    Just do the New Config, THEN do a parity check (as I outlined above).    Since you didn't have any failed disks when you started; and the other "failures" are due to a bad controller, it's a fairly safe bet your disks are all okay.  In any event, you don't really have any other choice.    You may want to compare your data against your backups when you get everything reconstructed (if you have backups).

Guys - thank you for input. Highly appreciated.

Solution #1. Make sure that a parity check is performed both before and after and disk changes.

 

As I noted earlier, that's definitely the way to go ... HOWEVER, you can't do a parity check, since you have 2 red-balled disks.    Just do the New Config, THEN do a parity check (as I outlined above).   

Yep, I`ll do this once the new controllers have proven reliable running preclears and copy tests.

You may want to compare your data against your backups when you get everything reconstructed (if you have backups).

Fortunately, I have offsite-backup for essential data (~10 Tb). The rest would be just an inconvenience to get again, but not a problem. That is again a reminder not to confuse protection (against hardware failures) with backups.