Possible data loss

I am using V 5.05 and this is what I have now. It was working good for around three months.

 

Parity  WDC_WD40EFRX-68WT0N0_WD-WCC4E0553678

Disk 1  ST3000DM001-9YN166_S1F0PQ9S

Disk 2  ST3000DM001-1CH166_Z1F1YREJ

Disk 3  WDC_WD20EARX-00PASB0_WD-WCAZAC998124

And a cache drive.

 

I rebooted unraid and had a big chaos…..IF you read this post you will see.

[ftp=ftp://lime-technology.com/forum/index.php?topic=32808.0]http://lime-technology.com/forum/index.php?topic=32808.0[/ftp]

 

The problem is that unraid thought that Disk 1 (ST3000DM001-9YN166_S1F0PQ9S) was my parity drive and started to do a parity…..apparently writing on disk one. I stopped the parity right away….one minute or so into it.

 

As of now Disk 1 appears as unformatted and I am missing a lot of files. I would say that all what was on disk 1 is lost.

 

I ran reiserfsck --check /dev/md1 and got:

Failed to open the filesystem

If the partition table has not been changed, and the partition is

valid  and  it really  contains  a reiserfs  partition,  then the

superblock  is corrupted and you need to run this utility with

--rebuild-sb.

 

IF I run a parity check without disk1…would I get the files back?

Shall I format disk1, add it to the array and do parity?

Shall I do -- rebuild-sb?

I need advice because I wouldn’t like to lose all the data.

See pics and syslog from this morning attached

Thank you

syslog201404110938.txt

One more pic. another minute of parity with the unformatted disk

You do have a mess on your hand. Looks like you have done something to make parity valid after screwing up your disk 1. Hopefully you did not you run a parity check (=parity-rebuild)?

 

If you did, that rebuild your parity to reflect the corrupted state of disk1 and effectively destroyed your good parity that you would have been able to use to rebuild disk 1. You will have to look at recovering files from disk1. Sorry, no clue how to do that. I bet you wish you had stopped and asked for help as soon as you realized you had corrupted disk1.

 

Hopefully, parity is only showing as valid because you forced it with console commands. Sorry, I'm not going to read your 3-page thread. If that is the case, you should be able to simply stop the array, unassign disk1, start the array (which will then invalidate the data on the physical disk1, since you may be making changes to the virtual disk1 that is now available via parity), then stop the array, the reassign the now invalid disk1, and start array. This should start rebuilding your disk1 from good data and you will rejoice with much happiness.

 

DO NOTHING - I AM RESPONDING AND MAY HAVE GOOD NEWS.

You do have a mess on your hand. Looks like you have done something to make parity valid after screwing up your disk 1. Hopefully you did not you run a parity check (=parity-rebuild)?

 

If you did, that rebuild your parity to reflect the corrupted state of disk1 and effectively destroyed your good parity that you would have been able to use to rebuild disk 1. You will have to look at recovering files from disk1. Sorry, no clue how to do that. I bet you wish you had stopped and asked for help as soon as you realized you had corrupted disk1.

 

Hopefully, parity is only showing as valid because you forced it with console commands. Sorry, I'm not going to read your 3-page thread. If that is the case, you should be able to simply stop the array, unassign disk1, start the array (which will then invalidate the data on the physical disk1, since you may be making changes to the virtual disk1 that is now available via parity), then stop the array, the reassign the now invalid disk1, and start array. This should start rebuilding your disk1 from good data and you will rejoice with much happiness.

 

I did not run parity. I started the array, saw that Disk1 was Unformatted and when I went to the Array operation tab, realized that parity had started by itself. I stopped it right there because I assumed that it was not properly to do it. I haven't force parity with commands.

I think you have left out a step in your description. Normally, parity would only be valid if you rebuild your parity, or forced it. Maybe you switched config files around and that is why it thinks parity is valid.

 

One step you can take without risk is to run a NO-CORRECTING parity check. If you get lots of errors in the first minute, then you probably have good parity and can rebuild disk1 as I described. But hold off on that step till you hear from bjp.

I think you have left out a step in your description. Normally, parity would only be valid if you rebuild your parity, or forced it. Maybe you switched config files around and that is why it thinks parity is valid.

 

One step you can take without risk is to run a NO-CORRECTING parity check. If you get lots of errors in the first minute, then you probably have good parity and can rebuild disk1 as I described. But hold off on that step till you hear from bjp.

If you read his other thread, switched config files around is exactly what happened. He booted from an old flash that had an old disk configuration on it. That is why unRAID thought disk1 was parity, because it used to be parity. Now he has a corrupt reiser partition as a result of writing parity to disk1.

Take a look at THIS POST. This is so close to what happened to me its like deja vu!

 

So let me see if I understand ...

 

1 - You started the array with the wrong disk in the parity slot, and a correcting parity check started. But all of the disks were configured in the array.

2 - You stopped the parity check within 1 minute.

 

What happened next that lead to that screenshot you posted:

- Did you rearranged the disks to be in the right slots?

- Did you then run another parity check with the correct disk in the parity slot?

 

Just a random thought - If you got the wrong drive in the parity slot, but all other disks were in the array, I would think a parity check would still be successful and find no issues. The real parity drive would certainly look unformatted as a data disk, but parity should still be fulfilled.  Maybe the partitioning affects the starting point on the parity disk.

 

But, getting back to this situation, assuming all of your drives are in your array, and your parity is back in its true and correct location, and no correcting parity check has been issued since this problem, you should be able to rebuild disk1 from parity and the other disks. Even if you wrote to other disks in the array after this incident you should be ok. One way to tell is to remove physical disk1 from the array (you could shut down and unplug the disk). unRAID will simulate the disk. If it looks formatted and you can see your files and verify that all looks good, you can rebuild it! You will be very lucky indeed if this works.

 

But if it still looks unformatted, then you are back to having to do reiserfsck. If you didn't already, I would still disconnect disk 1 from the array, and run reiserfsck on it following my earlier post as a general guide (doubt you'll need spinrite or anything). Make sure to run it on the first partition (e.g. /dev/sdz1) and not on the device itself (/dev/sdz). Trust me, you don't want to make this mistake. Search and find current instructions, as some of the value selections may have changed since my experience.

 

It will take a long time and you may need to reboot afterwards to see the results properly.

Take a look at THIS POST. This is so close to what happened to me its like deja vu!

 

So let me see if I understand ...

 

1 - You started the array with the wrong disk in the parity slot, and a correcting parity check started. But all of the disks were configured in the array.

2 - You stopped the parity check within 1 minute.

 

What happened next that lead to that screenshot you posted:

- Did you rearranged the disks to be in the right slots?

- Did you then run another parity check with the correct disk in the parity slot?

 

Just a random thought - If you got the wrong drive in the parity slot, but all other disks were in the array, I would think a parity check would still be successful and find no issues. The real parity drive would certainly look unformatted as a data disk, but parity should still be fulfilled.  Maybe the partitioning affects the starting point on the parity disk.

 

But, getting back to this situation, assuming all of your drives are in your array, and your parity is back in its true and correct location, and no correcting parity check has been issued since this problem, you should be able to rebuild disk1 from parity and the other disks. Even if you wrote to other disks in the array after this incident you should be ok. One way to tell is to remove physical disk1 from the array (you could shut down and unplug the disk). unRAID will simulate the disk. If it looks formatted and you can see your files and verify that all looks good, you can rebuild it! You will be very lucky indeed if this works.

 

But if it still looks unformatted, then you are back to having to do reiserfsck. If you didn't already, I would still disconnect disk 1 from the array, and run reiserfsck on it following my earlier post as a general guide (doubt you'll need spinrite or anything). Make sure to run it on the first partition (e.g. /dev/sdz1) and not on the device itself (/dev/sdz). Trust me, you don't want to make this mistake. Search and find current instructions, as some of the value selections may have changed since my experience.

 

It will take a long time and you may need to reboot afterwards to see the results properly.

 

I removed disk1 from the array. Started it and it says "Stopped. Missing disk"

I guess I am supposed to tick the check box

"Start will disable the missing disk and then bring the array on-line.

The disks data will be available, but the array will be unprotected; install a new disk as soon as possible.

Yes I want to do this"

and START the array.......I that true?

Take a look at THIS POST. This is so close to what happened to me its like deja vu!

 

So let me see if I understand ...

 

1 - You started the array with the wrong disk in the parity slot, and a correcting parity check started. But all of the disks were configured in the array.

2 - You stopped the parity check within 1 minute.

 

What happened next that lead to that screenshot you posted:

- Did you rearranged the disks to be in the right slots?

- Did you then run another parity check with the correct disk in the parity slot?

 

Just a random thought - If you got the wrong drive in the parity slot, but all other disks were in the array, I would think a parity check would still be successful and find no issues. The real parity drive would certainly look unformatted as a data disk, but parity should still be fulfilled.  Maybe the partitioning affects the starting point on the parity disk.

 

But, getting back to this situation, assuming all of your drives are in your array, and your parity is back in its true and correct location, and no correcting parity check has been issued since this problem, you should be able to rebuild disk1 from parity and the other disks. Even if you wrote to other disks in the array after this incident you should be ok. One way to tell is to remove physical disk1 from the array (you could shut down and unplug the disk). unRAID will simulate the disk. If it looks formatted and you can see your files and verify that all looks good, you can rebuild it! You will be very lucky indeed if this works.

 

But if it still looks unformatted, then you are back to having to do reiserfsck. If you didn't already, I would still disconnect disk 1 from the array, and run reiserfsck on it following my earlier post as a general guide (doubt you'll need spinrite or anything). Make sure to run it on the first partition (e.g. /dev/sdz1) and not on the device itself (/dev/sdz). Trust me, you don't want to make this mistake. Search and find current instructions, as some of the value selections may have changed since my experience.

 

It will take a long time and you may need to reboot afterwards to see the results properly.

 

I removed disk1 from the array. Started it and it says "Stopped. Missing disk"

I guess I am supposed to tick the check box

"Start will disable the missing disk and then bring the array on-line.

The disks data will be available, but the array will be unprotected; install a new disk as soon as possible.

Yes I want to do this"

and START the array.......I that true?

 

yes

Array started, It shows disk 1 with red dot.....Not Installed...I guess as it should.

Files are still gone.

Should I run a NO-CORRECTING parity check as suggested by Morten?

 

You can't. That was a step got before. Are you sure the filed are gone? What is at /mnt/disk1?

You can't. That was a step got before. Are you sure the filed are gone? What is at /mnt/disk1?

You are right, I cant...

\\TOWER\disk1..............WINDOWS CANNOT ACCESS\\Tower\disk1

 

I have access to \\TOWER\disk2

Try and confirm via console.

ls /mnt

ls /mnt/disk1

Does it still say the disk is unformatted?

 

Take a GUI shot.

Try and confirm via console.

ls /mnt

ls /mnt/disk1

 

root@Tower:~# ls /mnt

cache/  disk2/  disk3/  user/  user0/

root@Tower:~# ls /mnt/disk1

/bin/ls: cannot access /mnt/disk1: No such file or directory

 

Are you sure that all of the disks are in the array the way they were before this fiasco?

Does it still say the disk is unformatted?

 

Take a GUI shot.

 

See attachments

 

So let me see if I understand ...

 

1 - You started the array with the wrong disk in the parity slot, and a correcting parity check started. But all of the disks were configured in the array.

2 - You stopped the parity check within 1 minute.

 

 

Yes, to # 1. I started the array with an old flash drive because the current one was not letting me see unraid. See previous post.

With the old flash disk1 was parity...it took it as parity now. but the now parity 4tb...did not appear in the array at all. IT started the parity by itself and I stopped it.

Then after backing up the faulty flash drive, I deleted everything from it.

Copied the contents of the old flash into the faulty flash drive.

I copied the config folder from the faulty USB to the new "hybrid"USB and it gave me the same error.

Therefore From the faulty USB I kept :

Shares(folder)

disk.cfg

Plus.key

Super.dat

Everything else belongs to the old USB.

Are you sure that all of the disks are in the array the way they were before this fiasco?

 

I am 100% sure...but read my previous post. Perhaps there is another file that I should copy from the faulty USB backup to the Hybrid USB

 

EDIT: I can elaborate on what happened since the beginning but there is a thread about it.

[ftp=ftp://lime-technology.com/forum/index.php?topic=32808.0]http://lime-technology.com/forum/index.php?topic=32808.0[/ftp]

OK, thanks for fully describing the steps that led to this. Hmmm, You may want to let BJP confirm this is the right action but I think you should stop your array and shut down. Then unplug disk1 physically in the box. If you are not 100% sure, then note the serial numbers (they are in your screenshots here) and go by that. Then take your usbdrive, and put on a fresh copy of disk config files from your old non-working USB key. Hopefully you still have a copy on your computer. than fire it up again and see if you get a simulated disk1.

 

If you do (and I think you will), you're golden. Shut down and reinstall disk 1, and reassign to the array, it will then rebuild disk 1.

OK, thanks for fully describing the steps that led to this. Hmmm, You may want to let BJP confirm this is the right action but I think you should stop your array and shut down. Then unplug disk1 physically in the box. If you are not 100% sure, then note the serial numbers (they are in your screenshots here) and go by that. Then take your usbdrive, and put on a fresh copy of disk config files from your old non-working USB key. Hopefully you still have a copy on your computer. than fire it up again and see if you get a simulated disk1.

 

If you do (and I think you will), you're golden. Shut down and reinstall disk 1, and reassign to the array, it will then rebuild disk 1.

 

No. If the simulated disk is bad, the rebuilt disk would be bad. You'd overwrite your best chance of recovery. Do not to that.

 

USB doesn't matter so much. It saves the disk config and tells unRAID that parity is valid or not. Also is update to say whether there was a clean shutdown. Nothing about the USB is the issue now if the array is started and the disks are laid out as they should be.

 

If there should be 4 disks in your array - a 4T parity, 2 3T data, and 1 2T data, then we've done all we can do with this approach.

 

I'm assuming that when you started the array with the old USB, that the 4T was not in the array. That would explain the parity errors you got that corrupted your disk. I still believe if all of the disks were in the array and you had a data disk in the parity slot, a parity check would be clean. But each write would be shooting bullets in your data disk so definitely wouldn't recommend it!!

 

Conceptually disk1 should have been the only thing impacted by being in the parity slot. So this should have worked. Unless there was another disk involved I'm not sure why it didn't.

 

You might want to reach out to Tom. I would. Everything I've done can be undone. We have not made any writes to the drives in your array.

 

I believe the next step would be to run reiserfsck, but not sure whether to do on the physical disk (which I think it probably the right answer) on the stimulated disk (which I expect is junk), or both. If you told me it was all backed up and it didn't matter, I'd say to run on the physical disk. But if we're talking about lost baby pictures and irretrievable content, I'd go to the big guy for assurance there is not another path to try first. (If you don't own a license, and he helps you, you might conisder buying one ) You could use dd to clone disk1, and run the recovery on the clone. Because once you run recovery you have no ability to un-run it. The changes are made permanently. But I don't think there is any plan C if this doesn't work, so not sure there is a need. Again, I would talk to Tom.

 

Update: Sorry, at first I was thinking this was a free license but with 3 data disks its not.

Yeah, I just looked at the screenshot again and see disk 1 shows up as both "not installed" and "unformatted". However, nothing in his description of events explain why the simulated disk1 would be corrupted/unformatted. First I thought disk1 was not being simulated, are we 100% sure it is in fact being simulated here?

 

It may very well be BJP is correct, that restoring the disk config will not help, but on the other hand it is hard to see how it would do any harm (assuming you physically unplug disk1 so the array doesn't start automatically when you boot).

 

You hit it....all my pictures were in there....two kids....from 2001 - mid 2011

the rest is movies which I could them .

How do i talk to Tom? Do I send him an e-mail?

 

You hit it....all my pictures were in there....two kids....from 2001 - mid 2011

the rest is movies which I could them .

How do i talk to Tom? Do I send him an e-mail?

 

Yes

 

Yeah, I just looked at the screenshot again and see disk 1 shows up as both "not installed" and "unformatted". However, nothing in his description of events explain why the simulated disk1 would be corrupted/unformatted. First I thought disk1 was not being simulated, are we 100% sure it is in fact being simulated here?

 

It may very well be BJP is correct, that restoring the disk config will not help, but on the other hand it is hard to see how it would do any harm (assuming you physically unplug disk1 so the array doesn't start automatically when you boot).

 

Sorry, I saw this part and skimmed the earlier part too quickly. Rebuilding disk1 in the current state would be a disaster.

 

Shut down and reinstall disk 1, and reassign to the array, it will then rebuild disk 1.