After a fatal crash - now what?

It was only fatal to my unRAID tower, but it felt like somebody had murdered me.  Moved my tower to have the carpets cleaned.  Broke the USB key. 

Here's what happened next:

1. USB couldn't be booted up on unRAID server, but could copy contents onto a Windows platform folder.  Thought it MUST be the broken USB key, but it could be read...

2. Read that a clean install is required for an update from 4.7 to 6.1.  Crap, but ok.  I will follow the path of least resistance even if it is a ton of time

3. DL the latest package, put it on a USB that was formatted and made bootable on Windows 10

4. Plugged it into the back USB slot of the server.  No booting up.  "Put bootable media in..." etc.  Figured it might be because it was a USB3 thumb drive now, a SanDisk Ultrafit 32GB - overkill, but it was lying around and wouldn't get broken if bumped into.

5. No, it seemed that plugging it into both USB2 and 3 ports made no difference.  So I went into BIOS and set the ONLY boot option to be USB.  I should have thought about this a moment...

6. No luck booting.  Of course.

7. Tried a different USB.  Old 2GB, and new USB2 16GB approved list.  Maybe, I thought, Windows 10 doesn't make bootable with the .bat file for unRAID.  Tried on a Win7 machine.  Same

8. No USB would boot, and then, no getting back into BIOS.  Guess what?  The USB wireless keyboard and mouse quit working on initial boot.  No wired keyboard lying around.  So I cleared cmos by the jumper, and tried again.  No luck. 

9. Found a wired keyboard, still didn't work.  Cleared cmos again.  This time with a USB wired keyboard, got into BIOS ("Delete, Delete, Delete, F2, F2, F2, repeat ove and over....") after several tries.  Changed to default settings.  Now at least the keyboards and mouse worked. 

10. Took several minutes spinning up the SuperMicro SAS2LP - all 8 drives.  Every stinking time before I could go into BIOS or boot menu.  So I pulled the card temporarily.

11.Did a F11 to bootup menu, booted from the USB, and guess what - it started unpacking linux.  Went to the screen to go online to get key, but I had my SM card out, I thought I would put it back in and restart to let unRAID see my whole array (I was worried it would not see the disks correctly and screw up the install - the parity was onboard SATA, so I probably could have avoided this step...).

12. So I stopped and respun up my SM SAS2LP - all 8 drives. And no unRAID installation.  This became the pattern.  Spin up 8 drives (several minutes) ,then enter BIOS or boot menu.  Never got the unRAID key to fire up again.  No booting this machine apparently. 

13.  So, mobo maybe?  USB bus?  Problems with the SM SAS2LP?  I tried removing the PCI-e card again, faster boot but no unRAID, no USB device boot.

14. The BIOS reads that there is a USB device, that it is bootable (at least I can choose it in the boot menu in BIOS).  But no booting.

15. At this point I became worried about my data.  4 years of flawless unRAID with only 1 bad disk in that time. 

16.  It was backed up, maybe, from backblaze (I map my shares to my Windows 10 machine, and backblaze does 20TB backup without squawking.  Sloooowly.  But I didn't want to have to restore 23TB from backblaze, so I am doing the unthinkable - I am taking the drives out and using DiskInternals reader to save the movies, photos, music, and videos (I just use unRAID as a file server/NAS.  Backups and archival storage, streaming to Kodi).  I have much of the data on external HDD but have to compare contents and make sure.  Lucky I have a 5TB, 3x 4TB, and several 3TB Seagate Backup drives.  It takes time to read, save and copy the data from the ReiserFS to NTFS drives.

Now what?  Should I trash the mobo?  It is older (relatively speaking) but on a Gig network it was fast enough for streaming and never gave a problem for me.

I can guess what a decent rebuild would be:

1. SuperMicro  MBD-X10SL7-F-O

2. Haswell either i3-4330, i5-4460 (no HT), or all the way to Xeon E3-1231V3

3. 16GB Crucial ECC unbuffered 1600

4. Seasonic S12G 650W Gold (I think this is a single rail, have to check)

5. Existing case - old Xigametek some viking god name or something.  Like the Rosewil Thor now - 9 5.25 external bays.

6. Cages - 4-in-3 or 5-in-3 cages - I have 3 of the Cooler Master ones, wouldn't really buy them again, but as I don't replace drives regularly, they would work.  I have 1 IcyDock 5-in-3 MB155SP-B.  Maybe use this, but some people here are not fond of these I believe.  I like the looks of the IcyDock Black Vortex (maybe I just like the name).  I worry about the 5-in-3 temps.

7. HDD - I was planning a HDD rebuild anyway, so I have currently 4 Seagate 4TB, 1 WD Red 4TB (my parity?), 3 WD Red 3TB.  So 7 HDD that have more space than my 11 2TB array.  I would put the least worn 2TB in the array to fill it up.

8. Use the old SM SAS 8x card? There are enough SATA and SAS on the mobo for 14 drives I believe, so would this be better to use the PCI-e or to just let the mobo controllers do it?  Any bottleneck here?  I don't think so, but I am a doctor, not an IT expert.  That's why I just lurk.  I take too long to translate what you guys say here.

Not a huge expense since I have drives and case, but disappointing.  Next problem, what do I do with my array?

Should I:

      a) add in the newer drives and preclear and add empty, starting from scratch and using my backups to add 20TB to the array?

      b) try to keep my current array and replace the mobo, cpu and memory, use the new SAS onboard (or the old SAS2LP?) to try to get the array back first, then change out parity drive to largest and add in the bigger drives and take the oldest offline?

      c) Will that even be possible?  I was unclear on the clean installation (4.7 to 6.1) whether the data would remain on the reiserFS disks and if the old parity drive could be dded to a clean install and recognize where the disks were.  Or if a new parity could rebuild with the other disks staying intact. 

      d) If I do have to use the new disks or reformat the old ones anyway, should I use one of the new formats instead of reiserFS?  BTRFS?  No, I think XFS is the choice for most because of stability, right?

Am I missing something obvious?  I don't want to do extra work.  It already is a headache, but I don't like living without my unRAID!  Also I know from the past the I might be able to get Joe to give me a new key.  I partially broke the case of the stick a year ago and he said no proble.  I have paid for 2 licenses but back in the v4 days.  I can't blame limetech if I have to rebuy 2 licenses (Pro).  Worth it to me, a Windows user who is used to being ripped off by software (cf. Ad$$e, M$$$$$oft, etc.). 

I have no current syslog to attach (wish I did), as I can't exactly run that right now.  At least I have screenshots of unmenu and the GUI and all the smartdisk info so I know which disk was where and on what controller.  Sigh...

Thanks for any advice and help.  Maybe I have learned something, like have a backup server, a backup USB key with a 2nd license, and not to clean the carpet in the office again.

It it's somewhat common to have USB boot issues. Most motherboards allow you to boot from either hard disk or removable media. There are ways to configure the usb as one or the other. From USB BIOS settings, you can force the USB to emulate a floppy which will force it to be a removable, and then you can set your boot option to boot from the removable. This is actually a very good method because adding or removing hard drives has no effect on booting from the removable. But it doesn't always work with every motherboard / specific hardware configuration.

Configuring the USB as a hard disk (normally the default I believe) often requires individually disabling bootability from real hard drives. And shuffling disks or adding new ones often confuses the BIOS into trying to boot from a drive and not the USB. This becomes an ongoing maintenance irritant that most users just get used to.

Seems odd that breaking your usb sick would physically damage your motherboard unless it was powered on (assuming you weren't moving it when it was running). Fact that you were able to boot in some configurations and not others makes me think that MB is ok, and you are just dealing with persnickedy BIOS settings. But not sure. BTW, most BIOS have some USB legacy option you can set to use usb keyboard. If you completely reset your BIOS, it is possible that some option you set years ago and forgot about is now not set properly and causing problems.

I'm a little confused by your step 6 comment about "of course" it wouldn't boot. Implies you know why. Do you? Or was this a sarcastic comment?

Not reading anywhere that you have data loss. Stay calm and don't take unnecessary risks. Replacing mb is a relatively easy process, albeit the cost. But I think you will be able to get this one working once you get your usb settings right.

Good luck.

It it's somewhat common to have USB boot issues. Most motherboards allow you to boot from either hard disk or removable media. There are ways to configure the usb as one or the other. From USB BIOS settings, you can force the USB to emulate a floppy which will force it to be a removable, and then you can set your boot option to boot from the removable. This is actually a very good method because adding or removing hard drives has no effect on booting from the removable. But it doesn't always work with every motherboard / specific hardware configuration.

 

Configuring the USB as a hard disk (normally the default I believe) often requires individually disabling bootability from real hard drives. And shuffling disks or adding new ones often confuses the BIOS into trying to boot from a drive and not the USB. This becomes an ongoing maintenance irritant that most users just get used to.

 

Seems odd that breaking your usb sick would physically damage your motherboard unless it was powered on (assuming you weren't moving it when it was running). Fact that you were able to boot in some configurations and not others makes me think that MB is ok, and you are just dealing with persnickedy BIOS settings. But not sure. BTW, most BIOS have some USB legacy option you can set to use usb keyboard. If you completely reset your BIOS, it is possible that some option you set years ago and forgot about is now not set properly and causing problems.

 

I'm a little confused by your step 6 comment about "of course" it wouldn't boot. Implies you know why. Do you? Or was this a sarcastic comment?

 

Not reading anywhere that you have data loss. Stay calm and don't take unnecessary risks. Replacing mb is a relatively easy process, albeit the cost. But I think you will be able to get this one working once you get your usb settings right.

 

Good luck.

Sorry for the sarcasm.  Justa murphy's law thing I guess.  I will try to reconfigure bios to read the usb as a removable/floppy.  Sounds like that could work.  Will do that when I get home and report back.  The mobo may be ok, but the bios is definitely screwed up.  I don't know how to flash it or update it without being able to read usb or get to internet.  I'm not all that familiar with ASrock anyway.  It may have some other options.  The idea of replacing the mobo and cpu was to eliminate the bios problem entirely, and hopefully not have problems with usb stcks.  But trying your idea will be the 1st thing.

My question:  will I keep data integrity if I change mobo, cpu, AND clean install of unRAID 6?  That was my worry.  I don't know if the changes in unRAID will keep everything the same or not.  Maybe that is just rookie fears, but it scares me to mess with the OS that much.  I stayed in 4.7 becasue it was working perfect for me for so long.  Thanks for the help!

There were a few issues moving from 4.7 to 5.0. Don't remember the details. I'd recommend getting 4.7 running before upgrading.