Came home to a mess[SOLVED]

To start this off, I have rebooted multiple times so I don't have a log. I have unraid server pro running 5.0.4 with a 3tb parity, 5 3tb, a 1tb, and a 1tb cache drive. I had a 6th 3tb pre clearing in my drive over the last 24 hours. I came home tonight and noticed that my parity drive had a lot of read/writes, way more than normal. I checked the running syslog and discovered that one of my drives was doing that hard resetting link error. I have had issues in the past with loose cables or faulty power supply. I normally can shut it down, check all cables to make sure they are snug, then boot it back up with no problems, but not tonight. When I booted it back up disk 3 shows disabled and the parity is yellow/orange, but I didn't notice this until I started the array because it was telling me I have an unformatted disk in, which was the disabled one. So naturally I stop the array, power it down, check the cables again. I boot it back up and the same thing. I checked the smart report and it wouldn't show any information, so I thought that the drive actually died. So what better time to put in the disk I just got done pre clearing. I booted it up, started the array, proceeded to format the new disk, and it just comes back to the screen asking to format again. So I try again and same results. So I'm getting a little confused and powered the box down again and installed the drive I thought was faulty. I boot back up and it acts like it's a new disk. I decided I've lot data, I don't care, I just want my server back up. I start the array, try to format, same thing, just goes back to showing I have a drive that needs formatting. I have no idea what to do. I went ahead and ordered a highly rated power supply to hopefully solve future problems, but what can I do to get my array back going? I am going to power it down and leave  it off until I get a response.

Not clear from your description how you got to this point. In particular, when doing a rebuild, I don't think formatting the new drive is one of the steps since the file system is part of the rebuild. unRAID offers to rebuild the drive when you change it instead of offering to format it.

 

What new PSU have you ordered? Are you using locking SATA cables? What about the power connections to the drives?

 

A screenshot might give us a better idea of the current state.

 

Don't start up to create a syslog, but don't forget:

The next time you start the array, SAVE the SYSLOG before you shutdown...and post it.

 

Trying to understand where you're at. Is the following right:

Your original disk#3 and original Parity are in the array.

Parity is bad. AND the parity disk itself has read/write errors.

Disk#3 is showing up as unformatted.

 

Is that right?

 

@ trurl The power supply I ordered is http://www.newegg.com/Product/Product.aspx?Item=N82E16817153165 I'm not using locking sata cables on all but some. I assume you are asking if the power wires are on secure and they are. I do use a power splitter http://www.amazon.com/StarTech-com-Power-Splitter-Adapter-PYO4SATA/dp/B0086OGN9E/ref=sr_1_3?ie=UTF8&qid=1392647481&sr=8-3&keywords=sata+power+cable for all my power.

 

@DaleWilliams I hope the screenshots and syslog will assist in giving you the correct information. I started the array in maintenance mode because I was scared to start it and bring it online. If that is what is needed I will do it.

 

 

Thanks for yalls input so far.

 

syslog.zip

@ trurl The power supply I ordered is http://www.newegg.com/Product/Product.aspx?Item=N82E16817153165

That power supply appears to be a multi rail power supply - not recommended for unRAID.  I would cancel the order and get a single rail power supply if that is still possible.

 

For 5 more dollars here is a single rail Thermal Take power supply of almost same size (550W instead of 500W):

http://www.newegg.com/Product/Product.aspx?Item=N82E16817153172

@BobPhoenix  Order is canceled. Do you have one you can recommend?

 

 

Sent from my iPhone using Tapatalk

@BobPhoenix  Order is canceled. Do you have one you can recommend?

 

 

Sent from my iPhone using Tapatalk

Added a possibility above.  I like Seasonic personally but the ThermalTake I posted should work I would think.  Maybe garycase has a better idea for you however.

+1 for Seasonic and Corsair. I personally would not buy anything else.

 

Here's some options. They're a bit more than what you were originally looking at, but the PSU is not somewhere where you should be cutting corners:

 

http://pcpartpicker.com/parts/power-supply/#m=11,71&sort=a9&w=700,600,500

 

I'd recommend the TX650 from Corsair or the S12G-550 from Seasonic. The Seasonic is actually Gold certified, which means you might save a few bucks in electricity because of it's efficiency, but the Corsair is certainly no slouch...

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16817151136

http://www.newegg.com/Product/Product.aspx?Item=N82E16817139020

 

EDIT: Here's another Seasonic. A bit cheaper: http://www.newegg.com/Product/Product.aspx?Item=N82E16817151096&clickid=Qz1wgFzy1xcdxbkThnTJZSvmUkTUs6xD0Ql1xY0&iradid=97618&ircid=2106&irpid=79301&nm_mc=AFC-IR&cm_mmc=AFC-IR-_-na-_-na-_-na

 

HTH!

 

I ordered this one http://www.newegg.com/Product/Product.aspx?Item=N82E16817151136

You're right about not wanting to cut corners here. When I first built this server I didn't realize it was going to grow into this. I didnt go with the second Seasonic you recommend because I was scared your recommendation was going off me being cheap. I almost went ahead with the 650 watt version of the one you picked out but didnt. Thanks for your help and recommendation.

 

Edit: I also just ordered a pack of these http://www.amazon.com/gp/product/B00HE5VNFA/ref=oh_details_o00_s00_i00?ie=UTF8&psc=1

Shut down cleanly.

Power down.

Power on and enter BIOS setup.  (Delete, F2, F10, F12, or some such key.)

Make sure that ALL your hard drives are SATA or AHCI.

 

When unRAID starts to load enter SAFEMODE and start up in maintenance mode without all the plugins and add-ons.

It's an option on the boot screen (where you select unRAID OS or Memtest for example).

 

Stay in Maintenance Mode.

Post the new syslog.

Did the stoplights next to the disks change colors?

---

Edit: Nice locking SATA cables-and a good price...I ordered a set, too. 

I ordered this one http://www.newegg.com/Product/Product.aspx?Item=N82E16817151136

 

Congrats! The specs of that PSU are solid. The fact that it is Gold Rated at that price point is amazing. Coupled with a reliable brand name and a 5 year warranty, I think you made a great choice .

 

Also, It should also give you room to grow, as 45amps on the 12v rail should allow you to go up to 20 green drives or 13 7200rpm drives (and because it is gold-rated, it will still output power at around 90% efficiency with the smaller number of drives you currently have running).

 

Enjoy

@DoeBoye Thanks, and thanks for the help.

 

@DaleWilliams I know what you are talking about on SATA and/or AHCI mode but I cannot find this in my bios. I searched through every menu and didnt find it. I would assume it would be a sub menu under "Sata Operation" but I dont see that either. Either way, I booted unraid and I also didnt see a "Safe Mode" I took a picture of the unraid menu with the options I got. I watched it boot from there and never saw another option for it.

 

Cool, I hope you like your new cables. From experience, never ever order these. In theory they sound great, but I had problems with them pulling out of the mobo on their own after months of working fine.

http://www.amazon.com/gp/product/B003ZDNY02/ref=wms_ohs_product?ie=UTF8&psc=1

 

@DoeBoye Thanks, and thanks for the help.

 

@DaleWilliams I know what you are talking about on SATA and/or AHCI mode but I cannot find this in my bios. I searched through every menu and didnt find it. I would assume it would be a sub menu under "Sata Operation" but I dont see that either. Either way, I booted unraid and I also didnt see a "Safe Mode" I took a picture of the unraid menu with the options I got. I watched it boot from there and never saw another option for it.

 

Cool, I hope you like your new cables. From experience, never ever order these. In theory they sound great, but I had problems with them pulling out of the mobo on their own after months of working fine.

http://www.amazon.com/gp/product/B003ZDNY02/ref=wms_ohs_product?ie=UTF8&psc=1

What version of unRAID did you say you are running?  5.0.? release versions should all have it.  If you have 4.7 or 5.0 betas up through one of the RCs it didn't have safe mode available.  Also you may need to copy the menu file from the setup not just bzroot & bzimage if you've upgraded to 5.0.? release.

If you upgraded by only copying bzroot, bzimage then you would not have gotten the new boot menu. It is in the new syslinux. If you copy that from the distribution then I think you will have to run make_bootable agaiin.

 

Maybe better to get to no addons the old way, as seen in dgaschk's sig - Revert to stock system and Stock go file

@BobPhoenix and @DaleWilliams I did what trurl said and manually entered safe mode. I still have the same two orange/yellow dots beside the parity drive and drive 3.  A syslog is attached.

And yes, I upgraded by only copying the two files. Maybe when this all gets sorted I can do the fully copy.

syslog.txt

First thing I see in the syslog (which when I look at your graphic shows it too) is that you don't have your server setup to be AHCI in the bios for all of the drives (hda: ST3000DM001-9YN166 & hdb: WDC WD1001FALS-00J7B0).  That will slow your access to them but otherwise shouldn't cause any problems and the GUI shows that too as they are green status.  Looks like you have a SASLP-MV8(mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps)?  That controller looks to be connected to your problem drives.  I didn't see anything else that stood out to me but my log foo isn't very good so a second opinion is necessary.  Definately need someone that is linux and reiserfs savy to help further I think.

I'll look again for the ACHI mode option in the bios. That controller is hooked to disk 4 I know for sure because it is the only Hitachi I have. You figuring all that out in the syslog is 10 times better than I could reading that thing. Maybe someone could come along and figure something out for me. I think before I fire it with anyones ideas it may be best for me to get the new psu in I ordered. I think even if I lost data on the 3tb it will be a mild lesson learned IMO.

 

Edit: @BobPhoenix I finally found where to change it to AHCI. Now if I change this to AHCI are you sure this isnt going to mess up anything like it would if this had a Windows install on it?

Never had a problem doing that myself in unRAID but once again I would get a second opinion on that as I would hate to tell you it is no problem and I am wrong.  I know Windows would just blue screen and reboot.

Yeah, I dont know what this will do if I were to change it. But what is weird is you say that 2 of them are set to one thing and the others are set to something else. By the looks of the menu in the screenshot, it's either all or none. Hopefully someone will get an answer for me. BTW, my power supply has already shipped. Go Newegg!!!

 

Edit: Oh no, I just looked at the tracking on it and it is Fedex Smart post. I wont get this thing till the end of the week or possibly next week.

Replace the drive that was removed and set the drives to AHCI. Start the server and after 30 minutes copy then attach the syslog. Attach a screenshot.

@dgaschk  I think I should wait until the new power supply is in but I do want to get this straight in the meantime. I should change the Sata operation to AHCI mode. Replace disk 3 which I believe is my only Western digital 3tb drive with the newly pre cleared 3tb drive. Now here is where I want to be sure I am correct. Do you want me to start the array in normal mode or maintenance mode? I am pretty sure you also want me to start in safe mode with no addons ons running. Please clarify so I can do it exactly how you say. I'm sure I've already done enough damage on my own  

Thanks.

Yes, first put in the new PSU. Do you still have the original disk3 with the data on it? Do switch to AHCI on the next boot. While switching to AHCI should only effect performance and not correctness, I've seen switching to AHCI fix disk problems.

Ok I will wait until the psu is in. Yes, the original disk 3 is still in it and is plugged up. Should I leave it in or replace it with another one of the same size?

 

 

Sent from my iPhone using Tapatalk

If you want to attempt to recover the data then keep the one with data in.

Ok thanks. I will respond back with my findings once the new psu is in.

 

 

Sent from my iPhone using Tapatalk