Parity drive uninstalled itself after added newly pre-cleared drive to array ???

I've been running unRAID for over a year now without a single issue.  I recently built a bigger badder server and I think it hates me.  Here's my latest problem...

 

Info first:

unRAID 4.7 Pro

MB - SuperMicro X9SCM-F

3x SuperMicro SASLP cards (not all in use yet)

 

My server was up and running 6 data drives + 1 parity without any problems.  I have several more hard drives that I want to add to the array.  Tonight, I pre-cleared one of the drives using preclear_disk.sh v1.13 (./preclear_disk.sh -n /dev/sdX).  I used the -n option because I know these drives to be good and didn't want to wait the extra 12 hours for the pre-read and post-read cycles.  My MBR alignment is set to 4K aligned in the unRAID GUI.

 

After I pre-cleared the drive, I stopped my array, added the newly pre-cleared drive as disk7 and then restarted my array.  As usual, unRAID prompted me to format the newly installed drive.  I complied.  When the format completed, I noticed that my parity drive was now red-balled and listed as "Not Installed" in the main unRAID GUI.  Huh??  I rebooted my server and now the parity drive has gone from a red-ball to a blue-ball and claims a new parity disk was installed.  unRAID now wants to rebuild parity when I start my array.

 

What the heck happened?  I thought pre-clearing was a way that I could prevent having to rebuild my parity when a new disk is installed in the array?

 

Any ideas?  I've attached my syslog file as a zip file (32 MB unzipped / 233 KB zipped) if any of the experts have a chance to check it out.  

 

If any of the unRAID experts have a chance to check it out, my zipped up syslog is linked here (too big include as attachment).  The drive finished pre-clearing around timestamp "Oct 11 00:21:15".  I stopped the array and assigned the drived immediately after that.

 

[EDIT: More info in next post]

 

 

Been doing some digging and came across the following page:

 

http://lime-technology.com/wiki/index.php?title=The_Analysis_of_Drive_Issues

 

My syslog contains several of the errors listed on this page.  

Though, I'm still trying to determine exactly what the problem might be.

 

The errors that are in my syslog:

-----------------

10B8B error on my ata14 drive (this is my parity drive).

 Oct 10 17:46:56 media kernel: ata14: SError: { PHYRdyChg 10B8B DevExch }

-----------------

ATA bus error on my ata14 drive in a few places (followed by a DRDY):

 Oct 10 17:47:14 media kernel: ata14.00: cmd c8/00:10:20:96:00/00:00:00:00:00/e0 tag 0 dma 8192 in
 Oct 10 17:47:14 media kernel:          res 50/00:00:d7:00:01/00:00:00:00:00/e0 Emask 0x10 (ATA bus error)
 Oct 10 17:47:14 media kernel: ata14.00: status: { DRDY }

-----------------

I also get 'identify failed' in several places

-----------------

I see that ata14 also gets a 'frozen' error.

 Oct 10 17:46:46 media kernel: ata14.00: exception Emask 0x10 SAct 0x0 SErr 0x4090000 action 0xe frozen

 

 

If I'm reading this right, it looks like either (1) my Norco backplane is a piece of junk (maybe isn't getting enough power to the WD Black drive), or (2) I might have a bad SAS cable (it's a reverse breakout from Norco).

 

My new system is running an 850 watt seasonic power supply, so I'm pretty sure that it's not my power supply.  However, I am connecting my backplanes to the power supply through that really junky 1-7 molex splitter that is popular with the Norco 4224 cases.  It would seem odd that it be a power problem.  I did a parity build without any problems using this same setup.  It wasn't until I wanted to add a drive to the array that I had problems.

 

Any ideas on where I should start?  How I might determine where the problem lies?

 

 

 

 

What EXACT make/model power supply.  (I know you said 850 watt seasonic, but if multi-rail, it could easily be the issue if powering  many drives)  The first 850 watt seasonic I saw on newegg had two rails, each with 40 amps capacity.  That would not be enough if you fully loaded your norco, as usually only one rail is available for the disks, and frequently that is shared with the motherboard.

 

How many drives?  How many non-green?

 

Joe L.

What EXACT make/model power supply.  (I know you said 850 watt seasonic, but if multi-rail, it could easily be the issue if powering  many drives)  The first 850 watt seasonic I saw on newegg had two rails, each with 40 amps capacity.  That would not be enough if you fully loaded your norco, as usually only one rail is available for the disks, and frequently that is shared with the motherboard.

 

How many drives?   How many non-green?

 

Joe L.

 

 

Joe, I'm currently using the following power supply.  Newegg claims it to be a single rail supply:

SeaSonic X Series X-850

 

I have a Norco 4224 with the following drives installed:

1x WD Caviar Black 7200RPM

2x WD Caviar Green WD20EARS

2x Hitachi 7K2000

1x Hitachi 7K3000

6x Hitachi 5K3000 (took advantage of recent $59 sale at newegg )

 

In total there are 4 power hungry drives ... and 8 that are less so.  The remainder of the slots in my Norco are currently empty.

 

 

 

What EXACT make/model power supply.  (I know you said 850 watt seasonic, but if multi-rail, it could easily be the issue if powering  many drives)  The first 850 watt seasonic I saw on newegg had two rails, each with 40 amps capacity.  That would not be enough if you fully loaded your norco, as usually only one rail is available for the disks, and frequently that is shared with the motherboard.

 

How many drives?   How many non-green?

 

Joe L.

 

 

Joe, I'm currently using the following power supply.  Newegg claims it to be a single rail supply:

SeaSonic X Series X-850

 

I have a Norco 4224 with the following drives installed:

1x WD Caviar Black 7200RPM

2x WD Caviar Green WD20EARS

2x Hitachi 7K2000

1x Hitachi 7K3000

6x Hitachi 5K3000 (took advantage of recent $59 sale at newegg )

 

In total there are 4 power hungry drives ... and 8 that are less so.  The remainder of the slots in my Norco are currently empty.

 

 

 

Nice power supply.  (you obviously paid attention to many of the posts where the need for a single-rail supply when powering a large number of disks was stressed )

 

Joe L.

Nice power supply.  (you obviously paid attention to many of the posts where the need for a single-rail supply when powering a large number of disks was stressed )

 

Joe L.

 

Partly that ... partly that I just like to buy quality components.  Frankly, it's these Norco backplanes that have me a bit concerned.  I bought my case and only 2 out of the 6 backplanes worked.  Now I'm wondering how well even those two worked.

 

 

Are you powering all of your drives with one lead from the Seasonic?

 

I would recommend against using a single connection from a power supply to power 24 drives. Most power supply wiring is 18 gauge with some 20 gauge and 16 gauge wire occasionally used. During disk start-up the 12 volt line current draw is in excess of 50 amps. The "Handbook of Electronic Tables and Formulas" specifies maximum current draw for an 18 gauge wire at 16 amps. This is a very conservative figure, but exceeding it by a factor of three is not something I would do. Apart from the safety issues, the voltage drop from the wire alone is on the order of 1 volt. This does not include the connector resistances which at such high currents could also prove problematic. I wouldn't power more than 8 drives off a typical 18 gauge power supply connection. Request some power supply to Molex leads from Seasonic. Another user stated they got them for free.

 

http://www.powerstream.com/Wire_Size.htm

Are you powering all of your drives with one lead from the Seasonic?

 

I would recommend against using a single connection from a power supply to power 24 drives. Most power supply wiring is 18 gauge with some 20 gauge and 16 gauge wire occasionally used. During disk start-up the 12 volt line current draw is in excess of 50 amps. The "Handbook of Electronic Tables and Formulas" specifies maximum current draw for an 18 gauge wire at 16 amps. This is a very conservative figure, but exceeding it by a factor of three is not something I would do. Apart from the safety issues, the voltage drop from the wire alone is on the order of 1 volt. This does not include the connector resistances which at such high currents could also prove problematic. I wouldn't power more than 8 drives off a typical 18 gauge power supply connection. Request some power supply to Molex leads from Seasonic. Another user stated they got them for free.

 

http://www.powerstream.com/Wire_Size.htm

 

 

Thanks ohlwiler.

I've been thinking about that as well.  I'm currently only using a single molex connector into the Norco 1-to-7 splitter.  I currently only have 12 drives on the splitter, but I've been thinking even that is probably too many for the junky little splitter.  The only reason I even considered the splitter was because I know a lot of folks here have used it with success.  Maybe I'll give it the boot.

 

Ok, so let's assume that this is a power issue.  If I "unplug" my parity drive and then "plug" it back into my array does it exhibit the behavior that I've described in my OP?  That is, would it immediately want to re-sync?  That sounds like a reasonable and correct behavior given that data may have been written to the array while the parity drive was away.  However, I'm curious ... is this what unRAID is designed to do?

 

 

There have been numerous reports of the 4224 not working when a single power plug was used on each backplane. Run seperate cables from the power supply to each of the MOLEX plugs. Also, say you use a power cables from the supply which have 3 MOLEX on each cable. Connect the first cable to 3 of the backplanes in order 1-2-3 and the second cable in order 3-2-1. That way, the cabling connections are more evenly distributed to the backplanes.

 

A single MOLEX feeding a splitter for 24 drives is a recipe for failure.

 

Peter

 

I can possibly see using a 1 to 6 splitter if:

 

-The SATA controller supported staggered spin-up.

 

-Both Molex connectors (they are in parallel) were connected on each backplane (use two 1:6 splitters).

 

Since unraid does not support staggered spin-up, I can never see it working reliably.

 

With twelve drives on a single Molex connector you are going to get some voltage drop at spin-up. Not all drives respond the same to a voltage drop. If you want reliable operation, provide stable high current capable connections to the Norco backplanes.

 

Unraid does not support hot swapping. I don't know how unraid would respond if you yanked the parity drive and then plugged it back in. I have never tried it, but I am positive it will not re-sync.

FYI, if unRAID "loses" a drive due to a connection or power issue then it is red-balled. If the array is stopped and started or the machine is rebooted with the drive "lost" then unRAID "forgets" that it existed at all. Any drive that re-appears would be considered to be a replacement and not the original drive. This seems to be exactly what you experienced.

There have been numerous reports of the 4224 not working when a single power plug was used on each backplane. Run seperate cables from the power supply to each of the MOLEX plugs. Also, say you use a power cables from the supply which have 3 MOLEX on each cable. Connect the first cable to 3 of the backplanes in order 1-2-3 and the second cable in order 3-2-1. That way, the cabling connections are more evenly distributed to the backplanes.

 

A single MOLEX feeding a splitter for 24 drives is a recipe for failure.

 

Peter

 

 

 

I'll give this a try this evening.  I have plenty of molex connectors lying around for SeaSonic power supplies.  I should (almost) be able to run an single cable from the power supply to each of the backplanes.  I might have to double-up on one of the cables and connect it to two backplanes because the power supply I'm using only has inputs for 5 peripheral cables total.  If I double-up on any one cable, I'll be sure to put all green drives on those two backplanes.

 

 

FYI, if unRAID "loses" a drive due to a connection or power issue then it is red-balled. If the array is stopped and started or the machine is rebooted with the drive "lost" then unRAID "forgets" that it existed at all. Any drive that re-appears would be considered to be a replacement and not the original drive. This seems to be exactly what you experienced.

 

Yep.  I assumed that would be behavior of unRAID.  It only seems logical to assume that if the parity drive goes missing for any period of time (even a millisecond) then you must assume it is out of sync.

 

 

FYI, if unRAID "loses" a drive due to a connection or power issue then it is red-balled. If the array is stopped and started or the machine is rebooted with the drive "lost" then unRAID "forgets" that it existed at all. Any drive that re-appears would be considered to be a replacement and not the original drive. This seems to be exactly what you experienced.

 

Yep.  I assumed that would be behavior of unRAID.  It only seems logical to assume that if the parity drive goes missing for any period of time (even a millisecond) then you must assume it is out of sync.

 

 

It is assumed to be out of sync if a "write" to it fails. 

You'll be fine if you run 2 cables and power say 2 or 3 backplanes with those 2 cables. I was meaning before that if you get a power supply cable with 3 MOLEX on it that you start the cable so the MOLEX closest to the supply is in backplane 1 with the next one going to backplane 2 and the last one on the end to backplane 3.  Do that for the left MOLEX connector on each backplane. Then, run another power supply cable to the right MOLEX connector on each backplane but reverse the order of the power plugs so the one closest to the power supply goes to backplane 3.

 

Peter

 

You'll be fine if you run 2 cables and power say 2 or 3 backplanes with those 2 cables. I was meaning before that if you get a power supply cable with 3 MOLEX on it that you start the cable so the MOLEX closest to the supply is in backplane 1 with the next one going to backplane 2 and the last one on the end to backplane 3.  Do that for the left MOLEX connector on each backplane. Then, run another power supply cable to the right MOLEX connector on each backplane but reverse the order of the power plugs so the one closest to the power supply goes to backplane 3.

 

Peter

 

 

Ahh, I understand what you mean now.  I was a bit confused when I read the first description you wrote.  This is much clearer.  I'll give it a try when I get a chance... though it's looking like it might not happen until this weekend.