A Strange problem with 4 TB drive and Supermicro AOC-SASLP-MV8 card

I just received my first  4 TB drive. It is a Seagate ST4000DM (4TB). Prior to this I was running 4.7 and had only 2 TB drives. I just recently upgraded to V5 in order to support larger drive sizes.

 

My system seems fairly typical. I have a Norco 4224 case, a Gigabyte GA-MA785G-UD3H motherboard, and two Supermicro AOC-SASLP-MV8 cards. One is installed in a PCI-E x16 slot (the motherboard has on-board video) and the other is in a PCI-E x4 slot. Each card, of course, has two SAS connectors each of which is connected to a controller board on the Norco case. Both cards have the .15 firmware installed. Both boards have been operating without problems for some time.

 

However, when I place the new 4 TB drive in a slot connected to one SAS connector on one card, the system hangs during the initialization process for that card.

 

The drive is recognized and unraid boots just fine when the drive is placed in a slot connected to

 

-an on-board SATA port

 

-either of the SAS connectors on the card connected to the SASLP-MV8 card in the PCI-E x16 slot

 

-one of the two SAS connectors on the SASLP-MV8 card in the PCI-E x4 slot.

 

But if I put it in a slot connected to the other SAS connector on the card in the PCI-E x4 slot the system hangs during initialization.

 

I'd be inclined to think this is a hardware problem if no drive worked when put in a slot connected to that side of the card but that is not the case. I have two 2 TB drives connected to that side of the SASLP-MV8 card and they work just fine. It is only when the 4 TB drive is connected to either of the two empty slots connected to that side of the card that the system won't boot. And, just to reiterate, the 4 TB drive doesn't hang the system when connected to the other side of the same card.

 

Anyone have any insight into what might be going on and/or how to correct it?

 

Thanks

 

 

Harry

 

I'm guessing that the system is looking to this drive as a potential boot device, and the BIOS doesn't support booting from a device > 2.2TB.  Have you disabled INT13 on the SASLP-MV8 cards?

I'm guessing that the system is looking to this drive as a potential boot device, and the BIOS doesn't support booting from a device > 2.2TB.  Have you disabled INT13 on the SASLP-MV8 cards?

 

Nice thought but I just confirmed that isn' the problem. I set Int 13 off as soon as a got the cards, some time ago, but went back and looked anyway. Turned out one of the two didn't have Int 13 correctly (the one that was causing the problem) but changing the setting to disabled didn't change anything.

 

Also this problem bank of drives posts pretty late in the process. The first Supermicro posts all it's drives, then the first bank on the second supermicro board and finally the second bank where the problem is. I can snap a picture of what the screen is showing when the problem occcurs. if anyone is interested.

 

The system gives no indication of trying to boot off of that drive. Could it be that I should be using the .21 firmware or will that just make troubleshoooting more difficult.

 

Thanks.

I've been doing troubleshooting on this problem all day and I've narrowed things down without solving the problem.

 

To restate the problem: A 4 TB Seagate drive - the first drive larger than 2 TB on the system, hangs when connected to one "side" of an SASLP-MV8 card. During the posting process, when it gets to that drive it just hangs trying to deal with that drive until the routine times out and just stalls.  (see attached photo)

 

My system consists of Norco 4224 case with six controller boards each of which controls 4 drive bays. Connected to these Norco controllers I have 6 on-board SATA ports and two SASLP-MV8 controller cards one inserted into a PCI-E x4 slot and the other into  a PCI-E x16 slot. Each SASLP-MV8 has, of course, two SAS connectors which supports 4 SATA drives. It was one particular "path" - consisting of one connector on the SASLP-MV8 card in the x4 slot, an SAS cable, a NORCO controller board, and 4 drive bays that is probelmatic. The drive bays were easily eliminated since the system behaved the same regardless of which of four bays in a given row were used to house the 4 TB drive.

 

When the 4 TB disk was connected to an onboard SATA slot, either SAS connector on the card in the PCI-E x16 slot or the "good" side of the card in the x4 slot, unraid booted just fine and showed the drive as a new unformatted drive.

 

When placed in a slot connected to the other SAS connector on the card in the PCI-E x4 card the system wouldn't boot and wouldn't get past the posting process for the supermicro cards.

 

At that point all I knew was that one particular "path" wouldn't work. The culprit could have been the card, the cable or the controller card on the Norco case. There was also the question of the firmware on the cards (.15).

 

The first thing I tried was flipping the two SAS connectors on the card connected to the PCI-E x4 slot. This eliminated the controller card on the case, since the slots that wouldn't work moved to another controller board. This also eliminated the cable since the SAS cable were never removed from the Norco boards. Next I swapped the cards, putting the card that was in the x16 slot in the x4 slot and vice versa. The problem remained with the x4 slot, thus eliminating the card.

 

Next I tried updating the firmware to .21. After doing that the system wouldn't boot. It would say something about initializing RAID and hang. But curiously, with  the .21 firmware installed, the system had no problem recognizing the 4 TB drive in a slot where it had previously been unable to properly deal with the drive. It was after that point, where the system was getting confused regarding RAID that the system hung.

 

I tried to see if the .21 firmware had somehow changed some motherboard BIOS setting, but the system refused to allow me to get to the MB bios while booting. I had to disconnect all the SAS connectors from the Supermicro boards and physically remove one of the cards in order to investigate the MB bios (nothing had been changed) and to re-install the .15 firmware on the Supermicro card. Then I had to remove that card, replace it with the other and re-install the .15 firmware on the other card.

 

 

After re-installing the .15 firmware to both cards and re-attaching the SAS connectors to the Supermicro cards,  I could boot successfully to unraid and was back where I started. So what do I think I have learned?

 

-It is NOT the Supermicro card (Whichever card is in the x4 slot is the one that has the problem)

-It is NOT the Norco board (Swapping the SAS connectors on the card in the x4 slot moves the problem to a different Norco Board (and incidentally another SAS cable).

-It is not an INT 13 problem. (It has been disabled on both cards and the system shows no evidence of trying to boot from the 4 TB drive.)

- The system recognizes the 4 TB drive, wherever it is placed if I use the .21 firmware, but using the .21 firmware causes the system to hang thinking it is trying to load RAID.

 

It looks to me that this is basically some incompatibility between my motherboard (GIGABYTE GA-MA785G-UD3H) and the Supermicro .21 firmware.

 

It may be time to look for a new motherboard.

 

Does anybody read these results as saying something different, or can someone offer any solution short of replacing the motherboard.

 

 

Thanks for any help.

 

 

Harry

- The system recognizes the 4 TB drive, wherever it is placed if I use the .21 firmware, but using the .21 firmware causes the system to hang thinking it is trying to load RAID.

Did you have the other card plugged to the x16 slot with the .15 firmware at the same time the one with the .21 firmware was plugged into the x4 slot?  If so try it with only the card at .21 plugged in or flash both cards to .21 and see if that make a difference.  If that doesn't help then you are probably right .21 isn't compatible with your mother board.

- The system recognizes the 4 TB drive, wherever it is placed if I use the .21 firmware, but using the .21 firmware causes the system to hang thinking it is trying to load RAID.

Did you have the other card plugged to the x16 slot with the .15 firmware at the same time the one with the .21 firmware was plugged into the x4 slot?  If so try it with only the card at .21 plugged in or flash both cards to .21 and see if that make a difference.  If that doesn't help then you are probably right .21 isn't compatible with your mother board.

 

Thanks for the suggestion, but I don't think that was the problem. I installed the .21 firmware to both cards at the same time.

 

To be explicit: I started with both cards using the .15 firmware. I shut down the system, and rebooted with a flash drive that booted to DOS and contained the .21 firmware. I ran the batch file, installed the .21 firmware on both cards, shutdown the system and rebooted with the unraid flash drive installed. It was after that that the boot process would stall when trying to install RAID.

 

What blows my mind is that the x4 slot works for HALF of a supermicro card. I would get it immediately if I couldn't connect any 4 TB drive to anything connected to the x4 slot, but that one half of the supermicro card works but the other doesn't, and that it is not a problem with the card itself, blows my mind.

 

But I was very careful with my troubleshooting process and there is no question in my mind that the problem is not with the card, (and, incidentally, very thankful that it is not a problem with the rather tempermental Norco controller boards.)  Either card fails in exactly the same way when plugged into the x4 slot with the .15 firmware.

 

I've searched around most of the day but can't find anyone reporting similar problems. But, in the scheme of things, it's a pretty obscure problem.

 

Thanks for your feedback. I keep thinking I'm missing something.

 

Harry

- The system recognizes the 4 TB drive, wherever it is placed if I use the .21 firmware, but using the .21 firmware causes the system to hang thinking it is trying to load RAID.

Did you have the other card plugged to the x16 slot with the .15 firmware at the same time the one with the .21 firmware was plugged into the x4 slot?  If so try it with only the card at .21 plugged in or flash both cards to .21 and see if that make a difference.  If that doesn't help then you are probably right .21 isn't compatible with your mother board.

 

Thanks for the suggestion, but I don't think that was the problem. I installed the .21 firmware to both cards at the same time.

That's essentially what I was asking.  Others have posted in other threads that there were problems with mismatched firmware.  I've never had any but I'm not currently using multiple AOC-SASLP-MV8 in one box any more either.  I switched to M1015s I got off of eBay.  I think you have discovered an incompatibility as you suspected.  Other than changing MBs or controllers not sure what you can do.  One last possiblity that you probably already tried/looked for (but didn't see in quick scan of thread again):  Upgrade the MB bios to latest version. 

- The system recognizes the 4 TB drive, wherever it is placed if I use the .21 firmware, but using the .21 firmware causes the system to hang thinking it is trying to load RAID.

Did you have the other card plugged to the x16 slot with the .15 firmware at the same time the one with the .21 firmware was plugged into the x4 slot?  If so try it with only the card at .21 plugged in or flash both cards to .21 and see if that make a difference.  If that doesn't help then you are probably right .21 isn't compatible with your mother board.

 

Thanks for the suggestion, but I don't think that was the problem. I installed the .21 firmware to both cards at the same time.

That's essentially what I was asking.  Others have posted in other threads that there were problems with mismatched firmware.  I've never had any but I'm not currently using multiple AOC-SASLP-MV8 in one box any more either.  I switched to M1015s I got off of eBay.  I think you have discovered an incompatibility as you suspected.  Other than changing MBs or controllers not sure what you can do.  One last possiblity that you probably already tried/looked for (but didn't see in quick scan of thread again):  Upgrade the MB bios to latest version.

 

That's probably worth trying. I actually hadn't thought about doing that, having never flashed a BIOS before.

 

I'm in the middle of pre-clearing the 4 TB drive. It's gonna take a while. I've found the BIOS's Gigabyte has posted and downloaded all of them. When the pre-clear is done, I'll reboot and  see what BIOS I'm currently using. Given the dates of the BIOS's posted I suspect I'm not running the latest version.

 

Thanks.

 

Harry

I just saw this post and the issue you are having with not being able to boot using the .21 firmware and the Gigabyte motherboard is very very similar to the issue I had in this post:

 

http://lime-technology.com/forum/index.php?topic=12079.msg114857

 

The Gigabyte motherboard BIOS had insufficient space in conjunction with the .21 bios of the SAS cards. Luckily Gigabyte released a beta bios that was 10MB smaller for my motherboard and after that I was able to successfully flash the SAS cards to .21 and boot. I've been running this way now for many months and it has been stable.

 

hth

 

TheWombat

I just saw this post and the issue you are having with not being able to boot using the .21 firmware and the Gigabyte motherboard is very very similar to the issue I had in this post:

 

http://lime-technology.com/forum/index.php?topic=12079.msg114857

 

The Gigabyte motherboard BIOS had insufficient space in conjunction with the .21 bios of the SAS cards. Luckily Gigabyte released a beta bios that was 10MB smaller for my motherboard and after that I was able to successfully flash the SAS cards to .21 and boot. I've been running this way now for many months and it has been stable.

 

hth

 

TheWombat

 

Sometimes the world is really quite strange. I was just cruising around the forum today, noted your response to another thread on the Supermicro card and followed it back to the post you made about the card and the Gigabyte bios. Not two hours later you're responding to my old post.

 

I'm not sure it's the same problem. My two cards were working fine with .15 and even with one removed .21 wouldn't work. It would try to install some RAID functionality that would hang the boot process. And the problem I was having would only arise when trying to use 4 TB drives on the card in one of the two SAS connectors on either card placed in the x4 slot.

 

I decided I'd rather switch than fight so I wound up replacing the motherboard with an ASRock Z77 Extreme4. But the problem persisted even though the new board had an x8 instead of an x4. And again, the .21 firmware wouldn't allow the system to boot, thus probably eliminating the Gigabyte board as the culprit.

 

So that kind of left the Supermicro card and/or the card's firmware as the culprit. In any event, I've now migrated to two M1015 controllers that are working fine and repurposed the Gigabyte board into a backup PC. I'll probably wind up selling or trading the Supermicro controllers at some point since they work fine, although I'm convinced there is some obscure problem with using multiple cards and 4 TB drives.

 

I have a suspicion, totally unconfirmed, that one of the problems I had with the .21 firmware is that, unlike the .15 firmware, it did not have a configuration text file included in the package. The .15 firmware had a file named SMC_N.txt which contained what looked to me like a bunch of configuration settings one of which was "[RAID FEATURE]=DISABLE". My hunch is that using the right configuration file would have allowed the card to boot without a raid configuration using the .21 firmware, and that might have allowed the 4 TB drives to be handled correctly.

 

But I didn't have the expertise to adapt the entire file and saw way too much potential to do serious damage by trying to adapt the .15 configuration file in complete ignorance of all the settings in it. Oh, and incidentally before swapping motherboards I did check for updated BIOS's from Gigabyte, but I was already using the latest BIOS version.

 

Anyway thanks for the willingness to help.

 

 

Harry

 

 

I've figured out how to disable RAID on the .21 firmware. Your comments in the post above got me thinking and after a couple of hours experimenting it is working and all seems fine so far.

 

See:

http://lime-technology.com/forum/index.php?topic=27129.0

 

:-)

 

TheWombat

I've figured out how to disable RAID on the .21 firmware. Your comments in the post above got me thinking and after a couple of hours experimenting it is working and all seems fine so far.

 

See:

http://lime-technology.com/forum/index.php?topic=27129.0

 

:-)

 

TheWombat

 

Very cool! Ever since I noticed that configuration file missing in the .21 package I thought that would be the key to getting rid of the raid functionality, but had neither the guts nor skill you displayed to experiment with it.

 

It probably would have solved my previous problem. That's because while the .15 firmware wouldn't work because in one specific circumstance it couldn't post a 4 TB drive, the .21 firmware wouldn't work because, even through it would POST the 4 TB drive, it would hang on the RAID functionality.

 

Unfortunately I've now moved beyond these cards so operationally, for me, the point is moot, but I'm curious enough that when I get a spare moment I'll try to recreate what you've done on the PC that now contains the Gigabyte board. It's not an unraid system any longer but I can still install the cards to see if I can replicate what you did. If I do try it. I'll report back.

 

This is great work! If I might, I suggest you package it up and post a revised 3.1..0.21 package with the new configuration files and post it somewhere. I'm sure it will be of interest to other unraid users with Supermicro cards.