Fastest motherboard for writes and parity checks

I'm looking to replace my GIGABYTE GA-G31M-ES2L board with something that can calculate parity faster than 10MB-13MB a second.  It takes forever on a 1.5TB parity drive.  Currently I'm running 11 SATA drives off of two PCI-X SATA-MV8 boards and I'm looking to move to something with either 8 or more SATA ports built in or PCI-X to use my existing controllers and hopefully gain some speed.

Does anyone have a good suggestion for a top performing board that I can hit over 70MBps and has a decent amount of SATA ports?

Your problem is not the mobo, but the controller card.

I have a GA-G31M-ES2L and get much faster parity checks.  Make sure your parity disk and all the LARGEST disks are on the mobo ports.

@bubbaq

I will attempt moving my drives to the 4 onboard sata ports but I ran into HPA issues with the new 1.5TB drives I purchased appearing smaller than the rest.  I know it was mentioned there is a utility to fix this so I can attempt that. 

How many drives are you running?  At 11 I'm not sure how much moving 4 will help but I'm more than willing to attempt it to fix the slowness.  Do you have an estimate on how fast your parity calcs are?

The reason I was looking at a new motherboard was to alleviate all bandwidth issues on the controller side, either by using PCI-X slots or using onboard.  I still wouldn't mind getting some recommendations on a fast board with lots of SATA connections or PCI-X slots.

Most P45/X48 boards have 10 sata ports provided by the ICH10R southbridge. Or you can get a Supermicro X7SLA-H-O, it's a Atom-motherboard with two PCI-e x8 ports (one running at x4). Add two Adaptec AAR-1430SA controllercards ($99 each) and you have great perfomance.

http://lime-technology.com/forum/index.php?topic=2697.0

Peter

I will attempt moving my drives to the 4 onboard sata ports but I ran into HPA issues with the new 1.5TB drives I purchased appearing smaller than the rest.  I know it was mentioned there is a utility to fix this so I can attempt that.

I agree with BubbaQ.

I would suggest at the very least.

Move the parity drive to the motherboard port and resolve the HPA issue.

It probably won't make parity checks much faster, but it will help in day to day operations where parity is updated.

I saw an immediate improvement moving my parity drive from a 1TB drive to a 1.5TB drive.

So if yours is on the fastest port possible, you should see some improvement.

I realize that size is not the issue here, but the 1.5TB drives are very fast and if you are using one as parity on the PCI bus then there is a severe bottleneck.

http://lime-technology.com/forum/index.php?topic=2697.0

 

Peter

 

This looked like a really nice board albeit rather expensive.  I do love the idea of buying an IPMI card to do kvm over ip.  It pretty much has everything I want but might be outside my price range. 

Most P45/X48 boards have 10 sata ports provided by the ICH10R southbridge. Or you can get a Supermicro X7SLA-H-O, it's a Atom-motherboard with two PCI-e x8 ports (one running at x4). Add two Adaptec AAR-1430SA controllercards ($99 each) and you have great perfomance.

It seems to me that the ICH10R boards all have a couple weird ports that can only be used in a raid array.  So the actual usable would be 8 ports and that opens up a bunch of other boards that could be candidates.  Maybe the GIGABYTE GA-EP45-UD3P?

http://lime-technology.com/forum/index.php?topic=2721.0

As far as the Atom board is concerned I think it might be a nice configuration if I didn't already own two controllers.  Power was one of the things I paid close attention to with my current build and this would be even better.

I will attempt moving my drives to the 4 onboard sata ports but I ran into HPA issues with the new 1.5TB drives I purchased appearing smaller than the rest.  I know it was mentioned there is a utility to fix this so I can attempt that.

 

I agree with BubbaQ.

 

I would suggest at the very least.

Move the parity drive to the motherboard port and resolve the HPA issue.

It probably won't make parity checks much faster, but it will help in day to day operations where parity is updated.

I saw an immediate improvement moving my parity drive from a 1TB drive to a 1.5TB drive.

So if yours is on the fastest port possible, you should see some improvement.

 

I realize that size is not the issue here, but the 1.5TB drives are very fast and if you are using one as parity on the PCI bus then there is a severe bottleneck.

 

I will make the change this evening and see what kind of performance increases are gained.  I have noticed the 1.5TB drives are faster than the old 400GB Samsung drives they replaced.  I should be able to pull one of the controller cards as well.  Actually, I'll need to because that was one of the issues why I didn't use the onboard in the first place.  I can't use them with both controllers installed since they sit to close above the onboard ports. 

I will attempt moving my drives to the 4 onboard sata ports but I ran into HPA issues with the new 1.5TB drives I purchased appearing smaller than the rest.  I know it was mentioned there is a utility to fix this so I can attempt that.

You may decide to move to another board any way, but I would like to say that the Gigabyte HPA issue is not a blocker, just something to work around.  Since it is so determined to "help you", by saving a copy of the BIOS to an HPA on one of the first onboard drives, just let it.  The HPA is minuscule, practically invisible because it is so tiny.  The only real problem for us is when the board tries to put it on the parity drive, which results in the 'size too small' issue.  So always make sure the parity drive is connected to the last SATA port (or IDE connector), and that other drives are connected on the first ports or connectors.  That way, the HPA is installed on drives where it won't matter, and it won't try to install it on any other drive, once it can find it on one or more of the first drives it checks.  As far as I can tell, it appears to check for and install the HPA on the SATA or IDE ports in order, from the first one to the last.  I have never yet heard of it using one of the ports on a controller card, which is why that is often suggested as an alternative place to connect the parity drive.

By the way, attempting to remove the HPA from a drive connected to a Gigabyte onboard port may be futile, as it will probably want to put it right back on a later boot.  Better to just live with it.  You lose essentially nothing, except having to be careful where you connect the parity drive.  It really does not matter if this HPA is installed on any other drive, but the parity drive.

If a user does find it on their parity drive, then it will have to be removed.  But first you will need the Gigabyte board to install an HPA on another drive or drives, where it will be found first.  So temporarily remove the parity drive, and make sure that an HPA is there on other drives located on the first ports (may take one or more reboots), then re-install the parity drive on the last port possible, and run one of the tools to remove the HPA from it.

The HPA topic should be a good place for all HPA-related info, but I can't seem to get around to finishing it.  It is still missing a number of important links.

So always make sure the parity drive is connected to the last SATA port (or IDE connector),

I would recommend against putting parity on the last IDE connector.

A. IDE is on a PCI bus.

B. If you have more the one drive on an IDE cable, only one drive can be communicated with at a time while the channel is busy.

IDE is not like SCSI where the CPU sends a command and disconnects from the bus while the command is active.

When there is an outstanding command other commands to the same drive or to any other drive on the cable must waituntil the first is finished. This is why a raid array on consecutive P-ATA cables can be problems (I've had my fair share of this).

If a person must use a P-ATA and it cannot be the first, put it on the second IDE channel as the Master with no other drive on the cable. (If performance matters).

So always make sure the parity drive is connected to the last SATA port (or IDE connector),

 

I would recommend against putting parity on the last IDE connector.

A. IDE is on a PCI bus.

B. If you have more the one drive on an IDE cable, only one drive can be communicated with at a time while the channel is busy.

IDE is not like SCSI where the CPU sends a command and disconnects from the bus while the command is active.

When there is an outstanding command other commands to the same drive or to any other drive on the cable must wait until the first is finished. This is why a raid array on consecutive P-ATA cables can be problems (I've had my fair share of this).

 

If a person must use a P-ATA and it cannot be the first, put it on the second IDE channel as the Master with no other drive on the cable. (If performance matters).

Thanks for clarifying.

When I was researching this Gigabyte HPA issue, I found the first complaints about it involved IDE based systems, so I added the parenthetical comment about IDE connectors above, thinking only of all SATA or all IDE systems.  If a user has nothing but IDE drives, then they should follow the advice above.  But adding a SATA controller and drive(s) would be better.  If a user with IDE drives has any SATA ports at all, even older and slower ones, they should try to use a SATA port for the parity drive, for the reasons given above.

Wow, so I dropped my parity calculation time from 32 hours to 17.25.  Starting out the parity ran at around 15-16MB/sec and then after getting past the 750GB drives halfway through it jumped to about 53MB/sec since then it only had 2 1.5TB drives on the pci bus to contend for bandwidth instead of 7 total.  So if my calculations are correct I'm getting about 23.1MB/sec for my parity speed up from 11MB/sec. I will be replacing the 750GB drives since these 1.5TB drives have enough space and then I should be down to 6 drives total.  This should increase my speed due to that I'm only getting around 106MB/sec over the pci bus. This puts me at 106/2= 53MB/sec for the entire run. After running the numbers I should finish a parity check in 7.67 hours.

Now this has me thinking though, should I replace just my supermicro sata adapters then as they are the cause of my bandwidth issues?  Ideally I would like to have a four port SATA adapter since I have two 1.5TB drives that will be added to my array after the RMA.  The issue is, with my board I'm limited by the expansions slots.  I have a PCI-E X16, X1 and two regular PCI slots.  Only one PCI is being used now for my AOC-SAT2-MV8 8-port controller.  I've found some good 4-port cards but they are PCI-E x4 or x8 cards.  My concern is that there is no guarantee that the x16 slot will allow me to use it for anything but video.  I've seen many people report this for other boards not specifically mine.  I could use the PCI-E x1 slot but most cards are two port.  From what I've gathered it appears that you can get upwards of 500MB/sec over PCI-E 2.0 per lane so an X1 card would be just fine for 4 drives if it reached anywhere close to that number.  I'm sure it varies by mb chipset as well as the sata controller chipset.  I've include a link to a two-port below that might fit the bill and would allow me to test on the x16 slot without investing too much money.

http://www.monoprice.com/products/product.asp?c_id=104&cp_id=10407&cs_id=1040702&p_id=2530&seq=1&format=2

Thanks everyone for your help.

Did you try moving your parity drive onto the motherboard port?

If it's on the PCI bus, it is hampered.

The 4X Adaptec 1430SA will be fine for 4 ports.

A 1X SIL3132 will be fine for 2 more ports.

From what I've gathered it appears that you can get upwards of 500MB/sec over PCI-E 2.0 per lane so an X1 card would be just fine for 4 drives if it reached anywhere close to that number

This is bi-directional  250MB/s in one direction, 250MB/s in the other.

Still you have to consider multiple drives can be supported on a PCI bus just fine with 133MB/s.

So the PCI 1X in practice could support it too.. Just not with every drive running full speed ahead.

I think someone modded their PCI port to allow installation of the 1430SA (4x) in a 1X slot.

They said it was successful. (not saying you should do this).

Did you try moving your parity drive onto the motherboard port?

If it's on the PCI bus, it is hampered.

Yes it is now connected to the onboard sata controller along with three other 1.5TB Seagate drives.  This is what has greatly increased my speeds.

The 4X Adaptec 1430SA will be fine for 4 ports.

A 1X SIL3132 will be fine for 2 more ports.

I think the Adaptec would be the best fit if I can either get confirmation that you can use the pci-e x16 port.  The SIL3132 I will probably purchase to do my own testing on the x16 port as well as a temporary solution with 2 drives on the AOC-SAT2-MV8 adapter and two on the SIL3132.  This should keep me around 50MB/sec throughout the parity check.  This all hinges on me removing the five 750GB drives and running 8 1.5TB drives total.

 

From what I've gathered it appears that you can get upwards of 500MB/sec over PCI-E 2.0 per lane so an X1 card would be just fine for 4 drives if it reached anywhere close to that number

This is bi-directional  250MB/s in one direction, 250MB/s in the other.

 

Still you have to consider multiple drives can be supported on a PCI bus just fine with 133MB/s.

So the PCI 1X in practice could support it too.. Just not with every drive running full speed ahead.

 

My testing showed I was consistently hitting around 106-108MB/sec over the PCI bus.  I gauged this by noticing that with 7 drives running on the PCI bus I was at around 15-16MB/sec parity check and then when it dropped to two drives after the 750GB mark I was hitting around 53-54MB/sec.  I would assume that like all things there is overhead on the PCI bus so you never get close to 133MB/sec.  This is my reasoning for looking at moving away from anything tied to the PCI bus on my motherboard.  The adaptec card really would be the best fit if it works in my motherboard.

Most P45/X48 boards have 10 sata ports provided by the ICH10R southbridge. Or you can get a Supermicro X7SLA-H-O, it's a Atom-motherboard with two PCI-e x8 ports (one running at x4). Add two Adaptec AAR-1430SA controllercards ($99 each) and you have great perfomance.

I found the Adaptec 1430SA SATA2 RAID 4-Port PCI-E Controller Card for $85.00 at ewiz (http://www.ewiz.com/detail.php?p=AD-1430S&c=fr&pid=1a00e0e026973ce0f633839874b6a45234023502bda8bbd9094a71b158a1ddb7), but is this the cheapest 4 port PCI-e cards around?  Sorry if I am off topic, but I need a couple of 4 port cards, but the IRS nailed me good this year so any discounts I can find would be very helpful...

Most P45/X48 boards have 10 sata ports provided by the ICH10R southbridge. Or you can get a Supermicro X7SLA-H-O, it's a Atom-motherboard with two PCI-e x8 ports (one running at x4). Add two Adaptec AAR-1430SA controllercards ($99 each) and you have great perfomance.

 

I found the Adaptec 1430SA SATA2 RAID 4-Port PCI-E Controller Card for $85.00 at ewiz (http://www.ewiz.com/detail.php?p=AD-1430S&c=fr&pid=1a00e0e026973ce0f633839874b6a45234023502bda8bbd9094a71b158a1ddb7), but is this the cheapest 4 port PCI-e cards around?  Sorry if I am off topic, but I need a couple of 4 port cards, but the IRS nailed me good this year so any discounts I can find would be very helpful...

http://cgi.ebay.com/Adaptec-2241000-R-1430SA-4-Port-PCI-Express-x4-Serial_W0QQitemZ250417766359QQcmdZViewItemQQptZCOMP_EN_Networking_Components?hash=item3a4e0fdfd7&_trksid=p3286.c0.m14&_trkparms=65%3A12|66%3A2|39%3A1|72%3A1205|240%3A1318|301%3A1|293%3A1|294%3A50

http://cgi.ebay.com/NEW-Adaptec-1430SA-4-Port-Serial-ATA-RAID-Controller_W0QQitemZ350192300005QQcmdZViewItemQQptZLH_DefaultDomain_0?hash=item5189166fe5&_trksid=p3286.c0.m14&_trkparms=65%3A12|66%3A2|39%3A1|72%3A1205|240%3A1318|301%3A1|293%3A1|294%3A50

These have free shipping.