Can someone explain the connectivity IBM ServeRAID M1015 & Expander Card RES2SV2

Actually,

I need to understand the connectivity between these three components:

IBM ServeRAID M1015

Expander Card RES2SV240

Norco RPC-4224 Chassis

For a system with 24 HDDs.

Kindly mention each end's connector type, the cables, from and to.

thanksss

I just built a system with these exact components.

Here goes

Ok so the M1015 should be flashed to IT mode using these directions.

http://www.servethehome.com/ibm-serveraid-m1015-part-4/

Specifically this:

Convert LSI9240(IBM M1015) to a LSI9211-IT mode

Type in the following exactly:

megarec -writesbr 0 sbrempty.bin

megarec -cleanflash 0

<reboot, back to USB stick >

sas2flsh -o -f 2118it.bin -b mptsas2.rom (sas2flsh -o -f 2118it.bin if OptionROM is not needed)

sas2flsh -o -sasadd 500605bxxxxxxxxx (x= numbers for SAS address)

<reboot>

Done!

Once the M1015 is in IT mode you can hook up the expander. The expander uses a SFF-8087 cable to connect to the M1015.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816133034

Once connected you are able to connect the other 5 ports on the expander to the backplane on the norco using another 5 SFF-8087 cables.

This will connect all but one row of disks to your machine. To connect the last row you will use another SFF-8087 cable to the other port on the m1015 and the last port on the backplane of the Norco.

So in total you need 7 sff-8087 cables to connect everything.

Here is a crappy phone pic to give you an idea. Missing the 7th cable to connect the extra port on the m1015 to the backplane. Also ignore the power supply. Just there to test the motherboard, etc.

I just built a system with these exact components.

Impmonkey, thanks for the info...can you elaborate more on how "stable"/"usable" this combo is?

I've seen comments that the intel expander fires alarms quite frequently, if not connected to a "matched" controller

with the right sensors.

thanks Impmonkey

the expander doesn't need to plugin to the motherboard? that's what I see from your picture.

Also, the IBM M1015 connects to the expander only with 1 cable?

Well I have had it up and running for about two weeks now without an alarm. The intel is an lsi chip and so is the M1015 so maybe that is why they play nice.

The expander just needs power. It can either get it through the PCI-e connection or the molex connector. In this case i am using a mini-itx board with only a single slot thus I am powering it through a molex connector on the powersupply.

Only one cable needs to connect the card to the expander. This leaves the other slot available to the used on the backplane of the norco.

I will try to get some clearer pictures and start a build log soon.

LMK if you have anymore questions.

impmonkey,

well explained, thank you.

Current Build Photo with 32GB Ram, E-1240, 2x M1015, Expander, Corsair Pro SSD's, and custom power cables

here is how I have mine at the moment.

it is in a PCIe slot only because the top power plug fried i have an RMA out for my card right now.

I have 1 m1015 to the expander with 4 ports off the expander to the backplanes. 1 port straight from the m1015 to the backplane. (the second m1015 is for my ZFS array  and yes, one of my backplanes does not match...)

There is a lot of talk in the Atlas thread about the expander configurations.

there is some basics here also. http://lime-technology.com/forum/index.php?topic=16769.msg153296#msg153296

there are two ways t power it. PCIe or molex.. (not both at once)...

There are two ways to configure the SAS..

you can run 16 fast ports but using 2 of the m1015 ports jumped to the expander.

or 20 not so fast ports with 4 full speed ports... by jumping only one port.

as far as issues with the two cards.. they are both LSI.. intel sells a card that matches the m-1015 and its on the HCL as is the the LSI card from LSI that matches it. (I just talked to Intel's tech support they said it is fine. they just are not happy the mobo is not intel )

Current Build Photo with 32GB Ram, E-1240, 2x M1015, Expander, Corsair Pro SSD's, and custom power cables

 

here is how I have mine at the moment.

it is in a PCIe slot only because the top power plug fried i have an RMA out for my card right now.

I have 1 m1015 to the expander with 4 ports off the expander to the backplanes. 1 port straight from the m1015 to the backplane. (the second m1015 is for my ZFS array  and yes, one of my backplanes does not match...)

 

There is a lot of talk in the Atlas thread about the expander configurations.

there is some basics here also. http://lime-technology.com/forum/index.php?topic=16769.msg153296#msg153296

 

there are two ways t power it. PCIe or molex.. (not both at once)...

 

There are two ways to configure the SAS..

you can run 16 fast ports but using 2 of the m1015 ports jumped to the expander.

 

or 20 not so fast ports with 4 full speed ports... by jumping only one port.

 

 

as far as issues with the two cards.. they are both LSI.. intel sells a card that matches the m-1015 and its on the HCL as is the the LSI card from LSI that matches it. (I just talked to Intel's tech support they said it is fine. they just are not happy the mobo is not intel )

Hi johnm,

Yah, the first backplane (the top) doesn't match, why those 4 light blue cables? they are connecting where on the backplane?

and about the power, you have one power supply, and you are connecting each backplane to both power, i thought that's made if you have redundant power supplies (two). or am i wrong? i read it in one of the threads.

and the power cables look nice, you made them yourself?  I'm thinking to get the ones you suggested from svc.com

The top backplane is from a 4024.

It was what I used for my rmd and datastore drives in  esxi. Now that whole backplane is a zfs raidz going to my second n1015

I ran 2 sets of wires due to the fact that they are not the. Best gauge. You are correct its normally for redundant psu.

Sent from my iPhone using Tapatalk

I was wondering if anyone could tell me how impmonkey ran his M1015 without being plugged into PCI-e on the motherboard?

The mobo I'm using has only one PCI-e to use for my M1015, but am just now acquiring an HP SAS Expander and I have to decide how to upgrade to allow me to use it.  Not having to upgrade my mobo would be great...

...the M1015 is plugged into the motherboard....you can see the flex-riser card on the picture....would not make sense to not plug it in 

Edit: in order to power the expander, it depends on the model...the Intel RES2SV240 offers a PCIe and Molex Option at the same time to choose from.

....

There are two ways to configure the SAS..

you can run 16 fast ports but using 2 of the m1015 ports jumped to the expander.

 

or 20 not so fast ports with 4 full speed ports... by jumping only one port.

...

Hi,

can you maybe share some more details:

What is fast, not so fast and full speed? (What is the differences in Parity check speed and/or copying to/from the array?)

Thank you

....

There are two ways to configure the SAS..

you can run 16 fast ports but using 2 of the m1015 ports jumped to the expander.

 

or 20 not so fast ports with 4 full speed ports... by jumping only one port.

...

 

Hi,

 

can you maybe share some more details:

What is fast, not so fast and full speed? (What is the differences in Parity check speed and/or copying to/from the array?)

 

Thank you

...it's all about bandwidth shared between the different ports.

Assume that a M1015 alone, when seated in a PCIex8 slot, can drive 8 ports at 6GB/sec each...consider this being the "full speed" scenario.

The M1015 has two connectors, each offering 4 ports.

Also assume that the expander offers to drive each port with good bandwidth of 6GB/sec (or things/maths get more complicated when there is another bottleneck in the path)

When you connect a 24-port expander with one M1015 connector, the connection consumes 4 ports on the expander, leaving 20 ports.

However, the available bandwith is that of the original 4 ports, driven by the M1015 connection to the expander.

In this scenario, referred as "not so fast", the available 20 ports share 4x6GB/s physically....so a disk connected to one port ends up receiving 1.2GB/sec (24/20 = 6/5 = 1.2 GB/sec).

This is still good enough for a green 5.4k drive....maybe tight for a 7k drive.

The fast scenario is to use both connectors of the M1015 with the expander, consuming 8 ports, leaving 16 ports out of 24 ports available.

Here a single disk receives 3GB/sec (8x6/16=3 GB/sec) of bandwidth, what equals a standard S-ATA II connection.

However this scenario is normally used for multipathing, where two M1015 serve the same expander with one connection each...it is meant for redundancy, if one card fails.

Edit: corrected my maths

...the M1015 is plugged into the motherboard....you can see the flex-riser card on the picture....would not make sense to not plug it in 

 

Edit: in order to power the expander, it depends on the model...the Intel RES2SV240 offers a PCIe and Molex Option at the same time to choose from.

Ok next question...what's a "flex-riser card"?  All I saw was an M1015 plugged into something besides a mobo and was intrigued.

Ok next question...what's a "flex-riser card"?  All I saw was an M1015 plugged into something besides a mobo and was intrigued.

...it is a RiserCard, based on a flexible ribbon cable.

It allows you to physically move the slot location to another position inside your case.

See these for examples: http://www.google.de/search?q=flex+riser+card&tbm=isch&tbo=u&source=univ&sa=X&ei=jrz4UezJOoSmO5zBgRA&ved=0CEUQsAQ&biw=1920&bih=886

Interesting idea, I wasn't aware of these, but I don't think it helps my particular situation.  Thanks.

Interesting idea, I wasn't aware of these, but I don't think it helps my particular situation.  Thanks.

...AFAIU you're short of a slot as you'll be in need to power your expander.

Note that the expander just needs power, no data signal from the mobo...data is coming through the 8087 connection from your M1015.

As I already said, you should search for an expander that allows to be powered by a Molex-Connector ...you then don't need a slot in the mobo for it at all.

The Intel based Expanders do that....the RES2CVxx models only have power via molex, only the RES2SV24 has a PCIe connector for power as an alternative.

If you end up with an expander that only can be powered by a Slot connector, look for a (flex-)riser that converts between slot types (maybe your mobo has one PCI(-X) left?

I *think* that I've seen them. As you only need power and not good data connection from it you don't need to worry too much about its functuonality.

Correction, this may indeed work for me.  That looks like an x1 to x16 adapter, and I do have a spare x1 slot.  Ordered one off ebay for a bucks, hope it works for me.  Is anyone else using these flex card risers?  I did read that some risers may not properly handle all the circuitry correctly, such as "PWRGD".

Ah, posting at the same time.

I'm going with the HP expander because I got a fantastic deal on one yesterday ($80 brand new, yay me).  Otherwise I would probably be going with the Chenbro CK23601 or perhaps the Intel, since either could be powered by Molex.

I will try the x1 to x16 adapter.

FWIW, the 1x to 16x adapter I got works fine, powers the HP expander without issue and I didn't have to buy a mobo with more PCIe slots.  The only disadvantage I see is that the card is not as structurally sound without a solid attachment on the bottom.

I just built a system with these exact components.

Here goes

Ok so the M1015 should be flashed to IT mode using these directions.

http://www.servethehome.com/ibm-serveraid-m1015-part-4/

Specifically this:

Convert LSI9240(IBM M1015) to a LSI9211-IT mode

Type in the following exactly:

megarec -writesbr 0 sbrempty.bin

megarec -cleanflash 0

<reboot, back to USB stick >

sas2flsh -o -f 2118it.bin -b mptsas2.rom (sas2flsh -o -f 2118it.bin if OptionROM is not needed)

sas2flsh -o -sasadd 500605bxxxxxxxxx (x= numbers for SAS address)

<reboot>

Done!

 

Once the M1015 is in IT mode you can hook up the expander. The expander uses a SFF-8087 cable to connect to the M1015.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816133034

Once connected you are able to connect the other 5 ports on the expander to the backplane on the norco using another 5 SFF-8087 cables.

 

This will connect all but one row of disks to your machine. To connect the last row you will use another SFF-8087 cable to the other port on the m1015 and the last port on the backplane of the Norco.

 

So in total you need 7 sff-8087 cables to connect everything.

Here is a crappy phone pic to give you an idea. Missing the 7th cable to connect the extra port on the m1015 to the backplane. Also ignore the power supply. Just there to test the motherboard, etc.

How can the m1015 still work as you have plugged it to a PCI-E x1 slot?

Still work but only bandwidth is limited?

I am thinking if I can plug my PCI-E x4 NIC card to PCI-E x1 slot. 

 

How can the m1015 still work as you have plugged it to a PCI-E x1 slot?

 

Still work but only bandwidth is limited?

 

I am thinking if I can plug my PCI-E x4 NIC card to PCI-E x1 slot. 

Because of the riser card, the M1015 is physically fit into a x8 or larger slot.

It would not fit into anything smaller (if not equipped with open ends).

Yes, bandwidth would be pretty limited, I am sure. (and using an expander with it is adding to the problem).

I am actually not sure if this would be a working, satisfying setup at all.

When I fist saw this post, I assumed that this is a x4 slot which is proven to work for a M1015 (and this is what the SM Atom-boards have).

What NIC is this? A single, or even Dual Port GBit maybe, should do fine in a x1 slot and still deliver the bandwidth.

 

How can the m1015 still work as you have plugged it to a PCI-E x1 slot?

 

Still work but only bandwidth is limited?

 

I am thinking if I can plug my PCI-E x4 NIC card to PCI-E x1 slot. 

 

Because of the riser card, the M1015 is physically fit into a x8 or larger slot.

It would not fit into anything smaller (if not equipped with open ends).

Yes, bandwidth would be pretty limited, I am sure. (and using an expander with it is adding to the problem).

I am actually not sure if this would be a working, satisfying setup at all.

When I fist saw this post, I assumed that this is a x4 slot which is proven to work for a M1015 (and this is what the SM Atom-boards have).

 

What NIC is this? A single, or even Dual Port GBit maybe, should do fine in a x1 slot and still deliver the bandwidth.

The ASUS Hummingbird should only have a PCI-E x1 Slot.

http://www.asus.com/Commercial_Servers_Workstations/Hummingbird/#specifications

My NIC card is Broadcom 5709 Dual Port NIC Card (PCI-E x4 port).

PCI-E x1 have around 500MB/s bandwidth. It would be really nice if that NIC card can run on the x1 port.