N54L + N36L = N90L 14 Drive Macroserver

Hi all

Recently started looking into how to utilise my spare N36L microserver since i had upgraded to the N54L, see these posts in neilt0's HP Microserver thread for a little back ground info

http://lime-technology.com/forum/index.php?topic=11585.msg283759#msg283759

http://lime-technology.com/forum/index.php?topic=11585.msg284288#msg284288

So since that I have been thinking more about it and wanted to try and understand if my plan to gut the old N36L and use it as a disk store for extra drive storage for the N54L would actually work. Having the N36L coupled to the N54L... hence a N90L Macroserver

I certainly understand there are far easier, more elegant solutions but since when was that ever any fun? Plus i want to make use of the N36L and try and just have one unraid instance running.

Quick overview:

Masterplan:

Motherboard adapter plate to mount two further 3.5" SATA drives in the N36L:

Planned mounting location of the two sata drives in the N36L motherboard tray (a base plate will be created to mouth the drives to the motherboard tray itself)

The details

N36L

• Remove the motherboard, PSU and rear fan from the N36L leaving a bare shell with just the 4 drive bay cage and back pane with the HP mini SAS cable
   
  
• Use the original HP mini SAS cable to connect the 4 drives in the 4 bay cage to PORT1 of a SAS card installed in the N54L (needs confirmation this will work as I read all sorts about reverse SAS cables blah blah blah...)
  
• Mount two further 3.5" drives in a custom base plate on the motherboard tray giving a total of 8 drives in that N36 chassis (4 in the regular drive bays, 2 up top in the 5.25" space and the 2 down at the bottom in the motherboard space). The drives will be secured to the place via the four mounting points under the drives. The adapter plate itself will secure to the motherboard tray via two screws used to secure the original motherboard to the tray (the other two holes are locating holes which wont need any screws but help keep the tray aligned and steady.
   
  
• Use Mini SAS 36pin (SFF-8087) to 4x SATA Cable to connect the remaining 4 SATA drives to PORT 2 of a SAS card installed in the N54L (again need to confirm the cable choice is correct)
  
• Run a fanless 200W Pico PSU (Z3-ATX-200) in the N36L chassis to power all the drives. It will have a trigger wire feed from the N54L so when the N54L is powered on, the drives in the N36L will spin up (well that's the theory anyway!)
  
• Run a Norco or similar quiet 120mm fan in place of the original HP rear chassis fan. Since it wont be run off the motherboard it will probably just be set to around 900rpm constant and run off molex (will have to play with noise / cooling balance to see what works best as I live in Brisbane, Australia and it gets kinda toasty here)
  

N54L

• Remove the PSU from the N54L chassis.
  
• Run a fanless 160W Pico PSU (PSU-160-XT) in the N54L chassis. It will have a trigger wire feed going to the N36L (see above for info)
  
• Install a two port SAS card in the PCIe x16 slot (Really need to know which SAS card is suitable for my needs. Would the IBM BR10i be a good choice? If possible I'd also love to have a card which has at least two mini SAS ports on the rear pci slot for easy connectivity (that way the two mini SAS cables from the N36L would plug into the ports on the back of the pci-slot rather than having to feed them through a hole on the N54L chassis to ports on the SAS card. Or is something like this my only option (http://www.span.com/images/products/i/lightbox/ira-ps2.jpg)
  
• Future consideration : Run a quiet/silent 120mm fan in place of the original HP rear chassis fan. I'll want one which wont cause any powerdown issues when on the idle PWM signal off the motherboard.Problem is i cant find the suggested fan from the silentreview article : SY1225SL12HPVC.
  

Things to consider / unknowns:

• Will sending a 'powerdown/shutdown' command from unraid on the N54L turn off the power to the drives on the N36L??? Not sure if the trigger used to turn ON the PICO 200W PSU in the N36L will also turn the power OFF?
  
• Will the 200W Pico PSU be able to fire up 8 4TB drives in the N36L (initial calculations tell me it should... but that is yet to be seen). I plan on supplementing my four Hitachi 4TB drives living in the N54L with another 10 in the future [2 more in the N54L and 8 in the N36L). For now I don't think ill have any issues as at most ill have one 4TB Hitachi in the N36L and the rest will be 2TB's.
  
• Hopefully the 160W pico will be adequate for the N54L. Currently the 150W HP Power supply in there is running 4x4TB + 2x2TB drives without any issues - so i'm hoping the extra 10W is enough to cover the extra juice needed for the SAS card and the extra draw of two 4TB drives.
  
• Trying to source a suitable AC-DC power brick to feed the 200W pico psu is a little difficult. So far one i can see is 192W http://www.mini-box.com/12v-16A-AC-DC-Power-Adapter. There are some folks who have used a DA-2 MK394 which is a Dell 220W brick, but this needs some modifications to the end connector to make it usable
  
• Should I opt for a SAS card with more than two ports for expand ability options in the future? As mentioned, the ideal is having two external rear mini SAS ports on a half height plate, however having some internal SAS ports would be a bonus (as long as the one PCIe x16 would be able to handle more than the load of 8 drives?)
  

Immediate Questions I need help on

• Which SAS card best meets my needs?

[Dell 6Gbps SAS HBA]

• Will the original HP mini SAS cable connect directly to the SAS card, or am i not able to use it at all - which would be a pain and make cabling a lot trickier as id have to remove the backpane connectors from the 4 drive cage

[YES it seems like it will via a SFF-8087 to SFF-8088 adapter card]

• Will this mini SAS cable work to connect the other drives from the N36L to the SAS card? (http://www.ebay.com.au/itm/Mini-SAS-36pin-SFF-8087-to-4x-SATA-Cable-90cm-/251326590405?pt=AU_CablesConnectors&hash=item3a843b6dc5&_uhb=1)

[Went ahead and ordered the 50cm version of it - will find out if it works when it arrives!]

• what critical aspect am I overlooking!
  

Thanks in advance for your time and assistance!

UPDATES

[28/JAN/14]

v0.2 DRAFT - Incorporated feedback and advice from Stanza (from OCAU forum : http://forums.overclockers.com.au/showpost.php?p=15865123&postcount=9577)

Ordered the following items:

1 x Mini SAS 36pin (SFF-8087) to 4x SATA Cable 55cm [ebay item 251382719073 sold by seller tiktronic]

2 x HP 407344-001 0.5M External Mini SAS Cable SFF-8088 / SFF-8088 Connectors [ebay item 251389991511 sold by seller easysell119]

1 x Lycom ST187 Dual SFF8088 to SFF8087 Low profile PCIe Host Adapter [ebay item 360786025472 sold by seller scancomuters_int]

Still trying to find a decent price for the Dell SAS card - hopefully i'll be able to grab a cheap one soon

Also the SupermicroCSE-PTJBOD-CB1 JBOD card is proving to be quite difficult to source here in Australia. I've seen it being sold in the US, but the delivery charge kinda blows the price out. Anyone know where to source these locally for a decent price?

One reason I run both an N36L and an N54L is if the N54L dies (e.g. the motherboard), I can put the drives in the N36L and carry on as usual within a  few hours. I have 8 drives attached to my N54L and it wasn't too hard to do -- how many drives in total do you plan on attaching to the N54L and what is your total capacity (excluding parity drive)? I'm at 24TB with mine.

If you do go ahead, I'd be interested in seeing the results!

Cheers,

Neil.

You can slave one PSU from another.  There are even cables you can get to make it easy, I believe.  The On and Off operations should work - just try it with no drives connected to see what powers up and down.

What you should do, and I highly recommend, is provide addiional wires for each 0 volts connection from one supply outlet to the other.  For safe and reliable operation of the drives you need a good clean power supply in the second unit that has the same electronic signal ground as the first supply and the motherboard and SATA controllers, and that means close electrical coupling to eliminate elecrical noise.  So fit one wire (at least as thick as the existing power supply wires) from each black wire on the master PSU to the same wire on the slave PSU.  This will mean that you have many 0 volt wires in parallel with one another than that means lower electrical inductance which is what is needed.

that's great - just the kind of info I needed. I've updated the diagram to reflect the fact that ill run 8 separate ground wires between the two PSU's (got lazy and didn't draw them as 8 separate lines  but you get what i mean!)

Any idea on the SAS card or cables? I'm struggling with the whole SFF-8070, SFF-8088 and target vs host as well as trying to know how to wire it all up neatly. As far as my googling powers go, I dont think there is a two port external mini SAS half height PCI bracket (they all seem to come with the full height brackets). I guess i could use both of the half hight slots and just put in two single slot exernal SAS connector brackets???

Neil - ill certainly keep a photo record of the progress and update this thread. As for the total drive storage, i'm looking at initially doing :

N54L:

6 x 4TB drives in the N54L (1 being the parity)

N36L

1 x 4TB

6 x 2TB drives

That will give me 36TB of usable space which will certainly give me enough room for at least another year. Ultimately Ill be looking to swap in the Hitachi 4TB deskstar drives when prices are good or deals are on. Final spec with both servers stuffed with 4TB Hitachi's would be 52TB useable space 

Would something like this help?

http://www.pc-pitstop.com/sas_cables_adapters/AD8788LP.asp

I did something similar: My N40L was reaching its capacity (on 6 drives) and instead of migrating everything to a new hardware box, I expanded with a Norco 4220 I had sitting around... I'm up to a full 25 drives now.

Let me know if you want more details... off the top of my head, the Norco has the following:

Supermicro CSEPT-JBOD-CB1 Power supply controller

Supermicro CBL-0168L-LP 2-port SAS backplane

Intel RES2SV240 expander

and in the N40L

IBM M1015

Supermicro CBL-0168L-LP 2-port SAS backplane

How long does a parity check take? 

How long does a parity check take? 

N54Ls are not slow. With 8 drives (including 3 on a a 1430SA PCI-E card), my parity check starts at around 120MB/sec IIRC and doesn't drop off that much.

Some people have mounted the MicroServer mobo in a rack case: http://forums.overclockers.com.au/showthread.php?p=15366743

Or tower case (can't find the link). Then you just need to add a larger PSU a PCI-E HBA or two and a bunch of drives. I've considered doing this, as I have all the kit, but I like the MicroServer form factor and don't want to manage a lot of drives ever again. I sold off about 40 smaller drives and just bought bigger (4TB) drives. There's also a greater likelihood of multiple drive failure the more drives you have per array. My plan is to end up with 7x4TB drives (already in place) in one server and 6x4TB drives in the second.

If I need more space in future, I'll just upgrade to larger drives -- 6TB, 8TB or whatever the mfrs come up with.

just a quick update to the original post based on some further info and advice...

Got some items on order so will update the thread when the goodies arrive!

So a few of the parts I ordered have started to come through but unfortunately the DELL SAS card isnt here yet so i have not had the chance to make too much progress.

In the mean time i wanted to ask (moreso reconfirm) if the following migration / reconfiguration of disks is going to be possible. Firstly, here is a quick summary of how things are currently setup in my N54L (running 6 drives):

As you can see i have FIVE 4tb drives and ONE 2tb drive.

As part of the expansion I want to reconfigure some drives and the shares. Here is what i was thinking to do (take note of the serial numbers to keep track of which disk is going where!)

So the jist of it is that I want to replace the current disk3 and disk4 with two new 4tb disks to expand SHARE1 I want these disks physically housed in the N54L. The 'old' disk3 and disk4 move over to the N36L housing and become disk 6 and disk7 respectively.

So am i correct in thinking I can achieve this by following these steps:

1) Perform parity check on array

2) Take array offline and un-assign disk3 and disk4

3) Perform all the hardware changes and physical movement of drives

4) Fire up unraid and at the drive configuration screen:

(i) assign the two new 4tb drives the disk3 and disk4 slots

(ii) assign the 'old' disk3 [b2SS] to disk 6

(iii) assign the old disk4 [D2BG] to disk 7

5) Modify share configuration in unraid to include disk6 and disk7

6) Start array

7) Perform parity check which will rebuild the parity since it may have been invalidated???

Part of me thinks after step 1 I actually need to remove all the unraid drive configuration so that unraid doesn't start thinking disk3 and disk4 took a dump and need recovering? Is this correct, and if so do i just run the initconfig command?

Just looking for some guidance as losing any data due to a silly oversight on my part is going to drive me bonkers!

thanks folks

anyone able to take a quick look and confirm / deny my plan of action isn't going to cause me to have to down an entire bottle of whiskey after trashing all my data? 

thanks