Custom Xeon Dual Core 24 Bay SATA Ebay Server


jochs

Recommended Posts

What did you replace the fans with?

 

I got mine a couple of days ago.  Finally had some time to play around with it.  As others have said, it's a BEAST, and a loud one at that.  So far noise seems like the biggest issue to me.  I cannot imagine walking into a datacenter with all of the ones on ebay running at the same time.

 

I picked up a pair of 5440s to put in it that should be arriving on Monday.  Today I'm hoping to get a chance to try vmware on it and see how the esxi passthrough works. 

Link to comment

These servers have SuperMicro's IPMI card built in. Anyone successfully used IPMI with these servers? If so, tell me about it :)

 

The IMPI card in my system is the AOC-SIMLC. I downloaded the driver CD image [ftp=ftp://ftp.supermicro.com/CDR_Images/CDR-SIMIPMI_1.13_for_SIM_IPMI.iso]ftp://ftp.supermicro.com/CDR_Images/CDR-SIMIPMI_1.13_for_SIM_IPMI.iso[/ftp] & burned it to a CD from my desktop Windows 7 system.

 

If the above link does not work, visit the SuperMicro webpage for the IPMI http://www.supermicro.com/products/accessories/addon/sim.cfm. Scroll down to the Download links for the specific card, where you will find "User's Guide", "Driver" & "Driver CD Image".

 

Then I hooked up an IDE DVD/CD drive to my Xeon UnRAID server, edited the BIOS to boot from the CD drive & booted from the burned disk.

 

It loads FreeDOS & tells you to press any key within 5 seconds to continue booting from the "floppy". I did not press anything the first time (the word floppy in reference to a CD distracted me) & it hung. I rebooted & pressed a key within 5 seconds this time & got a command prompt.

 

This is a good website https://www.ahead-it.eu/support/?/Knowledgebase/Article/View/33/0/supermicro-ipmi-configuration-utility--ipmicfg for the IPMICFG commands.

 

The ones I used:

 

reset IPMI to default settings:

ipmicfg -fd

 

list the current IP of the IPMI card:

ipmicfg -m

 

set the IP to something on my network:

ipmicfg -m 192.168.0.25

 

set the gateway/router:

ipmicfg -g 192.168.0.1

 

On my Windows 7 system, installed IPMIVIEW [ftp=ftp://ftp.supermicro.com/utility/IPMIView/]ftp://ftp.supermicro.com/utility/IPMIView/[/ftp]

If the direct IMPIVIEW link does not work (as I had to click through to accept SuperMicro's terms, try the right side of this page http://www.supermicro.com/products/nfo/IPMI.cfm under Resources, towards the bottom of Resources.

 

Once IPMIVIEW was installed & I had configured the server with the IPMICFG disk, I could use DISCOVERY to find my UnRAID server.

 

I mostly use it to power up my UnRAID server, as mine does not have a power button in the back above the 2 reset buttons (just a hole). Took me some time to figure out how to get the server on when I first got it. Ended up downloading the motherboard manual, finding the header pins on the board for the power button, then using my screwdriver to touch the 2 power pins to start it.

 

This model IPMI does not have the KVM capabilities (from what I have read) that some other models have, but with the remote power on, unmenu, shared drives & ssh, I seem to be able to do everything remotely that I need to do.

 

Today I'm hoping to get a chance to try vmware on it and see how the esxi passthrough works.

 

I read in one of the forums that this setup did not support hardware passthrough. If you manage to get it working, please post what you did. I plan to install VirtualBox on top of my UnRAID following this guide http://lime-technology.com/wiki/index.php/Installing_VirtualBox_in_unRAID

Link to comment

You did it the hard way :) I just downloaded the linux setup file from here:

ftp://supermicro.com/CDR-SIMIPMI_1.13_for_SIM_IPMI/IPMI_Solution/Utility/IPMICFG/Linux/ipmicfg-linux.x86.static

 

copied ipmicfg-linux.x86.static to the root of my flash drive that boots unraid.

 

Ran this:

./boot/ipmicfg-linux.x86.static

 

you can set your ip and mac address from there and you are good to go.

 

Thanks though :)

Link to comment

You did it the hard way :) I just downloaded the linux setup file from here:

ftp://supermicro.com/CDR-SIMIPMI_1.13_for_SIM_IPMI/IPMI_Solution/Utility/IPMICFG/Linux/ipmicfg-linux.x86.static

 

copied ipmicfg-linux.x86.static to the root of my flash drive that boots unraid.

 

Ran this:

./boot/ipmicfg-linux.x86.static

 

you can set your ip and mac address from there and you are good to go.

 

Thanks though :)

 

I tried creating a directory named /boot/opt/ipmicfg/ & copied the ipmicfg files there first. When I tried to run it, received this:

/boot/opt/ipmicfg# ./ipmicfg-linux.x86_64.static 
-bash: ./ipmicfg-linux.x86_64.static: cannot execute binary file

 

Tried all 4 files ipmicfg-linux.x86.static, ipmicfg-linux.x86_64.static, ipmicfg-linux.x86 & ipmicfg-linux.x86_64 with the same results. So that is why I went with the bootable CD.

Link to comment

 

Today I'm hoping to get a chance to try vmware on it and see how the esxi passthrough works.

 

I read in one of the forums that this setup did not support hardware passthrough. If you manage to get it working, please post what you did. I plan to install VirtualBox on top of my UnRAID following this guide http://lime-technology.com/wiki/index.php/Installing_VirtualBox_in_unRAID

 

What I've read has lead me to believe that it doesn't support VT-d unfortunately, so no passthrough.  I may have to the virtualbox route as well, although I've never had any luck getting the kernel right in the past.

Link to comment

I just got my L5420 Cpus.  I installed them with standard aluminum heat sinks (no fans yet).  Initial temps seem to be in the low 50s which isn't bad given the current set up...plus I've read these cpus are known to have questionable temp sensors in them.

 

Planned changes:

Switch the 60mm fans with quieter ones

Cable/wiring management

Install norco fan wall

Removing the side fans if the fan wall generates enough air

 

Overall I'm really happy with the case.  It's well built and has enough space that you can modify it to fit future needs.

Link to comment
  • 3 weeks later...

Putting this at the root flash drive allows me to shutdown and start the server (thanks!). But I'm missing something somewhere.

 

Now I can't get access to the unraid page (or telnet). Windows can't seem to find it and it isn't showing up in OS X. If I hook up a monitor and run ifconfig eth0 I have the right address.

 

I tried setting the ipmicfg -m to the ip that unraid has and setting it to something else. Every time I can get to the ipmi page but not the unraid page.

Is there a setting I need to change in the ipmi stuff to regain access to the unraid server?

 

 

You did it the hard way :) I just downloaded the linux setup file from here:

ftp://supermicro.com/CDR-SIMIPMI_1.13_for_SIM_IPMI/IPMI_Solution/Utility/IPMICFG/Linux/ipmicfg-linux.x86.static

 

copied ipmicfg-linux.x86.static to the root of my flash drive that boots unraid.

 

Ran this:

./boot/ipmicfg-linux.x86.static

 

you can set your ip and mac address from there and you are good to go.

 

Thanks though :)

Link to comment

I tried setting the ipmicfg -m to the ip that unraid has and setting it to something else. Every time I can get to the ipmi page but not the unraid page.

Is there a setting I need to change in the ipmi stuff to regain access to the unraid server?

 

You want UnRAID & IPMI to have separate IP addresses.

ifconfig eth0 should be the IP for UnRAID. Change IPMI to something else using ipmicfg -m x.x.x.x then reboot.

Hopefully you will be able to access UnRAID from the eth0 IP & the IPMI from the newly set IP.

Link to comment
  • 1 month later...

Just wanted to document the results I have seen from these ebay Xeon servers.  The 220w baseline power draw is my main issue with these units.  I would like to cut the power draw in half.

 

I will be comparing this to another server based on a Norco 4224 with updated components Intel i3 2100 SUPERMICRO MBD-X9SCM-F-O, SUPERMICRO AOC-SASLP-MV8, and 80+ Antec NEO Eco 620C power supply.  The 3 Xeon ebay units were purchased as backup units for this one.

 

1.  They are loud, but after unplugging the 2 60mm fans on the back, noise is only mildly objectionable.

2.  They sure suck the electricity.  Only plugging in 1 power supply and with no drives, it pulls 220w compared with 40w from the updated server.

3.  Plugging in all 3 of the 3 redundant power supplies boosts power draw to 260w.

4.  Otherwise they seem to work fine.

 

This Xeon is only a 65w processor, so why the huge difference in power draw?  I can feel heat coming off the 8GB Ram area, which is surprising.  Also the heat sinks on the motherboard are hot suggesting the mother board is not as efficient, but 220w is still a lot of power.  Is it possible that the 3 raid cards are guilty?  Replacing the electronic guts would solve the problem, but I would like to keep the systems as stock as possible, since they are only backups.

 

Processor: Intel Dual Core E5130

•Speed: 2 Ghz

•Motherboard: SuperMicro X7DBE-X

•RAM: 8GB Fully Buffered DDR2 (4 x 2gb)

•3 x SAT2-MV8 Raid Cards

Link to comment

I have a motherboard that has PCI-X and PCIe slots. I too wondered which SATA card was most power efficient. I tried a single SAT2 card and my system drew 74 watts at idle. I replaced the SAT2 card with a SASLP and the system drew 77.5 watts at idle. I don't think the high power draw at idle is due to the SATA cards in your server. The small size of the heat sink is a clue that it doesn't consume much power. Have you tried popping out half the RAM to see how much power your RAM is drawing?

Link to comment

Have you tried popping out half the RAM to see how much power your RAM is drawing?

 

Good guess, dropping to 4gb ram reduced power usage by 30w.  Each of the 60 mm screamer fans behind the ram was pulling 10w so unplugged them.

 

I also had a Seasonic 460w X series fanless 80+ Gold rated power supply that I temporarily plugged in.  This creates a new base power draw of 120w with no drives powered.  Much better, but still not anywhere near the 40w base power draw of the newer i3 based system.

 

I guess the original power supplies are not nearly as good as a Seasonic X series.  Still this is not a sustainable solution as changing the power supply in this case will be difficult.  The 3 redundant emac power supplies are mounted under the motherboard, lifting the motherboard up 1 1/2 inches.  (http://www.rackmountnet.com/emacs-500w-power-module-p-2874.html)

 

I'm guessing that the cpu is pulling 20w more than the i3, and the motherboard and ram is wasting the other 60w compared to the newer system.

Link to comment

I saw the guide, and I think this is a bit more involved than I want to get for a backup system.  If I need to run it 24/7, I would need to replace the guts with something more energy efficient.  I would like a quick and easy way to cut the power, and is seems that dropping to 4gb ram, unplugging the 60mm fans behind the ram are all that can be done without major surgery. 

 

Not sure if this applies to unRaid, but on avsforum I was advised to make sure C1E, GV1/GV3 are setup to reduce power usage.  See this advice:

 

Intel E5130:
1) C1E ( can halt/minimize cpu clock only where no activity in general speaking)
2) GV1/GV3 , Intel speedstep( can clock down and up)
* C1E and GV1/GV3 can be controlled by Hardware only

check on AIC server that C1E and GV1/GV3 are enabled to reduced power usage

Link to comment
  • 2 weeks later...

I'm guessing that the cpu is pulling 20w more than the i3, and the motherboard and ram is wasting the other 60w compared to the newer system.

 

I purchased 3 of these systems for backing up my UnRaid systems.  The tremendous power draw wasn't making me real happy, but I have found a solution that I can share with those who have similar needs.

 

When shut right down with the 3 original redundant power supplies all plugged in, the unit still draws 10 watts.  This I can live with. 

 

I had some great help from Influencer getting the IPMI interface working unRaid to unRaid.  No I can have one unraid server start up another one from being totally off via command line or cron job by issueing the following command.

 

ipmitool -I lan -H 192.168.1.149 -U ADMIN -P ADMIN chassis power on

 

Of course IPMI View for Windows can also do this, but you need to do it manually.  I was looking for a scriptable solution.  Learn how to do it here:

 

http://lime-technology.com/forum/index.php?topic=22527.0

 

Now my master unRaid server will wake the other ones up when they have work to do, and shut them down when they are done.  In my case this will be to rsync changes to these backup servers.  Energy use will be still quite acceptable.

Link to comment
  • 2 weeks later...

What's the minimum depth needed for a server rack?  I'm looking at a rack that's 26.5 inches deep but that seems like it wouldn't be deep enough for this case.

 

Thanks

 

If the case is the same as the Xeon version (RSC-4ED2)

 

Dimensions: (W x D x H)

mm: 482.6 x 673.1 x 177.8

inches: 19 x 26.5 x 7

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.