Support > Lounge

Recommend a OS for a ESX NFS datastore

(1/2) > >>

NAS:
I am looking to build a low power box to mix of two 256GB SATAIII non RAID SSD and 4 500GB 7.2K in RAID 5 or 10. I dont mind if its hardware or software RAID but if hardware i would have to buy the card.

The current planned architecture is to have 3 NICs in the storage box , 1 for general LAN connection and the other two directly connected to a dedicated NIC in each ESX. However i have the abilty to bond nics to the switch if this would be a better route.

I am not set in stone with this architecture but my main aim is to be able to saturate each 1Gb NIC if needed or rather get local HDD like performance via ethernet.

The problem is, as with all things ESX, there is just too much information out there. I was hoping there would be some relatively simple plug and play OS for this as i simply dont have the time to fettle to the nth degree.

The SAN would likely have a cheap i3 CPU/MB with 8GB of ram and the OS would be on a USB3 8GB key.

I will have < 10 vms and they are relatively light usage so I dont need blazing speed but equally i dont want rubbish.

If anyone has any pointers or links for me to read it would be very much appreciated.

Ford Prefect:
....use a ZFS based solution, like OpenIndiana (OI) and export a pool for your ESX(i)s via NFS....use napp-it to get easy configuration option.via Web.
I can't make out a clear path from the disks you want to employ for having a datastore that would saturate at least 2 NICs at the same time.
Maybe you need to give it some more thought.
I doubt that OI will install and run safely on a 8GB USB3...>16MB is recommended.

NAS:
Thats great. I didnt want to put in too much detail as I wasnt hoping for a detailed reply. Just suggesitons i can use to good and look at specific things.

Thanks i will go and research

Johnm:
Ironically, I have spent a better part of the weekend doing just this.
(post is a little long winded since i am still fighting this. I'll remind you that i am a Windowz admin and hardware guy. *nix is my weak point. This does not answer your question at all. but gives you some insight and sharing my weekend experience.)

I have an option of putting the datastore in an outboard NAS with NFS or iSCSI (this would be a great use for my spare HP Microserver). A separate box would also be helpful if you have multiple ESXi servers. While I do have 2 ESXi servers, I do not need a central SANS type datastore..  Instead, I chose to try and put the NFS array on the primary ESXi itself as a guest, then use the NFS share as the datastore.

The real reason i went with this route was bandwidth (plus there is that Tim Allen mentality that we can put more stuff into one thing). My new datastore will be 4x 2TB samsung F4's in NFS (old datastore is 1x 2TB and 2x1.5TB JBOD). I can get a good 400MB/s sequential read/write with the F4's in raidz. a single gigabit would be the bottleneck. I also plan to expand this out to a second vdev that will speed it up even more. I am still leaving my 2x 256GB SSD's as traditional datastores (one will have the NFS OS that hosts the rest of the guests.).

I started out trying OI as a baseline for napp-it. I honestly got stuck configuring it, broke it and had to reinstall.
I figured as long as I was reinstalling, I would also look into freeNAS since I have used it in the past for this purpose with good success (and without hacking it).
I was on a bit of a time crunch. [I only had one afternoon to get this up and running and my main datastore drive keeps making a horrid crash sound and going offline. with requires a reboot. I backed up most of that drive I thought. that drive died in in the restore. I need to get migrated before I lose it all... meh.. its just data.. had I more time, I would have kept up with the IO route.]

I actually got that to work quite well.. I had to do about 6 different hacks to it to get it to work in the end. It used to work out of the box for me.
apparently it has issues with the m1015 (works with a hack). i had issues with the ESX 10GB nic (the e1000 still works at 10GB I found out). the NFS reads fast but wont write with ZIL enabled (another hack). If i were to have used the iSCSI connector instead, I think it would have been much easier and just as fast. It also has ESXi tools built in so thats a big plus.

The NFS share is reporting about 408MB/s in a guest with CrystalDiskMark. Not bad considering my SSD boot drive on the same guest is reporting about 420MB/s.(it was 408MB read and 4.5MB write before I turned off ZIL)

So far... I feel pretty good about about running freeNAS for the datastore inside ESXi. But, I am not done testing yet.
I would not run it without a UPS though since i had to disable ZIL. A power loss could be a total disaster (my UPS does do a clean shutdown, i tested this). I need to look to see if I can either disable synchronous writes on the ESXi box (assuming this is the issue) or toss an SSD into the freeNAS for ZIL. I'll also try iSCSI (the route I should have probably went from the beginning) and see if this just solves the issues without a loss of speed. I'll do this once the migration is complete in a few days.

Right now where I stand is, I have freeNAS running as a guest and I am migrating my failing JBOD datastores to the NFS shares. Once that is complete. I'll try patching up and fixing the freeNAS issues listed above. I'll also try to install OI again next to the freeNAS once I have spare time. I will see what one I like better and use that. then I'll  remove the second array (migrating again if i need to).

lboregard:

--- Quote from: Johnm ---The real reason i went with this route was bandwidth (plus there is that Tim Allen mentality that we can put more stuff into one thing). My new datastore will be 4x 2TB samsung F4's in NFS (old datastore is 1x 2TB and 2x1.5TB JBOD).

--- End quote ---

Johnm, do you expect a performance penalty in exposing the 4 drive zfs raidz as a nfs share ?

i remember you were using one of the guests in atlas as your usenet downloader, (sabnzbd maybe?). in any case, any usenet downloader writes a lot of small files and then does cpu/disk intensive parity repair ...

how do you think it would fare with shared disks on a zfs raid ?

Navigation

[0] Message Index

[#] Next page