4 gaming VM's in 1 PC storage configuration advice.

I'm fairly new at Unraid and how to set up the correct configuration for storage, i've managed to build a 2 in 1 gaming setup and i was wondering was storage setup would be ideal to crank up that number up to 4 VM's.

The whole setup is.

Intel 5960x 8 cores, 16 threads.

Gigabyte UD4P X99 Motherboard

Corsair Vengeance LPX 2400mhz 64gb memory.

2 Western Digital Black 1TB

2 SSD's Kingston UV400 480gb

2 GTX970 Gigabyte Windforce.

I can upgrade the ram to 128gb, the cpu to the 6950x and add more SSD's and 2 more gpu's, the question is, what would be the ideal configuration for them. Is there any way to get more performance out of those 4 SSD's set up as cache? From my understanding the 4 would work as Raid 1, giving me 960gb of space avaiable to assignti the VM´s, is there any way to convert that cache pool to raid 0 or something like that?

If you got the money to spend on that cpu, I'd look for a dual socket configuration. If you can install multiple cpus, you will most likely have a better result then with a single one even if yours has "a lot of cores". Might save some money as well.

As with that generation of cpus, the "highest" one is not the best for single os gaming - also as with more cores you get less power per core, so even if you use all of them, and especially while you do, the per core performance wont be that awesome.

I guess my dual e5 2680 Build will perform better if you aim for 4 or more gaming vms in one. My whole rig with everything but peripherals and gpu costs about half of your cpu alone.

Also, server boards are more likely to be able to serve all the rare function you might want to make use of. starting with multiple nics. 120 MB throughput in the home network is ok, but if you split it to 4 vms that gigabit line is down to effectively 30 MBs per vm. Then there is the question if you can use 4 x16 Pci-e cards together with all the other stuff you might want to add like a m2 drive and so on. One cpu might run out of lanes as your mainboard might run out of expansion slots and/or space.

To come back to your question, there is a option to change the "raid level" of the cache. That way you could have something like a stripe/raid 0. I currently use this configuration and am happy with it (2 ssds) If you got money to spend though, you might consider a pci-E ssd. Those often cant be booted from as in booted directly from the bios/uefi, but they certainly offer more speeds then some sata 3 drives.

Note: I would definitely keep my hands of any kinds of OC on that multi user platform.

If you got the money to spend on that cpu, I'd look for a dual socket configuration. If you can install multiple cpus, you will most likely have a better result then with a single one even if yours has "a lot of cores". Might save some money as well.

 

As with that generation of cpus, the "highest" one is not the best for single os gaming - also as with more cores you get less power per core, so even if you use all of them, and especially while you do, the per core performance wont be that awesome.

 

I guess my dual e5 2680 Build will perform better if you aim for 4 or more gaming vms in one. My whole rig with everything but peripherals and gpu costs about half of your cpu alone.

 

Also, server boards are more likely to be able to serve all the rare function you might want to make use of. starting with multiple nics. 120 MB throughput in the home network is ok, but if you split it to 4 vms that gigabit line is down to effectively 30 MBs per vm. Then there is the question if you can use 4 x16 Pci-e cards together with all the other stuff you might want to add like a m2 drive and so on. One cpu might run out of lanes as your mainboard might run out of expansion slots and/or space.

 

To come back to your question, there is a option to change the "raid level" of the cache. That way you could have something like a stripe/raid 0. I currently use this configuration and am happy with it (2 ssds) If you got money to spend though, you might consider a pci-E ssd. Those often cant be booted from as in booted directly from the bios/uefi, but they certainly offer more speeds then some sata 3 drives.

 

Note: I would definitely keep my hands of any kinds of OC on that multi user platform.

Thankfully the cpu did'nt cost me anything, i got one thanks to my job. The problem with server platforms, both xeons and motherboards is that they're not easy to find in my country, and import is not an option, i'm from Argentina.

I might be getting some M.2 drives in 3 weeks or so but i'm not sure about that, those 480gb drives are the best i can find for now. I've been testing setting the 4 drives in raid 0 as cache and they gave me quite the boost in performance until i booted up the 3rd VM.

For some reason GPU usage is hovering around 50-70% when running Crysis 3 in 2 VM's at the same time, i have set 4 cores per VM, 16gb of ram and 240gb of drive space located in the cache array and using only it. btw i've already tested 4 Nvidia gpu's on this config with the fix posted on this thread. http://lime-technology.com/forum/index.php?topic=43644.msg452464#msg452464

Network performance is not something that worries me since the whole set up is meant to play offline.

So with the cores performance dropping i guess running 4 vm's with 4 cores each and the rest assigned to Unraid with the 6950x would not be ideal?

Is it possible to assign one SSD per VM? I mean, to locate each VM in a separate SSD to get the best performance of each drive per vm?

Answers to your questions

Yes you can setup a raid 0 cache in btrfs

eg

-dconvert=raid0 -mconvert=raid1

this would give you data is striped for speed but the filesystem metadata  mirrored for redundancy.

You could also just passthrough physical ssd disks to use on each vm by using disk location by dev id 

eg

/dev/disk/by-id/xxxxxxxxxxxxxxx

I really dont think it is necessary to passthrough each disk. Using 4 vdisks off the cache would be fine when gaming im sure but if you wanted the fastest disk speed possible in a vm 4 passed through disks is going to be best performance.

As regards your cpu if its free hey you cant go wrong there. 6950x is a 10 core cpu with hyperthreading so the most you could give each machine is 2 hyperthreaded cores. (4 vcpus)

For best results isolate the 8 cores from unraid in the syslinuxconf file using isocpu

Then pin 2 hyperthreded pairs (4 vcpu) to each vm. Make sure you do choose the hyperthread pairs.

For 2 of the vms pin the emulation calls to one of the 2 cores you didnt isocpu in syslinux file, and the other vm to the other core.By editing xml and using emulatorpin cpuset=  see here http://lime-technology.com/forum/index.php?topic=49051.0

The single core speed will be fine for gaming the 6950x the cpu is awesome (although i think over priced compared to similar xeon...but hey its free for you!) the chip gets 20223 passmark with single thread of 2143!! thats pretty impressive. Yes all core turbo will not be as high as single core turbo but that would be the same in a dual cpu setup aswell but having 2 cpus obviously would give more cores to use but would still not give faster single core speeds.

OMG your post is so on point with things i was searching to see if they were possible or not, thank you so much! Here's some of the thoughts i was having.

Answers to your questions

 

Yes you can setup a raid 0 cache in btrfs

eg

-dconvert=raid0 -mconvert=raid1

 

this would give you data is striped for speed but the filesystem metadata  mirrored for redundancy.

 

You could also just passthrough physical ssd disks to use on each vm by using disk location by dev id 

eg

/dev/disk/by-id/xxxxxxxxxxxxxxx

 

I really dont think it is necessary to passthrough each disk. Using 4 vdisks off the cache would be fine when gaming im sure but if you wanted the fastest disk speed possible in a vm 4 passed through disks is going to be best performance.

I found exactly how to convert the array to raid 0, i was also testing raid 10 since technically would give me a certain security level and not the biggest penalty in performance. My thought initially was to do what you exactly said, to give to each VM one SSD, to get the best performance of each drive since they are not the fastest in world and up to a certain they perform the best that way. Will you explain how could i assign them to a VM using the method you said? I literally have no idea how to proceed.

As regards your cpu if its free hey you cant go wrong there. 6950x is a 10 core cpu with hyperthreading so the most you could give each machine is 2 hyperthreaded cores. (4 vcpus)

For best results isolate the 8 cores from unraid in the syslinuxconf file using isocpu

Then pin 2 hyperthreded pairs (4 vcpu) to each vm. Make sure you do choose the hyperthread pairs.

For 2 of the vms pin the emulation calls to one of the 2 cores you didnt isocpu in syslinux file, and the other vm to the other core.By editing xml and using emulatorpin cpuset=  see here http://lime-technology.com/forum/index.php?topic=49051.0

I had that thread on my bookmarks since yesterday to test how to properly assign cores and so far it's working perfectly, i only tried it with some cores and a single VM and it's working a lot more smoother than before. I don't get that choppy performance ingame and desktop usage has improved a lot. My original concern was that those problems were caused by the cache array i was using but it seems that it's not it alone and there's a lot of way i could improve the current setup.

This maybe silly but have you actually tried to plug the GPU on the last PCIe slot to see if you can still plug other stuff to the mobo? The GPU can block some important / required connections.

This maybe silly but have you actually tried to plug the GPU on the last PCIe slot to see if you can still plug other stuff to the mobo? The GPU can block some important / required connections.

Yes, i've already tested that in case i needed to use the internal USB ports. I'm planning to build the PC in a Thermaltake P5 case although for now it's on a bench table.