Merging Desktop and unRAID?


Recommended Posts

I am not sure if this belongs here; if it should be placed somewhere else, mods please move it.

If it is possible, I am considering merging my components in to one box with a view towards eliminating my separate desktop and relying upon one unified hardware box running unRAID (or an alternative) for all of my use cases. I am hoping to pin down the best plan of attack.

* I use my unRAID for SAB/SB/CP/Plex and storage of various media. Plex is usually transcoding at most one stream at a time (either my wife or myself but not both, if we are both watching something at the same time one of us is usually direct streaming).

* I use my desktop for web browsing, extremely light gaming (mostly old console emulation - NES, SNES, etc.), watching videos in Plex/YouTube, and viewing photos. I also occasionally rip/mux/re-encode Blu-Rays direct from disc.  I use dual generic 20” monitors at 1680x1050.

I have the following relevant hardware:

unRAID:

CPU: Celeron G1610 (side note - this is a tough little chip that is basically fine for transcoding in Plex)

RAM: 4GB (1 x 4GB)

Further specs in my signature if necessary

Desktop:

CPU: i3-3225

RAM: 8 GB (2 x 4GB)

Blu-Ray Burner

60 GB SSD

Integrated graphics - no GPU

I also have an extra 4 GB of RAM laying around (2 x 2GB). Neither processor supports VT-D.

Questions:

1. Is combining desktop and server a bad idea?

2. Is it possible to eliminate my desktop by putting the i3 into the unRAID server, upping the RAM, and instead use unRAID as a base with a Linux VM (NOT Windows)? (Since I don’t have VT-D, I am guessing this is a non-starter.)

3. Would it be better to instead run a different Linux Distro as the base for my “desktop” uses, and then run unRAID in a VM on top of it, or switch to something like Snapraid/AUFS? Or would I still have the same problem due to the lack of VT-D?

4. If these things are possible, do I need more horsepower than the i3-3225 to be able to do this? (I’m thinking the answer is probably yes.)

5. If I get a new processor (with VT-D), what is the better plan of attack from the above? I.E. unRAID base/Linux VM or Linux base/unRAID VM?

6. If I get a new processor, any preference between an i5-3470s and a Xeon E3-1245v2? ($80 price difference at MicroCenter but hyperthreading may be useful for future-proofing.) And can I use the integrated graphics or is that a waste?

To be clear, I am a fairly happy unRAID user and paid for Plus and then for Pro. However, if my use case has changed and a different alternative is now preferable (different Linux base, Snapraid, etc.), I can migrate away if need be. It would be with no hard feelings - I knew what I was getting and got everything I paid for.

Sorry for the brain dump. Any thoughts would be appreciated. If I can provide additional information, please let me know. Thanks!

Link to comment

I think a number of people are wondering if it will become possible to do this with unRAID v6 using KVM or Xen plus hardware passthru of the GPU.  However it may only be practical for those who have hardware that supports GPU passthru so may necessitate some hardware changes to make it a possibility.  At the moment the jury seems to be out as to whether this will be stable enough to use in reality.  Hopefully this will become clearer as v6 gets nearer a production release.

 

I must admit it is of interest to me if I can reduce the box count - although in my case I would want a Windows VM rather than a Linux VM.

Link to comment

I think a number of people are wondering if it will become possible to do this with unRAID v6 using KVM or Xen plus hardware passthru of the GPU.  However it may only be practical for those who have hardware that supports GPU passthru so may necessitate some hardware changes to make it a possibility.  At the moment the jury seems to be out as to whether this will be stable enough to use in reality.  Hopefully this will become clearer as v6 gets nearer a production release.

 

I must admit it is of interest to me if I can reduce the box count - although in my case I would want a Windows VM rather than a Linux VM.

 

Yeah I am thinking I don't have the right hardware because of the passthrough issue. I wasn't sure if using a different method (Linux base, unRAID VM) would change that - I'm guessing it does not because I would still need passthrough on the drives.

 

I made a conscious effort to use a low-powered processor (wattage and performance-wise) but it appears that may have been in error given where things are going with virtualization, both here and in the computing world in general.

Link to comment
I made a conscious effort to use a low-powered processor (wattage and performance-wise) but it appears that may have been in error given where things are going with virtualization, both here and in the computing world in general.

The difference in wattage between the low and high end of CPU's is such a small part of the overall system draw you would hardly notice it anymore, especially under mostly light use. The days of the 130W CPU are thankfully over for the moment.
Link to comment

Merging of the NAS and other computers SOUNDS like a great idea ... but when you look at the different requirements, it's not clear to me it's the right direction.

 

I'm leaning towards a low-powered server box that has plenty of storage capacity ("enough" obviously varies with the user ... I want a min of 25TB) ... and a second box for my primary Windows machine, which will also run VM's under VMWare Workstation for other things I want to do.  I think the total power consumption won't be appreciably different than a single box designed to "do it all" -- in fact, the Windows box can be in S3 when not in use, so its consumption will be effectively zero. 

 

Link to comment

On the other hand, if there is a vt-d equipped, medium to high performance machine running 24x7, it can be used to run unRAID and desktop. The downsides are often just the noise of disk drives, as the fans and heat are part of the workstation load. This is clearly not for everyone, but for those with a 24x7 workstation already running, it can work nicely.

 

A couple things to keep in mind;

it's a server, use a UPS,

locate it in a safe place for no spills, or vacuum cleaner bangs, etc.

The case should be large enough for all your drives, and cooling space for the graphics card, etc.

 

I ran workstations 24x7 for years. They were running "something" which keep me from turning them off overnight, etc. That's all virtualized now and still running 24x7.

 

As Gary says, if power optimization is your goal, the low power server and well designed workstation will have lower power consumption. But if you have reason to keep it all busy, a single machine is doable.

 

Link to comment

Agree ... if your desktop needs to be on 24/7 then doing it all in one machine makes more sense.    You'll need to build a fairly powerful machine; and use a case large enough to hold all of the disk drives with suitable cooling;  with a CPU that supports vt-X and vt-D.    It should also support ECC memory and certainly have a UPS.    You're putting all your eggs in one basket here, so be sure it's a SAFE basket  :)

 

A good SuperMicro server board with 16-32GB of ECC RAM and a Xeon processor with vt-x and vt-d is a good choice for the hardware.    UnRAID v6, with any add-ons supported via Docker containers and your VM's supported by whichever hypervisor ends up being supported in v6 is the best approach.    An alternative that some folks use is to run ESXi on the hardware, with UnRAID running in a VM.    Either approach should work fine once it's set up.  With UnRAID moving towards native hypervisor support, I'd probably favor that approach;  but ESXi has very strong support and is (at least at the moment) easier to add new VMs to if your inventory of "machines" grows.    Getting the UnRAID VM set up can be a bit of a challenge - but there are several threads here to help with that as well.

 

Link to comment

When I built my unRAID and got all of my media apps for downloading, organizing, and serving all running on unRAID, I decided I didn't really need a PC of my own anymore and instead built my wife a nice workstation for Photoshop. I have an account on that machine I use occasionally, but both of us use our notebooks and other portable devices a lot more than we do a desktop PC. And I have a small, quiet HTPC with the TV in another room. The Photoshop workstation is not to be tinkered with.

 

I have moved all those media apps to docker now, and I really like doing things with VMs and containers, but my unRAID will probably always be headless so I don't really need video passthrough for my scenario.

Link to comment

My thanks to all for the comments.

 

 

While I think I was looking for a discussion of the possibility merging my current components, I think we were able to go right past that based upon the lack of VT-D - I would need a hardware upgrade, and it would not just be as simple as plugging a processor in to my unRAID box, it would require a dedicated server mobo and ECC RAM to be done correctly. On top of that, my use cases do not require me to have the server on 24/7 (some nights for a Handbrake encode, sure, but otherwise no). As my wife barely tolerates the server we already have (despite being perfectly happy to watch her various garbage reality TV shows through Plex/Roku), an upgrade of that magnitude is not in the cards for me right now. I probably could have juuuuuuust squeaked by with a new CPU if factoring in reselling my current components. The itch to constantly upgrade/flip old parts is one I have to get over!

Link to comment

Clearly your current hardware isn't right for combining everything into a single box ... and it sounds like that's a deal breaker for your plans to consolidate.

 

It sounds like your setup is working reasonably well as is; so I'd just live with that, and save the consolidation project for your next major system refresh ... perhaps when you run out of space on your current server, or you have hardware issues.

 

As I noted earlier, consolidation doesn't necessarily mean power savings anyway ... particularly since you're not running your server 24/7.

 

Link to comment

Thought I'd offer some insight on this topic since it is one I myself have been spending a lot of time contemplating.  The trick to this puzzle is that the right approach varies on your specific situation.  For the OP's hardware, I'm afraid there is really no ideal solution to utilize that hardware.  The problem is in isolating the user operating environment from the server operating environment.  To make use of a "desktop OS" while also adding server functionality is inherently a bad design concept.  The more server applications you have running on a desktop PC, the worse the user experience becomes.  In addition, users can then even get in the way of the servers by using their own applications, some of which are dangerous (viruses and malware) which can overtake the system.  This is the risk of running a end-user OS and server OS as a single entity.  The users themselves make it a risky proposition.  Windows IT veterans can attest to this:  when you install Windows Server OS and launch IE and try to go to ANY website, you get a million warnings from IE about security unless you significantly reduce the default permissions which is a pain to do.  On a desktop IE instance, same sites, no problem.  Why?  Because Microsoft knows that just browsing the web can be "risky" and have implications.

 

On the flip side, if you are an EXTREMELY savvy user that is willing to live on the wild-side, it is technically possible to do so, but wouldn't recommend it to someone to do unless they REALLY knew what they were doing.

 

With unRAID, to achieve a true two in one solution, we require the use of virtualization to assist us in that objective.  There are numerous approaches that you can take with Xen or KVM, but at this time, these are all considered "experimental" in our implementation.  Many have had great success, but it isn't part of our webGui to automate the configuration of this capability as of today.

 

In short, by having unRAID run as the server operating system at a layer above the other operating functions, we are able to prioritize the most important aspect of unRAID:  managing your data.  We also can isolate how resources are allocated from the host towards specific functions such as applications or virtual machines.  This gives great flexibility and control over your system and, with the right hardware, allows you to do more advanced things such as run a full desktop operating system with display output through a GPU and control through USB devices (M&KB).  The key here is making sure that the core operating system itself (unRAID Server OS) is isolated from this usage to avoid conflict in resource demands.

 

Desktop and server processors today, when used in this scenario, go vastly underutilized most of the time in a headless scenario.  We accept this because when they are utilized, we can push on them pretty hard to do a number of things for us concurrently.  However, today's system processors are vastly more powerful than what our core operating system requires to do it's natural functions.  When combined with SSDs in cache pools, the performance improvement is noticed even more so.

 

All this being said, what I'm really driving at here is that if you have the right hardware, you can take advantage of these advanced capabilities.  And whether that hardware was purchased to be a NAS or a Desktop doesn't really matter.  All that matters is whether or not the hardware is capable and the desire is there to apply its capability to good use.  If you don't have the right hardware, while combining these capabilities is possible, it's not highly recommended.  In addition, these capabilities are experimental and are still in development and their advancement is detached from our release goals for version 6.  Expect to see continued improvements in the "Localized Virtualization Department", but they will remain experimental for the time being.

Link to comment

jon outlined the options nicely.

 

Note that the direction of v6 changed quite dramatically with the move to Docker.  Probably a GOOD thing, but nevertheless the focus on virtualization is no longer part of v6's development.

 

I have no insight into the future plans for UnRAID, but it's reasonable to assume that they haven't abandoned virtualization -- only pushed it out a bit in the future.  I'd think v7 is very likely to have embedded hypervisor support with a nice VM manager as part of the package.    THEN doing what you'd like here will be not only very possible; but also relatively simple.  But as jon noted, it will require some fairly capable hardware.

 

If I was building a new system for UnRAID, I'd absolutely use a server motherboard; a high end Xeon; 32-GB of ECC RAM; and be sure I had both vt-x and vt-d support.    That would be a box well-positioned for the future versions I expect to see in the next couple years ... and could almost certainly replace most of my systems with a single box.

 

 

 

Link to comment

Note that the direction of v6 changed quite dramatically with the move to Docker.  Probably a GOOD thing, but nevertheless the focus on virtualization is no longer part of v6's development.

 

I can see where one might think this, however, to say the "focus on virtualization is no longer part of v6's development" is absolutely not the case.

Link to comment

Note that the direction of v6 changed quite dramatically with the move to Docker.  Probably a GOOD thing, but nevertheless the focus on virtualization is no longer part of v6's development.

 

I can see where one might think this, however, to say the "focus on virtualization is no longer part of v6's development" is absolutely not the case.

 

Good to know, but it certainly seems like the focus on Docker has put further virtualization development on the back burner.    Note that I didn't say that "virtualization" was no longer included; just that the "focus" on it has been deferred in favor of other things (e.g. Docker).    With everything on your plate that needs to be done before v6.0, and a promised date of "3rd quarter", it certainly doesn't seem like a hypervisor management package is something that will be included.  Is that wrong?

 

Link to comment

Let's take a moment to clarify.  First and foremost, what is Virtualization?  Let's not get too high-level here, but quite simply Virtualization refers to a concept in technology whereby you are invoking some type of software emulation to get around a software limitation.  Example:  VLANs (virtual local area networks) are ways to create additional network ranges on individual pieces of switching/routing hardware that otherwise could not exist without using virtualization-minded engineering approaches.  Without virtualization, each network would require separate physical hardware to control.  In server virtualization, a hypervisor is employed to emulate the physical hardware to an extent to support the running of multiple operating systems (something impossible without the use of virtualization).

 

Each applied use of virtualization to a particular technology segment is sub-named for it's specific type.  The big categories still to this day are as follows:  application virtualization, server virtualization, network virtualization, storage virtualization, and desktop virtualization.  Docker actually falls somewhere in between application and server virtualization into a sub-segment called "operating system virtualization."  Technical Comment:  The interesting aspect of Docker's virtualized approach is that they do not actually have to emulate the Linux kernel itself or any of the hardware components of a physical machine.

 

So what does all this mean?  Xen, KVM, and Docker are all methods of "virtualization", but with important and subtle differences.  Docker does not require any specialized hardware requirements to run on unRAID and that is a huge advantage.  Docker also offers a more complete solution in terms of image management than what a virtual machine can do under the management of any hypervisor.  Xen and KVM have their place the in unRAID universe as well, and we intend on showing that off in future releases.

 

The key points I was trying to convey at the end of my post were simply this:

 

To the OP:  Your hardware is not able to support both a local desktop experience and unRAID concurrently.  To accomplish this, you would need to have hardware that supports VT-d and/or AMD Vi.

 

To all:  If you have ANY 64-bit capable computer that support VT-d and/or AMD Vi, regardless if you use it as a NAS or not, you might be interested in adding unRAID to that system at some point in the future as an extra layer of control, management, protection, and get further value out of your investment into that hardware.

 

To anyone building a new rig right now or thinking about it:  There are cost-effective solutions to custom-build hardware options that SHOULD work with the advanced virtualization capabilities of unRAID in the future.  Example:  http://pcpartpicker.com/part/intel-cpu-eu80570pj0806m.  That processors supports VT-d and is only $34.95.  Keep in mind, we haven't tested all of the different hardware options out there, but as always, pricing for these things comes down over time and VT-d support isn't just a brand new thing.

 

And lastly, with respect to development in this area, all I was originally trying to state was that we are not binding ourselves to machine-based virtualization management as a requirement for the release of unRAID 6.0.  Continued development is going to occur on VMs in unRAID, but this is not considered a core feature requirement for unRAID 6.0's release.

Link to comment

The key difference between Docker and a virtualized environment is that Docker runs on the base OS ... it simply manages "containers" that fully encapsulate all dependencies for the application(s) within that container ... eliminating potential inconsistencies with other code running on the OS (e.g. UnRAID).    Clearly it works very well ... and is a nice way to isolate UnRAID from the vagaries of all the various add-ons and plug-ins without the need to run a separate "machine" for those other applications [whether that "machine" is real or virtual is almost irrelevant].  But it's NOT virtualization ... it does not virtualize the environment; nor does it completely isolate the Docker container from the OS ... a crash in Docker can still take down the base OS.

 

Clearly when I made my comment r.e. virtualization I was referring to virtual machines, which WAS a key focus a few weeks ago until the move to Docker.  Prior to that point the discussion was largely on which hypervisor to use.  I certainly think it's accurate to say:

 

... the direction of v6 changed quite dramatically with the move to Docker.

 

It's nice to confirm that:

 

Continued development is going to occur on VMs in unRAID ...

 

 

Perhaps when I said "...  the focus on virtualization is no longer part of v6's development."  it would have been more accurate to say what jonp did:

 

Continued development is going to occur on VMs in unRAID, but this is not considered a core feature requirement for unRAID 6.0's release.

 

What I was basically saying was that full support for VMs was likely to move to a later release ... if that's wrong, and it's going to be in a v6.x release that's good news.

 

Link to comment

The key difference between Docker and a virtualized environment is that Docker runs on the base OS ... it simply manages "containers" that fully encapsulate all dependencies for the application(s) within that container ... eliminating potential inconsistencies with other code running on the OS (e.g. UnRAID).    Clearly it works very well ... and is a nice way to isolate UnRAID from the vagaries of all the various add-ons and plug-ins without the need to run a separate "machine" for those other applications [whether that "machine" is real or virtual is almost irrelevant].  But it's NOT virtualization ... it does not virtualize the environment; nor does it completely isolate the Docker container from the OS ... a crash in Docker can still take down the base OS.

 

http://en.wikipedia.org/wiki/Operating_system%E2%80%93level_virtualization

 

See the "Docker" part on there.  The word "virtualization" can be very ambiguous, so please understand that "virtualization" and "virtual machines" are different things altogether in terms of terminology.

Link to comment

Agree ... using the term "virtualization" to represent isolation techniques adds another meaning that's different than what IBM had in mind back in the 70's when they built the early hypervisors.  I'm old school - to me virtualization support means a hypervisor and virtual machines  :)

 

... but I agree the term has evolved to include less isolated instances that can still provide significant isolation from the underlying OS (but not by any means as complete as a true VM).

 

Link to comment

Agree ... using the term "virtualization" to represent isolation techniques adds another meaning that's different than what IBM had in mind back in the 70's when they built the early hypervisors.  I'm old school - to me virtualization support means a hypervisor and virtual machines  :)

 

... but I agree the term has evolved to include less isolated instances that can still provide significant isolation from the underlying OS (but not by any means as complete as a true VM).

 

I deal with the same thing all the time.  Folks would tell me they were doing "Desktop Virtualization" but were really just using Citrix XenApp with Microsoft Terminal Services to present a Windows Server Desktop session to a user.  I would turn my head sideways and say, "yeah, that's kinda desktop virtualization but not technically accurate.  That's really application virtualization presented through a desktop interface."  And that's when everyone else's heads turned sideways as well...

 

I thought of another good analogy today as well.  Would you call that Android or iOS device in your pocket a "phone" by the standards of what it was defined as being when it was first invented?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.