Virtualizing unRAID with Xen on Arch Linux, XBMC & Windows with VGA Passthrough

I received some absolutely sterling advice and help from grumpy at an unreasonable hour, which was absolutely appreciated. Whilst i had issues with amd's woeful driver support in linux regarding HD AUDIO passthrough, I am firmly of the belief that virtualisation is the way to go, luckily I have a small 250 GB slotted into a spare bay on my server that i can pull/push to give me a safe environment to muck about and explore possibilities without affecting my main system in any way.

 

Latest venture so far, I have followed the XMBC VDPAU installation instructions in ubuntu 13.10 minimal server install to get a kernel with some patches that allows AMD graphics card to passthrough HD AUDIO (AND IT WORKS !!!!!!) , i've got a xen hypervisor up and running. It's movie time for me now (using the main system again) , but tomorrow I plan on playing with getting unraid up and running on the hypervisor instance.

 

Glad you were successful. Are you using the Radeon Drivers (which I use) or does AMD have theirs out in beta yet?

 

I'm using the walkthrough that fritsch posted over on the XBMC forum in this thread, http://forum.xbmc.org/showthread.php?tid=174854

 

How did you get the output to go to the radeon (TV) rather than the virtual display adapter (console).  I followed that thread to the letter and never got out to the TV.  XBMC always reports that "gallium" display adapter in the Settings Info.  Did you have to do anything other than what is outlined in that thread?

 

John

I did all the stuff in that thread on a bare metal system on a spare drive away from my main array, then i installed the xen hypervisor using this thread that grumpy sent me https://help.ubuntu.com/community/Xen

I initially had an issue with the hypervisor not running at all, but copy pasted the error message into the great god google and found a debian thread that had the solution.

OK...you are running it on baremetal...not in a Xen VM.

OK...you are running it on baremetal...not in a Xen VM.

XBMC is running on ubuntu server install as per that thread, then on that install i put the xen hypervisor. so it boots up XBMC coming out of the HDMI port of the card, and i can get all the xen goodness via ssh.

OK...you are running it on baremetal...not in a Xen VM.

As an aside, and it may be relevant, i don't know. But when i first booted on bare metal with the radeon card in my super micro board i got no video output from either the vga onboard, the vga output on the card or the hdmi output.

so i popped the cmos battery to reset the bios, then went in and found a setting to favour off board vga output and i then got full output of post screens and ubuntu setup via HDMI.

Just trying to figure out where the SSD that you installed Arch/Xen on is?  You said that you passed through the onboard SATA to Unraid so wouldn't you lose your SSD to Unraid? 

Besides that, thanks for all your detailed explanations.  I am currently doing this via ESXi and I think I am going to move to Xen for better performance and more flexibility.

Sam

Just trying to figure out where the SSD that you installed Arch/Xen on is?  You said that you passed through the onboard SATA to Unraid so wouldn't you lose your SSD to Unraid? 

 

Besides that, thanks for all your detailed explanations.  I am currently doing this via ESXi and I think I am going to move to Xen for better performance and more flexibility.

 

Sam

I have a PCI sata card I picked up for £15.

Sent from my Nexus 5 using Tapatalk

Just trying to figure out where the SSD that you installed Arch/Xen on is?  You said that you passed through the onboard SATA to Unraid so wouldn't you lose your SSD to Unraid? 

 

Besides that, thanks for all your detailed explanations.  I am currently doing this via ESXi and I think I am going to move to Xen for better performance and more flexibility.

 

Sam

On my case, my board some Asrock Z77 has 2 Sata controllers, one Asmedia with 2 ports which I'm using as LVM storage directly on Arch, using for now only 1 SSD disk, and I'm passing through the whole Intel controller (6 ports) to Unraid.

My installation is on a USB 3.0 pen, which has some issues since can't boot on a USB 3.0 port, must be on a 2.0.

Was wondering if anyone can point to a virt-manager / xen setup guide for arch. Not sure if virt-manager is the best manager for Xen. Thoughts?

I keep thinking about this but I find the CLI so easy now I just don't need it. 4 months ago I was at level 0 in xen knowledge and would have agreed with you.

XL CREATE /path/to/config.cfg

Xl top

Xl list

Xl destroy unraid

Etc etc. Its all so easy I can do it from my phone over juicessh or prompt on iOS. Why do you want / need a GUI? You barely interact with the console anyway on a production level machine.

Sent from my Nexus 5 using Tapatalk

Getting after it again... Figured what the heck try once more. I thought to go a different route this time,  slightly.

I found some YouTube videos going through installing Xen using lvm partitions step by step,  it really helps walking through it all and makes one realize how simple things are.  However,  when I got to the point of reboot,  it barks at me about the uuid of my boot drive.  I looked at fstab and it looks right,  so I am not sure why it's mad.  Just wondering what's better,  start again or fix grub and whatever it is it doesn't like.  I did try searching the error,  but nothing stuck out. I know we did something with fstab before so I am hoping the fix is as simple as that.

Getting after it again... Figured what the heck try once more. I thought to go a different route this time,  slightly.

 

I found some YouTube videos going through installing Xen using lvm partitions step by step,  it really helps walking through it all and makes one realize how simple things are.  However,  when I got to the point of reboot,  it barks at me about the uuid of my boot drive.  I looked at fstab and it looks right,  so I am not sure why it's mad.  Just wondering what's better,  start again or fix grub and whatever it is it doesn't like.  I did try searching the error,  but nothing stuck out. I know we did something with fstab before so I am hoping the fix is as simple as that.

I remember having a similar issue while installing arch. I booted up with the usb install iso, dropped me to root shell and I set

use_lvmetad = 1

in /etc/lvm/lvm.conf. For some reason I had to set it to 0 to get past a grub issue.

And also check /etc/default/grub and add:

GRUB_DISABLE_SUBMENU=y

To get around a grub bug, rerun the grub command afterwards

Was wondering if anyone can point to a virt-manager / xen setup guide for arch. Not sure if virt-manager is the best manager for Xen. Thoughts?

The default libvirt package in Arch is compiled without Xen support enabled.

I will post a libvirt package later today that has it enabled and the other settings to make libvirt, Xen and Virt-Manager work.

I think grub menu issue is resolved because when the system boots,  grub has a timer.

Will try the lvm fix,  but first I am doing it all again without lvm.  Like grumpy says,  we are learning at first,  need to make sure I can just boot that,  once that is complete and bootable I can try with lvm again.  Currently I yanked the drive from my laptop just to play with a 8gb USB,  once things work I can do a normal hd and return to Xen and all the fun.  For now I just want to grasp arch and its goodness.

-

FYI,  just completed install again and once more,  error: no such device found "uuid"  so at least it is not lvm

---

Wanted to report back that this is resolved.  I abandoned the USB thinking it was that and tried a regular drive and ended up with the same thing.  So, I started my search again and found the fix.  Not sure why, but my grub.cfg was written as grub.cfg.new and grub.cfg was bad.  So, I booted back to the install media, mounted my disks, went to /boot/grub, renamed grub.cfg to grub.cfg.bak and then copied grub.cfg.new to grub.cfg, exited, unmounted, rebooted and now Arch loads.

As a result, I think I may try my hand with the USB again, just to confirm the fix since it has Arch installed, it just wont load well.

Compile problems:

Arch linux, compile goes along until I reach the following:

make -C firmware install
make[3]: Entering directory '/home/mykroft/xen/src/xen-4.3.1/tools/firmware'
GIT=git /home/mykroft/xen/src/xen-4.3.1/tools/firmware/../../scripts/git-checkout.sh git://xenbits.xen.org/seabios.git 3a28511b46f0c2af5fae1b6ed2b0c19d7913cee3 seabios-dir
Cloning into 'seabios-dir-remote.tmp'...
fatal: read error: Connection reset by peer
Makefile:21: recipe for target 'seabios-dir' failed
make[3]: *** [seabios-dir] Error 128
make[3]: Leaving directory '/home/mykroft/xen/src/xen-4.3.1/tools/firmware'
/home/mykroft/xen/src/xen-4.3.1/tools/../tools/Rules.mk:105: recipe for target 'subdir-install-firmware' failed
make[2]: *** [subdir-install-firmware] Error 2
make[2]: Leaving directory '/home/mykroft/xen/src/xen-4.3.1/tools'
/home/mykroft/xen/src/xen-4.3.1/tools/../tools/Rules.mk:100: recipe for target 'subdirs-install' failed
make[1]: *** [subdirs-install] Error 2
make[1]: Leaving directory '/home/mykroft/xen/src/xen-4.3.1/tools'
Makefile:74: recipe for target 'install-tools' failed
make: *** [install-tools] Error 2
==> ERROR: A failure occurred in package().
    Aborting...

Anyone have any ideas - google didnt help

Also using the compiled version from the dropbox link in the 1st post - looks for a bluez4, I cant not find that to install and have a bluez-5.....?

Packages (1): bluez-5.12-2

Thanks

Myk

Compile problems:

Use the package in my blog post.

http://blog.ktz.me/?p=219

Install it with

Pacman -U "pkgname"

As for the bluez thing just edit PKGBUILD to match your installed version.

Sent from my Nexus 5 using Tapatalk

Ironic, with everything else on your blog, feel free to post this fix there as well.

I experienced the error like 4 times, USB in a laptop and even a hdd on a normal computer.  I saw it with LVM and even normal partitions.  Each time I used the "dual" install media from Arch, not your noted "network" install media.  I would follow the install guides, get to the end reboot and boom, error: no such device found "uuid"

This prevented me from getting into the system and finally the only fix I found to work was booting to the CD again.  mounting the partitions as normal during the install, chroot into the system and then go to /boot/grub/  in the folder there will be a few grub.cfg files including a .new. 

I have now fixed this twice, once on my physical machine with normal hdd using LVM partitions and once on a USB stick on my laptop with normal partitions.

For both cases that I have fixed this issue, I renamed the existing .cfg to .bak and then renamed .new to .cfg/

mv /boot/grub/grub.cfg /boot/grub/grub.cfg.bak

cp /boot/grub/grub.cfg.new /boot/grub/grub.cfg

exit

umount -R /mnt

reboot

As noted, this fix worked for me twice now.  Maybe someone else can test the install and see if they can reproduce this.  Maybe it is a issue with the "dual" media, but in all instances I saw this.  So I am either doing something really wrong, there is an issue with the media or Arch simply does not like me...

Ironic, with everything else on your blog, feel free to post this fix there as well.

 

I experienced the error like 4 times, USB in a laptop and even a hdd on a normal computer.  I saw it with LVM and even normal partitions.  Each time I used the "dual" install media from Arch, not your noted "network" install media.  I would follow the install guides, get to the end reboot and boom, error: no such device found "uuid"

The problem is the fstab is not populated correctly.

That is due to a command that archboot (what Ironic uses in his blog to install Arch) runs that doesn't always populate the fstab with the correct UUIDs. It's hit or miss.

During an install using Archboot when you get to the configure your system screen:

Hit CTL + F1 or F2 until you get a screen that lets you drop to a command prompt.

When at the command prompt:

blkid

It will list all the UUIDs for your partitions.

Verify the correct UUID for /boot and root (/) or any other Arch partitions you set up (/var, /home, etc.) have the correct UUID in the fstab.

nano /install/etc/fstab

You can cut and paste the UUID from the blkid command one at a time.

If you already went through the install, you can easily fix it using the method below:

1. Reboot back into Archboot.

2. When the blue screen to install Archboot appears hit CTL+F1 or CTL+F2 until you get a screen where you can hit enter to get to a commmand prompt.

3. Mount the root (/) partition of where you installed Arch (I am assuming /dev/sda2):

mount /dev/sda2 /mnt

4. Prepare the boot partition to be mounted (I am assuming /dev/sda1).

mkdir -p /mnt/boot

5. Mount the boot (/boot) partition of where you installed it (I am assuming /dev/sda1):

mount /dev/sda1 /mnt/boot

6. Find out the UUIDs of your partitions.

blkid

7. Update the Arch install with the correct UUID for /boot.

Highlight the UUID for /boot with your mouse

nano /mnt/etc/fstab

Replace whatever UUID is there now with the correct UUID that you just cut and now need to paste (click the mouse button to paste).

Save it.

8.  Update the Arch install with the correct UUID for root (/).

Highlight the UUID for root (/) with your mouse.

nano /mnt/etc/fstab

Replace whatever UUID is there now with the correct UUID that you just cut and now need to paste (click the mouse button to paste).

Save it.

9. Do the above for any separate Arch partitions that you might have done during install (/home /var etc.)

10. We need to chroot into our Arch install.

arch-chroot /mnt /bin/bash

You are now logged into your Arch install. This is a VERY useful tool you might use in the future. I am just showing you how to fix grub and create a new one.

However, lets say you made a mistake and didn't load the correct driver into the ramfs and are unable to boot your system. You could go edit "/etc/mkinitcpio.conf" and add it and run "mkinitcpio -p linux" and update the ramfs with the correct driver loaded. It would save you from having to reinstall Arch.

11. Let's update grub with the new UUIDs we corrected earlier.

grub-mkconfig -o /boot/grub/grub.cfg

12. Exit out of your Arch Install

exit

13. umount your Arch Install

umount -R /mnt

14. Reboot

Everything sounds so much clearer when grumpy says it...

Ironic, have you removed step from your guide on your blog?  I thought it used to talk about the less privileged user setup for yaourt and xen, but that seems gone now along with a few other steps.  Just curious if I am just not seeing it or if things really got removed because the issues with the current package, hence the one you link special.

Also, is there a search option that I am also not seeing?  I thought there was one before, but I could be wrong.  Now that I got past the issue with booting my system, I wanted to proceed, with the guide, but I no longer see the steps I did the first pass.

Everything sounds so much clearer when grumpy says it...

I'm grumpy, fun, sarcastic and rude at times. Plus, I write good guides.

Probably the reasons people tolerate me and why I am not banned (yet).

Everything sounds so much clearer when grumpy says it...

 

Ironic, have you removed step from your guide on your blog?  I thought it used to talk about the less privileged user setup for yaourt and xen, but that seems gone now along with a few other steps.  Just curious if I am just not seeing it or if things really got removed because the issues with the current package, hence the one you link special.

 

Also, is there a search option that I am also not seeing?  I thought there was one before, but I could be wrong.  Now that I got past the issue with booting my system, I wanted to proceed, with the guide, but I no longer see the steps I did the first pass.

The yaourt stuff is in this post.

http://blog.ktz.me/?p=120

Search is done by typing the blog URL then a space. I haven't got round to fixing it yet...

Eg "blog.ktz.me yaourt"

Sent from my Nexus 5 using Tapatalk

Grumpy,  is your grub fix with the uuid any more right or better than the way I did it?

ok, used the package listed in the link:  http://blog.ktz.me/wp-content/uploads/2013/09/xen-4.3.1-2-x86_64.pkg_.tar.xz_.zip

used pacman -U to install it, seemed to go fine, but when trying to update the grub.cfg file with the following command I get:

[root@tcsserver ~]# grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found Xen hypervisor image: /boot/xen-4.3.1.gz
        Found linux image: /boot/vmlinuz-linux
        Found initrd image: /boot/initramfs-linux.img
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/initramfs-linux.img
error: out of memory.
error: syntax error.
error: Incorrect command.
error: syntax error.
Syntax error at line 183
Syntax errors are detected in generated GRUB config file.
Ensure that there are no errors in /etc/default/grub
and /etc/grub.d/* files or please file a bug report with
/boot/grub/grub.cfg.new file attached.done

I have used linux for several years, but this install is getting the better of me

Also I dont dare reboot now, as my installed kernel image is gone

[root@tcsserver boot]# ls -la
total 23692
drwxr-xr-x  3 root root     4096 Dec 26 20:12 .
drwxr-xr-x 17 root root     4096 Dec 25 16:21 ..
drwxr-xr-x  6 root root     4096 Dec 26 20:13 grub
-rwxr-xr-x  1 root root 16336033 Dec 25 22:50 initramfs-linux-fallback.img
-rwxr-xr-x  1 root root  3171662 Dec 25 22:49 initramfs-linux.img
-rwxr-xr-x  1 root root  3899664 Dec 12 04:58 vmlinuz-linux
-rwxr-xr-x  1 root root   829287 Dec 20 09:22 xen-4.3.1.gz
[root@tcsserver boot]#

suggestions on what to look for?

Thanks

Myk

Also I dont dare reboot now, as my installed kernel image is gone

It's bug. Well not really a bug. The Arch ISO isn't up to date with the new grub that installs. Arch Downloads the latest packages when it installs. Since they release an ISOs every month, the ISO you are using doesn't know a new setting in the new grub package. The fix will be in the new ISO on 01/14.

The solution until then...

echo GRUB_DISABLE_SUBMENU=y >> /etc/default/grub

then...

grub-mkconfig -o /boot/grub/grub.cfg

You should be good to go.

Grumpy,  is your grub fix with the uuid any more right or better than the way I did it?

If your boots into your system... Good! I would update grub again just to be on the safe side.

grub-mkconfig -o /boot/grub/grub.cfg

Thanks Grumpy - got that fixed, got my eno1 bridged...

question on that eno1 before the bridge is set to static 192.168.0.3

after the bridge activates eno1 no longer has a ip, and per the example I was following - xenbr0 gets a dhcp of 182.168.0.138.  Can I set that to static of the .3 and keep everything happy?

Thanks

Myk