htpcnewbie Posted December 7, 2014 Share Posted December 7, 2014 Hi, Continuing a discussion I started in another thread, opening up a new thread since the solution doesn't seem to be easy and didn't want to hijack another thread. The old thread where JustinChase had some suggestions is here http://lime-technology.com/forum/index.php?topic=36314.msg340969#msg340969 The status is I am setting up a new machine intended for GPU pass through and having issues with it. When I define the qemu parameters the VM does not start and get "error" status on the bottom of the VM window. This is a i7-4790 on a Asrock Z97 extreme 6 motherboard (both are vt-d capable and user JustinChase is using this mobo with good success). The complete system parts are below in my sig. The GPU is a Radeon 5450 test card, I intend to move my current R9 280x gaming+HTPC card to this new machine once everything starts working. I am trying to setup a Win7 Pro VM, Justin as well as others have suggested Win8 is more stable but given I had few spare licences of Win7, I am trying to set this up. Checked/changes I have made so far: 1. Ensured vt-d is enabled in bios 2. updated /boot/config/syslinux/syslinux.cfg to enable "append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot" 3. Created a vm template and checked that Win7 VM is running. Generated the xml and added sections toward the beginning and end as discussed in the above post 4. Removed from go file Added the following vfio-bind in /boot/config/go "/usr/local/sbin/vfio-bind 0000:01:00.0 0000:01:00.1" to reflect the pcie address of the video and hdmi audio addresses (ran it interactively in the terminal and also rebooted the system) I have spent most of the last 3 nights trying to figure this out and appreciate any suggestions to trouble shoot this issue. Is the last few lines of qemu, is the definition of "ioh3420" needed? I do not know what this refers to. My xml: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win7x64vtdxml2</name> <uuid>1ea9db75-b2ec-4990-9c1f-458275612529</uuid> <description>win7x64vtd_xml2</description> <memory unit='KiB'>13107200</memory> <currentMemory unit='KiB'>12091392</currentMemory> <vcpu placement='static'>4</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> </os> <features> <acpi/> <apic/> </features> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='yes'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/mnt/disk/SSD1/vm/image/Win7x64.qcow2'/> <backingStore/> <target dev='hda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/X17-24281.iso'/> <backingStore/> <target dev='hdc' bus='ide'/> <readonly/> <alias name='ide0-1-0'/> <address type='drive' controller='0' bus='1' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/virtio-win-0.1-81.iso'/> <backingStore/> <target dev='hdd' bus='ide'/> <readonly/> <alias name='ide0-1-1'/> <address type='drive' controller='0' bus='1' target='0' unit='1'/> </disk> <controller type='usb' index='0'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <interface type='bridge'> <mac address='52:54:00:0e:6d:2d'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/1'/> <target port='0'/> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/1'> <source path='/dev/pts/1'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc223'/> <address bus='3' device='4'/> </source> <alias name='hostdev0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc226'/> <address bus='3' device='5'/> </source> <alias name='hostdev1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0458'/> <product id='0x0007'/> <address bus='3' device='6'/> </source> <alias name='hostdev2'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/> </qemu:commandline> </domain> In the above xml if I remove all the qemu:arg definitions, the vm domain is created and I can remote desktop into this machine. The xml has embedded usb passthru I was trying to achieve and I think it works but cannot confirm. The keyboard and its attached usb mice that I intend to passthru are disabled as soon as I start the VM but I cannot confirm they are working unless I have an working screen. The output from my VM > devices > pci Device Identification Driver Vendor Product XML pci_0000_00_00_0 0x8086:0x0c00 hsw_uncore Intel Corporation Haswell DRAM Controller pci_0000_00_01_0 0x8086:0x0c01 pcieport Intel Corporation Haswell PCI Express x16 Controller pci_0000_01_00_0 0x1002:0x68f9 None AMD/ATI [Advanced Micro Devices, Inc.] Cedar [Radeon HD 5000/6000/7350 Series] pci_0000_01_00_1 0x1002:0xaa68 None AMD/ATI [Advanced Micro Devices, Inc.] Cedar HDMI Audio [Radeon HD 5400/6300 Series] pci_0000_00_01_1 0x8086:0x0c05 pcieport Intel Corporation Haswell PCI Express x8 Controller pci_0000_02_00_0 0x1b4b:0x9485 mvsas Marvell Technology Group Ltd. Unknown pci_0000_00_02_0 0x8086:0x0412 None Intel Corporation Haswell Integrated Graphics Controller pci_0000_00_03_0 0x8086:0x0c0c None Intel Corporation Haswell HD Audio Controller pci_0000_00_14_0 0x8086:0x8cb1 xhci_hcd Intel Corporation Unknown pci_0000_00_16_0 0x8086:0x8cba None Intel Corporation Unknown pci_0000_00_19_0 0x8086:0x15a1 e1000e Intel Corporation Unknown pci_0000_00_1a_0 0x8086:0x8cad ehci-pci Intel Corporation Unknown pci_0000_00_1b_0 0x8086:0x8ca0 None Intel Corporation Unknown pci_0000_00_1c_0 0x8086:0x8c90 pcieport Intel Corporation Unknown pci_0000_00_1c_2 0x8086:0x8c94 pcieport Intel Corporation Unknown pci_0000_04_00_0 0x10ec:0x8168 r8169 Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller pci_0000_00_1c_3 0x8086:0x8c96 pcieport Intel Corporation Unknown pci_0000_05_00_0 0x1b21:0x1184 pcieport ASMedia Technology Inc. Unknown pci_0000_06_01_0 0x1b21:0x1184 pcieport ASMedia Technology Inc. Unknown pci_0000_06_03_0 0x1b21:0x1184 pcieport ASMedia Technology Inc. Unknown pci_0000_08_00_0 0x1b21:0x0612 ahci ASMedia Technology Inc. ASM1062 Serial ATA Controller pci_0000_06_05_0 0x1b21:0x1184 pcieport ASMedia Technology Inc. Unknown pci_0000_06_07_0 0x1b21:0x1184 pcieport ASMedia Technology Inc. Unknown pci_0000_0a_00_0 0x1b21:0x0612 ahci ASMedia Technology Inc. ASM1062 Serial ATA Controller pci_0000_00_1c_6 0x8086:0x8c9c pcieport Intel Corporation Unknown pci_0000_0b_00_0 0x1b21:0x1142 xhci_hcd ASMedia Technology Inc. Unknown pci_0000_00_1d_0 0x8086:0x8ca6 ehci-pci Intel Corporation Unknown pci_0000_00_1f_0 0x8086:0x8cc4 None Intel Corporation Unknown pci_0000_00_1f_2 0x8086:0x8c82 ahci Intel Corporation Unknown pci_0000_00_1f_3 0x8086:0x8ca2 None Intel Corporation Unknown Link to comment
JustinChase Posted December 7, 2014 Share Posted December 7, 2014 Here is my XML file, which is what I'm currently using for my Windows8 VM. It's basically the same as the one I used for Windows7... <domain type='kvm' id='2' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>Windows8_Passthru</name> <uuid>2e151e44-f801-4773-a6df-bfb01b0812fb</uuid> <description>Windows8 with Passthru</description> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>2</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/cache/VM/Windows8.qcow2'/> <backingStore/> <target dev='vdc' bus='virtio'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/backup/Programs/WindowsISOs/en_windows_8_1_x64_dvd_2707217.iso'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <alias name='sata0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/backup/Programs/WindowsISOs/virtio-win-0.1-81.iso'/> <backingStore/> <target dev='sdd' bus='sata'/> <readonly/> <alias name='sata0-0-3'/> <address type='drive' controller='0' bus='0' target='0' unit='3'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb0'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb0'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb0'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x2'/> </controller> <controller type='sata' index='0'> <alias name='sata0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'> <alias name='pcie.0'/> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='00:16:3e:c8:c8:c8'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> </interface> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x050d'/> <product id='0x0201'/> <address bus='3' device='4'/> </source> <alias name='hostdev0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x045e'/> <product id='0x0745'/> <address bus='3' device='8'/> </source> <alias name='hostdev1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0a12'/> <product id='0x0001'/> <address bus='3' device='9'/> </source> <alias name='hostdev2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x147a'/> <product id='0xe03e'/> <address bus='3' device='6'/> </source> <alias name='hostdev3'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> <seclabel type='none' model='none'/> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=07:00.0,bus=pcie.0'/> </qemu:commandline> </domain> I don't know what the first line is for; but pretty much every XML I have seen has this, and it works fine with it, so I suggest you just keep it as is. The last 2 lines, with the host=07:00.0 part is for my TV Tuner card, so you can remove it. Any lines I have above which contain an address can almost certainly be removed; libvirt will add then back as necessary. Having them wrong is likely to cause problems. Also, you've got a lot of memory assigned. I've read/seen instances where too much can cause problems. I'd lower it to 2 or maybe 4GB of memory assigned, to eliminate that as a source of problems. You also have different settings for os type in your XML. I would change that to match what I have, again, to eliminate sources of your troubles. In fact, I'd suggest you take my XML, change the path information and name/description, remove all the address stuff, and the last 2 lines (to remove my TV Tuner card), and see if that works. Our video and audio addresses are the same. Better yet, try this one, which is the one I use to connect to my VM with VNC. I modified it to match your name and path info. <domain type='kvm'> <name>win7x64vtdxml2</name> <uuid>1ea9db75-b2ec-4990-9c1f-458275612529</uuid> <description>win7x64vtd_xml2 on VNC</description> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>2</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='cdrom'/> <bootmenu enable='yes'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/disk/SSD1/vm/image/Win7x64.qcow2'/> <backingStore/> <target dev='vdc' bus='virtio'/> <alias name='virtio-disk2'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/X17-24281.iso'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <alias name='sata0-0-0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/virtio-win-0.1-81.iso'/> <backingStore/> <target dev='sdd' bus='sata'/> <readonly/> <alias name='sata0-0-3'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x2'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:0e:6d:2d'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> </interface> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='5700' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='cirrus' vram='9216' heads='1'/> </video> <memballoon model='virtio'> <alias name='balloon0'/> </memballoon> </devices> </domain> If that works for your VNC connection, you could try this one for full passthru (after rebooting the server)... <domain type='kvm' id='2' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win7x64vtdxml2</name> <uuid>1ea9db75-b2ec-4990-9c1f-458275612520</uuid> <description>win7x64vtd_xml2 with Passthru</description> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>2</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/disk/SSD1/vm/image/Win7x64.qcow2'/> <backingStore/> <target dev='vdc' bus='virtio'/> <alias name='virtio-disk2'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/X17-24281.iso'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <alias name='sata0-0-0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/virtio-win-0.1-81.iso'/> <backingStore/> <target dev='sdd' bus='sata'/> <readonly/> <alias name='sata0-0-3'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb0'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb0'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb0'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x2'/> </controller> <controller type='sata' index='0'> <alias name='sata0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'> <alias name='pcie.0'/> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:0e:6d:3d'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> </interface> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc223'/> </source> <alias name='hostdev0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc226'/> </source> <alias name='hostdev1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0458'/> <product id='0x0007'/> </source> <alias name='hostdev2'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> </memballoon> </devices> <seclabel type='none' model='none'/> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/> </qemu:commandline> </domain> Link to comment
htpcnewbie Posted December 7, 2014 Author Share Posted December 7, 2014 Justin - Thanks for effort in modifying ^^^^ the xml's to match my setup! I copied the one with VNC and it worked. I use remote desktop than VNC (works even if vnc video part is not defined) since it is much faster and emulates the screen better. RD worked well but the VNC gave a blank page. I copied the next xml with the passthrough (changed the uuid), rebooted the server. With the passthru, it is a no-go. I get an error while trying to start the vm. In your qemu definitions you have a device with value "ioh3420". What does this correspond to? I looked in your pci device as well as mine and do not see this device value. I have video out plugged in DVI to this video card, does that matter? At this point, I am shooting darts in the dark and do not know which could remotely influence this setup. Anything else I can try? Thanks! Link to comment
JustinChase Posted December 8, 2014 Share Posted December 8, 2014 Where did you get an error, and what was the error? Are you using the vmMan plugin? Do you have virtman installed also? Are they both up to date? There was an older vmMan plugin, which was replaced by a newer version. Make sure you're using the newer version. The device with value "ioh3420" is something in all other XML examples I've seen. I don't know what it is, or why it's there, but it seems to be necessary. I removed it once before, and the VM didn't work. I suspect using DVI instead of the HDMI might be an/the issue. I suspect the VM needs/expects both video and audio thru the port, and DVI doesn't include audio, so that's likely a/the problem. Can you try with HDMI, to see if that works? If it does, then at least you know where to investigate next. Also, just to make sure; you're connecting to the video card, not the onboard video connection. Also, can you confirm that you don't have anything in your go file related to passthru? I think you said you had something in the other post; but it was only needed for XEN, not KVM, so it should be removed. As for the syslinux.cfg file, here is how/where I have modified mine... label KVM/unRAID OS menu default kernel /bzimage append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot I can't think of anything else that might need done/changed, other than the above, sorry. Link to comment
htpcnewbie Posted December 8, 2014 Author Share Posted December 8, 2014 Moved the videocard output to hdmi, still have issues. When I try to start the vm in the KVM page, I get an "error" message in the bottom of the page and nothing happens. I did do a fresh install of both the VM manager and virtman plugin from http://lime-technology.com/forum/index.php?topic=35858.0 couple of days ago. To try and see whether I can get more error output, I created the xml file and attempted to start the vm manually. I get the following error: root@Tower:/mnt/disk/SSD1/vm/virsh# virsh create win7vtdxml_JustNoVNC.xml error: Failed to create domain from win7vtdxml_JustNoVNC.xml error: internal error: early end of file from monitor: possible problem: 2014-12-08T03:01:04.357989Z qemu-system-x86_64: -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1: Bus 'pcie.0' not found The error seems to be indicating to the device "ioh3420". We both have the same MB and from what I understand, this is a generic definition. What gives, any pointers? I used the last pass thru xml you suggested. EDIT: When I comment out the first device in the qemu commands, I get the following error for the videocard definition root@Tower:/mnt/disk/SSD1/vm/virsh# virsh create win7vtdxml_JustNoVNC-2.xml error: Failed to create domain from win7vtdxml_JustNoVNC-2.xml error: internal error: early end of file from monitor: possible problem: 2014-12-08T03:07:29.347862Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Bus 'root.1' not found It seems to be pointing to an incorrect bus definition. Where did you get an error, and what was the error? Are you using the vmMan plugin? Do you have virtman installed also? Are they both up to date? There was an older vmMan plugin, which was replaced by a newer version. Make sure you're using the newer version. The device with value "ioh3420" is something in all other XML examples I've seen. I don't know what it is, or why it's there, but it seems to be necessary. I removed it once before, and the VM didn't work. I suspect using DVI instead of the HDMI might be an/the issue. I suspect the VM needs/expects both video and audio thru the port, and DVI doesn't include audio, so that's likely a/the problem. Can you try with HDMI, to see if that works? If it does, then at least you know where to investigate next. Also, just to make sure; you're connecting to the video card, not the onboard video connection. Also, can you confirm that you don't have anything in your go file related to passthru? I think you said you had something in the other post; but it was only needed for XEN, not KVM, so it should be removed. As for the syslinux.cfg file, here is how/where I have modified mine... label KVM/unRAID OS menu default kernel /bzimage append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot I can't think of anything else that might need done/changed, other than the above, sorry. Link to comment
JustinChase Posted December 8, 2014 Share Posted December 8, 2014 Hmmm... it seems it's not a problem with DVI (good for your future ) Can you please post your full syslinux.cfg, go file, passthru XML, and plugin versions? Also, your HDMI is connected to the video card, not the motherboard output; right? (sorry if that's a silly question; I'm grasping at things now) Oh, you said you have VT-d enabled in BIOS; but I believe there are 2 settings; one is for virtualization, an another for VT-d. Are you certain you have VT-d enabled, and not just virtualization. Last thing I can think of; do you have the primary display set in BIOS to onboard or PCI-e? I believe I have mine set to onboard; not sure it makes any difference though. Again, grasping here. It does seem like it's a problem with the bus setting; but I'm not sure where that problem is coming up. Also; I'm certainly no expert in this stuff, so hopefully someone else has some ideas for you. Link to comment
htpcnewbie Posted December 8, 2014 Author Share Posted December 8, 2014 Justin - Really appreciate your suggestions and help in troubleshooting this, thank you! I checked in bios again .. "Intel virtualization is enabled" in Advanced -> chipset and "VT-D" is enabled in "Advanced -> CPU". My current graphics settings were onboard as primary with multi IGPU turned on. I believe the default is off, I turned it off. I checked again, only a sngle hdmi cable is plugged in to the video card. I have attached the xml file (no vnc, gpu pass thru), go and syslinux.cfg files. I have also attached a pdf of the plugin page which shows all the plugins and their versions. I tried again after rebooting the server, still get the same error root@Tower:/mnt/disk/SSD1/vm/virsh# virsh create win7vtdxml_JustNoVNC.xml error: Failed to create domain from win7vtdxml_JustNoVNC.xml error: internal error: early end of file from monitor: possible problem: 2014-12-08T03:59:04.935878Z qemu-system-x86_64: -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1: Bus 'pcie.0' not found Hmmm... it seems it's not a problem with DVI (good for your future ) Can you please post your full syslinux.cfg, go file, passthru XML, and plugin versions? Also, your HDMI is connected to the video card, not the motherboard output; right? (sorry if that's a silly question; I'm grasping at things now) Oh, you said you have VT-d enabled in BIOS; but I believe there are 2 settings; one is for virtualization, an another for VT-d. Are you certain you have VT-d enabled, and not just virtualization. Last thing I can think of; do you have the primary display set in BIOS to onboard or PCI-e? I believe I have mine set to onboard; not sure it makes any difference though. Again, grasping here. It does seem like it's a problem with the bus setting; but I'm not sure where that problem is coming up. Also; I'm certainly no expert in this stuff, so hopefully someone else has some ideas for you. win7vtdxml_JustNoVNC.xml.txt go.txt syslinux.cfg.txt Tower_Plugins.pdf Link to comment
JustinChase Posted December 8, 2014 Share Posted December 8, 2014 Okay, a couple things. You seem to have "Virtual Machine Management" in there twice. The first one shown is outdated; the second one appears to be the same version I have installed. I'd try uninstalling the first one. This might actually uninstall both versions, so you may have to reinstall the new version when done. In fact, since it's so fast/easy to do, I might uninstall them both, reboot, then install the new one fresh. Second thing, did you try using the XML I posted earlier*? The one you posted above is not the one I modified for you. it's got too much memory, some lines I'm not sure about, and generally not at all what I posted. If you've not tried it yet, can you please try using the one I modified for you, as is, with no changes. I've modified it to point to your paths, and removed all the stuff I don't think you need. If that works, you can look at modifying it after, adding things one at a time until you find whatever is breaking it. Besides passing a lot of memory to the VM, you're using a different machine type than me. I'm not sure if that machine type will work. Another thing, which I know can cause issues, is your XML has lots of address stuff in it. if any of it is 'wrong' the VM will fail to start. The XML I posted for you has this all removed. libvirt will add all the address stuff back in upon running, so don't worry about removing it. I know my XML works, so modifying it as little as possible to see if it works for you seems a good start. Please create a new VM by clicking the Add XML tab, pasting the XML into the box and hitting save. Then when you go back to the Virtual Machines tab, you should see a new VM entitled win7x64-test The go file has a LOT of extra stuff in it. I'm sure much/most/all is necessary for the other stuff you're running, but it could also be causing the error you're seeing (not likely, but again, grasping). * I made one more change (the name) to make sure it doesn't conflict with any other of your VM's. <domain type='kvm' id='2' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win7x64-test</name> <uuid>1ea9db75-b2ec-4990-9c1f-458275612520</uuid> <description>win7x64vtd_xml2 with Passthru</description> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>2</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/disk/SSD1/vm/image/Win7x64.qcow2'/> <backingStore/> <target dev='vdc' bus='virtio'/> <alias name='virtio-disk2'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/X17-24281.iso'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <alias name='sata0-0-0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disk/SSD1/vm/media/virtio-win-0.1-81.iso'/> <backingStore/> <target dev='sdd' bus='sata'/> <readonly/> <alias name='sata0-0-3'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb0'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb0'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb0'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x2'/> </controller> <controller type='sata' index='0'> <alias name='sata0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'> <alias name='pcie.0'/> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:0e:6d:3d'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> </interface> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc223'/> </source> <alias name='hostdev0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc226'/> </source> <alias name='hostdev1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0458'/> <product id='0x0007'/> </source> <alias name='hostdev2'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> </memballoon> </devices> <seclabel type='none' model='none'/> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/> </qemu:commandline> </domain> Link to comment
JustinChase Posted December 8, 2014 Share Posted December 8, 2014 Also, the Docker Manager plugin is no longer needed, and LimeTech suggests it be removed. Certainly not related to your VM issues, just something I noticed. Link to comment
johnodon Posted December 8, 2014 Share Posted December 8, 2014 The go file has a LOT of extra stuff in it. I'm sure much/most/all is necessary for the other stuff you're running, but it could also be causing the error you're seeing (not likely, but again, grasping). Man...I have never seen so much crap in a GO file. Unless I am missing it (still waking up) or you are doing it manually for now, I do not see a vfio-bind line in your GO script. This is the contents of my GO script: #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp & /usr/local/sbin/vfio-bind 0000:83:00.0 0000:83:00.1 0000:86:00.0 0000:86:00.1 Here it is adapted for your vid card (unless you have since moved the card to a different PCIE slot): /usr/local/sbin/vfio-bind 0000:01:00.0 0000:01:00.1 If you want to test prior to adding that line to the GO script and rebooting, just type it into a terminal and try to fire up your VM again. the bind itself does not require a reboot. John Link to comment
JustinChase Posted December 8, 2014 Share Posted December 8, 2014 The go file has a LOT of extra stuff in it. I'm sure much/most/all is necessary for the other stuff you're running, but it could also be causing the error you're seeing (not likely, but again, grasping). Man...I have never seen so much crap in a GO file. Unless I am missing it (still waking up) or you are doing it manually for now, I do not see a vfiobind line in your GO script. This is the contents of my GO script: #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp & /usr/local/sbin/vfio-bind 0000:83:00.0 0000:83:00.1 0000:86:00.0 0000:86:00.1 That's not needed for KVM, only for XEN. Nope; I was totally wrong. You used to need to put that in the syslinux.cfg, which you don't now need for KVM, but you do, in fact, need that in the go file. htpcnewbie, here is what you need to add. I suggest adding it near the top of your go file, but I'm not sure it matters. Sorry for not thinking of this before /usr/local/sbin/vfio-bind 0000:01:00.0 0000:01:00.1 Link to comment
htpcnewbie Posted December 8, 2014 Author Share Posted December 8, 2014 Yayy, success!!! well .. key part. Thank you Justin and John. I uninstalled the libvrt and virt manager, rebooted the server and made a fresh install. Added the vfio bind to go script and rebooted again so that the KVM tab is populated. I used the xml that Justin sent in above post and worked like a charm. The video is passed through the hdmi but I don't seem to be getting the audio. From the vfio bind and and the xml, the hdmi audio also seems to be indicated for passthrough but having trouble with it (no sound for now). Needs some more debugging but the basics of having a videocard passthrough is working. Can you point me to a robust way of passing through usb devices - I have a Logitech keyboard with touchpad (ir dongle), HP IR remote controller for Jriver MC. EDIT: I plugged in a usb hub with few devices connected to it. Didn't have to restart the server and I see the following in the "usb devices" section. Can we pass all devices connected to a hub for passthrough or do they have to done individually? Are all usb devices treated equally (for passthrough) or need specific passthrough based on functionality? usb_3_9_4_4_4 0x05e3:0x0608 usb Unknown USB2.0 Hub usb_3_9_4_4_4_1 0x05e3:0x0608 usb Unknown USB2.0 Hub usb_3_9_4_4_4_2 0x0471:0x0815 usb Philips eHome Infrared Transceiver usb_3_9_4_4_4_4 0x046d:0x0b04 usb Logitech Logitech BT Mini-Receiver usb_3_9_4_4_4_1_1 0x046d:0x082d usb Unknown HD Pro Webcam C920 usb_3_9_4_4_4_1_2 0x04d8:0x003f usb DSP4YOU LTD. miniDSP v1.2 usb_3_9_4_4_4_4_2 0x046d:0xc713 usb Logitech Logitech BT Mini-Receiver usb_3_9_4_4_4_4_3 0x046d:0xc714 usb Logitech Logitech BT Mini-Receiver Link to comment
JustinChase Posted December 8, 2014 Share Posted December 8, 2014 Yayy, success!!! ... Thank you Justin and John. I used the xml that Justin sent in above post and worked like a charm. Excellent, happy to hear it. Sorry I forgot the go file addition; you could have probably been done sooner The video is passed through the hdmi but I don't seem to be getting the audio. From the vfio bind and and the xml, the hdmi audio also seems to be indicated for passthrough but having trouble with it (no sound for now). Needs some more debugging but the basics of having a videocard passthrough is working. A couple of ideas. If you already have JRiver installed, make sure the WDM driver is not selected; control panel > Sound > Make sure the HDMI is default (might have your TV name here). If you want to keep the WDM as default, JRiver has to be running to hear anything. You can set it up to launch Media Server at windows startup, which will work, if you don't want the full player running all the time. I do it this way, so that I can connect to the server library any time (including while away from the house, via Gizmo). If that's not the problem, go into device manager, and see if there are any sound devices installed. You probably have to install the video card drivers now that you've successfully passed the GPU thru to windows for the first time; which should also install the sound card drivers. Can you point me to a robust way of passing through usb devices - I have a Logitech keyboard with touchpad (ir dongle), HP IR remote controller for Jriver MC. EDIT: I plugged in a usb hub with few devices connected to it. Didn't have to restart the server and I see the following in the "usb devices" section. Can we pass all devices connected to a hub for passthrough or do they have to done individually? Are all usb devices treated equally (for passthrough) or need specific passthrough based on functionality? usb_3_9_4_4_4 0x05e3:0x0608 usb Unknown USB2.0 Hub usb_3_9_4_4_4_1 0x05e3:0x0608 usb Unknown USB2.0 Hub usb_3_9_4_4_4_2 0x0471:0x0815 usb Philips eHome Infrared Transceiver usb_3_9_4_4_4_4 0x046d:0x0b04 usb Logitech Logitech BT Mini-Receiver usb_3_9_4_4_4_1_1 0x046d:0x082d usb Unknown HD Pro Webcam C920 usb_3_9_4_4_4_1_2 0x04d8:0x003f usb DSP4YOU LTD. miniDSP v1.2 usb_3_9_4_4_4_4_2 0x046d:0xc713 usb Logitech Logitech BT Mini-Receiver usb_3_9_4_4_4_4_3 0x046d:0xc714 usb Logitech Logitech BT Mini-Receiver I don't know of a good way to pass thru the whole USB controller/hub. I'm currently passing thru each device individually. Here is the section of the XML doing that... <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc223'/> </source> <alias name='hostdev0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc226'/> </source> <alias name='hostdev1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0458'/> <product id='0x0007'/> </source> <alias name='hostdev2'/> </hostdev> I customized your XML to match the devices from your original XML; but the addresses could have changed since. Just replace the vendor id and product id for each line to match whatever device you want to pass thru. If you need to add more, just copy/paste the whole section (from hostdev mode to /hostdev) for each new device. In theory (I've not tried it yet myself, but plan to soon), you could pass thru the PCI controller the USB hub is connected to, and then anything you connect to the HUB (the whole PCI controller actually), should get passed thru without having to add it manually. I'm wanting to do this so I can connect a thumb drive, or external hard drive directly to my windows VM. The trick here is that I'm not sure exactly which USB ports go to which controller, and if you passthru a USB hub with the unRAID thumbdrive (for example) you'll hose the server until you reboot. So, it will take a bit of trial and error, but I'm pretty sure it will work. Keep us posted on your progress. **Also, remember the pay it forward rule. Now you have to help the next person having trouble, since you now know how to do all this fun/crazy stuff Link to comment
johnodon Posted December 8, 2014 Share Posted December 8, 2014 You probably have to install the video card drivers now that you've successfully passed the GPU thru to windows for the first time; which should also install the sound card drivers. +1 Link to comment
htpcnewbie Posted December 9, 2014 Author Share Posted December 9, 2014 Thanks John and Justin, I finally got things working. The issue with HDMI audio seems to be related to the HD5450 card, the audio drivers did not install. I spent a good 2 hours trying to uninstall, reinstall and upgrade the drivers but it didn't work. I finally gave up and moved my R9-280x to this machine, it worked like a charm. Everything is working well as expected, thanks again folks!! Justin, thanks for the reminder. I will do my best to give forward You mentioned about about Win7 being unstable, can you please explain on how this behaved for you? Any best practices you recommend to get this as stable as possible? I do not have a Win8 license and trying to minimize further spending on this setup. I have few extra Win7 licenses though! About the docker and the go files, I need to take sometime to clean them up, hardly get time at home. Thanks again guys, it would have taken forever if it weren't for your help. Link to comment
JustinChase Posted December 9, 2014 Share Posted December 9, 2014 Thanks John and Justin, I finally got things working. The issue with HDMI audio seems to be related to the HD5450 card, the audio drivers did not install. I spent a good 2 hours trying to uninstall, reinstall and upgrade the drivers but it didn't work. I finally gave up and moved my R9-280x to this machine, it worked like a charm. Everything is working well as expected, thanks again folks!! Justin, You mentioned about about Win7 being unstable, can you please explain on how this behaved for you? Any best practices you recommend to get this as stable as possible? good to hear you got it working finally. I'm not sure why the HD5450 didn't work for you. I had an HD5550 in mine a couple months ago, but it died, so I went to the only other card I had, my nVidia. I don't recall saying Windows7 was unstable for me; just that it didn't do passthru upon rebooting, without rebooting the whole server. I'm not sure if this was something specific to my hardware; or a windows7 thing. I think others have reported the same issue with windows7; but can't point to anyone specific though. I think once you get windows7 all setup and running, it should be plenty stable. I was having a lot of other issues with my server, so I bit the bullet and bought a windows8 license. i used my sons student discount, so it only cost me $70. I'd rather have not spent the money, and prefer windows7, but it was money well spent in the end. I pretty much only use this VM for running JRiver, and it's worked great since I got it all running. Anyway, I suspect you'll be fine with windows7. Also, you can try bumping up your memory if you want. I'm not positive that 12GB is too much for a KVM VM. I just wanted to eliminate any reasons things might not be working. Whatever you do though, do one thing at a time, so that if something stops working, you should know what caused it. Have fun! Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.