EdgarWallace Posted October 21, 2014 Share Posted October 21, 2014 I was trying to set up my Windows 8 VM (on my test machine) by using dmacias excellent Virtual Machine Management and the Web Virtual Manager support. The objective is to passthrough the NVIDIA GF106GL [Quadro 2000] video card along with the NVIDIA GF106 HD Audio Controller and making the DVD ROM available for that particular VM. vfio-bind 0000:01:00.0 0000:01:00.1 is working: Kernel driver in use: vfio-pci (please see below): root@Tower2:/mnt/cache/.VMs/win8# lspci -knn 00:00.0 Host bridge [0600]: Intel Corporation Core Processor DRAM Controller [8086:0040] (rev 12) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] 00:01.0 PCI bridge [0604]: Intel Corporation Core Processor PCI Express x16 Root Port [8086:0041] (rev 12) Kernel driver in use: pcieport 00:16.0 Communication controller [0780]: Intel Corporation 5 Series/3400 Series Chipset HECI Controller [8086:3b64] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] 00:1a.0 USB controller [0c03]: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller [8086:3b3c] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] Kernel driver in use: ehci-pci 00:1b.0 Audio device [0403]: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio [8086:3b56] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8435] 00:1c.0 PCI bridge [0604]: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 [8086:3b42] (rev 06) Kernel driver in use: pcieport 00:1c.2 PCI bridge [0604]: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 3 [8086:3b46] (rev 06) Kernel driver in use: pcieport 00:1c.3 PCI bridge [0604]: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 4 [8086:3b48] (rev 06) Kernel driver in use: pcieport 00:1c.4 PCI bridge [0604]: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 5 [8086:3b4a] (rev 06) Kernel driver in use: pcieport 00:1c.5 PCI bridge [0604]: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 6 [8086:3b4c] (rev 06) Kernel driver in use: pcieport 00:1d.0 USB controller [0c03]: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller [8086:3b34] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] Kernel driver in use: ehci-pci 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev a6) 00:1f.0 ISA bridge [0601]: Intel Corporation 5 Series Chipset LPC Interface Controller [8086:3b06] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] 00:1f.2 SATA controller [0106]: Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller [8086:3b22] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] Kernel driver in use: ahci Kernel modules: ahci 00:1f.3 SMBus [0c05]: Intel Corporation 5 Series/3400 Series Chipset SMBus Controller [8086:3b30] (rev 06) Subsystem: ASUSTeK Computer Inc. Device [1043:8383] Kernel modules: i2c_i801 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF106GL [Quadro 2000] [10de:0dd8] (rev a1) Subsystem: Hewlett-Packard Company Device [103c:084a] Kernel driver in use: vfio-pci 01:00.1 Audio device [0403]: NVIDIA Corporation GF106 High Definition Audio Controller [10de:0be9] (rev a1) Subsystem: Hewlett-Packard Company Device [103c:084a] Kernel driver in use: vfio-pci 03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller [10ec:8168] (rev 03) Subsystem: ASUSTeK Computer Inc. M4A785TD Motherboard [1043:83a3] Kernel driver in use: r8169 Kernel modules: r8169 05:00.0 Multimedia video controller [0400]: Micronas Semiconductor Holding AG nGene PCI-Express Multimedia Controller [18c3:0720] (rev 01) Subsystem: Micronas Semiconductor Holding AG Device [18c3:db02] 06:00.0 IDE interface [0101]: JMicron Technology Corp. JMB368 IDE controller [197b:2368] Subsystem: ASUSTeK Computer Inc. Device [1043:824f] Kernel driver in use: pata_jmicron Kernel modules: pata_jmicron 3f:00.0 Host bridge [0600]: Intel Corporation Core Processor QuickPath Architecture Generic Non-core Registers [8086:2c61] (rev 02) Subsystem: Intel Corporation Device [8086:8086] 3f:00.1 Host bridge [0600]: Intel Corporation Core Processor QuickPath Architecture System Address Decoder [8086:2d01] (rev 02) Subsystem: Intel Corporation Device [8086:8086] 3f:02.0 Host bridge [0600]: Intel Corporation Core Processor QPI Link 0 [8086:2d10] (rev 02) Subsystem: Intel Corporation Device [8086:8086] 3f:02.1 Host bridge [0600]: Intel Corporation Core Processor QPI Physical 0 [8086:2d11] (rev 02) Subsystem: Intel Corporation Device [8086:8086] 3f:02.2 Host bridge [0600]: Intel Corporation Core Processor Reserved [8086:2d12] (rev 02) Subsystem: Intel Corporation Device [8086:8086] 3f:02.3 Host bridge [0600]: Intel Corporation Core Processor Reserved [8086:2d13] (rev 02) Subsystem: Intel Corporation Device [8086:8086] Here's my xml file (I have no clue where I got the controller information from - this VM was running already a while ago): <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win8</name> <uuid>cc411d70-4463-4db7-bf36-d364c0cdaa9d</uuid> <description>None</description> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <loader>/usr/share/qemu/bios-256k.bin</loader> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> <pae/> </features> <cpu mode='custom' match='exact'> <model fallback='allow'>SandyBridge</model> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/mnt/cache/.VMs/win8/win8.qcow2'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </disk> <device> <name>block_sr0_HL_DT_ST_DVDRAM_GH24NS95_K3XD1JC0659</name> <path>/sys/devices/pci0000:00/0000:00:1c.0/0000:06:00.0/ata1/host1/target1:0:1/1:0:1:0/block/sr0</path> <parent>scsi_1_0_1_0</parent> <capability type='storage'> <block>/dev/sr0</block> <bus>ata</bus> <drive_type>cdrom</drive_type> <model>DVDRAM GH24NS95</model> <vendor>HL-DT-ST</vendor> <serial>HL-DT-ST_DVDRAM_GH24NS95_K3XD1JC0659</serial> <capability type='removable'> <media_available>0</media_available> <media_size>0</media_size> </capability> </capability> </device> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='02:16:3e:1a:b3:4a'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> </domain> Error code: XML error: PCI bus is not compatible with the device at 0000:00:03.0. Device requires a standard PCI slot, which is not provided by bus 0000:00 I was using the Node information for the DVD ROM <device> but I might need to completely start over again by using the existing win8.qcow2 file. So what is best practice? Quote Link to comment
EdgarWallace Posted October 24, 2014 Author Share Posted October 24, 2014 I used copy & past from an existing VM: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win8</name> <uuid>cc411d70-4463-4db7-bf36-d364c0cdaa9d</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> <pae/> </features> <cpu mode='host-passthrough'> <topology sockets='2' cores='2' threads='1'/> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/cache/.VMs/win8/win8.qcow2'/> <target dev='vda' bus='virtio'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <device> <name>block_sr0_HL_DT_ST_DVDRAM_GH24NS95_K3XD1JC0659</name> <path>/sys/devices/pci0000:00/0000:00:1c.0/0000:06:00.0/ata1/host1/target1:0:1/1:0:1:0/block/sr0</path> <parent>scsi_1_0_1_0</parent> <capability type='storage'> <block>/dev/sr0</block> <bus>ata</bus> <drive_type>cdrom</drive_type> <model>DVDRAM GH24NS95</model> <vendor>HL-DT-ST</vendor> <serial>HL-DT-ST_DVDRAM_GH24NS95_K3XD1JC0659</serial> <capability type='removable'> <media_available>0</media_available> <media_size>0</media_size> </capability> </capability> </device> <interface type='bridge'> <mac address='02:16:3e:1a:b3:4a'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> </interface> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <hub type='usb'> <address type='usb' bus='0' port='1'/> </hub> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline> </domain> The VM is starting fine when I exclude the following: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline> If I'm using this the error message is: root@Tower2:/mnt/cache/.VMs/win8# virsh create win8.xml error: Failed to create domain from win8.xml error: internal error: early end of file from monitor: possible problem: 2014-10-24T11:24:42.013243Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted 2014-10-24T11:24:42.013293Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 13 2014-10-24T11:24:42.013307Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 13 2014-10-24T11:24:42.013329Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed. 2014-10-24T11:24:42.013348Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized lspci -knn is showing, that the devices are using the vfio driver: 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF106GL [Quadro 2000] [10de:0dd8] (rev a1) Subsystem: Hewlett-Packard Company Device [103c:084a] Kernel driver in use: vfio-pci 01:00.1 Audio device [0403]: NVIDIA Corporation GF106 High Definition Audio Controller [10de:0be9] (rev a1) Subsystem: Hewlett-Packard Company Device [103c:084a] Kernel driver in use: vfio-pci Any idea? Quote Link to comment
EdgarWallace Posted October 24, 2014 Author Share Posted October 24, 2014 Just found something in syslog: Tower2 kernel: vfio_iommu_type1_attach_group: No interrupt remapping support. Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform I'm afraid that I have to change back the syslinux.cfg. This is what I'm having now: label KVM/unRAID OS menu default kernel /bzimage append pcie_acs_override=downstream initrd=/bzroot Let's see what the change will bring.... Quote Link to comment
EdgarWallace Posted October 24, 2014 Author Share Posted October 24, 2014 Yes, this was doing the trick. I can see the NVIDIA Quadro 2000 and fight with the device driver but it is looking promising. Will report later. @jonp, maybe you would consider to change your advise here: http://lime-technology.com/forum/index.php?topic=35732.0. This was related to the syslinux.cfg {EDIT}Issues with the device driver. The driver installation went well but the VM doesn't boot anymore. Quote Link to comment
rd_blair Posted October 24, 2014 Share Posted October 24, 2014 @EdgarWallace Give this a try: http://lime-technology.com/forum/index.php?topic=35732.msg334596#msg334596 Worked on my NVIDIA based Evga GTX650. Pass-through worked right from boot menu. I have run Windows 7, 8.1 and 10 with the above template. Reed Quote Link to comment
EdgarWallace Posted October 29, 2014 Author Share Posted October 29, 2014 Thanks rd_blair. I was able to install Windows 8.1 from scratch and installed the NVIDIA device driver. Unfortunately the VM isn't starting anymore. Here is my win8.xml: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>win8</name> <uuid>cc411d70-4463-4db7-bf36-d364c0cdaa9d</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='cdrom'/> <bootmenu enable='yes'/> </os> <features> <acpi/> <apic/> <pae/> </features> <cpu mode='host-passthrough'> <topology sockets='2' cores='2' threads='1'/> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/cache/.VMs/win8/win8.qcow2'/> <target dev='vda' bus='virtio'/> </disk> <disk type='file' device='cdrom'> <source file='/mnt/user/backup/Oliver/Archiv/IT/unRaid/ISOs/win81enterp64.iso'/> <target dev='hdb' bus='sata'/> <readonly/> </disk> <disk type='file' device='cdrom'> <source file='/mnt/user/backup/Oliver/Archiv/IT/unRaid/ISOs/virtio-win-0.1-81.iso'/> <target dev='hdc' bus='sata'/> <readonly/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb0'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb0'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb0'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x2'/> </controller> <interface type='bridge'> <mac address='02:16:3e:1a:b3:4a'/> <source bridge='br0'/> <model type='virtio'/> </interface> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline> </domain> Any additional ideas? Quote Link to comment
reluctantflux Posted October 29, 2014 Share Posted October 29, 2014 Did you change the disk type? I know I had to set mine to SATA instead of Virtio, but that was because I created my VM with VirtualBox. Quote Link to comment
EdgarWallace Posted November 11, 2014 Author Share Posted November 11, 2014 Here is how the issues have been resolved: http://lime-technology.com/forum/index.php?topic=35732.msg337070#msg337070 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.