RichardBoelens

VM Hangs on Launch when GPU Passthrough is Used

6 posts in this topic Last Reply

Recommended Posts

Posted (edited)

Goal
I just rebuilt my father's computer and he's not very computer savvy. The intent was to put him on a Hypervisor so that I can provision him Hardware as he needs it without him worrying about the physical layer. He Games on occasion, and GPU Mines, so Passthrough is important (which is why we went UnRAID instead of HyperV).

 

Problem

Whenever trying to start the VM, It shows the Tiannacore Logo. Then one of two different things happen. Either

1) If the VM was improperly shutdown previously, itll take a few minutes, but the windows spinning circles will appear and Windows Recovery pops up. Rebooting after recovery brings you to #2

2) The screen hangs at the Tiannacore Logo (before the spinning circles show up) OR The spinning circles show up, spin once, then hangs halfway through the spin. The VM does not recover from this and freezes.

 

I've been able to boot with VNC + 1 GPU a total of 2 times, in which both cases, I rebooted, and despite making no provisioning changes, after the reboot, the VM fails to boot again. Not sure why/how it booted those times, as I was not able to reproduce that success on demand.

 

Host Specs

UnRAID Ver 6.5.1
Asus ROG Maximus X (Has Onboard HDMI + DP used only for UnRAID)

Intel I7-8700k

8GB Kingston Memory (Yes I know its kinda low, I intend to replace it with better/more in the near future)

EVGA 1080ti FTW3

EVGA 1080ti FTW3
 

VM Specs
All 6 CPU Cores, All 12 Threads

4GB RAM

GPU : 1080ti 1.00.0 - HDMI

GPU : 1080ti 2.00.0

Audio : 1080ti 1.00.1 - HDMI

 

Troubleshooting done so far

1) Removing Both Passthrough GPUs and only using VNC allows me to boot normally without fail. Adding one or both cards causes it to fail. Tried many combinations. VNC + GPU 1, VNC + GPU 2, VNC + GPU 1+2, GPU 1, GPU 2, GPU 1 + 2 etc etc

2) Tried with and Without GPU's HDMI Audio to see if that affected it, it did not.

3) I had an issue previously with a different Rig that had similar symptoms, Here, Applied the same fix with VBIOS, but this did not resolve the issue. I should also note, that with the other Rig, There was no onboard, just 4x Dedicated GPUs, while in this case, there is an onboard, with 2x Dedicated GPUs. This issue shouldn't happen according to research on this rig because there is onboard VGA.
4) Increased/Decreased Resources such as RAM and CPU to see if there is a difference. No Difference.

 

VM Logs

2018-05-14 23:19:51.682+0000: starting up libvirt version: 4.0.0, qemu version: 2.11.1, hostname: CB-URHost01
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name guest=CB-01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-CB-01/master-key.aes -machine pc-i440fx-2.11,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/c1fc581a-40df-a97d-2753-43497501e9f6_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 4096 -realtime mlock=off -smp 12,sockets=1,cores=6,threads=2 -uuid c1fc581a-40df-a97d-2753-43497501e9f6 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-CB-01/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,p2=15,p3=15,id=usb,bus=pci.0,-3-CB-01/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device 'vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/user/ISO Repository/Graphics ROM BIOS/EVGA 1080ti FTW3 GPU BIOS.dump' -device vfio-pci,host=02:00.0,id=hostdev1,bus=pci.0,addr=0x8 -device vfio-pci,host=01:00.1,id=hostdev2,bus=pci.0,addr=0x9 -device usb-host,hostbus=1,hostaddr=5,id=hostdev3,bus=usb.0,port=1 -device usb-host,hostbus=1,hostaddr=2,id=hostdev4,bus=usb.0,port=2 -device usb-host,hostbus=1,hostaddr=7,id=hostdev5,bus=usb.0,port=3 -device usb-host,hostbus=1,hostaddr=8,id=hostdev6,bus=usb.0,port=4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa -msg timestamp=on
2018-05-14 23:19:51.682+0000: Domain id=3 is tainted: high-privileges
2018-05-14 23:19:51.682+0000: Domain id=3 is tainted: host-cpu
2018-05-14T23:19:51.723700Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/0 (label charserial0)

 

Edited by RichardBoelens

Share this post


Link to post

Anyone?

This is kinda a make or break for this. I'm having him run windows off a Temp SSD till i get this resolved.
But without being able to launch the VM, it makes the computer completely unusable for him.

Is there paid support options?

Share this post


Link to post
Posted (edited)

300$/h is a bit much for a home user unfortunately

Hopefully someone will be able to point me in the right direction.

Other than the Motherboard Model, I've done the exact same configuration with my other build and did not have any issues so I suspect that there is something else at play.

Edited by RichardBoelens

Share this post


Link to post

assuming its windows, you shouldn't boot vnc plus gpu. 

 

also assuming its windows, what guide did you follow to setup the vm?

 

best practices is to not assign all cpus to the vm. leave cpu 0 for unRaid to manage its business and then send remaining 11 to the vm.

 

make sure your onboard video is set in the bios to be primary display

 

 

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Copyright © 2005-2018 Lime Technology, Inc.
unRAID® is a registered trademark of Lime Technology, Inc.