Actually I think I answered my own question, I had it isloated with VFIO and that made it run at high idle power. By not doing that, installing Radeontop, the power draw came right down, I assume because drivers got loaded? Anyway I can still use it in my VMs and power usage is much lower so I'm happy.