Stupid question about NICs

Just a quick question.  I have two NICs in my server, can I use one for UnRAID file operations, and one for the KVM VMs?

Just a quick question.  I have two NICs in my server, can I use one for UnRAID file operations, and one for the KVM VMs?

In theory yes, although look at the postings archedraft and myself have made regarding pfsense to see that setting up bridges etc is not that straightforward.  I've got mine working but I have no real idea what Unraid does with the config and it's still not quite what I'd like it to be. 

Try by all means but expect trouble ahead.... 

OK, cheers, will try it when I'm at home - don't want to futz with it when connecting remotely from work.

Perhaps Jonp has tried this and can answer definitively => but in theory you should be able to pass one through to the VM, and use it natively.    The unknown is if you can do this with an on-board NIC, as it can be problematic to pass through onboard devices.    It would almost certainly work if you had an add-on NIC board.

archedraft did try passing his NIC through but didn't get a good result although I forget what the issue was exactly and I haven't tried it myself.

archedraft did try passing his NIC through but didn't get a good result although I forget what the issue was exactly and I haven't tried it myself.

Was it an onboard NIC?    As I noted, those are often problematic (in fact, simply don't work in many cases).

But as long as the system supports vt-d (both CPU and motherboard), a plug-in card should pass through with no problem.    [FWIW the OP's i5-3570S does support vt-d]

archedraft did try passing his NIC through but didn't get a good result although I forget what the issue was exactly and I haven't tried it myself.

 

Was it an onboard NIC?    As I noted, those are often problematic (in fact, simply don't work in many cases).

 

But as long as the system supports vt-d (both CPU and motherboard), a plug-in card should pass through with no problem.    [FWIW the OP's i5-3570S does support vt-d]

No it was an Intel dual or triple NIC.  His hardware I think is very similar to mine (I think we may even have the same motherboard)

Here's the thread

I'll be using the super-high quality and ultra-stable Broadcom NICs on my board (sarcasm overload there).

You could try passing through and see what results you get.

archedraft did try passing his NIC through but didn't get a good result although I forget what the issue was exactly and I haven't tried it myself.

 

Was it an onboard NIC?    As I noted, those are often problematic (in fact, simply don't work in many cases).

 

But as long as the system supports vt-d (both CPU and motherboard), a plug-in card should pass through with no problem.    [FWIW the OP's i5-3570S does support vt-d]

No it is a separate dual Intel PCIe NIC. I attempted to pass through the dual NIC to my pfSense VM but unRAID seems to want to claim all the NIC's as it's own, even if you exclude the NIC from unRAID in the syslinux file. When I excluded it at boot, unRAID never received an IP address because unRAID still managed to attach to my dual NIC before it attached to the onboard NIC?? Eventually I gave up and just made three separate bridges but I would still prefer to just pass through my NIC to pfSense.

Ok, this is possible and I have tested it once successfully, but haven't bothered with it since.  I don't really understand the alure of passing through a NIC to a VM like this when virtual networking can be just as effective, but for those that want to do it, "just cuz," here's the procedure.

WARNING:  If you do not have multiple NICs in your system, doing this will result in your server losing all network connectivity.

1 - Login to your server via ssh.

2 - Type the following command:

lspci

You will get a list like this:

00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 04)
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 04)
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d4)
00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d4)
00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350/8350 Series]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300 Series]
02:00.0 VGA compatible controller: NVIDIA Corporation GK110 [GeForce GTX 780] (rev a1)
02:00.1 Audio device: NVIDIA Corporation GK110 HDMI Audio (rev a1)
04:00.0 Multimedia video controller: Device 1a0a:6202 (rev 01)

Identify the Ethernet controller you wish to assign.  Note the PCI address for the device (from my list, it would be 00:19.0).  From my list, I only have one network card, so I shouldn't do this, but if you have multiple, either one SHOULD be fine to select.

3 - Type the following command:

lspci -n

00:00.0 0600: 8086:0c00 (rev 06)
00:01.0 0604: 8086:0c01 (rev 06)
00:01.1 0604: 8086:0c05 (rev 06)
00:02.0 0300: 8086:0412 (rev 06)
00:03.0 0403: 8086:0c0c (rev 06)
00:14.0 0c03: 8086:8c31 (rev 04)
00:16.0 0780: 8086:8c3a (rev 04)
00:19.0 0200: 8086:153b (rev 04)
00:1b.0 0403: 8086:8c20 (rev 04)
00:1c.0 0604: 8086:8c10 (rev d4)
00:1c.3 0604: 8086:8c16 (rev d4)
00:1f.0 0601: 8086:8c44 (rev 04)
00:1f.2 0106: 8086:8c02 (rev 04)
00:1f.3 0c05: 8086:8c22 (rev 04)
01:00.0 0300: 1002:68f9
01:00.1 0403: 1002:aa68
02:00.0 0300: 10de:1004 (rev a1)
02:00.1 0403: 10de:0e1a (rev a1)
04:00.0 0400: 1a0a:6202 (rev 01)

4 - Identify your network card by PCI address (first column of results).

5 - Obtain the vendor/product ID for that device from the last column.  00:19.0 from my example is 8086:153b.

6 - Edit your syslinux.cfg file and add the following after the append but before initrd=/bzroot.

pci-stub.ids=8086:153b

REPLACE THE VENDOR/PRODUCT ID FROM MY EXAMPLE ABOVE WITH THE ONE YOU OBTAINED IN STEP 5.

7 - Reboot your system.

8 - Edit your VM using the XML editor mode.

9 - Add the following between the <devices> and </devices> tags.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>
      </source>
    </hostdev>

MODIFY THE ADDRESS LINE ENTERING IN THE TWO DIGIT BUS, SLOT, AND FUNCTION FROM YOUR ID.  So 00:19.0 translates to what I have above.

DONE.

Thank you so much for posting this! I just got this running and the results might surprise you.

The major improvement:

Is my internet speed through a VPN. Before with bridges I would get around 40-50 Mbps download speed when the VPN was on. Now I am getting my max speeds again at 110 Mbps. That is fantastic, I have been going crazy trying to get my speeds back up to that for months...

Minor benefits of this:

1. My unRAID server only has 1 IP address now (with bridges it would respond to 2 IP addresses).

2. Peace of mind knowing that unRAID cannot accidentally access the WAN bridge.

Thank you so much for posting this! I just got this running and the results might surprise you.

 

The major improvement:

Is my internet speed through a VPN. Before with bridges I would get around 40-50 Mbps download speed when the VPN was on. Now I am getting my max speeds again at 110 Mbps. That is fantastic, I have been going crazy trying to get my speeds back up to that for months...

 

Minor benefits of this:

1. My unRAID server only has 1 IP address now (with bridges it would respond to 2 IP addresses).

2. Peace of mind knowing that unRAID cannot accidentally access the WAN bridge.

That does surprise me that you are seeing such a dramatic improvement.  Very interesting indeed.