Different PCI bus addresses is assigned during PCI pass through in the VM to Physical Functions of same PCI device in the host.

[Ali Hasan Ahmed Khan]

I have used the below command to install VM with two PFs passthrough.

virt-install --name vm1 --memory 4096 --vcpus 2 --disk
/var/lib/libvirt/images/test.img --import --os-variant centos8 --nographics
--network type=direct,source=eno1,model=virtio
--host-device=pci_0000_04_00_1 --host-device=pci_0000_04_00_2

I was having an issue when I was passing through two Physical Functions of
the same PCI device (Mellanox ConnectX-6) in VM. The issue is that the VM
is assigning different PCI bus addresses to passed through PFs.
which was:
pci at 0000:05:00.0 to PF0
pci at 0000:06:00.0 to PF1.

Although both belong to the same vendor or family. Due to this issue, I was
unable to ping the passthrough PFs of VM from a remote server that is
physically connected and has a wired connection with another server's
SmartNIC. Even loopback ping was not working.

I temporarily fixed this issue by editing an XML of the VM in the" " tag. I
have changed the PF1 bus address and function from "<address type='pci'
domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>" to "<address
type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>". and
then shut down the VM and start it again. The PCI address assignment
changed to
pci at 0000:05:00.0 to PF0
pci at 0000:05:00.1 to PF1.
And after that when I try to ping the PFs of VM from another connected
server it got pinged and loopback also worked. So I want to ask is there
any way or followed method that will assign a similar PCI address to PF of
the same PCI device. If not kindly fixed this issue so that others won't be
stuck in the problem. If you guys can tell me the pointers in python-code
of virt-manager I can try to fix them by myself And I would love to
contribute to it. Kindly do let me know if you guys can count me in this
case. Thank You! :)

NOTE: I have performed this test with two SmartNICs which was Mellanox and
Xilinx in both's PFs PCI address assigning in VM are different.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/virt-tools-list/attachments/20210930/8045b8a2/attachment.htm>