Bug/Feature: Finer control of bhyve pci-isa bridge and control flags

BUG:

libvirt is generating bhyve command line that result in VM failure:

# virsh domxml-to-native --format bhyve-argv --domain VM_DOMAIN
/usr/sbin/bhyve -c 1 -m 2048 -A -I -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s 5:0,xhci,tablet -s 2:0,ahci,hd:/usr/home/shared/documents/virtual-machines/drum.dev.ubuntu/drum-hd1-01.img -s 4:0,e1000,tap0,mac=52:54:00:88:46:2c -s 3:0,fbuf,tcp=127.0.0.1:0,vga=off -s 1,lpc -l com1,/dev/nmdm1A VM_DOMAIN

Where:

• "-s 1,lpc" causes failure with UEFI boot and
• there should not be -A flag (as bhyve uses UEFI generated ACPI tables).

DIAGNOSIS:

I been testing with running bhyve nested on top of Ubuntu 20.04 and raised the following bug on bhyve: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247636

The work around was to change CPU to 1, remove -A flag and put PCI-ISA onto PCI device 31. With libvirt XML I can change the CPU but there is now way to control the "-A" flag (which should not be provided with UEFI) and there is no way to control the implicit PCI-ISA bridge as this has no corresponding device within libvirt XML and libvirt always generates as being on PCI Device 1 (src/bhyve/bhyve_command.c):

static int
bhyveBuildLPCArgStr(const virDomainDef *def G_GNUC_UNUSED,
                    virCommandPtr cmd)
{
    virCommandAddArgList(cmd, "-s", "1,lpc", NULL);
    return 0;
}

PROPOSAL:

Provide way to explicitly control the PCI Device ID for PCI-ISA device and optionally either encode bhyve flag invocation rules based on configuration or provide way to control command flag invocation via XML.

NOTE: I cannot achieve control of flag via "arbitrary commands" (as per documented guidance below) as this provides way to add to command but not to remove from the command (as it needed in -A case).

<domain type="bhyve" xmlns:bhyve="http://libvirt.org/schemas/domain/bhyve/1.0">
  ...
  <bhyve:commandline>
    <bhyve:arg value='-somebhyvearg'/>
  </bhyve:commandline>
</domain>

END