ppc64 tcg guests get incorrect Capacity Entitlement values from SMP spapr machine's RTAS - examples given from AIX guest OS
Host environment
- Operating system: Fedora 37
- OS/kernel version: 6.1.13
- Architecture: x86
- QEMU flavor: qemu-system-ppc64
- QEMU version: 7.1.0
- QEMU command line:
qemu-system-ppc64 \ -M pseries,ic-mode=xics \ -smp cpus=2,sockets=1,cores=2,threads=1 \ -m 8192 \ -nographic \ -prom-env input-device="/vdevice/vty@71000000" \ -prom-env output-device="/vdevice/vty@71000000" \ -serial tcp::9019,server,nowait \ -monitor tcp::9020,server,nowait \ -netdev bridge,id=nic0 \ -device spapr-vlan,netdev=nic0,mac=52:54:00:49:53:14 \ -drive file=images/aix-ppc64.img,format=raw,if=none,id=hd0,media=disk,cache=unsafe \ -device virtio-scsi,id=scsi0 \ -device scsi-hd,drive=hd0
Emulated/Virtualized environment
- Operating system: AIX 7.2 TL5 SP5 2245
- OS/kernel version: 7.2.5.200
- Architecture: ppc64
Description of problem
Entitled Capacity of the guest OS is not equal to the number of vCPUs you configure with QEMU.
It only gives you 1/4 entitlements to your configured capacity, and if you increase the number of processors, it converts the LPAR capacity to hundredths of a core, throttling guest performance.
Steps to reproduce
Definition of terms from lparstat: Entitled Capacity: The number of processing units this LPAR is entitled to receive. Maximum Capacity: The maximum number of processing units this LPAR was defined to ever have. Entitled capacity can be increased up to this value.
Some examples:
1 QEMU vCPU: Entitled Capacity: 0.25 Maximum Capacity: 1.00
2 QEMU vCPUs (-smp cpus=2,sockets=1,cores=2,threads=1): Entitled Capacity = 0.50 Maximum Capacity: 0.02
3 QEMU CPUs (-smp cpus=3,sockets=1,cores=3,threads=1): Entitled Capacity = 0.75 Maximum Capacity: 0.03
4 QEMU CPUs (-smp cpus=4,sockets=1,cores=4,threads=1): Entitled Capacity = 1.00 Maximum Capacity: 0.04
I believe these Entitled and Maximum values are comming from the spapr machine's MaxEntCap, DesProcs and MaxPlatProcs settings which get affected by -smp passed to QEMU.
Additional information
Details:
1 QEMU vCPU:
kens@aix-ppc64$ lparstat -i | head -20
Node Name : aix-ppc64
Partition Name : IBM AIX - IBM POWER9
Partition Number : 0
Type : Shared
Mode : Capped
Entitled Capacity : 0.25
Partition Group-ID : 1
Shared Pool ID : 1
Online Virtual CPUs : 1
Maximum Virtual CPUs : 1
Minimum Virtual CPUs : 1
Online Memory : 8192 MB
Maximum Memory : 8192 MB
Minimum Memory : 8192 MB
Variable Capacity Weight : 128
Minimum Capacity : 0.01
Maximum Capacity : 1.00
Capacity Increment : 1.00
Maximum Physical CPUs in system : 1
Active Physical CPUs in system : 1
2 QEMU vCPUs:
kens@aix-ppc64$ lparstat -i | head -20
Node Name : aix-ppc64
Partition Name : IBM AIX - IBM POWER9
Partition Number : 0
Type : Shared
Mode : Capped
Entitled Capacity : 0.50
Partition Group-ID : 1
Shared Pool ID : 1
Online Virtual CPUs : 2
Maximum Virtual CPUs : 2
Minimum Virtual CPUs : 2
Online Memory : 8192 MB
Maximum Memory : 8192 MB
Minimum Memory : 8192 MB
Variable Capacity Weight : 128
Minimum Capacity : 0.02
Maximum Capacity : 0.02
Capacity Increment : 1.00
Maximum Physical CPUs in system : 2
Active Physical CPUs in system : 2
3 QEMU vCPUs:
kens@aix-ppc64$ lparstat -i | head -20
Node Name : aix-ppc64
Partition Name : IBM AIX - IBM POWER9
Partition Number : 0
Type : Shared
Mode : Capped
Entitled Capacity : 0.75
Partition Group-ID : 1
Shared Pool ID : 1
Online Virtual CPUs : 3
Maximum Virtual CPUs : 3
Minimum Virtual CPUs : 3
Online Memory : 8192 MB
Maximum Memory : 8192 MB
Minimum Memory : 8192 MB
Variable Capacity Weight : 128
Minimum Capacity : 0.03
Maximum Capacity : 0.03
Capacity Increment : 1.00
Maximum Physical CPUs in system : 3
Active Physical CPUs in system : 3
4 QEMU vCPUs:
kens@aix-ppc64$ lparstat -i | head -2
lparstat -i | head -2
Node Name : aix-ppc64
Partition Name : IBM AIX - IBM POWER9
kens@aix-ppc64$ lparstat -i | head -20
lparstat -i | head -20
Node Name : aix-ppc64
Partition Name : IBM AIX - IBM POWER9
Partition Number : 0
Type : Shared
Mode : Capped
Entitled Capacity : 1.00
Partition Group-ID : 1
Shared Pool ID : 1
Online Virtual CPUs : 4
Maximum Virtual CPUs : 4
Minimum Virtual CPUs : 4
Online Memory : 8192 MB
Maximum Memory : 8192 MB
Minimum Memory : 8192 MB
Variable Capacity Weight : 128
Minimum Capacity : 0.04
Maximum Capacity : 0.04
Capacity Increment : 1.00
Maximum Physical CPUs in system : 4
Active Physical CPUs in system : 4
So it seems to me like the OS assumes the following from the spapr machine settings: Entitled Capacity = cpus / 4 (OS is assuming smt=4 threads maybe, see below) Maximim Capacity = cpus / 100 (OS is assuming hundredths of a core, even though the Capacity Increment is 1.00, see below))
On a real Power system (POWER8, smt=8), it looks like this:
kens@aix72$ lparstat -i | head -20
Node Name : aix72
Partition Name : n1
Partition Number : 1
Type : Shared-SMT-4
Mode : Uncapped
Entitled Capacity : 6.00
Partition Group-ID : 32784
Shared Pool ID : 0
Online Virtual CPUs : 12
Maximum Virtual CPUs : 28
Minimum Virtual CPUs : 1
Online Memory : 131072 MB
Maximum Memory : 196608 MB
Minimum Memory : 1024 MB
Variable Capacity Weight : 128
Minimum Capacity : 0.50
Maximum Capacity : 14.00
Capacity Increment : 0.01
Maximum Physical CPUs in system : 80
Active Physical CPUs in system : 80