Memory size limitation under podman on Apple silicon

Host environment

  • Operating system: MacOS Ventura
  • OS/kernel version: Darwin host.com 22.5.0 Darwin Kernel Version 22.5.0: Mon Apr 24 20:52:24 PDT 2023;root:xnu-8796.121.2~5/RELEASE_ARM64_T6000 arm64
  • Architecture: ARM (Apple Silicon)
  • QEMU flavor: qemu-system-aarch64
  • QEMU version: QEMU emulator version 8.0.3
  • QEMU command line: See below for steps to reproduce via podman.

Emulated/Virtualized environment

  • Operating system: Fedora
  • OS/kernel version: Linux localhost 6.3.12-200.fc38.aarch64 #1 SMP PREEMPT_DYNAMIC Thu Jul 6 04:24:20 UTC 2023 aarch64 GNU/Linux
  • Architecture: aarch64

Containerized environment

  • Operating system: Red Hat Enterprise Linux
  • OS/kernel version: 8.8
  • Architecture: aarch64

Description of problem

We are using latest MacOS (Ventura) on M2 Ultra with 128Gb RAM (Mac Studio) to run our product Linux aarch64 builds in podman containers. This is cheaper than buying ARM server hardware, and we are not able to use cloud services.

The issue arises when we try to use the available RAM for the underlying QEMU machine. There seems to be a memory limit which looks like it is in QEMU not podman machine, since that is more of a wrapper in this process.

The use case is to init a Fedora Linux VM by QEMU which provides a Linux kernel. That kernel is then used to run podman containers.

When we set the memory limit to 64513Mb the podman machine (VM) start fails with "Error: HV_BAD_ARGUMENT". If we reduce the memory limit to "64512" it works as expected.

This is an example of how to reproduce:

` macstudio:~ build $ podman machine init --cpus="18" --memory="64513" podman-machine-default Extracting compressed file Image resized. Machine init complete To start your machine run:

podman machine start

macstudio:~ build $ podman machine start Starting machine "podman-machine-default" Waiting for VM ... Error: qemu exited unexpectedly with exit code -1, stderr: qemu-system-aarch64: Error: HV_BAD_ARGUMENT

macstudio:~ build $ podman machine rm --force macstudio:~ build $ podman machine init --cpus="18" --memory="64512" podman-machine-default Extracting compressed file Image resized. Machine init complete To start your machine run:

podman machine start

macstudio:~ build $ podman machine start Starting machine "podman-machine-default" Waiting for VM ... Mounting volume... /Users:/Users Mounting volume... /private:/private Mounting volume... /var/folders:/var/folders

This machine is currently configured in rootless mode. If your containers require root permissions (e.g. ports < 1024), or if you run into compatibility issues with non-podman clients, you can switch using the following command:

podman machine set --rootful

API forwarding listening on: /Users/build_ci/.local/share/containers/podman/machine/qemu/podman.sock

The system helper service is not installed; the default Docker API socket address can't be used by podman. If you would like to install it run the following commands:

sudo /opt/homebrew/Cellar/podman/4.6.0/bin/podman-mac-helper install podman machine stop; podman machine start

You can still connect Docker API clients by setting DOCKER_HOST using the following command in your terminal session:

export DOCKER_HOST='unix:///Users/build/.local/share/containers/podman/machine/qemu/podman.sock'

Machine "podman-machine-default" started successfully macstudio:~ build $ podman machine ls NAME VM TYPE CREATED LAST UP CPUS MEMORY DISK SIZE podman-machine-default* qemu About a minute ago Currently running 18 63GiB 100GiB

`

Steps to reproduce

  1. Initialise the VM with a RAM limit of 64513Mb, then start it.

Additional information

Feel free to ask for more information. Unfortunately, these machines are our production platform, so further testing will not have a rapid turn around. We are open to taking a machine out of production for testing, it just needs scheduling.