qemu user space emulator handles syscall `setsockopt()` with `optlen=0` incorrectly

Host environment

• Operating system: Arch Linux
• OS/kernel version: Linux 6.7.6-arch1-1 x86_64 GNU/Linux
• Architecture: x86_64
• QEMU flavor: qemu-riscv64-static
• QEMU version: 8.2.1
• QEMU command line:

  qemu-riscv64-static ./alg-test

Emulated/Virtualized environment

• Operating system: Arch Linux rootfs
• OS/kernel version: (userspace only)
• Architecture: RISC-V 64

Description of problem

Note that despite I have only tested with the parameters/environments above, this problem probably affects ALL architectures on Linux.

When user program calls setsockopt(fd, SOL_ALG, ALG_SET_KEY, NULL, 0), qemu intercepts the syscall and returns -1 with errno = ENOMEM, which should have completed successfully returning zero.

Steps to reproduce

1. compile this code to binary executable:

#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <linux/if_alg.h>

int create_alg(const char *alg)
{
        struct sockaddr_alg salg;
        int sk;

        sk = socket(PF_ALG, SOCK_SEQPACKET | SOCK_CLOEXEC, 0);
        if (sk < 0)
                return -1;

        memset(&salg, 0, sizeof(salg));
        salg.salg_family = AF_ALG;
        strcpy((char *) salg.salg_type, "hash");
        strcpy((char *) salg.salg_name, alg);

        if (bind(sk, (struct sockaddr *) &salg, sizeof(salg)) < 0) {
                close(sk);
                return -1;
        }

        return sk;
}

int main() {
        int fd = create_alg("hmac(sha1)");
        char buf[10];
        int ret = setsockopt(fd, SOL_ALG, ALG_SET_KEY, NULL, 0);
        if(ret < 0){
                perror("err");
        }
        else{
                puts("SUCCESS!");
        }
        return 0;
}

2. run it in any qemu user space emulator

On real Linux kernel, this program outputs a SUCCESS! while in qemu it prints err: Cannot allocate memory.

The error is neither informative nor intuitive and could be misleading for user programs.

Additional information

I already have a patch which fixes the issue and I'm willing to send it to mailing list as soon as I have done the testing.