fix(gamma-wl): fix integer overflow

I noticed Gammastep crashed with SIGBUS when I plugged a display out. The coredump included the following stacktrace. It happened on the Arch Linux 2.0.9 build.

#0  wayland_set_temperature (setting=<optimized out>, state=<optimized out>)
    at /usr/src/debug/gammastep/gammastep-v2.0.9/src/gamma-wl.c:344
344				b_gamma[i] = value;                                                              
(gdb) bt
#0  wayland_set_temperature (setting=<optimized out>, state=<optimized out>)
    at /usr/src/debug/gammastep/gammastep-v2.0.9/src/gamma-wl.c:344
#1  wayland_set_temperature (state=<optimized out>, setting=0x7ffed9e04b70)
    at /usr/src/debug/gammastep/gammastep-v2.0.9/src/gamma-wl.c:275
#2  0x0000560c4df8f47f in run_continual_mode
    (preserve_gamma=1, use_fade=1, method_state=0x560c4fb49350, method=0x7ffed9e04d00, scheme=0x7ffed9e04bb8, location_state=0x560c4fb46fd0, provider=0x7ffed9e04c80)
    at /usr/src/debug/gammastep/gammastep-v2.0.9/src/redshift.c:724
#3  main (argc=<optimized out>, argv=<optimized out>)
    at /usr/src/debug/gammastep/gammastep-v2.0.9/src/redshift.c:1199

GDB revealed this happened in an iteration where output->gamma_size is 1337263648. It seems output->gamma_size may take random values when turning off or plugging out a display. The next instruction overflows int which has undefined behavior. This may be a hardware issue, but Gammastep should be able to survive it.

I couldn’t determine if the overflow was the direct cause of the SIGBUS on line 344. Still, as a precaution, I propose the following changes.

  1. Make both colorramp_fill functions take ssize_t instead of int for the size. This type is more appropriate and allows for bigger sizes on GNU/Linux.
  2. Change the type of size in wayland_set_temperature to ssize_t.
  3. Check for integer overflow before multiplying. Skip the current output as unsupported if overflow would happen.

Please let me know if change 1 should be separated from changes 2 and 3.

Merge request reports

Loading