fix(gamma-wl): fix integer overflow
I noticed Gammastep crashed with SIGBUS when I plugged a display out. The coredump included the following stacktrace. It happened on the Arch Linux 2.0.9 build.
#0 wayland_set_temperature (setting=<optimized out>, state=<optimized out>)
at /usr/src/debug/gammastep/gammastep-v2.0.9/src/gamma-wl.c:344
344 b_gamma[i] = value;
(gdb) bt
#0 wayland_set_temperature (setting=<optimized out>, state=<optimized out>)
at /usr/src/debug/gammastep/gammastep-v2.0.9/src/gamma-wl.c:344
#1 wayland_set_temperature (state=<optimized out>, setting=0x7ffed9e04b70)
at /usr/src/debug/gammastep/gammastep-v2.0.9/src/gamma-wl.c:275
#2 0x0000560c4df8f47f in run_continual_mode
(preserve_gamma=1, use_fade=1, method_state=0x560c4fb49350, method=0x7ffed9e04d00, scheme=0x7ffed9e04bb8, location_state=0x560c4fb46fd0, provider=0x7ffed9e04c80)
at /usr/src/debug/gammastep/gammastep-v2.0.9/src/redshift.c:724
#3 main (argc=<optimized out>, argv=<optimized out>)
at /usr/src/debug/gammastep/gammastep-v2.0.9/src/redshift.c:1199
GDB revealed this happened in an iteration where output->gamma_size is 1337263648. It seems output->gamma_size may take random values when turning off or plugging out a display. The next instruction overflows int which has undefined behavior. This may be a hardware issue, but Gammastep should be able to survive it.
I couldn’t determine if the overflow was the direct cause of the SIGBUS on line 344. Still, as a precaution, I propose the following changes.
- Make both
colorramp_fillfunctions takessize_tinstead ofintfor the size. This type is more appropriate and allows for bigger sizes on GNU/Linux. - Change the type of
sizeinwayland_set_temperaturetossize_t. - Check for integer overflow before multiplying. Skip the current output as unsupported if overflow would happen.
Please let me know if change 1 should be separated from changes 2 and 3.