SLES 12.5 builds fail intermittently because ngx_security_headers builds from master

We've seen this failure crop up recently:

make[1]: Entering directory '/var/cache/omnibus/src/nginx'
cc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -L/opt/gitlab/embedded/lib -I/opt/gitlab/embedded/include  -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I objs -I src/http -I src/http/modules -I src/http/v2 \
	-o objs/addon/src/ngx_http_security_headers_module.o \
	/var/cache/omnibus/src/ngx_security_headers/src/ngx_http_security_headers_module.c
objs/Makefile:1473: recipe for target 'objs/addon/src/ngx_http_security_headers_module.o' failed
make[1]: Leaving directory '/var/cache/omnibus/src/nginx'
Makefile:10: recipe for target 'build' failed
Error:
    /var/cache/omnibus/src/ngx_security_headers/src/ngx_http_security_headers_module.c: In function 'ngx_http_security_headers_filter':
/var/cache/omnibus/src/ngx_security_headers/src/ngx_http_security_headers_module.c:256:9: error: 'for' loop initial declarations are only allowed in C99 mode
         for (size_t i = 0; i < hide_headers_count; ++i) {
         ^
/var/cache/omnibus/src/ngx_security_headers/src/ngx_http_security_headers_module.c:256:9: note: use option -std=c99 or -std=gnu99 to compile your code
make[1]: *** [objs/addon/src/ngx_http_security_headers_module.o] Error 1
make: *** [build] Error 2
/builds/gitlab-org/omnibus-gitlab/gems/ruby/3.2.0/bundler/gems/omnibus-1593bf413088/lib/omnibus/util.rb:139:in `rescue in shellout!'
  /builds/gitlab-org/omnibus-gitlab/gems/ruby/3.2.0/bundler/gems/omnibus-1593bf413088/lib/omnibus/util.rb:134:in `shellout!'
  /builds/gitlab-org/omnibus-gitlab/gems/ruby/3.2.0/bundler/gems/omnibus-1593bf413088/lib/omnibus/builder.rb:911:in `shellout!'
  /builds/gitlab-org/omnibus-gitlab/gems/ruby/3.2.0/bundler/gems/omnibus-1593bf413088/lib/omnibus/builder.rb:85:in `block in command'
  /builds/gitlab-org/omnibus-gitlab/gems/ruby/3.2.0/bundler/gems/omnibus-1593bf413088/lib/omnibus/builder.rb:1101:in `instance_eval'
  /builds/gitlab-org/omnibus-gitlab/gems/ruby/3.2.0/bundler/gems/omnibus-1593bf413088/lib/omnibus/builder.rb:1101:in `run'

https://github.com/GetPageSpeed/ngx_security_headers/pull/23 was merged recently, and this requires C++99.

Somehow the 0.0.9 tag isn't being used in https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/4485d172235be9cfa80ab7964772c73063209527/config/software/ngx_security_headers.rb#L22.

Neither @apatterson2 nor I could reproduce it locally.

@apatterson2 just launched a new pipeline in https://gitlab.com/gitlab-org/omnibus-gitlab/-/pipelines/1785137393, and it passed.

I don't understand yet how this can happen. Things I looked at:

  • Flags that might cause this recipe to build from master. Didn't see any?
  • Omnibus cache
Edited by Stan Hu