Skip to content

Disable stop-writes-on-bgsave-error on Redis nodes

It looks like we're starting to upgrade the Omnibus package on the Redis nodes (https://gitlab.com/gitlab-com/gl-infra/chef-repo/-/merge_requests/653). That's good! From https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5547#note_685452011, this would be a good opportunity add this setting from gitlab-org/omnibus-gitlab!5616 (merged):

redis['stop_writes_on_bgsave_error'] = false

Redis configuration notes on this setting:

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.

Since we have a Redis cluster with good monitoring, I think we prefer to disable this setting.

/cc: @ahanselka, @msmiley

Edited by Stan Hu