elastic: Make number_of_shards matching logic in total_shards_per_node tuner threshold based

This gives us more flexibility when growing our ES cluster.

elastic/scheduled/hot_index_shards_per_node.sh is a scheduled job that runs on an hourly cadence to ensure we don't run into shards that cannot be moved to the warm fleet.

The lower bound / limiting factor is the number of warm nodes. Since we have fewer nodes there that are more densely packed.

We are likely to grow the hot fleet more quickly than the warm. And when we grow it, we also want to increase number_of_shards on high-throughput indices to spread load across all hot nodes.

When we do, we want to ensure that indices of both the previous shard count as well as the new shard count will be adjusted by this scheduled job.

Note: We grew the ES cluster in response to gitlab-com/gl-infra/production#4653 (closed), and this will enable us to increase number_of_shards safely.