[gprd] replace max_size (deprecated) with max_primary_shard_size and update ILM allocation settings

This is the production equivalent of !4971 (merged).

The aim of this MR is to:

• Replace max_size with max_primary_shard_size. It looks like max_size is deprecated, but regardless max_primary_shard_size makes more sense so you don't have to think about how many shards each index has to work out max_size.
• Remove the use of _tier_preference and require.data to move shards from hot to warm nodes. This is in-line with Elastic's recommendation, and it simplifies our configs.

Once this MR is merged & deployed, we need to update existing indexes using the following:

ES_URL="https://user:pass@<prod ES cluster>:9243"

HOT

1. Get list of indices that are in the hot phase:

curl -sSL -H 'Content-Type: application/json' -X GET "$ES_URL/_all/_ilm/explain" | jq -r '.indices | to_entries | .[].value | select(.phase == "hot") | .index' | sort -V > ~/tmp/hot-indices.txt

2. Split file into chunks of 20 lines:

split -l 20 ~/tmp/hot-indices.txt ~/tmp/hot-indices-

3. Update index settings:

for i in ~/tmp/hot-indices-*; do
  INDICES="$(cat $i | paste -sd, -)"
  echo "--> Pushing index settings to: ${INDICES}"
  curl -sSL -H 'Content-Type: application/json' -X PUT "${ES_URL}/${INDICES}/_settings" -d '{ "index.routing.allocation.require.data" : null, "index.routing.allocation.include._tier_preference": "data_content" }'
  echo
done

WARM

1. Get list of indices that are in the warm phase:

curl -sSL -H 'Content-Type: application/json' -X GET "$ES_URL/_all/_ilm/explain" | jq -r '.indices | to_entries | .[].value | select(.phase == "warm") | .index' | sort -V > ~/tmp/warm-indices.txt

2. Split file into chunks of 20 lines:

split -l 20 ~/tmp/warm-indices.txt ~/tmp/warm-indices-

3. Update index settings:

for i in ~/tmp/warm-indices-*; do
  INDICES="$(cat $i | paste -sd, -)"
  echo "--> Pushing index settings to: ${INDICES}"
  curl -sSL -H 'Content-Type: application/json' -X PUT "${ES_URL}/${INDICES}/_settings" -d '{ "index.routing.allocation.require.data" : null, "index.routing.allocation.include._tier_preference": "data_warm,data_hot" }'
  echo
done