Allow configuring the cache path for mailboxes
Issue description
Right now dovecot always writes the cache files into the maildir. That is fine for small-ish setups. As soon as your setups grows - and your usage pattern changes - you might want to use faster/ephemeral storage for some of the caching.
I am right now at a stage where my index files are >5GB in total (across ~120 Maildirs) and whenever I add/modify/delete a file from such a mailbox Dovecot rewrites the entire file. This is expected as the implementation for index caching is slow and doesn't scale well (as per documentation).
For now I've started to mitigate the issue by limiting the size of each of the cache files to 10MB which obviously has different issues (search on mobile/fat desktop clients becomes slow(er)).
Proposal
Right now we configure dovecot with mail_location = maildir:/var/vmail/%d/%n
while we could also set the cache directory like so: mail_location = maildir:/var/vmail/%d/%n:INDEX=/var/cache/dovecot/index/%u
(see https://doc.dovecot.org/configuration_manual/mail_location/#index-files). This would allow me to move the indexes to either tmpfs or faster (local SSD) storage to avoid the costs involved in writing to the underlying CEPH volume... (which is also 10GBit/s + SSD's but network latency & replication are costly).
We might be able to append an additional :INBOX=/var/vmail/%d/%n
to let dovecot fallback to the "old" (previous) locations on servers that already contain state.
Considerations
I've only looked into this for the last 45 or so minutes as I've been trying to make some IMAP operations faster that are otherwise likely to take days to complete.