Should every message be stored in the MessageStore?

Currently every message that Mailman sees is stored in the MessageStore. Its Message-ID is stored in the database and the contents of the message are stored on disk. In !116 (merged) the only place where messages were deleted from the store was removed (this was where held messages were disposed of and not explicitly preserved).

There are several use cases for at the least storing every Message-ID including that it's a useful anti-spam defense to reject duplicates. For forensic reasons it can be useful to store the contents of every message too. In fact, Mailman 2 does this by storing everything in an mbox file and an HTML copy in the Pipermail archives.

Aurelien points out that by storing messages both in the MessageStore and in HyperKitty, it can be wasteful of resources. This issue should track whether we want to provide knobs that can be turned to reduce the disk consumption of the store. One possibility could be to retain the Message-IDs but compress or remove the contents.