Caching in federation
I was thinking more about caching and maybe we could store the entities for a specified period, but not the relations. This means, in our example #16, that we could store Community B (name, description, images, etc) and the members (name, description, etc), but the server should always ask for the most recent members of the community because it may change very quickly. The same when we want to see the activities in the communities, we can have cached some activities (name, description, images, etc) but probably the last activities in a community are new ones.
So, the bad thing about cache is that we are storing old data and it is possible not to receive updates. The TTL is very important here, a low one it makes cache less useful, and a high one shows old data for a longer time.
I can see two different kinds of external data:
A user requests external data which is not related with any local data: I think in this case the best solution is only store it in the cache layer, because we never receive any updates. So after TTL, the data has to be gone.
A user requests external data which is followed by a local entity: In this case, we should have this information store in the database and we will receive updates, but will we receive all the information we need? I believe if the name is changed, we will receive that information so we can suppose it is updated. But what about the number of followers or likes? And more private data like moderators or the refuse the application to join a community?
So the final questions we want to solve are:
- How long should we store the cache data?
- What kind of data we can trust is always updated and not?
- Which cases we are going to use cache and which cases we are always making an external request?