notes cache are not invalidated correctly

We store pre-rendered HTML code with our notes, which help us minimize N+1 queries to render references. The problem is that whenever a cached data changes, like an issue title or merge request title, there is no easy way to invalidate them.

There are two possible alternatives to solve this:

  • keep a reverse graph of places that refers to the target item, in order to trigger cache invalidations
  • don't cache mutable data (like titles, descriptions etc) into pre-rendered HTML code and try to reduce requests by smart caching on client-side

For the reverse graph, as we now have to support only Postgres, we can use either jsonb or array in our favor. An alternative using array would be to add (to Issues,MRs and Epics):

  • mentioned_in_issues
  • mentioned_in_merge_requests
  • mentioned_in_epics

And keep unique ids of places that mention that entity, so whenever title/description/status is changed, we know what to invalidate cache for.

For the JSONB approach, we would create a mention field on the notes, and we can store either only ids, which we can use to search for later on and flag things that need to be refreshed, or store the mutable data, which we can update as well, in a single transaction.

For the smart cache idea on client-side, we can do something on this line:

  1. Keep each mentionable item stored on users browser IndexedDB, with all required data and a cache key.
  2. Whenever a user loads an issue/MR/epic, we look for the items we already have on local IndexedDB and use them right away for a faster UI response
  3. We do re-validate items on background by batching up a query to a GraphQL endpoint listing each primary_key for each type, along with the cache key. Ex: {issues: [{id: 25: cache_key: '...'}]}, the response would be either something like cache_fresh: true for each item/primary key, or content to update to.
  4. After the cache is refreshed we feed that back to the Vue app and let it handles re/rendering stuff

We can even go further and keep a TTL as well to prevent refreshing it every page hit. Also another thing we can do is, whenever you update an issue/MR/epic, your browser session should invalidate that local cache you have, so if you go to another window that is still open, when you hover over a mention, as that would be missing on the local indexedDB, it will fetch for the freshest version on the API.

Edited by 🤖 GitLab Bot 🤖