Use sorted sets to dynamically generate feeds in Redis backend
Benefits:
- Reducing complexity, allowing me to get rid of the feed processor
- Speeding up post creation process overall
- Atomically constructed feeds (one could never interfere with the process of a sorted set insertion)
Drawbacks:
- Feed presentation may or may not get slow
- May be solved by caching feed views and then busting the cache once the feed gets updated
- How this would get implemented in other backends?
- With a file-based implementation one would need a lock on a feed index
- Maybe there are some formats that will allow this to be atomic too
- How exactly does Redis store sorted sets? Maybe I could learn something from it
- One could just make it an implmentation detail of the Redis backend and make feed storage in other backends require a lock on the feed
- With a file-based implementation one would need a lock on a feed index
Draft implementation:
- Post creation (inserting into feeds)
timestamp = toUnixTimestamp(post.published[0]) ZADD ${timestamp} ${post.uid[0]}
- Requesting the feed:
feed = <the feed requested> user = <the user reported by the frontend or null, in case of anonymous access> limit = <the feed's post count> page = <the page number> offset = (${page} - 1) * ${limit} function checkAccess(post: MF2Data) -> boolean nowTimestamp = toUnixTimestamp(<request time>) feedList = [] while feedList.length < limit: feedList += filter(lambda post: checkAccess(post, user), ZREVRANGEBYSCORE feed_${feed} -inf LIMIT ${offset} ${limit}) offset += limit return feedList[0:${limit}]