Index feeds for abouts
As a part of #745, we should start using index feeds to support partial replication, in other words, to replicate less data and make the database more lean, and improve the user experience.
As a first use case, when we replicate distant peers, we can fetch only their about
messages (via index feeds), not their full feed. This allows the user to see a distant peer's profile without having to replicate their posts and votes. This only applies to distant peers that are using metafeeds, i.e. Manyverse and hopefully others.
Note about subfeeds and indexed feeds:
Indexed feeds are a transitional aid between "main" world and "metafeed" world. Indexed feeds are handy for about
s, but not for vote
s, because there are too many votes and spams the log with too many msgs (one for the actual OOO vote and another for the indexing msg). We should have a dedicated subfeed for votes, and I publish my votes on both the main and the subfeed for votes. Whoever I replicate who has a metafeed tree I can assume they have this dedicated subfeed for votes. This is what we'll refer to as "vote-subfeed". Similarly, "post-subfeed". Contrast that with "index-about" and "index-contact".
-
Change ssb-replication-scheduler so that it also deletes what is not requested - It should cover all the features of ssb-friends-purge, which should be deprecated & archived
- Requirement: do not EBT request a feed while its delete is in progress, queue it instead
- Requirement: begin deleting a feed only after its EBT request has been cancelled
- Requirement: delete only one feed at a time
- Requirement: don't delete while db2 indexing is in progress
-
Install ssb-index-feeds
in the backend -
Enable writing index feeds for about
,contact
-
Remove customization of hops setting - Hops setting has been a footgun. As soon as users move it to 3 or 4, they are in OOM territory. Setting it to 1 is also contrived, it makes the network suddenly super small and has zero discovery because you don't know who you don't know.
- Removing this setting also allows us to control and guarantee good user experience, especially now with partial replication, see below. This also provides predictable outcomes for users.
-
Configure ssb-replication-scheduler - Hops 0: replicate everything
- Hops 1: replicate main + index-about + index-contact
- Hops 2: replicate main + index-about + index-contact (dont drop votes, because
ssb:message/sha256/BF-PCfUVRutRsHvzeyUGwV98tuifYPCnZNLu_8qDwmo=
) - Hops 3: replicate about+contact, and if they dont have root metafeed, dont replicate
main
- Hops 4: replicate about, and if they dont have root metafeed, dont replicate
main
-
Test that Profile screen works when replicating index feeds of a distant peer -
Lots of cross-computer testing in a staging environment