Skip to content

The page listing a user's subscriptions is not scalable

This has been reported here: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/IQ56EMPB3NA7DL2J5ZXJS6HHP6TDN7T7/

Basically, the page listing a users's subscription (/accounts/subscriptions/) does not scale, because for each subscription it does the following REST queries:

GET /3.0/addresses/user@example.com
GET /3.0/members/123456789012345678901234567890/preferences
GET /3.0/lists/listname.example.com
GET /3.0/lists/listname.example.com/roster/moderator
GET /3.0/lists/listname.example.com
GET /3.0/lists/listname.example.com/roster/owner

So when a user has more than 15 subscriptions on a busy server, the page takes tens of seconds to load, and our reverse proxy timeouts.

IMHO this could be optimized in multiple ways:

  • don't request the same URL twice
  • make more data available in the base subscription endpoint so that additional queries are not necessary
  • cache the results in Django's cache, and invalidate using signals
  • load each subscription asynchronously via a separate HTTP request using Javascript.

The latter seems to me like the best effectiveness/complexity ratio, and it will scale better.