Tracking adoption / retention / reactivation in GitLab
Dev: https://dev.gitlab.org/gitlab/gitlabhq/issues/2113
If you're running GitLab as a SaaS, it is important to know where your users are in adopting GL (1), how (often) they use it (2) and you should be able to reactivate them (3).
We can build features for these points into GitLab for our own profit (GitLab.com) and others'. We can do this iteratively, quite easily. Below each point, which can be subdivided in separted features
1. Tracking adoption + user info
For a user (has signed up) you want to know whether they:
- confirmed their email
- (optional) synced an ssh key
- created a project
- pushed a commit
- (optional) created a group
- (optional) added people to a group
This information has to be presented as a funnel of sorts, so it's clear what the next step is. Also, general information should be available to highlight hardpoints (if we see that no one actually pushed a commit, maybe that doesn't work well). In addition, worthwile information:
- OS
- browser
- location
2. Retention
We want to see whether people keep using GitLab. If there is a 'typical' slope down, for instance, that would mean that people for some reason start using it less and less.
In this case, we mainly want to see a general measure, as we already have things like 'last logged in'.
"Last commit made at" would be nice too.
3. Reactivation
This is the most important one. For the steps in (1) we should have triggers if they do not go to the next step / depending on their status. For instance when:
- a user didn't log in for more than a week
- a user doesn't push a commit after more than a day of having created a project
- a user created a group, but never added anyone to it
- a user signed up, but never confirmed their email
- etc
In those cases we can easily send them an email. Very important at that moment is that we track what happens after emailing, similar to (1).
Implementation/Proposal
We create a field for an admin to see all the steps in (1). Preferably it shows the current step they're at and when that was made, the previous and the next one. This would be best combined with a general measure for these things so we can see for GitLab.com what happens with those 70k inactive users.
After that is done, we should revisit and consider how to properly measure retention. This is typically done with these graphs:
I think the reactivation is extremely important, but also quite a hefty feature. Apps such as Intercom.io do this very well and by that time it might be worth to look into plugging something like that in or rolling it ourselves.
(note: Intercom actually does all of the above, AFAIK; it would be quite an investment to implement them, but it's definitely an option)
Sytse
Thanks for the great issue, lets focus on retention first. Activation might be different per person (if I'm only discussing issues I don't need to push a commit) and Activation is really hard. First goal should be a cohort graph such as show in your screenshot.
Job
Not a bad idea!
There are two strategies:
- measure any action and make a graph like above for any action
- measure (/filter by) different actions and make the graph filterable
I'd prefer the second one obviously, so we can see whether people are just visiting or also actually committing. Might give some other interesting data.
Sytse
can we reuse existing data? start with comments?