Cycle analytics research
Analyzing cycle time is useful to understand how teams work, and evaluate how quickly ideas reach production. There are bugs in the current cycle analytics implementation which cause it to frequently be missing data; these should be fixed. But what assumptions have we made in the design of cycle analytics and how can we test them?
Ultimately, cycle analytics should be more than pretty numbers, but provide direction to how process can be improved so that ideas reach production faster, and should detect if measurable improvements are being made.
Variation between teams
Hypothesis
I expect that cycle time varies meaningfully by team in a project like GitLab. This may be caused by product managers using different processes, differences in backend and frontend teams, and also the variation in the codebase between teams.
Method
Calculate cycle times for Platform, Discussion and CI/CD for gitlab-ce
.
Proposal
If it is shown that there is significant variation between each team, we should build improvements to help customers determine which teams are most efficient. We should also compare the practices applied at GitLab to replicate the practices of our most productive teams.
Variation over time
Hypothesis
I expect the Code, Test and Review cycle times to be consistent month to month in a project like gitlab-ce, but will vary week to week as the development lifecycle progresses. In the early stages of the cycle, I would expect cycle time to be longer as larger changes are worked on, which due to their size and complexity will require Review changes, which will result in more runs of the Test suite, increasing cycle time. After the feature freeze, I would expect cycle time to drop significantly as small bugs are fixed, reviewed and merged quickly. I would anticipate a similar pattern in teams doing weekly or fortnightly sprints too.
Method
Calculate weekly and monthly cycle times for gitlab-ce
over the last 12 months.
Proposal
If it is shown that there is cyclic pattern within the development lifecycle ...
Relationship between size of change and cycle time
Hypothesis
I expect larger merge requests to disproportionately more time to Test and Review due to their complexity. Although slow tests can create an incentive to keep adding to a merge request, I expect expect even small increases in merge request size result in an overall increase in cycle time.
I also expect multiple smaller commits, particularly the first commit, is indicative of more methodically planned change, rather than a single large commit with many tiny fix up commits.
Method
Compare Test and Review cycle time by merge request size. The size of the merge request should be measured by multiple methods: total lines changes, net lines changed, files changes, number of commits, average total lines changed per commit, average net lines changed per commit, total lines changed in first commit, net lines changed in first commit.
Proposal
If large merge requests are shown to increase cycle time disproportionately we should consider adding metrics to allow users to quantify this, and estimate the time savings by changing team practices towards smaller merge requests.