The problem manifests in the Groups::MilestonesController#show action. I'm concerned Projects::MilestonesController#show could be a problem as well, as it also lists issues (for example, see the Milestone for 10.0).
We have 3 columns under the Issues tab. Would we paginate the columns separately? That means we need 3 pagination parameters page_unstarted, page_ongoing, page_completed.
Dennis had a great idea when I was talking with him earlier. Given that the issue "List" page has no performance issue, and can be filtered by milestone, why don't we direct users there? We could show a limited number of issues on the milestone page (maybe 50), with a link to the issue "List" (filtered by the given milestone). For example, see the issue list for the Backlog milestone.
To expand on the UX bit of it, the link could say "Show all issues" and then take them to the issue list view.
I don't think that there's a lot of value out of showing 300+ or 10k+ issues in a small-width columned list, so bringing them to a view where they can do all the filtering and sorting as needed would be more useful.
Thanks for looking into this, I really appreciate it. I think that we should involve the devopsplan stage since Milestones is the primary responsibility of that team.
@kokeefe: Would this fall into your world? What do you think of the proposal? Is this something your team might consider prioritizing?
I wonder if the issues need to be loaded to properly render the burndown chart
Capping the number of issues and linking to the issue list page seems like a reasonable and relatively simple solution. That said, we should be avoiding N+1s.
We could schedule the fix and backlog an issue to remove the N+1?
@felipe_artur This is ready to be worked on. We should take the first solution and hard limit the number of issues we load, linking to the issue list for the rest. Would you like to take it?
I opened !23102 (merged) that should fix the problem, however we should consider a couple of changes:
I think it is better to limit the issues to 20 which is our default page size in the issues list view, also it will make the page load faster.
The issues on milestones show page are not being loaded async. User has to wait all issues to be fetched and rendered before the page loads which hurts performance, even after !23102 (merged) gets merged it can still be slow. Merge requests, participants and labels tabs are already being rendered async, so i think we should create a follow up issue to do the same for issues tab.
Also to keep things simple and clean i took liberty to do frontend part on !23102 (merged) because it is small, very tied up to backend and don't require javascript changes.
@donaldcook What do you think of the 20 issue limit included in this change? When we follow up and make this an async call we could raise the limit to something higher that might accommodate more use-cases.
@felipe_artur I've scheduled that for %12.9 but based on @gweaver's comment below we can see what the plan for milestones is at that time and reschedule or update the issue.
If it matters, we will be overhauling milestones in a few releases which will likely involve getting rid of the three columns and collapsing into one...if we need to temporarily link to the issue list, that is fine but I don't believe it will be the ideal long term solution.
So... this seems to be live ? Even though it's labelled as %12.8.
There are many issues with this, I'm surprised this change got accepted.
What's the point of the milestone page if it's limited to 20 issues ? I'm curious to know whether it is because no-one is using milestones seriously or something else. Anyway, if you're overhauling milestones, why put some half-finished solution in the mean time ? If a customer has 11,015 issues in their backlog milestone, they should know better.
I believe a big change in UX like this should at least be backed by hard data like '99.9% of active milestones have less than 20 issues' - was it studied ?
Priority issues broke, they are not listed first in the columns anymore.
Weights and various counts seem to be broken as well: #199123 (closed)
Yes this is live. It was merged in the previous milestone but still not verified.
I agree this is not a perfect solution, but it is needed for now. We have a problem with how milestones are rendered that can be fixed with #197227 (closed) if we don't change milestones view before, like @gweaver pointed out.
If a customer has 11,015 issues in their backlog milestone, they should know better
We used 20 because is the default page size for issues list, also because of the problem described on #197227 (closed) for now we cannot show much more or it will increase the page loading time.
There are three places that shows how many issues are assigned to the milestone right now in the same page, but yes the link is pointing to the unfiltered issues list. I think we should change it to filter by milestone. Should be a quick fix. I will leave a comment in #199123 (closed).
Priority issues broke, they are not listed first in the columns anymore.
Thanks for pointing this out.
I am not sure why but looks like this is happening only when milestones are over limit. I will take a look. Also the problem with weights described on #199123 (closed) looks unrelated.
Hi GitLab team, I really appreciate your work on this platform, but I have to provide negative feedback that this change has been poorly done. There are many crucial problems with the way this change has been implemented:
Overall Workflow Effectiveness
My own workflow has been really negatively impacted by no longer being able to see an overview of my milestone in a single page. This is really frustrating as someone who relies almost exclusively on this view to manage the execution of milestones which typically contain between 40 and 80 issues. I understand there were some performance concerns for large queries and slow page load times, but I have never experienced that issue with milestone lists of that size. In fact, I now have to browse much more, resulting in what I imagine must be far more requests than what would be incurred by simply fetching all the issues once.
Lack of any intelligence in which issues are displayed
If you are going to limit the number of issues displayed (which I am not in favor of), a huge issue with this change is it doesn't apply any sanity or logic in which issues are displayed. The absolute worst impact of this is that in-progress issues do not display in the "Ongoing Issues (open and assigned)" lane unless they happen to be in the first page of 20 issues returned. This makes it impossible to use the milestone view to track ACTIVELY ONGOING WORK which is completely crazy.
If you must limit the number of issues returned, at the very least the pagination should be prioritized by:
Ongoing Issues - most important, prioritize these above any other issue type and sort within ongoing by issue weight
Unstarted Issues - second most important, work that still needs to be done
Completed Issues - also important, but least so. At the moment my current milestone page shows me mostly completed issues and hides issues that actually require work. This is horrible!
Priority/Weight broken
Self explanatory and already known.
Overall, this was a really frustrating move - given the damage to user experience I would really appreciate if you would consider reverting the change until a more thoughtful design can be implemented that uses data and a bit of user research to understand how customers are using the milestone view for planning and execution.
Thanks for considering my criticism - I appreciate what you do, but am feeling very frustrated at the moment about how this has hampered my own productivity.
@aaclayton Thanks for taking the time to provide this feedback.
It's clear we've broken workflows with this change. We have to address the performance problem but until we have a fix that doesn't impact workflows like yours we'll revert this change.
@felipe_artur I don't see an easier way to solve this that doesn't have negative side-effects. For example, if we implemented the original solution but added issue counts it would still have the problem that some issues wouldn't be visible. And it could end up being the same amount of work anyway.
#197227 (closed) was moved to %Backlog but since it's the only solution we have to this so far I'll move it to %12.9.
@tkuah That's really interesting. We merged !23102 (merged) and reverted it when we had feedback from customers that it was breaking their workflows. For a lot of customers, experiencing a limit on that page makes it completely useless.
However, if we're the only ones experiencing it then I think we could reinstate !23102 (merged) with the limit set to 3000 to at least get the page to load.
@tkuah This is nice.
Yes we could implement it and it is simple.
However #197227 (closed) backend part is already merged on !25794 (merged). If you think that it is not going to finish in time i will be glad to implement this.
@felipe_artur Loading issues async with GraphQL looks like a great idea. Does that still load all issues at once or in batches ?
Though this issue is about group milestones, but I see only project issues (with milestone filter) implemented in !25794 (merged). Do we need group issues too ?
@tkuah Graphql has built in pagination. We can load it all or do it in batches.
Though this issue is about group milestones, but I see only project issues (with milestone filter)
Good catch. Yes we need group milestones, also if we are going to use GraphQL i think we could expose issues in the milestone object.
Anyway this should take a while to be done. @johnhope Do you think we should implement @tkuah's solution for now to at least have something in this release?
@felipe_artur Yeah I think it's a great idea. We can still implement #197227 (closed) but we shouldn't consider it as blocking this one anymore. I reckon it's a weight 1 now, what do you think?
@gweaver The summary here is that we can re-instate the work we reverted before but with a much higher limit that we can prove only affects GitLab. Doing so would allow milestone pages to load for affected users but shouldn't affect customers.