We've done a lot of work to create GraphQL APIs for the GitLab Package Registry. With that work wrapping up in 14.1, it would be great to start updating the Package Registry UI to leverage the new API. This will help with performance and help to unblock future feature development.
Proposal
Refactor the package details page from using a mix of controllers/rest API to use the new GraphQL API.
@trizzi@dcroft I won't be able to complete this work for 14.1, following a small retrospective on why:
Wrong estimation
I've estimated this work a frontend-weight3 based on my experience with the Container Registry similar refactor, this was wrong because of a few points:
The Infrastructure registry is tied to the package app, requiring more planning and work
The sheer number of Package managers that we support and all the nuances around them, I have much more moving parts to take in account, test and double check
A few bug that we currently have that I stumbled upon doing the refactor, forcing me to stop and address them
Missing data from the API
While the package related GraphQL API is done, in the details API we rely on data coming from other part of the models/schema in particular we lack some info from the pipeline schema, I will be opening an MR to add those.
Unplanned time off on my part
Due to some personal circumstances I had to take more time off, I was less focused, than I had thought this milestone, leading to a small than usual throughput
Since the work started already, I am confident that this can be closed in 14.2
After 13 MRs of which 11 frontend and 2 backend that spun: 2800 lines moved and 4300 lines changed this work seems to be reaching the end!
Big big thank you to: @rob.hunt@ohoral that followed me in the first part of this work and @cngo and @ntepluhina for the reviews on the second part.
Special thanks to @10io for advices, listening to my rants and helping however he could.
I will create a retrospective around the miss-estimation of this work, which was clearly not a frontend-weight3 with the aim to highlight why this huge ~"feature::maintenance" work is really important for the future of the frontendPackage Repositories codebase.
I think I've defined this issue as one of the most off estimate I have ever given.
This work was initially estimated as a frontend-weight3 but it took two milestones and 13 MRs to be completed, so it was clearly a frontend-weight5.
I believe it is important to reflect on why this weight estimation was wrong and also why this work is even more important of what I thought.
What went not so well
Sheer amount of components, the package codebase has a lot of components, many are simple, but they all needed to be updated
Sheer amount of data points coming from the API that are bound in the frontend implementation
Each package manager has its quirks, custom implementation, weird conditions, that all needed to be re-tested both manually and by updating our unit tests
The frontend code was a bit of a Frankenstein situation, where we had multiple approaches to solve the same problem (with each package manager implementing its own)
Lack of a single source of truth, with data half coming from the REST API and half from the controller
A plethora of tiny bugs that became clear during the refactor, mostly coming from ☝
Despite a very thorough planning of the GraphQL API we still lacked some data there that we had to add during the refactor
The review process was longer because many small things from the original code needed to be updated to our latest best practices
What went well
We now have a single source of truth, the API
We have addressed several performance issues of the page - not going into details for security reasons -
We now have a clear and documented contract of what is available to the frontend, the GraphQL API.
We can now chop the page data loading in pieces, use the UI to defer some loading, show loaders and add pagination to all our lists
Action items
We should always keep in mind that the package registry has an inherent complexity due to the sheer and growing amount of package managers, even if the UI and UX is somehow uniform among them
We should be more proactive to create and schedule tech-debt issues when the code goes stale in the codebase, specifically when we should update something to the latest best practices
We should create better feature tests to test the UI of each package manager, specifically to address the differences between each, this would increase the confidence in the code and reduce potential bugs, while we have a very good set of unit test, we need feature tests because the UI has a lot of dynamic pieces that turns on or off based on the presence of certain data points.
We should probably create a bit of documentation around adding new pieces from new package manages, so that everything respects the same architecture