Refactor project importers / gitlab migration / history to re-use same codebase

At ~"group::import" there are several codebase parts which are important for us

• importers (GitHub, BitBucket, etc.)
• gitlab migration
• history for importers
• history for gitlab migrations

For historical reasons these are organized as different parts of the code (previously even each project importer was different) and this results in additional efforts to implement same functionality important for different parts of the project

Latest examples (which lead to this issue) were Group Migration list - 'To new group' should de... (#354694 - closed) and Importers - Default to the current group when i... (#359130 - closed) however with our increased team efforts towards pushing GitHub importer forward this will be more often:

• Cancel group imports (#250347) and Cancel GitHub project imports - frontend (#247325 - closed) (note, that we might definitely want to expand this to other project importers as well)

• GitHub Importer - Map GitHub Teams to GitLab du... (#353939) and GitLab Migration - Multi-select group(s) from t... (#267952) (they are actually about selecting thing inside a tree-like structure)

Additionally, I'm pretty sure that from UX perspective we're not happy that we're not creating consistent experience across multiple importers:

• some importers expose pagination (bitbucket), some infinite scroll (github), some expose nothing (gitlab.com importer)
• we have duplication of displaying completed imports in both importer pages of github and history
• while history page is intended to display imports which are currently in progress, we're missing realtime updates there (applies both to history of project imports and gitlab migration)
• we have re-import in group migration but not in project importers for no good reasons (backend is already capable of doing reimports)

By unifying our codebases we will create consistent experience for our users, greatly decreasing amount of code we should maintain

Why now?

With GitHub importer moving forward we will be implementing several issue mentioned earlier. By considering refactoring in mind this will be great opportunity to meet both results and iteration values - let's assume that will be 20% additional effort if new code will be written with this decision in mind and certain time spent to support this direction

How to proceed

If we decide to go with this approach first of all we will needfigure out expected "general behavior" for all these things (pagination, sorting, etc.). Engineering (both backend and frontend) will contribute with current limitations while our great UX will help us to do it in the sanest possible way. This could be the very first step of the process

After that I will be happy to create issues related to frontend engineering perspective (refactoring several parts) to make sure we have proper visibility of refactorings and making sure that all our refactoring are well aligned with our Deliverable and priorities

/cc @wortschi @danmh

added frontend meta workflowsolution validation + 2 deleted labels

assigned to @xanf

mentioned in merge request !90899 (merged)

changed the description

@xanf, thank you so much for creating this fantastic issue. I love how you've laid out the sections and made a strong case.

I 100% support this initiative.

I also expect shared patterns will create efficiency in any user testing, as large parts of the flow will be the same. And also benefits in quicker and wider reach of any learnings and improvements, as it makes it easier to improve all importers at once.

FYI @badnewsblair @vkarnes this is a good example of a great initiative for UX in ~"group::import".

Creating consistency on the backend with the codebase is the perfect opportunity to make the User Experience consistent!

Setting label(s) devopsmanage sectiondev based on ~"group::import".

added devopsmanage sectiondev + 1 deleted label

@xanf, please can you add a type label to this issue to help with issue discovery in issue reports.

---

(improve this comment?)

added auto updated + 1 deleted label

@xanf Thank you for raising this, I think this is a great proposal from a maintainability aspect but also towards creating a unified/consistent user experience.

we will needfigure out expected "general behavior" for all these things (pagination, sorting, etc.).

I agree, we need to define the desired behavior first but also keep an the technical limitations some of the 3rd party APIs might have.

Also, I wonder if we need to consider all importers or if we can exclude some importers from this effort as they are not being used by a larger user base or if the external service is not being maintained anymore?

Also, I wonder if we need to consider all importers or if we can exclude some importers from this effort as they are not being used by a larger user base or if the external service is not being maintained anymore?

Great question. Most of the importers have "bare minimum" functionality so there will not be too much effort to support something. However, I think as soon as consistency does not require too much effort we should unify look&feel as much as possible - for example if gitlab.com importer does not use server pagination - we can do it client-side (low effort) just for consistency

added typemaintenance + 1 deleted label

changed the description

mentioned in epic &8217

mentioned in issue #370734 (closed)

mentioned in issue gitlab-design#2062 (closed)

closed

promoted to epic &8785

added groupimport and integrate label and removed 1 deleted label