Add relations export API for Projects
What does this MR do and why?
Relates to #340309 (closed)
Adds API endpoints
Everything is behind the bulk_import
FF enabled by default.
The API endpoints will be ready with this MR, but the real intent of these endpoints is to allow projects to be migrated during the GitLab Group Migration. So, the API here is just a ground work for this. More work will be done on later MRs to support importing this data that we're making exportable here.
API endpoints being added:
-
POST /projects/:id/export_relations
: export Project relations to compacted NDJSON files -
GET /projects/:id/export_relations/status
: check the status of the export -
GET /projects/:id/export_relations/download
: download the NDJSON files
Adds API docs
doc/api/project_relations_export.md
Finally, I'll take care of #340309 (comment 677591237) in a separate MR.
Screenshots or screen recordings
I've tested it with the bellow commands and got the below results
How to set up and validate locally
-
Make sure FF is enabled on your GDK:
Feature.enable!(:bulk_import)
-
Create a project and add issues to it.
-
Export the project relations and check the "accepted" response. Remember to set your
private_token
with your GitLab access token with theapi
scope.λ gitlab git:(jcunha-add-relations-export-api-for-projects) ✗ curl --header "Content-Type: application/json" \ --request POST \ "https://gdk.joaocunha.eu:3443/api/v4/projects/24/export_relations?private_token=yrak6Tvi4NfGUyEbp7L3" {"message":"202 Accepted"}%
-
Check the status of the export (it can take a while for it to finish):
λ gitlab git:(jcunha-add-relations-export-api-for-projects) curl --header "Content-Type: application/json" \ --request GET \ "https://gdk.joaocunha.eu:3443/api/v4/projects/24/export_relations/status?private_token=yrak6Tvi4NfGUyEbp7L3" [{"relation":"releases","status":1,"error":null,"updated_at":"2021-09-16T15:10:57.596Z"}, {"relation":"project_feature","status":1,"error":null,"updated_at":"2021-09-16T15:11:01.787Z"}, {"relation":"issues","status":1,"error":null,"updated_at":"2021-09-16T15:10:56.795Z"}, ...
-
Download the file
λ gitlab git:(jcunha-add-relations-export-api-for-projects) curl --header "Content-Type: application/json" \ --request GET \ --output issues.ndjson.gz \ "https://gdk.joaocunha.eu:3443/api/v4/projects/24/export_relations/download?r relation=issues&private_token=yrak6Tvi4NfGUyEbp7L3" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 678 100 678 0 0 623 0 0:00:01 0:00:01 --:--:-- 623
-
Decompress the file
λ gitlab git:(jcunha-add-relations-export-api-for-projects) ✗ gunzip issues.ndjson.gz
-
Read the files contents
λ gitlab git:(jcunha-add-relations-export-api-for-projects) ✗ cat issues.ndjson {"id":551,"title":"New test issue","author_id":1,"project_id":24,"created_at":"2021-09-16T14:18:25.190Z","updated_at":"2021- 09-16T14:18:25.190Z","description":"export issue test description","iid":1,"updated_by_id":null,"weight":null,...}
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #340309 (closed)