POC - Replace REST with GraphQL API requests in GitHub Import
I believe we can speed up the migration process if we replace REST requests with GraphQL requests
For instance, the GraphQL queries below can request several pieces of information that would require multiple REST requests in just one request
Issues query
query {
repository(owner: "rails", name: "rails") {
issues(first: 100) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
title
body
comments(first: 100) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
body
author {
login
}
createdAt
}
}
},
timelineItems(first: 100) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
__typename
... on IssueComment {
author {
login
}
createdAt
}
... on ClosedEvent {
actor {
login
}
createdAt
}
# Add more event types as needed
}
}
}
}
}
}
}
}
Merge request query
{
repository(owner: "rails", name: "rails") {
pullRequests(first: 10) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
title
mergedBy {
login
}
reviewRequests(first: 5) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
requestedReviewer {
... on User {
login
}
... on Team {
name
}
}
}
}
}
reviews(first: 5) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
author {
login
}
state
body
createdAt
}
}
}
timelineItems(first: 5) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
__typename
... on IssueComment {
author {
login
}
body
createdAt
}
... on ClosedEvent {
actor {
login
}
createdAt
}
# Add more event types as needed
}
}
}
comments(first: 10) {
pageInfo {
endCursor
hasNextPage
}
edges {
node {
body
author {
login
}
createdAt
}
}
}
}
}
}
}
}
Paginating nested resources can be a challenging task. However, in most cases, pagination for nested resources is not needed, as we can request up to 100 resources at a time. The majority of issues and MRs typically do not have that many nested resources.
I think that adopting this approach can decrease the migration time by 6 times or more
Edited by Rodrigo Tomonari