Latest update hammers gitlab with API requests until the rate limit hits

The extension continuously polls the endpoint no matter what, so it just instantly hits the rate limits I have set (7200 requests per hour). It doesn’t stop polling once it gets back error 429 either, so the extension does not work at all anymore.

@ReagentX Thank you for filing the issue. Can you please provide us with more details?

• Are you using gitlab.com or a different GitLab instance?
• Do you know which endpoint is the extension hitting 7200 times per hour?
• Could please include any relevant logs?

The issue persists across both our self hosted instance and the gitlab website.

The extension doesn’t say what endpoint it’s hitting, it just endlessly pops up new notifications saying “Gitlab Workflow: failed to perform your operation” and “429 - ‘retry later\n’”

There are no logs.

Hi @ReagentX ,

the extension should try to get the last pipeline status and branch MR each 5 seconds (so 1440 times per hour) https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/blob/master/src/status_bar.js#L121 - https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/blob/master/src/status_bar.js#L184.

Also the extension pull some other data when you inspect issues, etc. but I don't think they are too many API call.

Are you using the same access token with many instances (multiple windows) of VS Code at the same time?

@viktomas maybe we could set the refresh timeout configurable or even disabled for users that don't need this functionality?

Thanks for the explanation @massimeddu. I can see more than two requests happening every 5 seconds. When I put console.log into gitlab_service.fetch() I see every 5 seconds the following requests:

https://gitlab.com/api/v4/projects/278964/pipelines?ref=master
https://gitlab.com/api/v4/projects/278964/merge_requests?state=opened&source_branch=master
https://gitlab.com/api/v4/projects/278964/pipelines/162016174
https://gitlab.com/api/v4/projects/278964/merge_requests/34986/closes_issues
https://gitlab.com/api/v4/projects/278964/pipelines/162016174/jobs

I think that we should increase the interval and maybe see how the Web IDE polls for pipeline updates.

In the version 2.2.2 the interval was

• 30 seconds for refreshPipeline()
• 60 seconds for fetchBranchMR()

2.2.2 version of status_bar.js

Should we be using a different token for each separate window? At most I have 4 open at a time. If we should, that should probably be documented.

Even if I do use a different token in each window, it instantly gets rate limited.

created merge request !75 (merged) to address this issue

mentioned in merge request !75 (merged)

added Category:GitLab CLI Editor ExtensionsVS Code devopscreate groupeditor [DEPRECATED] typebug labels

added priority1 severity2 labels

@viktomas For reference, I think the Web IDE is every 10 seconds, but it's also only making a single request not sending 5 requests every time. Reverting to the old timing makes sense, and I don't think we should make this user configurable based on some of our other non-configurable application limits.

I'm wondering if you've reached out to anyone from infrastructure to see what kind of abuse we're seeing to GitLab.com.

There has been a noticeable increase of gitlab.com traffic on the pipelines and merge_requests endpoints:

Dasboard accessible to GitLab team members

I'm going to release the interval increase change today or early next week.

Hi @viktomas I suppose that reverting to old values is the right choice if you don't want to make them configurable. I guess a configurable option could be better, and I've actually implemented it in !54 (closed) !54 (diffs) (maybe we could set a default value and a minimum value for this setting that makes more sense !54 (diffs) ). Actually, if a user for some reason, opens a lot of instances of the extension, it could get a 429 error even with 30/60 seconds timeout. In those cases it could be useful to give to the user the option to increase the value. On the other hand, most of the users that use only one instance of the extension, can prefer to have a faster refresh over the pipeline status (10 seconds as WebIDE looks reasonable as minimum value to me).

Btw, on !54 (closed) using vscode git extension we have a way to mitigate this problem, triggering the updates related to current branch only when the branch is actually changed: !54 (diffs)

Thanks for the suggestions @massimeddu, I went with the simplest fix (reverting to 30/60 timeout) to reduce the load on GitLab API for now.

Same as @phikai I would prefer not to make this configurable and instead, try to optimize in different ways (some of which you suggested and even implemented in !54 (closed)).

Actually, if a user for some reason, opens a lot of instances of the extension, it could get a 429 error even with 30/60 seconds timeout. In those cases it could be useful to give to the user the option to increase the value

We should only run the polling if the extension is active (the same way the official git extension does it). #212 (closed)

Btw, on !54 (closed) using vscode git extension we have a way to mitigate this problem, triggering the updates related to current branch only when the branch is actually changed: !54 (diffs)

But the pipeline status should be polled even when there is no branch change right?

But the pipeline status should be polled even when there is no branch change right?

Yes correct.

marked this issue as related to gitlab-com/gl-infra/production#2363 (closed)

closed via merge request !75 (merged)

closed via commit 94f0370d

mentioned in issue #212 (closed)

mentioned in issue create-stage#12685 (closed)

The API endpoint traffic is back to normal:

changed milestone to %13.2

assigned to @viktomas

mentioned in issue #238 (closed)

added groupide label and removed groupeditor [DEPRECATED] label

added sectiondev label