Add pagination to Bitbucket Server PR Notes Importer

• Please check this box if this contribution uses AI-generated content (including content generated by GitLab Duo features) as outlined in the GitLab DCO & CLA

What does this MR do and why?

See full context in !149302

This is Part 3 from a series of MRs to handle issues: #451129.

• Part 1 - Moving HTTP calls
• Part 2 - Separate Importers Logic
• Part 3 - Add Paging on HTTP Calls <-- This MR

This MR adds pagination to Bitbucket Server Activities HTTP call, so that the importer job can resume from the last page. This MR also changes the feature flag from development into beta

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After

How to set up and validate locally

1. Initial environment setup can follow this
2. Prepare some data in Bitbucket Server
3. Patch the code to inject interruption:

diff --git a/lib/gitlab/bitbucket_server_import/importers/notes_importer.rb b/lib/gitlab/bitbucket_server_import/importers/notes_importer.rb
index 6d06ab72c430..288fb244071d 100644
--- a/lib/gitlab/bitbucket_server_import/importers/notes_importer.rb
+++ b/lib/gitlab/bitbucket_server_import/importers/notes_importer.rb
@@ -58,6 +58,15 @@ def import_notes_individually
                          "page #{page} using batch-size #{concurrent_import_jobs_limit}"
               )

+              if merge_request.iid == 1
+                Gitlab::Redis::SharedState.with do |redis|
+                  temp_key = 'test-bitbucket-pr'
+                  temp_counter = redis.incr(temp_key)
+                  redis.expire(temp_key, 5.minutes)
+                  raise "purposely interrupt" if temp_counter == 2
+                end
+              end
+
               activities = client.activities(
                 project_key, repository_slug, merge_request.iid,
                 page_offset: page, limit: concurrent_import_jobs_limit
@@ -160,6 +169,10 @@ def merge_request_collection
         def page_counter_id(merge_request)
           "merge_request/#{merge_request.id}/#{collection_method}"
         end
+
+        def concurrent_import_jobs_limit
+          2
+        end
       end
     end
   end

4. Tail the log file: tail -f log/importer.log
5. Go to http://127.0.0.1:3000/import/bitbucket_server/status
6. Click Import