1. 28 Nov, 2020 1 commit
    • Kassio Borges's avatar
      Github importer - import pull request merged by · 7fb468f0
      Kassio Borges authored
      The merged by field is not available on the list of pull requests, which
      is used to import the merge request. Therefore, for each merged pull
      request we need to do another request to Github to fetch this field.
      
      Once fetched, if the user can be mapped to a GitLab user, it's added as
      to the `MergeRequest::Metrics#merged_by`. Otherwise, a note is added to
      the merge request with the text: 'Merged by "login"', where "login" is
      the Github login of the user who merged the pull request in Github.
      7fb468f0
  2. 25 Nov, 2020 1 commit
  3. 23 Nov, 2020 1 commit
  4. 19 Nov, 2020 1 commit
  5. 18 Nov, 2020 1 commit
  6. 13 Nov, 2020 1 commit
  7. 11 Nov, 2020 2 commits
  8. 10 Nov, 2020 1 commit
  9. 29 Oct, 2020 1 commit
    • Vladimir Shushlin's avatar
      Destroy pages deployments · 28780fa2
      Vladimir Shushlin authored
      Add worker for destroying pages deployments
      
      Schedule DestroyPagesdeploymentsworker on new deployment
      
      Remove pages deployment when pages are destroyed
      28780fa2
  10. 26 Oct, 2020 1 commit
    • David Fernandez's avatar
      Throttle the cleanup policies execution · 0b4849db
      David Fernandez authored
      By using a limited capacity worker and dedicated database column on
      container repositories to mark them as:
      
      * cleanup_scheduled: waiting for a worker pickup (happens after a
      cleanup policy execution)
      * cleanup_ongoing: worker cleaning up this repository
      * cleanup_unfinished: worker cleaned part of the repository but had to
      stop
      * cleanup_unscheduled: nothing to do
      0b4849db
  11. 23 Oct, 2020 2 commits
    • George Koltsov's avatar
      Add Group Import via GraphQL · 27daa621
      George Koltsov authored
      - Introduce Group Import ETL pipeline to import
        groups from source GitLab instance using GraphQL
      - Add sidekiq jobs & workers to execute group import
        in a distributed fashion
      27daa621
    • Arturo Herrero's avatar
      Propagate integration to group descendants · 88d7f03b
      Arturo Herrero authored
      After propagate group-level integrations
      !44023, we missed
      one scenario where the integration could inherit from another
      integration from the ancestors, not the closest one, this is something
      that we also fixed finding the closest integration
      !45022.
      
      Example: Having a group, a subgroup, and a project belonging to the
      subgroup. If we update the group, it propagates the settings to the
      subgroup and project (both having inherit_from_id = group.id (pointing
      to the group) but if after that, we update the settings of the subgroup,
      it doesn't propagate them to the project. This is because the project's
      inherit_from_id is still pointing to the group id from the first
      propagation.
      
      This always updates the integrations of any descendants for a group
      integration.
      88d7f03b
  12. 15 Oct, 2020 3 commits
  13. 13 Oct, 2020 2 commits
  14. 07 Oct, 2020 2 commits
    • John Skarbek's avatar
      Mark sidekiq queues that require local shared disk access · ee37dd26
      John Skarbek authored
      * Queues marked with this require some form of access for data
      that is shared among various components.  An example might be that the
      queue writes data to a local directory that is quickly displayed to the
      user.
      * In scenarios where large installations exist, this requires some form
      of shared stored, commonly NFS, to be configured where sidekiq can write
      the data, but then a set of web front ends have the same NFS share such
      that they are capable of reading that data and vice versa.
      * This commit marks all queues _known_ to have such a dependency
      * As this dependency is removed, this tag should be removed
      ee37dd26
    • Stan Hu's avatar
      Delete project and system hook logs in batches · 78632ed8
      Stan Hu authored
      When a user attempts to destroy a Web hook, the database will attempt to
      delete all the associated Web hook logs. However, as we have seen in
      #21940, the table may be
      too bloated or the number of rows too large that this deletion can time
      out due to a 15-second statement timeout.
      
      We rectify this situation by deleting these logs in batches of 1000
      outside of a transaction. That should be acceptable since old logs get
      pruned periodically anyway, and when a Web hook is destroyed it's more
      important that the destruction makes progress and eventually removes the
      hook.
      
      Relates to #21940
      78632ed8
  15. 02 Oct, 2020 1 commit
  16. 28 Sep, 2020 3 commits
    • Luke Duncalfe's avatar
      Copy designs to new issue when issue is moved · 0a8a30d3
      Luke Duncalfe authored
      Adds a set of services and workers to allow an asynchronous copy of
      designs from one issue to another.
      
      It happens asynchronously from the in-request issue move as the copy
      can take around 15s for 50 designs.
      
      The copy involves copying designs, versions and discussions on designs.
      
      #13426
      0a8a30d3
    • Arturo Herrero's avatar
      Update integrations using batching and queue · 1336ebea
      Arturo Herrero authored
      This updates all inherited integrations using the same approach of
      batching and Sidekiq queue that we use when bulk create integrations for
      project and group integrations.
      1336ebea
    • Arturo Herrero's avatar
      Propagate integrations using batching and queues · 163ad707
      Arturo Herrero authored
      This is an important change in the architecture to propagate
      integrations. We can now propagate instance-level integrations and
      templates using batching and Sidekiq queues.
      
      The problem before is the performance of the worst-case scenario, where
      if there are no matching records and the anti-join. With the new
      approach, each job in the new queues handles a batch of projects/groups;
      rather than having a single job for all of them.
      
      This is what we do right now in the complex case of propagating an
      instance-level integration:
      - Update inherited integrations,
      - Create integration for all projects without integration.
      - Create integration for all groups without integration.
      
      BEFORE:
      
                       Save integration
                              ↓
                  ┌┬───────────────────────┬┐
                  │| Propagate integration |│
                  └┴───────────────────────┴┘
                              ↓
                 Update inherited integrations
        Create integration for all projects without integration
        Create integration for all groups without integration
      
      AFTER:
      
                       Save integration
                              ↓
                  ┌┬───────────────────────┬┐
                  │| Propagate integration |│
                  └┴───────────────────────┴┘
                     ↓                   ↓
        ┌┬─────────────────────┬┐ ┌┬───────────────────────┬┐
        │| Propagate to groups |│ │| Propagate to projects |│
        └┴─────────────────────┴┘ └┴───────────────────────┴┘
                  Update inherited integrations
      163ad707
  17. 25 Sep, 2020 1 commit
  18. 23 Sep, 2020 1 commit
  19. 22 Sep, 2020 1 commit
  20. 18 Sep, 2020 1 commit
  21. 14 Sep, 2020 1 commit
  22. 11 Sep, 2020 1 commit
    • Patrick Bajao's avatar
      Clean up stale merge request HEAD ref · a2f003ab
      Patrick Bajao authored
      14 days after a merge request gets closed/merged, its HEAD ref
      will be deleted and moved as a `keep-around` ref (if not yet
      added).
      
      This is to ensure that the commit won't be deleted when GC runs
      and still being able to reduce the number of advertised refs on
      git push/pull.
      a2f003ab
  23. 10 Sep, 2020 2 commits
    • Amy Troschinetz's avatar
      Adds CI Platform Metrics bookkeeping worker · 9ff77fce
      Amy Troschinetz authored
      **app/workers/all_queues.yml:**
      
      - Automatically updated.
      
      **app/workers/ci_platform_metrics_update_cron_worker.rb:**
      
      - Adds a cron worker to update the CI Platform Metrics.
      
      **config/initializers/1_settings.rb:**
      
      - Cron settings for the new worker.
      
      **spec/workers/ci_platform_metrics_update_cron_worker_spec.rb:**
      
      - Spec tests for the new cron worker.
      
      **config/gitlab.yml.example:**
      
      - Adds cron settings to example config.
      9ff77fce
    • Jacopo's avatar
      Removes unused classes on initial Ci::Ref implementation · b70f5e5c
      Jacopo authored
      Removes unused classes introduced in inicial Ci::Ref implementation
      and deprecated afterwards.
      b70f5e5c
  24. 04 Sep, 2020 2 commits
  25. 01 Sep, 2020 1 commit
  26. 27 Aug, 2020 1 commit
  27. 26 Aug, 2020 2 commits
    • Alex Kalderimis's avatar
      Add worker queue definition · 029d6152
      Alex Kalderimis authored
      This adds the required worker queue definition for sidekiq, with its
      configuration and generated defintion.
      029d6152
    • Sean McGivern's avatar
      Make Gitlab::PagesTransfer possible to call async · 5a378f14
      Sean McGivern authored
      When we rename a project, we call Gitlab::PagesTransfer to move any
      pages directory on disk. This makes it convenient to use that class
      through Sidekiq instead, so that Puma no longer needs access to Pages
      NFS mounts.
      5a378f14
  28. 25 Aug, 2020 1 commit
  29. 21 Aug, 2020 1 commit