1. 22 Oct, 2019 4 commits
    • Victor Zagorodny's avatar
      Make SAST compatible with private dependencies · 42bb0907
      Victor Zagorodny authored
      Enable the propagation of all of the job ENV
      vars to the sast docker container and thus to
      the analyzer containers being started by it.
      This enables passing credentials for private
      repos being accessed by underlying analyzers
      via UI-set environment variables. Exclude a
      bunch of system-related ENV vars from
      propagation.
      42bb0907
    • Adrien Kohlbecker's avatar
      Add elastic stack as cluster app · ecb16107
      Adrien Kohlbecker authored
      Add a new gitlab-managed app for the elastic stack
      ecb16107
    • Vladimir Shushlin's avatar
      Add rake task for fixing pages access control on .com · 9dd7424f
      Vladimir Shushlin authored
      We've enabled access control for pages on .com 2.5 months
      after the migration fixing settings were run.
      
      All private projects created withing this timeframe have
      access control enabled but users do not know that.
      
      We add rake task which will find such projects and reset
      settings to the anticipated state.
      9dd7424f
    • Matthias Käppler's avatar
      More fine-grained metrics for sidekiq workers · 73b14405
      Matthias Käppler authored
      We currently only measure the global (node-level) business in terms
      of jobs being processed. We would instead like to be able to know
      for each sidekiq process how saturated it is in terms of actual
      concurrency vs requested/max concurrency.
      
      This patch does the following:
      
      - add a worker/process dimension to all existing metrics
      - introduce a new metric to observe concurrency per process
      
      by relating "actual job count per process" to "max job count per
      process" we can then obtain some sort of saturation metric.
      73b14405
  2. 21 Oct, 2019 3 commits
    • Nels Nelson's avatar
      Increase timeout for FetchInternalRemote RPC call · ca13bb71
      Nels Nelson authored
      This call is currently only used by `UpdateRepositoryStorageService`,
      and it can easily time out after 30 seconds for large projects. Increase
      it to a long timeout to ensure this doesn't fail prematurely.
      
      Encountered in
      gitlab-com/gl-infra/production#1250
      ca13bb71
    • Fabio Pitino's avatar
      Expose arbitrary artifacts via MR widget · 5c168885
      Fabio Pitino authored
      * Allow user to specify `artifacts:expose_as` in CI config
      * Save :has_exposed_artifacts in job metadata for queries
      * Find exposed artifacts in build metadata model
      * Expose API endpoint for frontend to fetch data
      
      Fix unlrelated controller specs
      
      Use default has_exposed_artifacts NULL
      
      Avoid using a background migration to change NULL
      to false. It's not needed.
      
      Feedback from review
      
      * add links to issue for follow up refactoring
      * preload job artifacts and metadata associations
      * merge DisallowedRegexInArrayValidator into existing
      ArrayOfStringsValidator
      * other minor changes
      
      Rename params to match frontend code
      
      Feedback from review
      
      Feedback from review
      5c168885
    • Sarah Yasonik's avatar
      Add service to fetch and format a grafana embed · 3857ab87
      Sarah Yasonik authored
      Adds a new Metrics Dashboard Service which fetches a grafana
      dashboard and datasource based on an input url, reformatting
      the output to be compatible with our existing dashboard
      rendering behavior on the FE. Ultimately, this will be used
      as a part of embedding prometheus metrics in GFM via grafana.
      
      Note that this service introduces usage of ReactiveCaching in
      the service as a result of external API calls. This will mean
      that usage of this new service will need to be polled.
      3857ab87
  3. 19 Oct, 2019 1 commit
  4. 18 Oct, 2019 6 commits
  5. 17 Oct, 2019 13 commits
  6. 16 Oct, 2019 13 commits
    • Winnie Hellmann's avatar
      1a18a942
    • Fabio Pitino's avatar
      Introduce new Ansi2json parser · 1bbe1605
      Fabio Pitino authored
      This introduces a new class Ansi2json which
      similarly to Ansi2html converts job logs
      to a format that can be parsed by frontend.
      1bbe1605
    • Patrick Bajao's avatar
      Fix showing diff when it has legacy diff notes · 6a79e237
      Patrick Bajao authored
      Do not try to unfold `LegacyDiffNote`s as they don't support
      `position` even though they're in the same `notes` table as
      `DiffNote`s.
      
      This is done by using `Gitlab::Diff::PositionCollection` which
      will be responsible for filtering the positions.
      6a79e237
    • Mario de la Ossa's avatar
      Add CI job for checking stale GraphQL docs · d7ab6d21
      Mario de la Ossa authored
      Adds a new job, `graphql-docs-verify`, that fails if the autogenerated
      graphql reference docs are not up to date.
      d7ab6d21
    • Tiger Watson's avatar
      Add AWS cluster provider · 8568e729
      Tiger Watson authored
      Similar to the existing GCP cluster data model, introduce
      models to represent an AWS user role, and an EKS cluster
      that has been created by GitLab.
      8568e729
    • Kamil Trzciński's avatar
      Introduce and use `refresh_state` · bb1b81c8
      Kamil Trzciński authored
      This updates metrics on demand,
      and tracks the current state as gauge.
      
      This also improves tests execution time.
      bb1b81c8
    • Qingyu Zhao's avatar
      Add Sidekiq memory killer prometheus metrics · 62fb215e
      Qingyu Zhao authored
      62fb215e
    • Yorick Peterse's avatar
      Add API for manually creating deployments · 77831f78
      Yorick Peterse authored
      This API can be used for manually creating deployments, instead of being
      limited to creating deployments using CI pipelines.
      
      As part of these changes, I refactored some parts of the existing
      deployments code. This ensures we reuse the same logic for creating
      deployments in different places. We also move the deployments service
      classes to a Deployments namespace, now that there are two service
      classes.
      
      We also make some changes to which deployments are displayed. Prior to
      these changes, only deployments that were successful were displayed.
      This can get very confusing when using deployments from external
      systems, so we now show deployments regardless of their status. The
      table to display deployments has also had some style/content changes to
      display the deployments data in a more meaningful way.
      77831f78
    • Kamil Trzciński's avatar
      Add `on_before_phased_restart` lifecycle event · 4514945a
      Kamil Trzciński authored
      Unicorn/Puma Cluster: This will be called before a graceful
      shutdown of workers starts hapepning.
      This is called on `master` process.
      4514945a
    • Igor's avatar
      Fix caching of submodule links for shas with dashes · 1281ac55
      Igor authored
      1281ac55
    • Kamil Trzciński's avatar
      Improve service level health-checks · 256a3072
      Kamil Trzciński authored
      This improves the behavior of health-checks
      introducing two levels of them:
      
      - service level: checks served by `sidekiq|web_exporter:'
        served on separate port
      
      - application level: checks served by `/-/metrics` endpoint
        of `HealthController`
      
      The changes makes the:
      
      - `/liveness` always indicate that application is ready
        and not deadlocked
      - `/readiness` to indicate:
      
      The changes for `/readiness`:
      
      - on service level: the status of the service if it can accept
        and process connections
      
      - on application level: the status of external components if they
        are accessible
      
      The `on service level` and `on application level` do not share
      any checks, they have different checks exposed.
      256a3072
    • Kamil Trzciński's avatar
      Make Daemon thread-safe · f5b1317a
      Kamil Trzciński authored
      This introduces `#run_thread` method.
      
      The `#start_working` is executed in lock context,
      the same as `#stop_working`. This allows safe
      initialisation of resources before `#run_thread`
      consumes them.
      f5b1317a
    • Kamil Trzciński's avatar
      Add unicorn/puma health checks · 03cba0fe
      Kamil Trzciński authored
      These checks do validate amount of workers
      available to process requests.
      
      It allows the readiness to indicate a time when
      the requests can be safelly send to the web server.
      03cba0fe