GitLab issueshttps://gitlab.com/gitlab-org/gitlab/-/issues2024-03-27T18:19:22Zhttps://gitlab.com/gitlab-org/gitlab/-/issues/372054Measuring SPACE framework metrics in GitLab2024-03-27T18:19:22ZHaim SnirMeasuring SPACE framework metrics in GitLab<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.-->
### Overview
The [SPACE framework](https://queue.acm.org/detail.cfm?id=3454124) presents five categories to measuring dev...<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.-->
### Overview
The [SPACE framework](https://queue.acm.org/detail.cfm?id=3454124) presents five categories to measuring developer productivity:
**S – Satisfaction & Well Being\
P – Performance\
A – Activity\
C – Collaboration & Communication\
E – Efficiency & Flow**
### Proposal
1. Adding [SPACE framework](https://queue.acm.org/detail.cfm?id=3454124) metrics into to value-stream mapping.
2. Add SPACE Metrics report to Insights.
#### SPACE metrics mapping (WIP)
- Satisfaction
- number of code reviews per developer
- Performance
- Code-review velocity - both an individual- and a team-level metric
- Activity
- how many reviews have been completed in a given time frame
- number of commits, coding time (total time spent or times of day), and number of code reviews completed.
- Communication and collaboration
- Code reviews
- PR Approvers & Commenters
- Efficiency and flow
- Pull Request Response Time
- Code Review Cycle Time
- Cycle Time by Stage
- Work Patterns
### how DORA and SPACE can be used together?
![Screenshot_2024-02-25_at_14.40.01](/uploads/c3db9f3eb61d0b293ccedacefdd3c976/Screenshot_2024-02-25_at_14.40.01.png)
https://www.youtube.com/watch?v=Hydf67rDUrY
### Open questions
1. Clarify SPACE & DORA in the context of value-stream
### Validation
#### Competition
- faros-ai: View well known engineering metrics such as DORA and SPACE --> https://github.com/faros-ai/faros-community-edition#-features
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
<!-- Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
-->
<!-- Label reminders
Use the following resources to find the appropriate labels:
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/371982Horizontal scrollbar is not easily accessible or visible on Boards2022-12-07T13:19:50ZGabe Weavergweaver@gitlab.comHorizontal scrollbar is not easily accessible or visible on Boards## Problem
In the words of a customer:
> Got a +1 vote on this from an internal user today who was also talking about how the horizontal scroll bar on a board is not visible unless you scroll vertically to the bottom, which is just a d...## Problem
In the words of a customer:
> Got a +1 vote on this from an internal user today who was also talking about how the horizontal scroll bar on a board is not visible unless you scroll vertically to the bottom, which is just a daily irritant.
## Proposal
Make the horizontal scrollbar more easily accessible/visible within Boards.15.6Simon KnoxSimon Knoxhttps://gitlab.com/gitlab-org/gitlab/-/issues/370887MVC design for the new Exec Dashboards page with metrics comparative view2023-05-15T13:49:33ZHaim SnirMVC design for the new Exec Dashboards page with metrics comparative view## Overview
Following the feedback from https://gitlab.com/gitlab-org/gitlab/-/issues/353651+ and https://gitlab.com/gitlab-org/gitlab/-/issues/354653+ here is issue to discuss the Proposal for customizable Exec Dashboards.
Here is the...## Overview
Following the feedback from https://gitlab.com/gitlab-org/gitlab/-/issues/353651+ and https://gitlab.com/gitlab-org/gitlab/-/issues/354653+ here is issue to discuss the Proposal for customizable Exec Dashboards.
Here is the [dogfooding POC for this MVC](https://gitlab.com/groups/gitlab-org/-/analytics/dashboards), and here are the details about [how to join the Exec Dashboard Beta](https://gitlab.com/groups/gitlab-org/-/epics/9182).
### Release notes
### Problem to solve
As a software Exec working to optimize my SDLC lifecycle, I need centralized UI where I can see and compare all of the developer productivity metrics. It is also important for me to understand whether projects and/or group(s) are improving or not.
Using the current UI user can't:
1. Tell the story whether their metrics are improving month over month.
2. Understand patterns in their metrics trend overtime.
3. Compare performance between groups, projects and teams.
4. Ability to view reports via a saved filter, allowing everyone to see the same metrics in the same view.
### Intended users
1. [Erin (Application Development Executive)](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/buyer-persona/#erin---the-application-development-executive-vp-etc)
2. [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director)
3. [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/product/personas/#delaney-development-team-lead)
### User experience goal
1. **High-level custom view** of multiple DevOps and Business metrics.
2. For each metric to show the current time period **compared** to previous time period.
3. For each metric to show the **change rate** % from previous time period, with badge indication.
4. For each metric to show the **historical trend**.
5. Each metric need to **filters** for different combinations of groups, projects and environments.
6. Name and description can be customized.
7. The view need to be **shared as a single link** for everyone to see the same metrics in the same context.
8. Each metric will have a default **drill-down link** for further investigation on other analytics reports. This link can be customized by users.
### Proposal
<details>
<summary>Click to expand the proposal details:</summary>
1. Add a new page with "Metrics comparative table".
2. Page name - "Executive Dashboard", path /analytics/dashboards
3. "Metrics comparative table" columns headers:
| Name | MTD | Period 1 | Period 2 | Past 90 days | Description |
|------|-----|----------|----------|--------------|-------------|
| Deployment Frequency for project A | 2.6/day (+1.5%) | 2.6/day (+1.5%) | 2.6/day (+1.5%) | Sparkline | DF for project A |
| Deployment Frequency for project B | 2.6/day (+1.5%) | 2.6/day (+1.5%) | 2.6/day (+1.5%) | Sparkline | DF for project B |
| Lead Time for Changes for project A | 0.5 days (-3%) | 0.5 days (-3%) | 0.5 days (-3%) | Sparkline | LTFC for project A |
| Lead Time for Changes for project B | 0.5 days (-3%) | 0.5 days (-3%) | 0.5 days (-3%) | Sparkline | LTFC for project B |
- MTD - current month-to-date, period starting at the beginning of the current calendar month and ending at the current date. (e.g. `2022-09-01 --> 2022-09-15`)
- `period 1` - last completed calendar month (e.g. `2022-08-01 --> 2022-08-31`)
- `period 2` - previous completed calendar month (e.g. `2022-07-01 --> 2022-07-31`)
- `change_rate\_%` - `period 2` / `period 1` .
- `Past 90 days` - Sparkline to show 90 days history of the data trends.
5. Each value need to be displayed with `value_unit` and `change_rate\_%`. For example: Deployment Frequency - "2.6/day (+1.5%)" \* Lead Time for Changes - "0.5 days (-3%)" \* Time to restore service - "1.4 days (-0.9%)" \* Change Failure Rate - "3% (+0.2%")
6. Allow users to add projects, groups, and subgroups to the view.
7. Allow users to filter specific metric by projects, groups, and subgroups.
8. Each metrics should have a drill-down link into other Gitlab analytics reports.
9. The table can be customized via yaml or json.
</details>
#### Suggested design:
![MVC](/uploads/4c49eb66059500898ecae5cbac5d2401/MVC.jpg)
### Acceptance criteria
1. Compare all 4 DORA metrics MoM `value` and `change rate %` .
2. Compare DORA Deployment Frequency `value` and `change rate %` across 2 different projects.
### Productivity metrics examples:
1. DORA deployment_frequency
2. DORA lead_time_for_changes
3. DORA change_failure_rate
4. DORA time_to_restore_service
5. VSA lead time
6. VSA Cycle time
7. VSA New issues
8. VSA Deploys
### Open questions
1. Can we POC the new [Product analytics dashboards and visualizations](https://gitlab.com/groups/gitlab-org/-/epics/8466 "Product analytics first sets of dashboards and visualizations")?
2. Can we leverage the [Operations Dashboard](https://gitlab.com/-/operations) cards for the Exec Dashboards?
3. Build this in [Grafana](https://gitlab.com/gitlab-org/gitlab/-/issues/353651#note_1078607572 "Customer Executive Dashboards [formerly called Customizable analytics dashboard]")?Next 1-3 releaseshttps://gitlab.com/gitlab-org/gitlab/-/issues/370459gitlab 12.1 ci editor and view page syntax indent error2023-12-11T03:03:45Z하명진gitlab 12.1 ci editor and view page syntax indent errormy gitlab version 15.2.1-ee self hosted
12.2 version upgrade after gitlab ci editor and lint " - | " broken ci lint
![image](/uploads/57e90140a4e393665f085c0b7558770a/image.png)
edit page is ok only read page display error syn...my gitlab version 15.2.1-ee self hosted
12.2 version upgrade after gitlab ci editor and lint " - | " broken ci lint
![image](/uploads/57e90140a4e393665f085c0b7558770a/image.png)
edit page is ok only read page display error syntax
![image](/uploads/55566a81f4fc7291beed146b2ef9523e/image.png)
indent display error
view page
![image](/uploads/1965404687cc59f14cead31025359016/image.png)
edit page
![image](/uploads/68274098d6225ca6cde2cfd225cac9c4/image.png)Awaiting further demandhttps://gitlab.com/gitlab-org/gitlab/-/issues/371427Setting up slash commands with external Mattermost server2023-05-12T11:37:20ZGhost UserSetting up slash commands with external Mattermost serverWhen using the GitLab omnibus docker package you can not configure Mattermost Slash commands with an external Mattermost Instance. It seems to be expected to use the integrated Mattermost instance. This might not be desirable if users al...When using the GitLab omnibus docker package you can not configure Mattermost Slash commands with an external Mattermost Instance. It seems to be expected to use the integrated Mattermost instance. This might not be desirable if users already have a Mattermost server, want more control over the server (using newer versions) and for seperation of intrests.
See the following Screenshot:
![Screenshot](https://i.imgur.com/nTwNfly.png)
The left side shows Gitlab.com (Gitlab SAS), the right side a fresh self hosted GitLab instance via Docker.
Is there a option to use an external Mattermost Instance?Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/370046services and "docker run" side by side2022-08-17T02:04:58ZRainer Hahnekampservices and "docker run" side by sideI need to use `docker run...` inside my CI jobs. I've followed the instructions on https://docs.gitlab.com/ee/ci/services/#using-services-with-docker-run-docker-in-docker-side-by-side. Unfortunately I was not successful.
I am getting th...I need to use `docker run...` inside my CI jobs. I've followed the instructions on https://docs.gitlab.com/ee/ci/services/#using-services-with-docker-run-docker-in-docker-side-by-side. Unfortunately I was not successful.
I am getting the error message:
```bash
curl: (6) Could not resolve host: tutum-wordpress
```
I am running GitLab 15.1 with docker-compose. My .gitlab-ci.yml:
```yaml
stages:
- docker
cluster:
stage: docker
services:
- docker:dind
- tutum/wordpress:latest
image: docker:20.10.16
variables:
FF_NETWORK_PER_BUILD: "true"
script:
- docker run --rm --network=host curlimages/curl:7.74.0 curl http://tutum-wordpress
```
config.toml
```toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "runner-1"
url = "ci-url"
token = "some-token"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "docker:dind"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache:/cache", "/builds:/builds"]
shm_size = 0
```
I am adding the issue here, because I've been having with this problem for several months and also didn't get any help in the GitLab community forum.
Thank you very much for your help.https://gitlab.com/gitlab-org/gitlab/-/issues/369444Recognize MulanPSL-2.0 license in a project2022-08-29T04:45:10ZKun Qianqiankunfaith@gmail.comRecognize MulanPSL-2.0 license in a project## Description
Even when the project has `MulanPSL-2.0` license, it will be recognized as "other" rather than its name.
![image](/uploads/bb80cdba383c1e29d546e41b8aeb57f7/image.png)
I see GitLab is using `Licensee` gem 9.14.1 (https:...## Description
Even when the project has `MulanPSL-2.0` license, it will be recognized as "other" rather than its name.
![image](/uploads/bb80cdba383c1e29d546e41b8aeb57f7/image.png)
I see GitLab is using `Licensee` gem 9.14.1 (https://gitlab.com/gitlab-org/gitlab/-/blob/20d7bd3fcef35c4808caf8d6900973ca597d0cde/Gemfile#L275), and this gem has included `MulanPSL-2.0` license since 9.15.2 (see its [changelog](https://github.com/licensee/licensee/releases/tag/v9.15.2)), shall we upgrade this gem to recognize `MulanPSL-2.0` license?
## Goal
1. ~~If copy the text of `MulanPSL-2.0` manually and create the `LICENSE` file on root path, it can be recognized correctly rather than `others` **in all plans**~~ ( **Update: it works now** )
Here is another example for `Mozilla Public License 2.0`, I just copied its text to `LICENSE` on root path, it's correctly recognized:
![image](https://gitlab.com/gitlab-org/gitlab/uploads/73469f121badc44281f147f57fb4f551/image.png)
## References
As relevant to this feature request, there is yet another feature:
- #37204115.4Kun Qianqiankunfaith@gmail.comQian Zhang (Gary)Kun Qianqiankunfaith@gmail.comhttps://gitlab.com/gitlab-org/gitlab/-/issues/368554Gitlab rake not working and preventing gitlab to upgrade2023-04-24T00:52:42ZTierra ComúnGitlab rake not working and preventing gitlab to upgrade## Description
When upgrading to a minor version (14.8.6), coming from 14.8.3 version, on my way to upgrade to a major one (14.9.0), the gitlab-ctl reconfigure got stuck on action run step, and sent the next error:
````
Running handlers...## Description
When upgrading to a minor version (14.8.6), coming from 14.8.3 version, on my way to upgrade to a major one (14.9.0), the gitlab-ctl reconfigure got stuck on action run step, and sent the next error:
````
Running handlers:
There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
````
Searching how to solve this problem, I found this [gitlab issue](https://gitlab.com/gitlab-org/gitlab/-/issues/360377#note_928527713) where its proposed to make a gitlab backup manually using various rake tasks.
But when trying to use these tasks -almost every rake task- this error message is sent:
````
rake aborted!
NameError: uninitialized constant API::Clusters::Cluster
/opt/gitlab/embedded/service/gitlab-rails/lib/api/metrics/dashboard/annotations.rb:15:in `<class:Annotations>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/metrics/dashboard/annotations.rb:6:in `<module:Dashboard>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/metrics/dashboard/annotations.rb:5:in `<module:Metrics>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/metrics/dashboard/annotations.rb:4:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/metrics/dashboard/annotations.rb:3:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:224:in `block in <class:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:151:in `<class:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:4:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:3:in `<top (required)>'
(eval):6:in `draw_route'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/patch/draw_route.rb:30:in `instance_eval'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/patch/draw_route.rb:30:in `draw_route'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/patch/draw_route.rb:17:in `draw_ce'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/patch/draw_route.rb:11:in `draw'
/opt/gitlab/embedded/service/gitlab-rails/config/routes.rb:272:in `block in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/routes.rb:7:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:7:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:version => db:load_config => environment
````
So, it looks more a rake issue than db issue.
Any idea on how can I fix this??
It has been very difficult for me to debug as I have no ruby skills.
Thanks on advance!
## Gitlab version:
14.8.3
## System:
Debian 9.13
## Steps to reproduce
Updating from version 14.8.3 to 14.9.0, failing on the gitlab-ctl reconfigure, and then, trying to use any rake task.
## Current behavior:
Failed upgrade and unable to use any rake task.
## Expected behavior:
Enable the use of rake tasks to go forward with the gitlab upgrade.https://gitlab.com/gitlab-org/gitlab/-/issues/368312Design: Explore removing the icons from the incident timeline view2022-07-26T01:02:16ZAmelia BauerlyDesign: Explore removing the icons from the incident timeline view<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated us...<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab.
The goal of this template is brevity for quick/smaller iterations. For a more thorough list of considerations for larger features or feature sets, you can leverage the detailed [feature proposal](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md). -->
### Release notes
<!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " -->
### Problem to solve
<!-- What is the user problem you are trying to solve with this issue? -->
Jeremy reviewed incident timelines and suggested we could potentially remove the icons from the timeline view:
> I'm guessing you explored something similar, but was thinking along the lines of no icon to help simplify the layout.
![image](https://gitlab.com/gitlab-org/gitlab/uploads/62e331569af2d5661d28e32972f2b29e/image.png)
When we did user testing, people thought the additional information the icons added was helpful, and we are thinking of introducing filters to the timeline view, which might make the icons additionally useful. However, at present the icons aren't super informative, as we only have two types. We could consider improving the icon use or perhaps removing them altogether?
### Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
### Intended users
<!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
* [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager)
* [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager)
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
* [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer)
* [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator)
* [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
### Feature Usage Metrics
<!-- How are you going to track usage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
-->
<!-- Label reminders
Use the following resources to find the appropriate labels:
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/368244Jobs stuck in `after_script` and timeout when `script` spawns a background pr...2024-03-28T21:19:35ZKonstantin SpasovJobs stuck in `after_script` and timeout when `script` spawns a background process with k8s executor<!-- This template is used as a starting point for understanding and articulating a customer problem.
Learn more about it in the handbook: https://about.gitlab.com/handbook/product-development-flow/#validation-phase-2-problem-validation ...<!-- This template is used as a starting point for understanding and articulating a customer problem.
Learn more about it in the handbook: https://about.gitlab.com/handbook/product-development-flow/#validation-phase-2-problem-validation
-->
## Problem Statement
<!-- What is the problem we hope to validate? Reference how to write a real customer problem statement at https://productcoalition.com/how-to-write-a-good-customer-problem-statement-a815f80189ba for guidance. -->
We have a job that has script and after_script sections. When the script section finishes and the after_script section is supposed to be executed nothing happens. It times out after 5 minutes resulting in failed job. Even the simple commands like echo don't get executed. Once it times out it is marked as failure.
## Reach
<!-- Please describe who suffers from this problem. Consider referring to our personas, which are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ -->
In certain scenario section of the job is never executed. This is inconvenient and is unexpected behavior.
<!-- Please also quantify the problem's reach using the following values, considering an aggregate across GitLab.com and self-managed:
10.0 = Impacts the vast majority (~80% or greater) of our users, prospects, or customers.
6.0 = Impacts a large percentage (~50% to ~80%) of the above.
3.0 = Significant reach (~25% to ~50%).
1.5 = Small reach (~5% to ~25%).
0.5 = Minimal reach (Less than ~5%). -->
## Impact
<!-- How do we positively impact the users above and GitLab's business by solving this problem? Please describe briefly, and provide a numerical assessment:
3.0 = Massive impact
2.0 = High impact
1.0 = Medium impact
0.5 = Low impact
0.25 = Minimal impact -->
Bug fix.
## Confidence
<!-- How do we know this is a problem? Please provide and link to any supporting information (e.g. data, customer verbatims) and use this basis to provide a numerical assessment on our confidence level in this problem's severity:
100% = High confidence
80% = Medium confidence
50% = Low confidence -->
Support ticket - https://support.gitlab.com/hc/en-us/requests/301579
## Effort
<!-- How much effort do we think it will be to solve this problem? Please include all counterparts (Product, UX, Engineering, etc) in your assessment and quantify the number of person-months needed to dedicate to the effort.
For example, if the solution will take a product manager, designer, and engineer two weeks of effort - you may quantify this as 1.5 (based on 0.5 months x 3 people). -->
## Definition of Done
- [ ] The problem is well understood by the PM to have an understanding summarized in a RICE score
- [ ] The problem is well understood by the PM to decide if they want to move forward with this idea or drop it
- [ ] The problem is well described and detailed with necessary requirements for product design to understand the problem
- [ ] The problem is well described and detailed with necessary requirements for engineering to understand the problem
## Research Issue
<!-- Link to the Problem Validation Research issue that will be executed by the UX Researcher. https://gitlab.com/gitlab-org/ux-research/ -->Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/367617Sidekiq keeps getting restarted2023-01-18T17:33:51ZPatrick Bajaoebajao@gitlab.comSidekiq keeps getting restartedAs reported in https://gitlab.com/gitlab-org/gitlab/-/issues/229352#note_1024233214, since upgrading from 14.9 to 15.1.2, Sidekiq stopped performing on jobs.
From customer's report:
![image](/uploads/041148d42d2c25140498bf0a3e313e01/im...As reported in https://gitlab.com/gitlab-org/gitlab/-/issues/229352#note_1024233214, since upgrading from 14.9 to 15.1.2, Sidekiq stopped performing on jobs.
From customer's report:
![image](/uploads/041148d42d2c25140498bf0a3e313e01/image.png)
There are no jobs running, and 148 enqueued. Tailing it's log, I've noticed the daemon keeps getting restarted:
```
[2022-07-12 15:19:47] INFO WEBrick 1.6.1
{"severity":"INFO","time":"2022-07-12T18:19:47.576Z","class":"Gitlab::SidekiqDaemon::MemoryKiller","action":"start","pid":7558,"message":"Starting Gitlab::SidekiqDaemon::MemoryKiller Daemon","retry":0}
[2022-07-12 15:19:47] INFO ruby 2.7.5 (2021-11-24) [x86_64-linux]
{"severity":"WARN","time":"2022-07-12T18:19:47.594Z","class":"Gitlab::SidekiqDaemon::MemoryKiller","action":"stop","pid":7558,"message":"Stopping Gitlab::SidekiqDaemon::MemoryKiller Daemon","retry":0}
{"severity":"INFO","time":"2022-07-12T18:19:50.298Z","message":"A worker terminated, shutting down the cluster"}
{"severity":"INFO","time":"2022-07-12T18:19:57.583Z","message":"Starting cluster with 1 processes"}
{"severity":"INFO","time":"2022-07-12T18:19:57.600Z","message":"Starting metrics server on port 8082"}
```
![image](/uploads/eb79d25534948b47993fb797f7f8db32/image.png)
Other things in Gitlab also stopped working (pipelines, for instance). Looks like that, at least in my case, the problem is not restricted to merge requests being stuck...https://gitlab.com/gitlab-org/gitlab/-/issues/366870ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key val...2023-10-24T01:42:59ZBotActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_ci_pipeline_ar...https://sentry.gitlab.net/gitlab/gitlabcom/issues/3357560/?referrer=gitlab_plugin
```
PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_ci_pipeline_artifacts_on_pipeline_id_and_file_type"
DETAIL: Key (p...https://sentry.gitlab.net/gitlab/gitlabcom/issues/3357560/?referrer=gitlab_plugin
```
PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_ci_pipeline_artifacts_on_pipeline_id_and_file_type"
DETAIL: Key (pipeline_id, file_type)=(580077258, 1) already exists.
lib/gitlab/database/load_balancing/connection_proxy.rb:120:in `block in write_using_load_balancer'
connection.send(...)
lib/gitlab/database/load_balancing/load_balancer.rb:115:in `block in read_write'
yield connection
lib/gitlab/database/load_balancing/load_balancer.rb:191:in `retry_with_backoff'
return yield
lib/gitlab/database/load_balancing/load_balancer.rb:111:in `read_write'
retry_with_backoff do
lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `write_using_load_balancer'
@load_balancer.read_write do |connection|
...
(213 additional frame(s) were not displayed)
ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_ci_pipeline_artifacts_on_pipeline_id_and_file_type"
DETAIL: Key (pipeline_id, file_type)=(580077258, 1) already exists.
PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_ci_pipeline_artifacts_on_pipeline_id_and_file_type"
DETAIL: Key (pipeline_id, file_type)=(580077258, 1) already exists.
```
This error seems to start occurring about the time https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89950 reached production.
### Proposal
Adding an exclusive lock to the service from CoverageReportWorker.
create_or_replace_for_pipeline has the potential of creating long transactions since it does non-database related work inside the transaction, communicating with the object store for delete and upload. We've had this problem before on job artifacts.Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/366558The issue details sidebar doesn't render or is grayed out on safari 15.5 macOS2022-10-27T09:40:16ZChris OrtmanThe issue details sidebar doesn't render or is grayed out on safari 15.5 macOSThis happens when in the kanban board view. If I add an issue and then click on other issues instead of updating with the issue details I will just see a gray rectangle. I don't have this problem in chrome.
![image](/uploads/f83dcb0ec3b...This happens when in the kanban board view. If I add an issue and then click on other issues instead of updating with the issue details I will just see a gray rectangle. I don't have this problem in chrome.
![image](/uploads/f83dcb0ec3b2cf706b943407c7d584bd/image.png)Simon KnoxSimon Knoxhttps://gitlab.com/gitlab-org/gitlab/-/issues/366448Runner IP Address Partially Obscured in Self-Managed UI2024-03-12T00:07:02ZSteve SharerRunner IP Address Partially Obscured in Self-Managed UI<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://g...<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug
and verify the issue you're about to submit isn't a duplicate.
--->
### Summary
<!-- Summarize the bug encountered concisely. -->
Customer using a self-managed 14.10 instance notified me that they are unable to see the full IP address in the "IP" column within the Admin Area -> Overview -> Runners page. Only the first two octets are fully shown.
### Steps to reproduce
<!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. -->
1. Login to the self-managed instance with instance admin creds (note, instance must have runners registered)
2. Navigate to Admin Area -> Overview -> Runners
3. Observe the cut-off IP address octets similar to
### Example Project
<!-- If possible, please create an example project here on GitLab.com that exhibits the problematic
behavior, and link to it here in the bug report. If you are using an older version of GitLab, this
will also determine whether the bug is fixed in a more recent version. -->
### What is the current *bug* behavior?
<!-- Describe what actually happens. -->
Current UI cuts of IP address information that's critical to customer workflows. It occurs on a variety of screen sizes and resolutions.
### What is the expected *correct* behavior?
<!-- Describe what you should see instead. -->
The admin should be able to see the entire IP when reviewing their runner fleet
### Relevant logs and/or screenshots
<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code
as it's tough to read otherwise. -->
![Screen_Shot_2022-06-28_at_12.56.13_PM](/uploads/bec10b5865c92070f6f515b08b5cc3ba/Screen_Shot_2022-06-28_at_12.56.13_PM.png)
### Output of checks
<!-- If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com -->
#### Results of GitLab environment info
<!-- Input any relevant GitLab environment information if needed. -->
Occurs on 14.10 and 14.8.2
<details>
<summary>Expand for output related to GitLab environment info</summary>
<pre>
(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)
(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
</pre>
</details>
#### Results of GitLab application Check
<!-- Input any relevant GitLab application check information if needed. -->
<details>
<summary>Expand for output related to the GitLab application check</summary>
<pre>
(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:check SANITIZE=true`)
(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`)
(we will only investigate if the tests are passing)
</pre>
</details>
### Possible fixes
<!-- If you can, link to the line of code that might be responsible for the problem. -->Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/365527Ability to filter Analytics by Sub Groups2022-11-24T00:09:32ZBethany FooteAbility to filter Analytics by Sub Groups### Proposal
We have a monorepo and therefore have several teams (Sub Groups) contributing to a single project/repo. We would like to be able to filter GitLab Analytics by Sub Group, so that we can see the the data on a per team basis (...### Proposal
We have a monorepo and therefore have several teams (Sub Groups) contributing to a single project/repo. We would like to be able to filter GitLab Analytics by Sub Group, so that we can see the the data on a per team basis (currently we get results for all engineers, which isn't very useful).
Feature Example:
When a user goes to an Analytics page, such as Code Review, there would be the option to filter the data by Sub Group.
Related:
https://gitlab.com/gitlab-org/gitlab/-/issues/284611
https://gitlab.com/gitlab-org/gitlab/-/issues/353651Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/365212Terraform state search / pagination2024-03-01T00:18:21ZViktor Nagy (GitLab)vnagy@gitlab.comTerraform state search / pagination<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated us...<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab.
The goal of this template is brevity for quick/smaller iterations. For a more thorough list of considerations for larger features or feature sets, you can leverage the detailed [feature proposal](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md). -->
## Release notes
<!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " -->
## Problem to solve
As a development team lead, when I want to administer Terraform states owned by my projects, I want to easily find a state file by its name.
> I ran into this last week with our Terraform images project, which has >700 states (it creates a new one for every MR). Currently the list is paginated, but there's no way to change the default sort order or skip to later pages, so if the state you're looking for is near the end you have to click through every page to get to it. Maybe we could copy the search/order nav bar seen in lots of other pages
## Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
### Filter
Allow filtering for the name of the state file
### Improve pagination
Allow skipping to any page from any page using the pagination
### Questions
- Is an improved pagination a good enough solution? It seems to be much simpler to implement.
- Is the problem project scoped or are users more interested in searching for state files within a group?
## Intended users
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
## Feature Usage Metrics
<!-- How are you going to track usage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
-->
<!-- Label reminders
Use the following resources to find the appropriate labels:
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->Awaiting further demandhttps://gitlab.com/gitlab-org/gitlab/-/issues/364765Branch and Tags missing after upgradation from GItLab 13.6.7 to 15.0.12022-07-05T20:08:22ZManoharan MBranch and Tags missing after upgradation from GItLab 13.6.7 to 15.0.1```
GitLab update performed from 13.6.7 to 15.0.1
In the current version, repos are migrated from legacy storage to hashed storage.
```
**Post upgrade below are observed.**
- Latest commits are missing. Before previous update whatever ...```
GitLab update performed from 13.6.7 to 15.0.1
In the current version, repos are migrated from legacy storage to hashed storage.
```
**Post upgrade below are observed.**
- Latest commits are missing. Before previous update whatever updated is available.
- The branches created after previous upgrade are not available.
- Images uploaded in Issues are not available.
- Images uploaded in wiki are not available.
```
Updates performed in below manner to reach latest version
13.6.7 (Current Version)
13.9.2
13.12.15
14.0.12
14.5.0
14.9.0
14.10.4
15.0.1 (Latest Version)
```
During 14.5.0 to 14.5.9 upgrade we get error.
```
Error 500 After LDAP Login in GUI
gitlab_rails['auto_migrate'] = true
gitlab-ctl reconfigure
DB Migration Failed.
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
Caused by:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "batched_background_migration_job_transition_logs" does not exist
LINE 8: WHERE a.attrelid = '"batched_background_migration_job_trans...
Referred Link: https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
gitlab-rake db:migrate
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
Caused by:
PG::UndefinedTable: ERROR: relation "batched_background_migration_job_transition_logs" does not exist
LINE 8: WHERE a.attrelid = '"batched_background_migration_job_trans...
gitlab-ctl reconfigure
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds,id,'[["id"\, "stage_id"]\, ["id_convert_to_bigint"\, "stage_id_convert_to_bigint"]]']
Refer: https://gitlab.com/gitlab-org/gitlab/-/issues/353927
gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count'
gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining'
[root@cm etc]# gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count'
0
[root@cm etc]# gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining'
0
[root@cm etc]#
https://docs.gitlab.com/ee/update/index.html
gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining'
gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count'
https://docs.gitlab.com/ee/update/index.html
# Start the rails console
gitlab-rails c
# Execute the following in the rails console
scheduled_queue = Sidekiq::ScheduledSet.new
pending_job_classes = scheduled_queue.select { |job| job["class"] == "BackgroundMigrationWorker" }.map { |job| job["args"].first }.uniq
pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) }
[root@cm etc]# gitlab-rails c
--------------------------------------------------------------------------------
Ruby: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
GitLab: 14.9.0 (77d81d7c756) FOSS
GitLab Shell: 13.24.0
PostgreSQL: 12.7
------------------------------------------------------------[ booted in 19.22s ]
Loading production environment (Rails 6.1.4.6)
irb(main):001:0> scheduled_queue = Sidekiq::ScheduledSet.new
=> #<Sidekiq::ScheduledSet:0x00007fc2f903cdc0 @name="schedule", @_size=0>
irb(main):002:0> pending_job_classes = scheduled_queue.select { |job| job["class"] == "BackgroundMigrationWorker" }.map { |job| job["args"].first }.un
iq
=> []
irb(main):003:0> pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) }
=> []
irb(main):004:0>
=====================
# Start the rails console
gitlab-rails c
# Execute the following in the rails console
Gitlab::Database::BackgroundMigrationJob.pending.find_each do |job|
puts "Running pending job '#{job.class_name}' with arguments #{job.arguments}"
result = Gitlab::BackgroundMigration.perform(job.class_name, job.arguments)
puts "Result: #{result}"
end
[root@cm etc]# gitlab-rails c
--------------------------------------------------------------------------------
Ruby: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
GitLab: 14.9.0 (77d81d7c756) FOSS
GitLab Shell: 13.24.0
PostgreSQL: 12.7
------------------------------------------------------------[ booted in 19.12s ]
Loading production environment (Rails 6.1.4.6)
irb(main):001:1* Gitlab::Database::BackgroundMigrationJob.pending.find_each do |job|
irb(main):002:1* puts "Running pending job '#{job.class_name}' with arguments #{job.arguments}"
irb(main):003:1* result = Gitlab::BackgroundMigration.perform(job.class_name, job.arguments)
irb(main):004:1* puts "Result: #{result}"
irb(main):005:0> end
=> nil
irb(main):006:0>
gitlab-rake db:migrate
gitlab-psql
select job_class_name, table_name, column_name, job_arguments from batched_background_migrations where status <> 3;
[root@cm etc]# gitlab-psql
psql (12.7)
Type "help" for help.
gitlabhq_production=# select job_class_name, table_name, column_name, job_arguments from batched_background_migrations where status <> 3;
job_class_name | table_name | column_name | job_arguments
---------------------------------------+---------------------+-------------+--------------------------------------------------------------------------
----------
CopyColumnUsingBackgroundMigrationJob | ci_builds_metadata | id | [["build_id"], ["build_id_convert_to_bigint"]]
CopyColumnUsingBackgroundMigrationJob | push_event_payloads | event_id | [["event_id"], ["event_id_convert_to_bigint"]]
CopyColumnUsingBackgroundMigrationJob | ci_job_artifacts | id | [["id", "job_id"], ["id_convert_to_bigint", "job_id_convert_to_bigint"]]
CopyColumnUsingBackgroundMigrationJob | taggings | id | [["id", "taggable_id"], ["id_convert_to_bigint", "taggable_id_convert_to_
bigint"]]
CopyColumnUsingBackgroundMigrationJob | ci_stages | id | [["id"], ["id_convert_to_bigint"]]
CopyColumnUsingBackgroundMigrationJob | events | id | [["id"], ["id_convert_to_bigint"]]
CopyColumnUsingBackgroundMigrationJob | ci_builds | id | [["id", "stage_id"], ["id_convert_to_bigint", "stage_id_convert_to_bigint
"]]
(7 rows)
gitlabhq_production=#
gitlab-rake db:migrate:up VERSION=20211123135255
[root@cm etc]# gitlab-rake db:migrate:up VERSION=20211123135255
== 20211123135255 CreateBatchedBackgroundMigrationJobTransitionLogs: migrating
-- execute("CREATE TABLE batched_background_migration_job_transition_logs (\n id bigserial NOT NULL,\n batched_background_migration_job_id bigint NOT NULL,\n created_at timestamp with time zone NOT NULL,\n updated_at timestamp with time zone NOT NULL,\n previous_status smallint NOT NULL,\n next_status smallint NOT NULL,\n exception_class text,\n exception_message text,\n CONSTRAINT check_50e580811a CHECK ((char_length(exception_message) <= 1000)),\n CONSTRAINT check_76e202c37a CHECK ((char_length(exception_class) <= 100)),\n PRIMARY KEY (id, created_at)\n) PARTITION BY RANGE (created_at);\n\nCREATE INDEX i_batched_background_migration_job_transition_logs_on_job_id\n ON batched_background_migration_job_transition_logs USING btree (batched_background_migration_job_id);\n\nALTER TABLE batched_background_migration_job_transition_logs ADD CONSTRAINT fk_rails_b7523a175b\n FOREIGN KEY (batched_background_migration_job_id) REFERENCES batched_background_migration_jobs(id) ON DELETE CASCADE;\n")
-> 0.0028s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_000000")
-> 0.0004s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_000000 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM (MINVALUE) TO ('2022-06-01')\n")
-> 0.5548s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202206")
-> 0.0005s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202206 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-06-01') TO ('2022-07-01')\n")
-> 0.6750s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202207")
-> 0.0004s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202207 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-07-01') TO ('2022-08-01')\n")
-> 0.5426s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202208")
-> 0.0005s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202208 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-08-01') TO ('2022-09-01')\n")
-> 0.5090s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202209")
-> 0.0006s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202209 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-09-01') TO ('2022-10-01')\n")
-> 0.6168s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202210")
-> 0.0005s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202210 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-10-01') TO ('2022-11-01')\n")
-> 0.4915s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202211")
-> 0.0004s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202211 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-11-01') TO ('2022-12-01')\n")
-> 0.5423s
-- table_exists?("gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202212")
-> 0.0005s
-- execute("CREATE TABLE gitlab_partitions_dynamic.batched_background_migration_job_transition_logs_202212 PARTITION OF batched_background_migration_job_transition_logs\nFOR VALUES FROM ('2022-12-01') TO ('2023-01-01')\n")
-> 0.5915s
== 20211123135255 CreateBatchedBackgroundMigrationJobTransitionLogs: migrated (4.5314s)
[root@cm etc]#
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds,id,'[["id"\, "stage_id"]\, ["id_convert_to_bigint"\, "stage_id_convert_to_bigint"]]']
[root@cm etc]# gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds,id,'[["id"\, "stage_id"]\, ["id_convert_to_bigint"\, "stage_id_convert_to_bigint"]]']
Done.
[root@cm etc]#
gitlab-ctl reconfigure
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
[root@cm etc]# gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
Done.
[root@cm etc]#
gitlab-rake db:migrate:up VERSION=20211123135255
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds,id,'[["id"\, "stage_id"]\, ["id_convert_to_bigint"\, "stage_id_convert_to_bigint"]]']
gitlab.rb file modified as below
gitlab_rails['auto_migrate'] = false
gitlab-ctl reconfigure
gitlab-ctl restart
```
` gitlab-ctl pg-upgrade`
Upgrade logs and Postgresql steps attached.
*NOTE: *
Currently all data available under `/var/opt/gitlab/git-data/repositories` path.
In UI it referred `@hashed` path.
Read only repo state verified.
```
[root@cm backups]# gitlab-rails console
--------------------------------------------------------------------------------
Ruby: ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]
GitLab: 13.6.7 (4d1fc8bce72) FOSS
GitLab Shell: 13.13.1
PostgreSQL: 11.10
--------------------------------------------------------------------------------
Loading production environment (Rails 6.0.3.3)
irb(main):001:0> projects = Project.where(repository_read_only: true)
=> #<ActiveRecord::Relation [#<Project id:444 cosmoss/cosmoss-mason>>, #<Project id:446 oss/google-dcn>>, #<Project id:214 manoharan/test-automat...
irb(main):002:1* projects.each do |p|
irb(main):003:1* p.update!(repository_read_only:nil)
irb(main):004:0> end
=> [#<Project id:444 cosmoss/cosmoss-mason>>, #<Project id:446 oss/google-dcn>>, #<Project id:214 manoharan/test-automation>>, #<Project id:213 sakthi/point>>]
irb(main):005:0>
[root@cm home]# gitlab-rake gitlab:storage:migrate_to_hashed
Enqueuing migration of 46 projects in batches of 200. Done!
[root@cm home]#
```
`Executed below script to convert from Legacy to Hashed`
```
# Find all projects that GitLab thinks is in legacy storage
Project.without_storage_feature(:repository).find_each(batch_size: 10) do |p|
# Generate the hashed path
hash_path = Storage::Hashed.new(p).disk_path
storage = Gitlab.config.repositories.storages[p.repository.shard]
# Enable access
Gitlab::GitalyClient::StorageSettings.allow_disk_access do
# Put a wildcard on the end so we find .git, .wiki.git and .design.git
full_hashed_path = File.expand_path(File.join(storage.legacy_disk_path, hash_path + '*'))
# If any repos already exist in hashed storage, complete the migration
if(!Dir.glob(full_hashed_path).empty?)
puts "Fixing ID:#{p.id} #{p.full_name} - #{full_hashed_path}"
# Set the repo to read only if it isn't already
if(!p.repository_read_only)
p.set_repository_read_only!
end
p.storage_version = ::Project::HASHED_STORAGE_FEATURES[:repository]
p.reload_repository!
p.write_repository_config
p.track_project_repository
p.repository_read_only = false
p.save!(validate: false)
end
end
end
```
Logs attached.
[prod_postgresql_update_09062022.txt](/uploads/570d4a95c58f2957be3bb93baf053717/prod_postgresql_update_09062022.txt)
[4.gitlab_update_14.5.0_to_14.9.0_07062022.txt](/uploads/80db72d01d925041496816376d5c6ec5/4.gitlab_update_14.5.0_to_14.9.0_07062022.txt)https://gitlab.com/gitlab-org/gitlab/-/issues/364688Unable to use variable in include.ref2023-09-07T12:21:35ZThomas WongUnable to use variable in include.refGitlab build will throw an "configuration is invalid" error if variable defined in the gitlab-ci.yml is used in the include.ref section in the same yml file:
```
This GitLab CI configuration is invalid: Project XXX reference `` does not ...Gitlab build will throw an "configuration is invalid" error if variable defined in the gitlab-ci.yml is used in the include.ref section in the same yml file:
```
This GitLab CI configuration is invalid: Project XXX reference `` does not exist! Learn more
```https://gitlab.com/gitlab-org/gitlab/-/issues/364601Notify an organization of its top level groups when they are created on gitla...2024-02-12T18:39:48ZEric StraitNotify an organization of its top level groups when they are created on gitlab.comCustomer would be interested in receiving notification when a new top level group is created by an user in their organization. This would be identified by the users' enterprise owned email.
> ...[we] are interested in knowing if a new...Customer would be interested in receiving notification when a new top level group is created by an user in their organization. This would be identified by the users' enterprise owned email.
> ...[we] are interested in knowing if a new top level group is created on gitlab.com that is related to OURCOMPANY. Obviously it’s not always easy to tell, but for example if the person who creates the group has an email address ending in “@ourcompany.com” then it’s _probably_ related to us. Do you have any mechanism to alert us if someone creates a new group that looks OURCOMPANY-related? We have some other cloud providers doing this for us where they monitor new accounts/groups/etc and if they see it belonging to an OURCOMPANY person, they alert us via email so we can bring the new group into the fold and into our enterprise org. Is this something GitLab would be able to help us with?
### How it will work.
A user@ourcompany.com creates a new top level group. A notification would be sent to the GitLab admin@ourcompany.com that identifies the new group url and the user@ourcompany.com creator.
### Some related issues & epics:
[Problem Validation: How are non-invited users finding their team's namespace + groups/projects when they create new accounts?](https://gitlab.com/gitlab-org/gitlab/-/issues/340755)
[Group domain verification](https://gitlab.com/groups/gitlab-org/-/epics/5299)Backloghttps://gitlab.com/gitlab-org/gitlab/-/issues/364494Mistake version of GitLab (mentioned by prospect)2022-06-07T12:59:12ZMaëlys Savymsavy@gitlab.comMistake version of GitLab (mentioned by prospect)Hello there!
https://docs.gitlab.com/ee/update/index.html#upgrade-paths
A prospect mentioned that there is an error: Current version of gitlab is 15.0.1, and on that webpage there is 15.1.0 - there is an error, when they use that vers...Hello there!
https://docs.gitlab.com/ee/update/index.html#upgrade-paths
A prospect mentioned that there is an error: Current version of gitlab is 15.0.1, and on that webpage there is 15.1.0 - there is an error, when they use that version number in docker.
Thanks
(let me know if I need to add a specific label please :smile: )
<!-- Don't edit below this line -->Jason Youngjay@rambleon.orgJason Youngjay@rambleon.org