When you are troubleshooting a pipeline that is behaving unexpectedly you may want to look at all the jobs with a particular status. That would require looking at the full list of jobs or using the GitLab API to get all the data and manipulate it outside the application which is a bad workflow and takes you out of GitLab where you can fix the issue.
Now you can filter for Job Status directly on the Job Page so you can see all jobs with the same status in GitLab.
Problem to solve
Users are expecting the same experience as the Pipelines page, where they can filter or search pipelines by:
User/Trigger
Branch name
Tag name
Status
Source
the Jobs page has no search or filtering capability, making it difficult to search for occurrences of a specific job in case of troubleshooting.
User experience goal
Add Job Status filtering capabilities in the Jobs index page.
This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Category:Fuzz Testing
GitLab Ultimate
devops
application security testing
feature flag
frontend
fuzzing
coverage
group
dynamic analysis
missed:14.7
section
sec
type
feature
workflow
in dev
Category:Fuzz Testing
GitLab Ultimate
backend
devops
application security testing
direction
fuzzing
coverage
group
dynamic analysis
missed:14.3
missed:14.4
section
sec
type
feature
workflow
in dev
Category:Fuzz Testing
Deliverable
GitLab Ultimate
backend
devops
application security testing
direction
fuzzing
coverage
group
dynamic analysis
section
sec
type
feature
workflow
in dev
Category:Fuzz Testing
Deliverable
GitLab Ultimate
backend
devops
application security testing
direction
fuzzing
coverage
group
dynamic analysis
section
sec
type
feature
workflow
in dev
Category:Fuzz Testing
Deliverable
GitLab Ultimate
backend
devops
application security testing
direction
fuzzing
coverage
group
dynamic analysis
section
sec
type
feature
workflow
in dev
Please also add filtering by job status and runner name. It would greatly help to troubleshoot runner issues. Nice to have would be filtering by log content or job time.
Chiming in to say that trying to sleuth through thousands of jobs looking for patterns of failures is very difficult without filtering... Looking forward to this being an enhancement!
Internal customer would appreciate this feature as well ;-) GitLab Support often needs to review the latest job log for this internal project. But because only one's own job logs are relevant, that page is less useful than /pipeline?page=1&scope=finished which OTOH definitely requires 2 clicks to load the actually relevant job log. #22962 would alternatively help as well.
From another internal customer. As part of the devopssecure stage, we often need to debug or check for a subset of pipeline jobs, namely those relevant to the secure stage (where status = "failed" and job_name in ("sast", "secret_detection", "dependency_scanning", "dast", "container_scanning", "api_fuzzing")).
Thanks @jreporter.
I do find the description insufficient at the moment so I'll take out sometime to populate it referencing the Zendesk comments and the customer requirements before I work on this
Maybe for first iteration we introduce filter by status and pipeline id?
clarifying question from a lurker if yall don't mind: Since the above proposal in based on a filtered search component rather than just filters, what are the expectations for the "search" capability?
I'm hoping to eventually be able to search by job name but since that's not a part of the proposed filters maybe that's already a given for the "search" capability?
Backing up what Lucas just said now, also jumping in... I think that by pipelineId isn't quite what the biggest painpoint is, it's trying to troubleshoot a particular troublesome job, so having a name filter is what would be important.
Thanks for the feedback status and job name sound like a great first implementation. The intentions of this feature are to work just like the filtered search we have for pipelines but tailored to jobs
Ah I see that text search isn't actually supported on the pipeline page after all . I'm not sure where that sits in our %Backlog but looks like we are limited to only tokenized filters right now for equivalent functionality
Also need some backend feedback on this cc @cheryl.li
@jivanvl Actually do you have any insights into the backend implementation? I saw you provided some context on GraphQL requirements from the blocking issue #320942 (comment 526471011) for the jobs view refactor.
This looks something I can take of, just one thing searching the way Lucas suggests in #293862 (comment 672595376) is non trivial work, there's no prior work done in the rest of the codebase to use full text search and GraphQL together. So we'd have to stick to tokenized search for now
@pburdette@jheimbuck_gl I added a design to the issue. That said, do you think based on the comment from Lucas in #293862 (comment 672595376) we should open an issue for adding text support in the filters(both pipeline and job) and add it to the epic: &6690 ?
@bgadberry we are scoping this issue down to just filtering on job status. The customer may want to follow the epic for further iterations and if they are interested in participating in solution validation to look at those early let us know!
This does not fit the very first use-case I have for filtering → checking how often a given job succeed.
The second use-case is to check if the failure happens on specific workers. The proposal to filter on job status only does not fit that use-case either.
Thanks for the extra case @marmoute - feel free to open an issue that describes your use case. I think filtering by job name would work for your first case eg; show me all the jobs named JobName. For your second case you might consider the jobs API which includes the runner attribute today as a workaround.
cc @v_mishra for an interesting use case for this.
@marmoute thanks for sharing your thoughts. This is the very first iteration for the filter and we will continue building the capability as learn of more use-cases. As @jheimbuck_gl please open an issue and outline your use-case.
This does not fit the very first use-case I have for filtering → checking how often a given job succeed.
Just so we are clear, can you also mention the persona that you're sharing this concern as?(eg. Developer, DevOps engg, Team lead etc) It would help us design the feature better.
@v_mishra@pburdette I've updated the issue to focus this is only filtering by job status but the proposal and the user experience seem to be saying different things. Is this happening on the Jobs page or the Pipelines page?
@pburdette@jivanvl@samdbeckham with the blockers for this done I think we can get an estimate on it for an upcoming milestone. I'm tentatively putting it into %15.0 but if you're unblocked and have capacity for it feel free to pull it forward
@jheimbuck_gl@jivanvl@samdbeckham I think %15.0 is perfect start time for this. That will give us time to rollout the feature flag in %14.10 that will make this feature possible.
@jheimbuck_gl nothing is needed from BE on this first iteration.
We have the ability in GraphQL to filter by statuses as an argument. I plan to take some time before %15.0 to review the implementation. It's been a while since I did the work on the pipelines filtered search, so I need to catch myself up.
Speaking as a developer for an Ultimate licensed instance installed at Lockheed:
I have a pipeline that runs nightly to build a simulation, then run a number of scenarios for pass/fail performance criteria. At times, we'll need to make a change to one of the models that will have a cascading effect on the performance tests. We then need examine each test and determine if parameters need to be adjusted or if there was a compatibility issue. Due to the size of the overall system, it may be weeks before we get a chance to examine a failing scenario.
At this point, I would want to filter by Job Name (a specific scenario) and Pass/Fail status so that I could find the last pipeline that the particular job was successful. That's what I need. For now, we run a python script that uses the API, but it would be far better to have it directly on the page.
The other most useful feature would be to list all manual jobs and see when they were executed. Honestly, I was surprised that manual jobs didn't have their own sidebar menu.
Without a filter, I'm not really sure what purpose the Jobs window is even supposed to fulfill. We have so many jobs in a pipeline that a single pipeline fills the first page. So, is it supposed to be a flat-view of a pipeline? But, I can't easily tell if I'm seeing a full, partial, or multiple pipelines. A list is an incomplete feature without sort and filter.
@pburdette
Any ideas if the ability to filter the Jobs by stage name is added? Would be great to have that ability - helps in knowing deployments done to specific stages.
@pburdette If you want to add some of the issues listed in future work on the epic it'd be great. I left a comment for Nikhil there about what other use cases they have we could solve. Search by name is likely the last one we'd implement if i remember our discussion correctly about performance of that search.
cc @v_mishra who may have insights from research to influence order of implementation.
The addition of a filter for status to the jobs page was wonderful & very useful, but are there more filters coming? If so, when & for what? @pburdette
Some beneficial choices for them:
Stage
Name
tag(s)
Start Date, End Date
Duration
Creator
Also, with the addition of the status filter, there's no way for them to do a raw text search.
Is there no support (besides the API) for filtering Jobs by which Runner performed the work? I am looking for an easy search token to be able to name a runner, or a tag associated with the runner.