Get all projects (including subgroups) for a given namespace

@Spenhouet Thanks for the proposal.

I don't think it's possible to list this with one call right now. It is possible to determine the subgroups beneath a group and query each individual subgroup for it's own projects.

https://gitlab.com/api/v4/groups/issue-reproduce%2Fsub-group/subgroups
https://gitlab.com/api/v4/groups/issue-reproduce%2Fsub-group%2Fsub-sub-group/projects

added Manage [DEPRECATED] api ~2975006 project labels

Are you proposing to allow the existing /projects endpoint to list projects in all subgroups if a particular parameter is passed?

added awaiting feedback label

@markglenfletcher I guess that would be one option. I'm not specifically proposing which solution would be best in the gitlab API since I don't know the internals.

/api/v4/projects/ currently only works for a namespace/project query where a project is mandatory. In general a /api/v4/projects/:namespace endpoint would be intuitive. The question is if this endpoint would then return all projects including the projects of subgroups and so on or if for that a special flag would have to be set.

Another solution could be to add such a flag to /api/v4/groups/Group1%2FSubgroup1/projects and instead of just returning the direct child projects to also return all projects of all depths.

What would be the most logical implementation API wise in your opinion?

@Spenhouet Thanks for the suggestions

@winh Please can you take a look here?

removed awaiting feedback label

changed the description

@Spenhouet Thank you for creating this issue! I think it is a duplicate of https://gitlab.com/gitlab-org/gitlab-ce/issues/38071:

I expect GET /api/v4/groups/parent_group/projects doesn't return projects in subgroups when it should, either.

Would you agree?

added awaiting feedback label

@winh I'm not sure. I have a hard time understanding the issue Nick Thomas describes. I think the issue he describes is unclear. He talks about subgroups and not projects and doesn't give any specific example on how a example structure looks like, what he did and what result he expected. Based on this unclear and example free issue I don't expect his issue to ever get picked up by anyone. Because of that I would be sad if the issue I describe would be tracked with https://gitlab.com/gitlab-org/gitlab-ce/issues/38071.

If you see it as a duplicate I would propose to rewrite the description and title of the other issue or I believe it will never get solved.

marked this issue as related to #38071 (moved)

@Spenhouet I understand your concerns. Let's keep both issues for the time being and see if the solution to https://gitlab.com/gitlab-org/gitlab-ce/issues/38071 will resolve this issue, too.

@winh Okay, thanks.

mentioned in issue #58136 (closed)

Hi @Spenhouet,

First of all, thank you for raising an issue to help improve the GitLab product. We're sorry about this, but this particular issue has gone unnoticed for quite some time.

We're posting this message because this issue meets the following criteria:

No activity in the past 6 months (since 2018-08-20T09:32:42.594Z)
No milestone (unscheduled)

We'd like to ask you to help us out and determine how we should act on this issue.

If this issue is reporting a bug, please can you attempt to reproduce on the latest version of GitLab or GitLab.com, to help us to understand whether the bug still needs our attention.

If this issue is proposing a new feature, please can you verify whether the feature proposal is still relevant.

Please mention the Triage team (@gitlab-org/triage) for help with applying labels

Thanks for your help!

You are welcome to help improve this comment.

added auto updated label

Judging from https://gitlab.com/api/v4/groups/reproduction-group%2Fsubgroup/projects / https://gitlab.com/reproduction-group/subgroup, this still seems to be relevant.

@jeremy Would you mind adding a milestone here?

removed awaiting feedback label

My problem does not fit exactly to the description, but title describes what I want. Today I created Sentry→Gitlab integration but Sentry does not list all projects within configured group. I thought it's Sentry issue but I tested in their source code what resource is called and called /api/v4/groups/<my-group>/projects by hand and the result was the same as in Sentry:

I don't get all available projects
I get projects from totally different namespace

Gitlab 11.8.2 CE.

Hello Guys!

I've found this thread after some time, because I haven't found on gitlab API documentation this functionality of listing subgroups of groups, and so on.

Since I haven't found on official documentation, I'd like to ask if I could migrate my python code to ruby and submit a PR.

I do believe it's an important functionality because I had one issue last week, i needed to update all my projects under groups and subgroups, but I'll need to do it one by one, but if we had this possibility to get all projects under one specific group we could implement those settings updates for all projects easily.

Cheers peeps.

changed milestone to %Backlog

added backend label

added [deprecated] Accepting merge requests label

added devopsmanage label

added groupoptimize label

mentioned in merge request gitlab-org/quality/triage-ops!268 (closed)

added 1 deleted label and removed devopsmanage groupoptimize labels

Setting devopsmanage based on Manage [DEPRECATED].

added devopsmanage label

There is now include_subgroups=true in https://docs.gitlab.com/ee/api/groups.html#list-a-groups-projects which resolves this issue.

It was added in 11.6: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22612

closed

can someone pls confirm if setting include_subgroups=true will get projects of direct subgroups of selected group and subgroups of subgroups (the entire subgroups tree is traversed)?

This include_subgroups=true is working perfectly! Thanks everybody <3

Can i get the Full URL

@MMallik5 Try this: https://gitlab.com/api/v4/groups/:groupid/projects/?include_subgroups=true

thanks, but in that we are only getting 20 projects per page, we need every project under the sub-groups, and also we need focus if that sub-group may have another sub-group, we should get those under project list also with covering all there tree-structured. group->sub-group->sub-group->projects (FYI tree structure may like this)

Hello @madhu.am321, I believe the 20 results per page can be remedied using the pagination links provided by the API response (look for the 'link' key in the response header and use the 'next' URL to pull the next set of response if they exist). I believe for groups there is a reference to the parent id and if there is something similar for projects you could use this to build a tree with some scripting work. Working on something myself at the moment and glad I stumbled upon this issue. Thanks all for the discourse!

Because it's not working for me

This script with generate a csv list of projects in the GROUP_PATH, including the subgroups, you can access. It uses pagination to extract the full list.

extracting the following data: Project Name, Project Path, Visibility, Archived, Creation Date

#!/bin/bash 

# Replace ${GITLAB_PRIVATE_TOKEN} with your GitLab access token or `export GITLAB_PRIVATE_TOKEN="your token"` before running
# Replace ${GROUP_PATH} with the path of the top-level group (e.g., "your-group" or "your-group/sub-group") or `export GROUP_PATH="path"` before running

base_url="https://gitlab.com/api/v4/groups/${GROUP_PATH}/projects"
page=1
per_page=100

echo "Project Name, Project Path, Visibility, Archived, Creation Date"

while true; do
    response=$(curl -s --header "PRIVATE-TOKEN: ${GITLAB_PRIVATE_TOKEN}" "${base_url}?include_subgroups=true&page=${page}&per_page=${per_page}")
    projects=$(echo "$response" | jq -r '.[] | [.name, .path_with_namespace, .visibility, .archived, .created_at] | @tsv')

    if [ -z "$projects" ]; then
        # No more projects, break out of the loop
        break
    fi

    # Fetch user details for each project
    while IFS=$'\t' read -r name path visibility archived created_at created_by; do
        user_response=$(curl -s --header "PRIVATE-TOKEN: <YOUR_PRIVATE_TOKEN>" "https://gitlab.com/api/v4/users/${created_by}")
        user_name=$(echo "$user_response" | jq -r '.name // empty')

	echo "$name, $path, $visibility, $archived, $created_at"
    done <<< "$projects"

    page=$((page + 1))
done

@comeh

Nice. I am trying to figure out a way to list all project/group access tokens which are part of my namespace and figure out which ones are about to expire. Looks like I can use your script as a base.

Just pushed it here in case it needs some more refinement in the future : https://gitlab.com/gitlab-tooling/gitlab-list-of-projects/-/blob/main/gitlab-api-list-projects.sh

feel free to add enhancements if need be !

mentioned in commit gitlab-tooling/gitlab-list-of-projects@8f5760d2

Get all projects (including subgroups) for a given namespace

Designs

Child items ...

Activity

Get all projects (including subgroups) for a given namespace

Relates to

Activity