How do I remove a linked directory?

I'm trying to remove my linked ImGUI directory from my gitlab, and I can't figure out why it doesn't work. I can't simply delete each file, because when pressing on the ImGUI folder, it refers to the ImGUI gitlab, which I don't have access to. So how can I close this link?