Gem Sockets GUI
I think that the best way to implement gemstones properly, fixing a lot of issues, too, would be to implement a GUI that opens gem slots for an item.
Pros: 1 - you won't need an item to remove gems if you don't want to, just drag them away. 2 - you won't need to make a giant tooltip to view gemstones effects, you could just open the GUI to remember what you put in there. 3 - complete customizability. you will be able to change the gemstones GUI title to customize graphics
It's such a simple change, yet it would greatly improve the gemstones idea. A great example of what you could achieve, is something like crests in Silksong, simple but effective wat to implement power-ups for weapons.