To quickly remove a label in the sidebar, show on hover an × in the label bubble to remove the label.
Something like this:
