Multiple user popovers added when repeatedly hovering a user

On current master I'm seeing multiple user popups. If you mouseover/out a few times there are extra shadows:

Screen_Shot_2020-02-03_at_4.14.07_pm

And this in devtools:

Screen_Shot_2020-02-03_at_4.14.16_pm

I think related to !23904 (merged) - reverting 11680481 and it doesn't happen