MWPS did not merge even though pipeline succeeded

Description

Noticed on this MR !32149 (merged). "Merge when pipeline succeed" was set but it no merge happened for a while after the pipeline succeeded

Screen_Shot_2020-05-15_at_1.03.31_PM

I've seen this happen when a thread is opened when it's about to merge, but that doesn't seem to be the case here.