During dragging we want to check both the place the mouse is now (during dragging) and the place the mouse was when the click happened.
The mouse down is actually way more important than the location of the motion event so that should go first.
Example of problem: