pointer reappear on bottom right corner on sway with multiple monitors

I have sway running on archlinux with two monitors (one on top of the other).

When the cursor hides nothing happens but when I move the mouse it reappears on the bottom right corner, making me switch focus between windows, which is really annoying.

I had to disable hideaway for now.