Window drifts when toggling to fullscreen and back

Apparently, SDL_SetWindowPosition sets the position with decorations and border, while SDL_GetWindowPosition returns the top left corner of the drawable area.