Screen/Display switching still works in Virtual Touchpad mode

Steps to reproduce

1. Connect a phone to an external display
2. Press Ctrl + Alt Up/Down

Expected results

Nothing should happen since there's only one display with a shell

Actual results

You can select between the external and internal displays and when you select the internal display, you switch to it and can't interact with the external display anymore