Timeout on signon 2 when changing server

I don't have a reliable way to reproduce it, but sometimes trying to hop from one server to another timeouts.

Loading screen get stuck on signon 2 for 10s before user is kicked back to main menu.

Log:

Trying to connect...
"challenge " received, sending connect request back to [2a01:4f8:251:191:c9:0:0:2]:26000
Got challenge response
Accepted
CL_Disconnect
CSQC unloaded
Unloading pack: dlcache/pushmoddm1.pk3
Unloading pack: dlcache/zzz_newhook_r2.pk3
Unloading pack: dlcache/zzz-xmodelsplayer_04.pk3
Unloading pack: dlcache/zzz-sounds-troll_v2.7.pk3
Unloading pack: dlcache/zzz-sounds-Jingle_v1.pk3
Unloading pack: dlcache/zzz-pack-lootcrates_06.pk3
Unloading pack: dlcache/zzz-pack-items_02.pk3
Unloading pack: dlcache/zzz-okmale_01.pk3
Unloading pack: dlcache/zzz-newsounds-v.8.2.pk3
Unloading pack: dlcache/zzz-mod-weapons-v.8.1.pk3
Unloading pack: dlcache/zzz-misc-v.8.1.pk3
Unloading pack: dlcache/csprogs-12122b5-jeff.pk3
Completed demo
Sending clc_disconnect
Disconnected
Client using an automatically assigned port
Client opened a socket on address 0.0.0.0:0
Client opened a socket on address [0:0:0:0:0:0:0:0]:0
Authenticated connection to [2a01:4f8:251:191:c9:0:0:2]:26000 has been established: server is cfTWCVMoaEXWtSvJyWhWp04abgwA4KVO9NJYhleDZzg=@Xon//Ks, I am ON60DZv@Xon//Ks
Connection accepted to [2a01:4f8:251:191:c9:0:0:2]:26000
Draw_Frame(15384): Unloading "gfx/menu/luma/border"
Draw_Frame(15384): Unloading "gfx/menu/luma/button_n"
Draw_Frame(15384): Unloading "gfx/menu/luma/scrollbar_s"
Draw_Frame(15384): Unloading "gfx/menu/luma/closebutton_n"
Draw_Frame(15384): Unloading "gfx/menu/luma/button_c"
Draw_Frame(15384): Unloading "gfx/menu/luma/checkbox_n0"
Draw_Frame(15384): Unloading "gfx/menu/luma/inputbox_n"
Draw_Frame(15384): Unloading "gfx/menu/luma/checkbox_n1"
Draw_Frame(15384): Unloading "gfx/menu/luma/cursor"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_aeslevel1"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_mod_instagib"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_stats1"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_mod_jeff's"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_ipv4"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_pure1"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_ipv6"
Draw_Frame(15384): Unloading "gfx/menu/luma/icon_mod_xpm"
Draw_Frame(15384): Unloading "gfx/menu/luma/scrollbar_f"
Draw_Frame(15384): Unloading "gfx/menu/luma/clearbutton_n"
Draw_Frame(15384): Unloading "gfx/blood"
Draw_Frame(15384): Unloading "gfx/menu/luma/background_ingame"
Draw_Frame(15384): Unloading "gfx/menu/luma/background_ingame_l2"
Draw_CachePic("gfx/conback"): frame 15596: loading pic
--> client to server keepalive
Draw_CachePic("gfx/conback2"): frame 15596: loading pic
Draw_CachePic("gfx/conback3"): frame 15596: loading pic
CL_Disconnect
Disconnect: Connection timed out