401 when accessing gitlab pages in Chrome after upgrading to v17.2.0

Summary

After we updated our selfhosted Gitlab instance to v17.2.0 we can't access any Gitlab Pages that are protected with access control.

Steps to reproduce

1. Go to project's Pages
2. On Oauth screen click Authorize Gitlab Pages
3. Get 401

Example Project

The issue happens on self hosted instance.

What is the current bug behavior?

After upgrading Gitlab from 17.1 to 17.2 we noticed that each time we try to access gitlab pages of any project that has access control enabled we are ending up with 401 screen. No matter which project or user - even instance admins are getting 401.

Also what's interesting - after clicking Authorize Gitlab Pages the app is not being authorized for user and the next time you visit gitlab pages it asks you again to authorize.

What is the expected correct behavior?

Access to pages should be possible without any errors.

Relevant logs and/or screenshots

and the logs that are relevant:

==> /var/log/gitlab/gitlab-pages/current <==
{"Namespace in path":"","Request host":"projects.pages.redacted","Session host":null,"correlation_id":"01J41F5QRM1XWFG7QRYF9W3AZV","host":"projects.pages.redacted","level":"info","msg":"Resetting session values","path":"/auth","state":"Zpp08KZrBUQBZU4Fy-sp7g==","time":"2024-07-30T09:22:36Z"}
{"correlation_id":"01J41F5QRM1XWFG7QRYF9W3AZV","host":"projects.pages.redacted","level":"info","msg":"Receive OAuth authentication callback","path":"/auth","state":"Zpp08KZrBUQBZU4Fy-sp7g==","time":"2024-07-30T09:22:36Z"}
{"correlation_id":"01J41F5QRM1XWFG7QRYF9W3AZV","host":"projects.pages.redacted","level":"warning","msg":"Authentication state did not match expected","path":"/auth","state":"Zpp08KZrBUQBZU4Fy-sp7g==","time":"2024-07-30T09:22:36Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01J41F5QRM1XWFG7QRYF9W3AZV","duration_ms":0,"host":"projects.pages.redacted","level":"info","method":"GET","msg":"access","pages_https":false,"proto":"HTTP/1.1","referrer":"https://gitlab.redacted/","remote_addr":"172.18.0.2:60812","remote_ip":"172.18.0.2","status":401,"system":"http","time":"2024-07-30T09:22:36Z","ttfb_ms":0,"uri":"/auth?code=7c56e510c1cc0fec43a705a9a17dd2f938f5b69003885a901981e94f8661ca84\u0026state=Zpp08KZrBUQBZU4Fy-sp7g%3D%3D","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36","written_bytes":2872}

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

$ gitlab-rake gitlab:env:info System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 3.1.5p253 Gem Version: 3.5.11 Bundler Version:2.5.11 Rake Version: 13.0.6 Redis Version: 7.0.15 Sidekiq Version:7.1.6 Go Version: unknown GitLab information Version: 17.2.1-ee Revision: 88793996279 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 14.11 URL: https://gitlab.redacted HTTP Clone URL: https://gitlab.redacted/some-group/some-project.git SSH Clone URL: git@gitlab.redacted:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: azure_activedirectory_v2 GitLab Shell Version: 14.37.0 Repository storages: - default: unix:/var/opt/gitlab/gitaly/gitaly.socket GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Gitaly - default Address: unix:/var/opt/gitlab/gitaly/gitaly.socket - default Version: 17.2.1 - default Git Version: 2.45.2

Results of GitLab application Check

Expand for output related to the GitLab application check

$ gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ... Checking GitLab Shell ... GitLab Shell: ... GitLab Shell version >= 14.37.0 ? ... OK (14.37.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Gitaly ... Gitaly: ... default ... OK Checking Gitaly ... Finished Checking Sidekiq ... Sidekiq: ... Running? ... yes Number of Sidekiq processes (cluster/worker) ... 1/1 Checking Sidekiq ... Finished Checking Incoming Email ... Incoming Email: ... Reply by email is disabled in config/gitlab.yml Checking Incoming Email ... Finished Checking LDAP ... LDAP: ... LDAP is disabled in config/gitlab.yml Checking LDAP ... Finished Checking GitLab App ... Database config exists? ... yes Tables are truncated? ... skipped All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Cable config exists? ... yes Resque config exists? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units) Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units) Projects have namespace: ... 2244/1 ... yes 20/2 ... yes 20/3 ... yes 36/5 ... yes 44/8 ... yes 44/9 ... yes 49/10 ... yes 49/11 ... yes 49/12 ... yes 49/13 ... yes 49/14 ... yes 56/15 ... yes 56/16 ... yes 56/17 ... yes 44/18 ... yes 44/19 ... yes 50/20 ... yes 56/21 ... yes 50/22 ... yes 56/23 ... yes 50/24 ... yes 74/25 ... yes 47/26 ... yes 50/27 ... yes 50/28 ... yes 50/29 ... yes 50/30 ... yes 50/31 ... yes 45/32 ... yes 45/33 ... yes 52/34 ... yes 53/35 ... yes 54/36 ... yes 74/37 ... yes 45/38 ... yes 89/39 ... yes 47/40 ... yes 46/41 ... yes 52/42 ... yes 46/43 ... yes 45/44 ... yes 47/45 ... yes 55/46 ... yes 50/47 ... yes 50/48 ... yes 50/49 ... yes 50/50 ... yes 50/51 ... yes 104/52 ... yes 50/53 ... yes 50/54 ... yes 104/55 ... yes 104/56 ... yes 104/57 ... yes 50/58 ... yes 48/59 ... yes 48/60 ... yes 48/61 ... yes 48/62 ... yes 117/63 ... yes 117/64 ... yes 50/65 ... yes 120/66 ... yes 120/67 ... yes 120/68 ... yes 9/69 ... yes 9/70 ... yes 36/119 ... yes 36/120 ... yes 36/126 ... yes 36/127 ... yes 36/128 ... yes 36/129 ... yes 49/131 ... yes 36/133 ... yes 20/866 ... yes 45/868 ... yes 47/870 ... yes 1618/871 ... yes 1618/872 ... yes 20/873 ... yes 1622/874 ... yes 2128/877 ... yes 2127/879 ... yes 2126/880 ... yes 2123/881 ... yes 2123/882 ... yes 1618/883 ... yes 1629/884 ... yes 2140/885 ... yes 2140/886 ... yes 1672/887 ... yes 2168/889 ... yes 1645/892 ... yes 1645/893 ... yes 1645/894 ... yes 1645/895 ... yes 1645/896 ... yes 1645/897 ... yes 1645/900 ... yes 1646/901 ... yes 1650/902 ... yes 1649/903 ... yes 1652/904 ... yes 1652/905 ... yes 1652/906 ... yes 1653/907 ... yes 1654/909 ... yes 1654/911 ... yes 1654/912 ... yes 1655/914 ... yes 1655/915 ... yes 1657/916 ... yes 1659/917 ... yes 1712/918 ... yes 1661/919 ... yes 2125/920 ... yes 2125/921 ... yes 1678/922 ... yes 1678/923 ... yes 1678/924 ... yes 1772/925 ... yes 1678/926 ... yes 1678/927 ... yes 1629/928 ... yes 2125/929 ... yes 2168/931 ... yes 2163/932 ... yes 2124/933 ... yes 2162/934 ... yes 2162/935 ... yes 2166/936 ... yes 2162/937 ... yes 2125/939 ... yes 2125/940 ... yes 2125/941 ... yes 2125/942 ... yes 2125/943 ... yes 2125/944 ... yes 2125/945 ... yes 1626/946 ... yes 2125/947 ... yes 2125/948 ... yes 2125/949 ... yes 2125/950 ... yes 2125/951 ... yes 2125/952 ... yes 2125/953 ... yes 2125/954 ... yes 2125/955 ... yes 2125/956 ... yes 2125/957 ... yes 2125/958 ... yes 2125/959 ... yes 1672/960 ... yes 2125/961 ... yes 1672/962 ... yes 2125/963 ... yes 2125/964 ... yes 2125/965 ... yes 2125/966 ... yes 2125/967 ... yes 2125/968 ... yes 2125/969 ... yes 2125/970 ... yes 2125/971 ... yes 2125/972 ... yes 2125/973 ... yes 2125/974 ... yes 2125/975 ... yes 2125/976 ... yes 2125/977 ... yes 2125/978 ... yes 2125/979 ... yes 2125/980 ... yes 2123/981 ... yes 2125/982 ... yes 2125/983 ... yes 2125/984 ... yes 2124/986 ... yes 2124/987 ... yes 2166/988 ... yes 2166/989 ... yes 2168/990 ... yes 2168/991 ... yes 2162/992 ... yes 2124/993 ... yes 2163/994 ... yes 2162/995 ... yes 2124/996 ... yes 2166/999 ... yes 2124/1001 ... yes 1772/1002 ... yes 2162/1003 ... yes 2166/1004 ... yes 2163/1005 ... yes 2124/1007 ... yes 2124/1008 ... yes 2162/1010 ... yes 2124/1011 ... yes 2124/1012 ... yes 2124/1013 ... yes 2123/1014 ... yes 2163/1015 ... yes 2124/1016 ... yes 2162/1017 ... yes 2124/1018 ... yes 2166/1020 ... yes 1678/1021 ... yes 2166/1022 ... yes 2124/1024 ... yes 2124/1025 ... yes 2124/1026 ... yes 2124/1027 ... yes 2166/1028 ... yes 2124/1029 ... yes 2124/1030 ... yes 2124/1031 ... yes 2124/1032 ... yes 2124/1033 ... yes 2166/1034 ... yes 2124/1035 ... yes 2124/1036 ... yes 2124/1037 ... yes 2124/1038 ... yes 2124/1039 ... yes 1707/1040 ... yes 1707/1041 ... yes 1707/1043 ... yes 1707/1044 ... yes 1707/1045 ... yes 1707/1046 ... yes 1709/1047 ... yes 1709/1048 ... yes 1710/1049 ... yes 1710/1050 ... yes 1710/1051 ... yes 1710/1052 ... yes 1710/1053 ... yes 1711/1054 ... yes 1711/1055 ... yes 1711/1056 ... yes 1711/1057 ... yes 1711/1058 ... yes 1711/1059 ... yes 1711/1060 ... yes 1711/1061 ... yes 1711/1063 ... yes 1711/1064 ... yes 1711/1066 ... yes 1711/1067 ... yes 1711/1068 ... yes 1711/1069 ... yes 1711/1070 ... yes 1711/1071 ... yes 1711/1072 ... yes 1711/1073 ... yes 1711/1074 ... yes 1711/1075 ... yes 1711/1077 ... yes 1712/1078 ... yes 1712/1079 ... yes 1712/1080 ... yes 1712/1081 ... yes 1712/1082 ... yes 1712/1083 ... yes 1712/1084 ... yes 1712/1085 ... yes 1712/1086 ... yes 1712/1087 ... yes 1712/1088 ... yes 1712/1089 ... yes 1713/1090 ... yes 1713/1091 ... yes 1713/1092 ... yes 1714/1093 ... yes 1714/1094 ... yes 1714/1095 ... yes 1714/1096 ... yes 1714/1098 ... yes 1714/1099 ... yes 1714/1100 ... yes 1714/1101 ... yes 1714/1102 ... yes 1714/1103 ... yes 1715/1104 ... yes 1715/1106 ... yes 1716/1107 ... yes 1715/1108 ... yes 1715/1110 ... yes 1715/1112 ... yes 1716/1113 ... yes 1715/1114 ... yes 1715/1115 ... yes 1716/1118 ... yes 1716/1119 ... yes 1716/1120 ... yes 1717/1122 ... yes 1717/1123 ... yes 1717/1124 ... yes 1717/1125 ... yes 1717/1126 ... yes 1728/1127 ... yes 1728/1128 ... yes 1728/1129 ... yes 1728/1130 ... yes 1728/1131 ... yes 1729/1132 ... yes 1729/1135 ... yes 1729/1137 ... yes 1730/1138 ... yes 1730/1139 ... yes 1730/1140 ... yes 1730/1141 ... yes 1730/1142 ... yes 1731/1143 ... yes 1732/1144 ... yes 1732/1145 ... yes 1732/1146 ... yes 1739/1148 ... yes 1739/1149 ... yes 1739/1150 ... yes 1739/1151 ... yes 1739/1152 ... yes 1739/1153 ... yes 1747/1154 ... yes 1747/1155 ... yes 1747/1156 ... yes 1747/1157 ... yes 1749/1158 ... yes 1749/1159 ... yes 1749/1160 ... yes 1749/1161 ... yes 1749/1162 ... yes 1749/1163 ... yes 1752/1164 ... yes 1752/1167 ... yes 1755/1168 ... yes 1755/1173 ... yes 1755/1174 ... yes 1756/1176 ... yes 1756/1177 ... yes 1756/1178 ... yes 1756/1179 ... yes 1756/1180 ... yes 1756/1181 ... yes 1756/1182 ... yes 2168/1183 ... yes 1680/1184 ... yes 1680/1185 ... yes 1680/1186 ... yes 1680/1188 ... yes 1678/1189 ... yes 1680/1190 ... yes 2168/1191 ... yes 2168/1192 ... yes 1680/1193 ... yes 1772/1195 ... yes 2140/1196 ... yes 1772/1197 ... yes 1772/1198 ... yes 1774/1199 ... yes 2162/1200 ... yes 2162/1201 ... yes 2162/1204 ... yes 2123/1205 ... yes 2162/1206 ... yes 1626/1208 ... yes 2405/1209 ... yes 2123/1212 ... yes 2167/1213 ... yes 2167/1214 ... yes 2167/1215 ... yes 2167/1216 ... yes 2167/1217 ... yes 1960/1218 ... yes 1960/1219 ... yes 1960/1220 ... yes 1960/1221 ... yes 1987/1222 ... yes 1987/1223 ... yes 2007/1226 ... yes 2007/1227 ... yes 2007/1228 ... yes 2007/1229 ... yes 2007/1230 ... yes 2086/1231 ... yes 2085/1232 ... yes 2087/1233 ... yes 2163/1244 ... yes 2162/1245 ... yes 2162/1246 ... yes 2162/1247 ... yes 2124/1248 ... yes 2140/1249 ... yes 1680/1250 ... yes 2124/1251 ... yes 1707/1252 ... yes 1729/1253 ... yes 1729/1254 ... yes 1729/1255 ... yes 1711/1256 ... yes 1711/1257 ... yes 1987/1258 ... yes 1714/1259 ... yes 1987/1260 ... yes 1716/1261 ... yes 1716/1262 ... yes 1716/1263 ... yes 1716/1264 ... yes 1716/1265 ... yes 1716/1266 ... yes 1755/1267 ... yes 1755/1268 ... yes 1755/1269 ... yes 1755/1270 ... yes 1755/1271 ... yes 1752/1272 ... yes 1752/1273 ... yes 1641/1274 ... yes 2130/1275 ... yes 1645/1276 ... yes 1645/1277 ... yes 1732/1278 ... yes 1654/1279 ... yes 1654/1280 ... yes 2123/1282 ... yes 2206/1285 ... yes 2206/1286 ... yes 2206/1287 ... yes 55/1289 ... yes 55/1290 ... yes 55/1291 ... yes 2223/1293 ... yes 49/1294 ... yes 1712/1296 ... yes 2206/1298 ... yes 1712/1299 ... yes 2223/1300 ... yes 45/1301 ... yes 2239/1303 ... yes 2239/1304 ... yes 2239/1305 ... yes 2239/1306 ... yes 2244/1307 ... yes 2140/1308 ... yes 6/1309 ... yes 2124/1313 ... yes 2124/1314 ... yes 1712/1315 ... yes 1712/1316 ... yes 36/1317 ... yes 1712/1318 ... yes 50/1319 ... yes 50/1320 ... yes 2180/1321 ... yes 2239/1322 ... yes 2168/1323 ... yes 2163/1324 ... yes 44/1325 ... yes 1772/1326 ... yes 2239/1327 ... yes 36/1328 ... yes 36/1330 ... yes 2166/1331 ... yes 2239/1332 ... yes 23/1333 ... yes 23/1334 ... yes 2287/1335 ... yes 6/1336 ... yes 49/1337 ... yes 10/1338 ... yes 36/1340 ... yes 1715/1342 ... yes 1715/1343 ... yes 2180/1344 ... yes 1715/1345 ... yes 1715/1346 ... yes 1715/1347 ... yes 50/1348 ... yes 1712/1349 ... yes 1752/1350 ... yes 1752/1351 ... yes 1707/1352 ... yes 1707/1353 ... yes 1752/1354 ... yes 1752/1355 ... yes 1711/1356 ... yes 1711/1357 ... yes 2316/1358 ... yes 1712/1359 ... yes 2181/1360 ... yes 2322/1364 ... yes 2326/1365 ... yes 1707/1366 ... yes 1689/1367 ... yes 1716/1368 ... yes 1716/1369 ... yes 1716/1370 ... yes 1730/1371 ... yes 1712/1373 ... yes 2326/1374 ... yes 2339/1375 ... yes 2339/1376 ... yes 2326/1378 ... yes 1730/1381 ... yes 1631/1382 ... yes 1711/1383 ... yes 1728/1384 ... yes 1728/1385 ... yes 6/1386 ... yes 2316/1387 ... yes 6/1388 ... yes 36/1389 ... yes 1657/1390 ... yes 49/1394 ... yes 2180/1395 ... yes 44/1396 ... yes 1728/1397 ... yes 50/1398 ... yes 1729/1399 ... yes 2374/1400 ... yes 36/1401 ... yes 55/1402 ... yes 53/1403 ... yes 53/1404 ... yes 2297/1405 ... yes 2297/1406 ... yes 2297/1407 ... yes 2297/1409 ... yes 20/1410 ... yes 2297/1411 ... yes 2297/1412 ... yes 2484/1413 ... yes 1707/1415 ... yes 2484/1416 ... yes 2484/1417 ... yes 2320/1418 ... yes 1755/1419 ... yes 1716/1420 ... yes 2162/1421 ... yes 2230/1423 ... yes 2124/1424 ... yes 2482/1425 ... yes 1657/1426 ... yes 2326/1427 ... yes 2297/1428 ... yes 2339/1429 ... yes 1695/1430 ... yes 1730/1431 ... yes 2418/1432 ... yes 2168/1433 ... yes 2206/1434 ... yes 2130/1436 ... yes 2130/1437 ... yes 2130/1438 ... yes 2130/1439 ... yes 2130/1440 ... yes 2130/1441 ... yes 1717/1443 ... yes 2130/1444 ... yes 1711/1446 ... yes 49/1447 ... yes 1711/1449 ... yes 20/1450 ... yes 47/1451 ... yes 2445/1452 ... yes 2168/1453 ... yes 117/1454 ... yes 2320/1456 ... yes 1712/1458 ... yes 2457/1459 ... yes 2457/1460 ... yes 53/1461 ... yes 1711/1462 ... yes 1717/1463 ... yes 50/1464 ... yes 2469/1465 ... yes 2418/1468 ... yes 1626/1469 ... yes 49/1471 ... yes 9/1472 ... yes 2482/1473 ... yes 2485/1474 ... yes 1712/1475 ... yes 2488/1476 ... yes 2488/1477 ... yes 2488/1478 ... yes 2488/1479 ... yes 14/1480 ... yes 36/1482 ... yes 36/1483 ... yes 56/1484 ... yes 45/1485 ... yes 36/1486 ... yes 36/1487 ... yes 51/1488 ... yes 1711/1489 ... yes 1630/1490 ... yes 1626/1491 ... yes Redis version >= 6.2.14? ... yes Ruby version >= 3.0.6 ? ... yes (3.1.5) Git user has default SSH configuration? ... yes Active users: ... 62 Is authorized keys file accessible? ... skipped (authorized keys not enabled) GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes Elasticsearch version 7.x-8.x or OpenSearch version 1.x ... skipped (Advanced Search is disabled) All migrations must be finished before doing a major upgrade ... skipped (Advanced Search is disabled) Checking GitLab App ... Finished Checking GitLab subtasks ... Finished

Gitlab pages configuration

Just a note that we are using a reverse proxy in front of our gitlab instance. The configuration of gitlab pages is as follows (all other settings are default):

Gitlab Pages config

# pages config 
pages_external_url "https://pages.redacted"
gitlab_pages['access_control'] = true
gitlab_pages['external_http'] = ['0.0.0.0:80']
pages_nginx['enable'] = false
gitlab_pages['auth_scope'] = 'read_api'

Tried options:

I tried to follow comments from gitlab-pages#1118 (closed) and I added env to disable feature flag: FF_ENABLE_PROJECT_PREFIX_COOKIE_PATH: "false" but it didn't work. I also tried to recreate pages oauth application, disabling and reenabling gitlab_pages['access_control'] - without any success. I also wasn't able to find any logs that would help in investigation.

Possible fixes

For HTTPS Pages URL only:
If you have HTTPS termination at proxy level and pages running on HTTP, then instead of using external_http, you should use listen_proxy.

Workarounds

While you're waiting for the patch release for 17.2, you can try one of the following solutions:

1. Use https for your pages instance
2. Use Firefox or other not-chromium-based browser
3. Disable authentication for pages