Anubis seems to be messing with API authentication

We have an internal error appearing for users occasionally (https://gitlab.cee.redhat.com/kernel-qe/sustaining-engineering/seqe/-/issues/15) which comes down to our auth code raising a KeyError looking for the csrftoken

    _dw_session.get(DW_OIDC_AUTH_URL,
                    auth=requests_gssapi.HTTPSPNEGOAuth(),
                    allow_redirects=True)
    # Fill in headers required for CSRF validation
    _dw_session.headers.update({
        "Referer": DW_BASE_URL,
        "X-CSRFToken": _dw_session.cookies["csrftoken"],
        "User-Agent": "seqe",
    })

I'm not defending that code - half the problem is the calling code around it doesn't handle that well, leading to confusing user responses.

But, despite the issues on my side of the code ... the csrftoken isn't set. The problem is it's a Heisenbug, you rerun things with debugging and it works. But I think I've managed to catch it in the act, because on one of my debugging runs I dumped the headers and they were

{'cache-control': 'no-store', 'content-encoding': 'gzip', 'content-type': 'text/html; charset=utf-8', 'set-cookie': 'techaro.lol-anubis-auth=; Path=/; Domain=datawarehouse.cki-project.org; Expires=Wed, 18 Feb 2026 01:20:45 GMT; Max-Age=0; Secure; SameSite=None, techaro.lol-anubis-cookie-verification=...; Path=/; Domain=datawarehouse.cki-project.org; Expires=Wed, 18 Feb 2026 01:51:45 GMT; Secure; SameSite=None, ...=...; path=/; HttpOnly; Secure; SameSite=None', 'date': 'Wed, 18 Feb 2026 01:21:45 GMT', 'transfer-encoding': 'chunked'}

This strongly makes me suspect anubis is involved. That this disappears after some retries also seems to fit the pattern. I don't really understand how anubis works at this level, I mean I know I see a page and get redirected when I use a browser. We have allow_redirects on the get() but perhaps it's using a <meta> tag or something that requests can't follow?

If that is the problem; at first I was going to suggest the login endpoints shouldn't be protected by anubis, but then I wonder if that just moves the problem to the next API call. Unless anubis knows to get out of the way of authenticated sessions, which it might? Otherwise, is there something I should do to reliably "pre-authenticate" these calls as not originating from a soul-sucking Tessier-Ashpool AI farm?

Jira: CKI-6521