Investigate making *Send nothing as Referer* the default *Rewrite Mode*
Based on recent user input I've come to believe that many sites that many sites that do referer checking skip their check if the Referer header field is empty / not present. This could be used to simplify the whitelist and reduce the number of issues discovered in the future.
It's hard to properly decide this since we cannot know the total number of websites doing referer checking and which of these properly handle Send URL of target page vs Send nothing as Referer values. However an empty referer value is afaik also sent when loading HTTP content in HTTPS pages and definitely on direct hit. It also makes sense from a programmers perspective to write something like
if(referHeader.present()) {
checkRefererOrThrow(refererHeader.value(), ["sourceDomain.a", "sourceDomain.b"]);
}
to do the referer checking: The check the presence of the header would likely already be added during development since the header is not present on direct-hit and during development you often have direct-hits while testing; adding the target domain as valid source domain on the other hand, is not something that would be needed during testing or real-life use-cases so it's much less likely. This is of course just a guess, but it's likely what I would do during development as well, simply due to how getting stuff done efficiently when programming works.
Due to lack of a better benchmark, I'll mostly go with the reasoning above if it turns out that at least 30% of the current whitelist entries are unnecessary when changing to direct-hit mode.
-
Check existing whitelist entries on whether they are necessary when changing to direct-hit -
Figure out the necessary migration code if results turn out to be promising