Commit d8999fc1 authored by Chris Moberly's avatar Chris Moberly
Browse files

Added a short tips page on bypassing Cloudflare DDoS

parent 21ebceda
......@@ -13,6 +13,10 @@ At GitLab one of our six [core values](https://about.gitlab.com/handbook/values/
* [K8s-GKE Attack Notes](/K8s-GKE-attack-notes/)
* [Firefox for Android LAN-Based Intent Triggering](/firefox-android-2020/)
### Quick Tips
* [Scraping Cloudflare Anti-Bot](/cloudflare-notes)
### Talks
* [Black Hat USA 2020 - Token-Hunter and gitrob: Hunting for Secrets](./blackhat2020-hunting-for-secrets/)
......@@ -29,8 +33,4 @@ At GitLab one of our six [core values](https://about.gitlab.com/handbook/values/
### Red Team Exercises
* [RT-011 Phishing Campaign - Fake Laptop Upgrade](https://gitlab.com/gitlab-com/gl-security/security-operations/gl-redteam/red-team-tech-notes/-/tree/master/RT-011%20-%20Phishing%20Campaign)
* [RT-011 Phishing Campaign - Fake Laptop Upgrade](https://gitlab.com/gitlab-com/gl-security/security-operations/gl-redteam/red-team-tech-notes/-/tree/master/RT-011%20-%20Phishing%20Campaign)
\ No newline at end of file
# Bypassing Cloudflare's "Under Attack" Mode
## Intro
Working on the Red Team is all about testing defenses by simulating malicious activity. A good deal of actual malicious activity is performed by automated tools, or "bots". These bots might try things like guessing passwords for legitimate accounts, creating new accounts to abuse resources, or just crawling a website and scraping resources in search of secrets.
While working on a recent project, we came up against a fairly common situation - the site we were auditing was protected by Cloudflare's [Under Attack](https://support.cloudflare.com/hc/en-us/articles/200170076-Understanding-Cloudflare-Under-Attack-mode-advanced-DDOS-protection-) mode. This mode is meant to protect against distributed denial-of-service (DDoS) attacks. While we had no intention of conducting this type of attack, the mechanism itself still prevented our tooling from interacting with the website.
This write-up explains the behaviour we observed and the solution we used to quickly deploy automated tooling that bypasses this particular protection measure.
No zero-days or new vulnerabilities are being disclosed here, and there is nothing earth-shattering or particularly new. This is being shared simple because others may find it useful or interesting.
**Technical TL;DR** <br>
- First GET to an Under Attack enabled site responds 503 with a complex/obfuscated JavaScript math challenge.
- Browser POSTs the solution back to the site after a random delay of 1-5 seconds.
- Site responds with `Set-Cookie: cf_clearance=xxxx`.
- Subsequent requests with `cf_clearance` cookie are not challenged
- The User-Agent **must** be consistent between cookie acquisition and ongoing requests!
- To bypass, a headless browser can be used to solve the initial challenge and grab the cookie, which can then be used by standard libraries to perform further HTTP requests.
## Why not use an existing library?
A [quick search](https://duckduckgo.com/?q=cloudflare+scraping+library) will reveal that many libraries already exist to bypass the Cloudflare browser check.
After closely reviewing the source, we tried a few of these tools. They didn't work. The HTTP interactions we observed didn't match up with what the tools expected. Fixing them appeared to be a bit more work than tweaking some regular expressions.
Maybe our particular target had some unique qualities. Maybe Cloudflare had made some breaking changes recently, or maybe we were just doing something incredibly silly. Whatever the reason, this seemed like a good opportunity to take a step back and get a better understanding of exactly what we were up against.
## Dynamic Analysis
The first thing to understand was what happens when everything works.
You may have noticed that when you visit some sites, there is a slight delay while an image like the one below is displayed.
![Checking your browser](checking.png)
If you were proxying your browser traffic through something like [ZAP](https://owasp.org/www-project-zap/), you would see that at this point you receive an HTTP reply with headers like this:
```
HTTP/1.1 503 Service Temporarily Unavailable
Date: Thu, 01 Oct 2020 01:36:07 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
X-Frame-Options: SAMEORIGIN
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 01 Jan 1970 00:00:01 GMT
cf-request-id: 028356780f008675309ea09200002004
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 3ebb736caf4bde30-MEL
```
And inside the body of that page is some JavaScript that looks like this:
```
setTimeout(function(){
var s,t,o,p, b,r,e,a,k,i,n,g
...
```
That function is much longer and contains some obfuscation, but that famous "stop breaking" is a clear indicator of the Cloudflare DDoS protection.
A bit below the JavaScript is an HTML form that has characteristics like the following:
```
<form class="challenge-form" id="challenge-form" action="/protected_page?__cf_chl_jschl_tk__=xxx" method="POST" enctype="application/x-www-form-urlencoded">
<input type="hidden" name="r" value="xxx"/>
<input type="hidden" value="xxx" id="jschl-vc" name="jschl_vc"/>
<input type="hidden" name="pass" value="xxx"/>
<input type="hidden" id="jschl-answer" name="jschl_answer"/>
</form>
```
What your browser does here is solve a math challenge buried inside that obfuscated JavaScript function, sleep for a few seconds, and then `POST` the response into the HTML form via the field named `jschl_answer`.
If the answer was correct, the web server will respond back with an `HTTP 200` including a header like the following:
```
`Set-Cookie: cf_clearance=xxxx`
```
And the page you originally intended to visit will be rendered. All subsequent requests you make to that domain will include the `cf_clearance` cookie and you will not be bothered with the process again.
## So how do we bypass it?
The existing libraries we've seen use things like complex regular expressions to extract the JavaScript and pass it off to third-party libraries to solve. This is great when it works, but the second Cloudflare changes a single character in their code everything breaks.
We had an idea to use a headless browser not for the entire scraping experience, but merely to grab the initial token and hand it back to our tooling. The benefit here is that it would adapt automatically to changes at Cloudflare and would minimize the overhead of a headless browser by using it for a maximum of three HTTP requests.
The internal tool we needed this for is written in Go, so we went looking for a compatible library to drive a headless browser, like Chromium. [chromedp](https://github.com/chromedp/chromedp) looked robust, actively maintained, and was actually recommended by other libraries which had been retired.
We wrote a quick function using chromedp that solved the initial challenge and returned the required cookie. From there, we simply included the cookie in all our `net/http` requests.
That function is available to use in a package located [here](https://gitlab.com/gitlab-com/gl-security/security-operations/gl-redteam/cfclearance). If you find it useful or have ideas, please let us know via an issue there. Thanks!
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment