GitLab Recon
In CE to make our security product more popular.
The ideal flow is: enter a domain name 'example.com' and get a list of vulnerabilities with the most likely ones on top (sorted by signal to noise ratio).
To do this there are two components, Recon (this issue) and DAST gitlab-ee#3956 (closed)
The first iteration user flow will be:
- Create a new project with the 'Recon' GitLab CI template.
- Enter the TARGET_DOMAIN environmental variable.
- Receive a list of subdomains in a yaml compatible format.
Rest of the workflow, not part of Recon:
- Copy (part of) that list to the clipboard
- Create a new project with the 'DAST' GitLab CI template.
- Past the list into the .gitlab-ci.yml file to run the DAST in parallel against all subdomains.
- Get a list of vulnerabilities with the most likely ones on top
The GitLab Recon CI template has two stages:
- Running a number of subdomain discovery tools in parallel.
- Running a task that aggregates their output and deduplicates it.
The subdomain discovery tools would be based on https://news.ycombinator.com/item?id=15676951 Example commands:
-
/cdx-index-client.py -c CC-MAIN-2017-43 '*.example.com'
with http://index.commoncrawl.org/ -
python sublist3r.py -d example.com
with https://github.com/aboul3la/Sublist3r -
./ct.py example.com | ./bin/massdns -r resolvers.txt -t A -q -a -o -w
with https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration -
python dnsrecon.py -n ns1.example.com -d example.com -D subdomains-top1mil-5000.txt -t brt
with https://github.com/darkoperator/dnsrecon - `dig +multi AXFR @ns1.exaple.com example.com
-
ldns-walk @ns1.example.com example.com
with https://www.nlnetlabs.nl/projects/ldns/