remove submoduleable vendor resources

parent 535a83d9
## Version 0.8.9
### Date: 1/14/14
### Changes:
## Version 0.8.8
- Minor bug fixes in parsing tool and dnsrecon.
### Date: 4/14/14
### Changes:
- Support for saving results to a JSON file.
- Bug fixes for:
- Parsing SPF and TXT records when saving to XML, CSV and SQLite3.
- Filtering of wildcard records when brute forcing a forward lookup zone.
- Several typos and misspelled words.
## Version 0.8.5
### Date: 5/25/13
### Changes:
- Changed the way IP ranges are handled.
- Greatly improved speed and memory use in a reverse lookup of large networks.
## Version 0.8.4
### Date: 5/19/13
### Changes:
- Improved Whois parsing for ranges and organization.
- Better Whois record and request handling for RIPE and APNIC.
- Several bug fixes.
- Added print messages when saving output to files.
## Version 0.8.1
### Changes:
- Improved DNSEC zone walk.
- Several bug fixes for exporting data and parsing records in zone transfers.
- DigiNinja Edition for all his hard work in making dnsrecon better.
## Version 0.7.8
### Date: 7/8/12
### Changes:
- CSV files now have a proper header for better parsing on tools that support them like Excel and PowerShell.
- Windows System Console printing is now managed properly.
- CNAME records are now saved in SQLite3 and CSV output.
- Fixed an error when performing zone transfers due to improper indent.
- Fixed mislabeling of -c option in the help message.
- If a range or CIDR is provided and no scan type is given, dnsrecon will perform a reverse lookup against it. When other types are given, the rvl type will be appended to the list automaticaly.
- Improved NSEC type detection to eliminate possible false positives.
- Added processing of LOC, NAPTR, CERT and RP records of zone transfers returned. Proper information saved on XML output with proper field names in the attributes for these.
- Fixes on Google enumeration parsing.
- Fixed several typos.
- Better handling and canceling of threaded tasks.
## Version 0.7.3
### Date: 5/2/12
### Changes:
- Fixes for Python 3 compatibility.
- Fixed key values when saving results to XML and CSV.
## Version 0.7.0
### Date: 3/2/12
### Changes:
- Fixes to zonewalk option.
- Query for _domainkey record in standard enumeration.
## Version 0.6.8
### Date: 2/15/12
### Changes:
- Added tool folder with Python script for parsing results in XML and CSV format.
- Added the ability to filter and extract hostnames and subdomains.
- Added Metasploit plugin for importing into Metasploit the CSV and XML results in a very fast manner using Nokogiri for XML. It will add hosts, notes for hostnames and service entries.
-Improvements on the zone transfer code:
- Handling of zones with no NS records.
- Proper parsing of PTR records in returned zones.
- De-duplication of NS records IP addresses.
- Provide additional info on failure.
- Provide more info on actions being taken.
- Bug fixes reported by users at RandomStorm and by Robin Wood.
- Zone walking has been greatly improved including the accuracy of the results and the formatting to extract the information in a manner more useful for a pentester.
## Version 0.6.6
### Date: 1/20/12
### Changes:
- Does not for a Origin Check for zones transferred since some admin may have configured their zones without NS servers as experienced by a user.
- Handles exception if NS records cannot be resolved when performing a zone transfer test.
- Will always ??? for a test for the SOA and test it for zone transfer.
- Fixed a problem when generating an XML file from a zone transfer with the new parsing method. Info type was added to the XML output.
## Version 0.6.5
### Date: 1/16/12
### Changes:
- Fixed problem with get_ns.
- Python 3.2 support.
- Color printing of messages like Metasploit.
- New library for printing color messages.
- Improved parsing of records when there is a zone transfer.
## Version 0.6.1
### Date: 1/14/12
### Changes:
- IPv6 support for ranges in reverse lookup.
- Enhanced parsing of SPF records ranges to cover includes and IPv6.
- Specific host query for TXT RR.
- Better handling and logging of TXT and SPF RR.
- Started changes for Python 3.x compatibility.
- Filtering of wildcard records when saving brute force results.
- Show found records after brute force of domain is finished.
- Manage Ctrl-c when doing a brute force and save results for those records found.
- Corrected several spelling errors.
## Version 0.6
### Date: 1/11/12
### Changes:
- Removed mDNS enumeration due to the pybonjour library has been abandoned and faster ways are available to achieve enumeration of mDNS records in a sub-net.
- Removed unused variables.
- Applied changes for PEP8 compliance.
- Added comma delimited value to a file for the results.
## Version 0.5.1
### Date: 1/8/12
### Changes:
- Additional fixes for XML formatting.
- Ability to end a zonewalk with Ctrl-c and not lose data.
- Initial Metasploit plug-in to be able to import data from XML file generated by dnsrecon.
## Version 0.5
### Date: 1/8/12
### Changes:
- Will check in standard enumeration if DNSSEC is configured for the zone by checking for DNSKEY records and checking if the zone is configured as NSEC or NSEC3.
- With the get_ip() method it will also check for CNAME records and add those to the list found hosts.
- Will perform a DNSSEC zonewalk if NSEC records are available. It currently identifies A, AAAA, CNAME, NS and SRV records. For any other, it will just print the RDATA info.
- General record resolver method added.
- Changes to the options.
Known Issues:
- For some reason, the Python getopt is not parsing the options correctly in some cases. Considering changing to optparse even if it is more complicated to manage.
- When running Python 3.x the Whois query does not show the organization.
This diff is collapsed.
This diff is collapsed.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (C) 2010 Carlos Perez
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; Applies version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
import urllib
import re
import time
try:
url_opener = urllib.FancyURLopener
except AttributeError:
import urllib.request
url_opener = urllib.request.FancyURLopener
class AppURLopener(url_opener):
version = 'Mozilla/5.0 (compatible; Googlebot/2.1; + http://www.google.com/bot.html)'
def scrape_google(dom):
"""
Function for enumerating sub-domains and hosts by scrapping Google. It returns a unique
list if host name extracted from the HREF entries from the Google search.
"""
results = []
filtered = []
searches = ["100", "200", "300", "400", "500"]
data = ""
urllib._urlopener = AppURLopener()
user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'
headers = {'User-Agent': user_agent, }
#opener.addheaders = [('User-Agent','Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)')]
for n in searches:
url = "http://google.com/search?hl=en&lr=&ie=UTF-8&q=%2B" + dom + "&start=" + n + "&sa=N&filter=0&num=100"
try:
sock = urllib.urlopen(url)
data += sock.read()
sock.close()
except AttributeError:
request = urllib.request.Request(url, None, headers)
response = urllib.request.urlopen(request)
data += str(response.read())
results.extend(unique(re.findall("href=\"htt\w{1,2}:\/\/([^:?]*[a-b0-9]*[^:?]*\." + dom + ")\/", data)))
# Make sure we are only getting the host
for f in results:
filtered.extend(re.findall("^([a-z.0-9^]*" + dom + ")", f))
time.sleep(2)
return unique(filtered)
def unique(seq, idfun=repr):
"""
Function to remove duplicates in an array. Returns array with duplicates
removed.
"""
seen = {}
return [seen.setdefault(idfun(e), e) for e in seq if idfun(e) not in seen]
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (C) 2010 Carlos Perez
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; Applies version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
import pybonjour
import select
import re
def mdns_browse(regtype):
"""
Function for resolving a specific mDNS record in the Local Subnet.
"""
found_mdns_records = []
domain = None
browse_timeout = 1
resolve_timeout = 1
results = []
resolved = []
def resolve_callback(
sdRef,
flags,
interfaceIndex,
errorCode,
fullname,
hosttarget,
port,
txtRecord,
):
n = re.compile(u'(\x00|\x07|\x1A|\x16|\x06|\x08|\x1f|\xdb|\xb2|\xb0|\xb1'
u'\xc9|\xb9|\xcd|\u2019|\u2018|\u2019|\u201c|\u201d|\u2407)')
t = re.compile(r'[\x00-\x1f|\x7f|\x0e]')
if errorCode == pybonjour.kDNSServiceErr_NoError:
results.append({
'type': 'MDNS',
'name': n.sub(" ",fullname),
'host': str(hosttarget).replace('\032'," "),
'port': str(port),
'txtRecord': t.sub(" ",txtRecord.strip())
})
resolved.append(True)
def browse_callback(
sdRef,
flags,
interfaceIndex,
errorCode,
serviceName,
regtype,
replyDomain,
):
if errorCode != pybonjour.kDNSServiceErr_NoError:
return
if not flags & pybonjour.kDNSServiceFlagsAdd:
# Service removed
return
resolve_sdRef = pybonjour.DNSServiceResolve(
0,
interfaceIndex,
serviceName,
regtype,
replyDomain,
resolve_callback,
)
try:
while not resolved:
ready = select.select([resolve_sdRef], [], [],
resolve_timeout)
if resolve_sdRef not in ready[0]:
# Resolve timed out
break
pybonjour.DNSServiceProcessResult(resolve_sdRef)
else:
resolved.pop()
finally:
resolve_sdRef.close()
browse_sdRef = pybonjour.DNSServiceBrowse(regtype=regtype,
domain=domain, callBack=browse_callback)
try:
while True:
ready = select.select([browse_sdRef], [], [],
browse_timeout)
if not ready[0]:
break
if browse_sdRef in ready[0]:
pybonjour.DNSServiceProcessResult(browse_sdRef)
_results = results
for result in _results:
found_mdns_records = [result]
finally:
browse_sdRef.close()
return found_mdns_records
#!/usr/bin/env python
import sys
import platform
# -*- coding: utf-8 -*-
# Copyright (C) 2012 Carlos Perez
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; Applies version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
def print_status(message=""):
if sys.stdout.isatty() and platform.system() != "Windows":
print("\033[1;34m[*]\033[1;m {0}".format(message))
else:
print("[*] {0}".format(message))
def print_good(message=""):
if sys.stdout.isatty() and platform.system() != "Windows":
print("\033[1;32m[+]\033[1;m {0}".format(message))
else:
print("[+] {0}".format(message))
def print_error(message=""):
if sys.stdout.isatty() and platform.system() != "Windows":
print("\033[1;31m[-]\033[1;m {0}".format(message))
else:
print("[-] {0}".format(message))
def print_debug(message=""):
if sys.stdout.isatty() and platform.system() != "Windows":
print("\033[1;31m[!]\033[1;m {0}".format(message))
else:
print("[!] {0}".format(message))
def print_line(message=""):
print("{0}".format(message))
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (C) 2010 Carlos Perez
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; Applies version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
import re
from netaddr import *
import socket
WHOIS_PORT_NUMBER = 43
WHOIS_RECEIVE_BUFFER_SIZE = 4096
def get_whois(ip_addrs):
"""
Function that returns what whois server is the one to be queried for
registration information, returns whois.arin.net is not in database, returns
None if private.
"""
whois_server = None
ip = IPAddress(ip_addrs)
info_of_ip = ip.info
if ip.version == 4 and ip.is_private() is False:
for i in info_of_ip['IPv4']:
whois_server = i['whois']
if len(whois_server) == 0 and i['status'] != "Reserved":
whois_server = "whois.arin.net"
elif len(whois_server) == 0:
whois_server = None
return whois_server
def whois(target, whois_srv):
"""
Performs a whois query against a arin.net for a given IP, Domain or Host as a
string and returns the answer of the query.
"""
response = ""
counter = 1
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((whois_srv, WHOIS_PORT_NUMBER))
if whois_srv == "whois.arin.net":
s.send(("n " + target + "\r\n").encode("utf-8"))
else:
s.send((target + "\r\n").encode("utf-8"))
response = ''
while True:
d = s.recv(WHOIS_RECEIVE_BUFFER_SIZE)
response += str(d)
counter += 1
if str(d) == '' or counter == 5:
break
s.close()
except Exception as e:
print(e)
pass
return response
def get_whois_nets(data):
"""
Parses whois data and extracts the Network Ranges returning an array of lists
where each list has the starting and ending IP of the found range.
"""
pattern = '([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) - ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})'
results = re.findall(pattern, data)
return results
def get_whois_orgname(data):
org_pattern = "OrgName\:\s*(.*)\n"
result = re.findall(org_pattern, data)
# Lets try RIPENET Format
if not result :
org_pattern = "netname\:\s*(.*)\n"
result = re.findall(org_pattern, data)
if not result:
result.append("Not Found")
return result
module Msf
class Plugin::PostCommand < Msf::Plugin
class MultiCommand
include Msf::Ui::Console::CommandDispatcher
# Set name for command dispatcher
def name
"DNSR_Import"
end
# Define Commands
def commands
{
"import_dnsrecon_xml" => "Import DNSRecon XML output file.",
"import_dnsrecon_csv" => "Import DNSRecon CSV output file."
}
end
# Method for adding a host
def report_host(ip)
print_good("Importing host #{ip}")
framework.db.report_host({:host => ip})
end
# Method for adding a service
def report_service(name, proto, port, target)
print_good("Importing service #{name} for host #{target}")
framework.db.report_service({
:host => target,
:name => name,
:port => port,
:proto => proto
})
end
# Method for parsing and importing the XML file
def xml_import(xm_file)
require "nokogiri"
reader = Nokogiri::XML::Reader(IO.read(xm_file))
reader.each do |node|
if node.name == 'record'
case node.attribute('type')
when /PTR|^[A]$|AAAA/
report_host(node.attribute('address'))
# Since several hosts can be tied to one IP, specially those that are vhosts
# or are behind a balancer the names are saved as notes.
framework.db.report_note({
:host => node.attribute('address'),
:ntype => "dns.info",
:data => "A #{node.attribute('name')}",
:update => :unique_data
})
when /NS/
report_host(node.attribute('address'))
report_service("dns", "udp", 53, node.attribute('address'))
framework.db.report_note({
:host => node.attribute('address'),
:port => 53,
:proto => "udp",
:ntype => "dns.info",
:data => "NS #{node.attribute('target')}"
})
when /SOA/
report_host(node.attribute('address'))
report_service("dns", "udp", 53, node.attribute('address'))
framework.db.report_note({
:host => node.attribute('address'),
:port => 53,
:proto => "udp",
:type => "dns.info",
:data => "NS #{node.attribute('mname')}"
})
when /MX/
report_host(node.attribute('address'))
report_service("smtp", "tcp", 25, node.attribute('address'))
framework.db.report_note({
:host => node.attribute('address'),
:port => 25,
:proto => "tcp",
:type => "dns.info",
:data => "MX #{node.attribute('exchange')}"
})
when /SRV/
srv_info = []
srv_info = node.attribute('name').strip.scan(/^_([A-Za-z0-9_-]*)\._(tcp|udp|tls)\./)[0]
port = node.attribute('port').strip.to_i
if srv_info[1] == "tls"
proto = "tcp"
else
proto = srv_info[1]
end
report_host(node.attribute('address'))
report_service(srv_info[0], proto, port, node.attribute('address'))
framework.db.report_note({
:host => node.attribute('address'),
:port => port,
:proto => proto,
:type => "dns.info",
:data => "SRV #{node.attribute('target')}"
})
end
end
end
end
# XML Import command
def cmd_import_dnsrecon_xml(*args)
# Define options
opts = Rex::Parser::Arguments.new(
"-f" => [ true, "XML file to import."],
"-h" => [ false, "Command Help"]
)
# set variables for options
xml_file = ""
# Parse options
opts.parse(args) do |opt, idx, val|
case opt
when "-f"
xml_file = val
when "-h"
print_line(opts.usage)
return
end
end
if not ::File.exists?(xml_file)
print_error "XML File does not exists!"
return
else
xml_import(xml_file)
end
end
# Tab completion for the XML import command
def cmd_import_dnsrecon_xml_tabs(str, words)
tab_complete_filenames(str, words)
end
# Method for parsing and importing the XML file
def csv_import(csv_file)
require "csv"
CSV.foreach(csv_file) do |row|
case row[0]
when /PTR|^[A]$|AAAA/
report_host(row[2])
# Since several hosts can be tied to one IP, specially those that are vhosts
# or are behind a balancer the names are saved as notes.
framework.db.report_note({
:host => row[2],
:ntype => "dns.info",
:data => "A #{row[1]}",
:update => :unique_data
})
when /NS/
report_host(row[2])
report_service("dns", "udp", 53, row[2])
framework.db.report_note({
:host => row[2],
:port => 53,
:proto => "udp",
:ntype => "dns.info",
:data => "NS #{row[1]}"
})
when /SOA/
report_host(row[2])
report_service("dns", "udp", 53, row[2])
framework.db.report_note({
:host => row[2],
:port => 53,
:proto => "udp",
:type => "dns.info",
:data => "NS #{row[1]}"
})
when /MX/
report_host(row[2])
report_service("smtp", "tcp", 25, row[2])
framework.db.report_note({
:host => row[2],
:port => 25,
:proto => "tcp",
:type => "dns.info",