Commit 518465ec authored by Sophie Brun's avatar Sophie Brun

Imported Upstream version 1.2

parent bd324728
# ChangeLog
## 1.2 _(July 16, 2015)_
- Switched to Arachni Public Source License v1.0.
- `UI`
- `CLI::Framework`
- Fixed timeout enforcement.
- `OptionParser`
- Added `--browser-cluster-wait-for-element`.
- `Output`
- `#error_log_fd` -- Catch `Errno` system errors (like `Too many open files`)
to avoid crashing.
- `OptionGroups`
- `HTTP`
- `#request_queue_size` -- Lowered from `500` to `100`.
- `BrowserCluster`
- `#wait_for_elements` -- Wait for element matching `CSS` to appear when
visiting a page whose URL matches the `PATTERN`.
- `#job_timeout` -- Increased from 15 to 25 seconds.
- `Framework`
- `#pause` -- Pause is now near instant.
- `#audit` -- Substantially simplified and optimized the consumption of URL
and page queues.
- `#audit_page` -- Application of DOM metadata now happens asynchronously
and uses the `BrowserCluster` instead of an independent `Browser`.
- `HTTP`
- `Client`
- Updated cookie setting from `OptionGroups::HTTP#cookies` `Hash`.
- Trigger garbage collections before and after performing the queued
requests to prevent large RAM spikes.
- `Dynamic404Handler`
- Account for cases where the server returns intermittent errors
that can lead to signature corruption and possibly false positives.
- Updated training scenarios for cases where `~` are ignored.
- Disable platform fingerprinting during the gathering of signatures.
- `Request`
- Ignore proxy-related traffic (`CONNECT`) when capturing raw traffic data.
- Added `#fingerprint` option to enable/disable platform fingerprinting
on a per request basis.
- `#response_max_size` -- In addition to setting the `maxfilesize` for
the `Typhoeus::Request`, stream bodies and manually abort if the
buffer exceeds the limit -- covers cases where no `Content-Type`
is set.
- `Headers`
- Merge values of headers with identical normalized names (i.e.
`set-cookie` and `Set-Cookie` in the same response).
- Cache header name canonicalization.
- `ProxyServer`
- Cache header name canonicalization.
- SSL interceptor now automatically generates certificate/key pairs
based on Arachni CA.
- `Page`
- `#has_script?` -- Detect using the body instead of the parsed document.
- `Parser`
- Optimized to avoid HTML parsing if it contains no indication of elements.
- `#headers` -- Updated to include headers from the HTTP request in addition
to common ones.
- `Extractors` -- Optimized to avoid HTML parsing if it contains no
indication of elements.
- `Element`
- Cleaned up per-element input value encoding.
- Enforce a `MAX_SIZE` on acceptable values during parsing.
- Optimized to avoid HTML parsing if it contains no indication of elements.
- `Server`
- `#log_remote_file_if_exists?` -- Flag issues as untrusted at that point
if possible, instead of at the end of the scan.
- `#remote_file_exist?` -- Disable platform fingerprinting when dealing
with a dynamic handler.
- `Capabilities`
- `Inputtable` -- Added cache for `#inputtable_id` calculation.
- `Analyzable`
- `Taint` -- Added match cache based on signatures and haystacks.
- `Timeout` -- Override user audit options that don't play nice with this technique.
- `Check::Auditor`
- `#log_remote_file` -- Assign `HTTP::Response#status_line` as proof.
- `Issue`
- `#signature` -- Store `Regexp` source instead of converting it to String.
- `Browser`
- Updated to extract and whitelist CDNs from response bodies.
- `#cookies` -- Normalize cookies with quoted values since Watir doesn't take
care of that bit.
- `Javascript`
- `#inject` -- Inject `TaintTracer` and `DOMMonitor` update calls in
requested JS assets.
- `TaintTracer`
- Limited data and execution flow sinks to a max size of 50 entries.
- Don't trace functions known to cause issues:
- Anonymous functions.
- `lodash()`
- `DOMMonitor`
- Keep track of `jQuery` delegated events.
- `Support`
- `Cache`
- `RandomReplacement` -- Removed extra key `Array`.
- `Signature` -- Cache token generation.
- Checks -- Added `Issue#proof` to as many issues as possible.
- Active
- `xss`
- When the case involves payloads landing in `textarea`s, break out of
them to prevent possible FPs.
- Added double-encoded payloads.
- `xss_dom_inputs`
- Don't perform redundant audits.
- Don't process custom events.
- Updated to handle cases where a button needs to be clicked after
filling in the inputs.
- Added progress messages.
- `unvalidated_redirect`
- Escalated severity to 'High'.
- Only perform straight payload injections.
- `unvalidated_redirect_dom`
- Escalated severity to 'High'.
- `path_traversal`, `file_inclusion`, `os_cmd_injection`, `xxe`
- Updated `/etc/passwd` content matching pattern.
- Passive
- Added
- `common_admin_intefaces` -- By Brendan Coles.
- `backdoors`, `backup_directories`, `backup_files`, `common_files`,
`directory_listing`
- Added MVC frameworks as exempt platforms since they do their own routing.
- Plugins
- Added
- `restrict_to_dom_state` -- Restricts the audit to a single page's DOM
state, based on a URL fragment.
- `metrics` -- Captures metrics about multiple aspects of the scan and
the web application.
- `autologin` -- Updated to fail gracefully in cases of an invisible form DOM elements.
- `login_script` -- Added support for Javascript login scripts.
- `proxy`
- Updated to show JSON and XML inputs in the inspection page.
- Added output message with instructions for server that use SSL.
- `vector_feed` -- Updated to support XML and JSON elements.
- Reporters
- `xml`
- Fixed bug causing vector `affected_input_name` to be blank.
- Fingerprinters -- Optimized across the board to prefer less resource intensive checks.
- Frameworks
- Rack -- Expanded signatures.
- Languages
- JSP renamed to Java and expanded signatures.
- PHP -- Expanded signatures.
- Python -- Expanded signatures.
- Servers
- Tomcat -- Expanded signatures.
- Added
- Frameworks
- Django
- Rails
- ASP.NET MVC
- CakePHP
- JSF
- CherryPy
- Servers
- Gunicorn
- Path extractors
- Added
- `data_url` -- Extracts paths from `data-url` attributes of `a` tags.
## 1.1 _(May 1, 2015)_
- `gemspec` -- Require Ruby >= 2.0.0.
......@@ -96,6 +253,8 @@
- Added `.full_and_absolute_url?`.
- `Browser`
- Updated to extract JSON and XML input vectors from HTTP requests.
- `#cookies` -- Normalize cookies with quoted values since Watir doesn't take
care of that bit.
- `#shutdown` -- Fixed Selenium exceptions on dead browser process.
- `#to_page` -- Apply DOM metadata to page elements.
- `#spawn_phantomjs` -- Enabled `--disk-cache` option for `phantomjs`.
......
This diff is collapsed.
......@@ -3,7 +3,7 @@
<table>
<tr>
<th>Version</th>
<td>1.1</td>
<td>1.2</td>
</tr>
<tr>
<th>Homepage</th>
......@@ -42,17 +42,16 @@
</tr>
<tr>
<th>License</th>
<td>Dual-licensed (Apache License v2.0/Commercial) - (see LICENSE file)</td>
<td>Arachni Public Source License v1.0 - (see LICENSE file)</td>
</tr>
</table>
![Arachni logo](http://arachni.github.com/arachni/logo.png)
![Arachni logo](http://www.arachni-scanner.com/large-logo.png)
## Synopsis
Arachni is an Open Source, feature-full, modular, high-performance Ruby framework
aimed towards helping penetration testers and administrators evaluate the security
of web applications.
Arachni is a feature-full, modular, high-performance Ruby framework aimed towards
helping penetration testers and administrators evaluate the security of web applications.
It is smart, it trains itself by monitoring and learning from the web application's
behavior during the scan process and is able to perform meta-analysis using a number of
......@@ -325,15 +324,22 @@ Currently, the following platforms can be identified:
- Nginx
- Tomcat
- Jetty
- Gunicorn
- Programming languages
- PHP
- ASP
- ASPX
- JSP
- Java
- Python
- Ruby
- Frameworks
- Rack
- CakePHP
- Rails
- Django
- ASP.NET MVC
- JSF
- CherryPy
The user also has the option of specifying extra platforms (like a DB server)
in order to help the system be as efficient as possible. Alternatively, fingerprinting
......@@ -380,23 +386,23 @@ Active checks engage the web application via its inputs.
- PHP
- Ruby
- Python
- JSP
- ASP.NET
- Java
- ASP
- Blind code injection using timing attacks (`code_injection_timing`).
- PHP
- Ruby
- Python
- JSP
- ASP.NET
- Java
- ASP
- LDAP injection (`ldap_injection`).
- Path traversal (`path_traversal`).
- *nix
- Windows
- Tomcat
- Java
- File inclusion (`file_inclusion`).
- *nix
- Windows
- Tomcat
- Java
- PHP
- Perl
- Response splitting (`response_splitting`).
......@@ -441,6 +447,7 @@ Passive checks look for the existence of files, folders and signatures.
- Allowed HTTP methods (`allowed_methods`).
- Back-up files (`backup_files`).
- Backup directories (`backup_directories`)
- Common administration interfaces (`common_admin_interfaces`).
- Common directories (`common_directories`).
- Common files (`common_files`).
- HTTP PUT (`http_put`).
......@@ -475,14 +482,14 @@ Passive checks look for the existence of files, folders and signatures.
#### Reporters
- Standard output
- [HTML](http://downloads.arachni-scanner.com/dev/reports/report.html/)
([zip](http://downloads.arachni-scanner.com/dev/reports/report.html.zip)) (`html`).
- [XML](http://downloads.arachni-scanner.com/dev/reports/report.xml) (`xml`).
- [Text](http://downloads.arachni-scanner.com/dev/reports/report.txt) (`text`).
- [JSON](http://downloads.arachni-scanner.com/dev/reports/report.json) (`json`)
- [Marshal](http://downloads.arachni-scanner.com/dev/reports/report.marshal) (`marshal`)
- [YAML](http://downloads.arachni-scanner.com/dev/reports/report.yml) (`yaml`)
- [AFR](http://downloads.arachni-scanner.com/dev/reports/report.afr) (`afr`)
- [HTML](http://www.arachni-scanner.com/reports/report.html/)
([zip](http://www.arachni-scanner.com/reports/report.html.zip)) (`html`).
- [XML](http://www.arachni-scanner.com/reports/report.xml) (`xml`).
- [Text](http://www.arachni-scanner.com/reports/report.txt) (`text`).
- [JSON](http://www.arachni-scanner.com/reports/report.json) (`json`)
- [Marshal](http://www.arachni-scanner.com/reports/report.marshal) (`marshal`)
- [YAML](http://www.arachni-scanner.com/reports/report.yml) (`yaml`)
- [AFR](http://www.arachni-scanner.com/reports/report.afr) (`afr`)
- The default Arachni Framework Report format.
#### Plugins
......@@ -514,6 +521,9 @@ core remains lean and makes it easy for anyone to add arbitrary functionality.
which are within the scan scope.
- Headers collector (`headers_collector`) -- Collects response headers based on specified criteria.
- Exec (`exec`) -- Calls external executables at different scan stages.
- Metrics (`metrics`) -- Captures metrics about multiple aspects of the scan and the web application.
- Restrict to DOM state (`restrict_to_dom_state`) -- Restricts the audit to a single page's DOM
state, based on a URL fragment.
##### Defaults
......@@ -522,7 +532,6 @@ Default plugins will run for every scan and are placed under `/plugins/defaults/
- AutoThrottle (`autothrottle`) -- Dynamically adjusts HTTP throughput during the scan for
maximum bandwidth utilization.
- Healthmap (`healthmap`) -- Generates sitemap showing the health of each crawled/audited URL
- Resolver (`resolver`) -- Resolves vulnerable hostnames to IP addresses.
###### Meta
......@@ -594,5 +603,4 @@ need to follow in order to contribute code:
## License
Dual-licensed (Apache License v2.0/Commercial) -- please see the _LICENSE_ file
for more information.
Arachni Public Source License v1.0 -- please see the _LICENSE_ file for more information.
......@@ -22,7 +22,7 @@ Gem::Specification.new do |s|
s.homepage = 'https://www.arachni-scanner.com'
s.email = 'tasos.laskos@arachni-scanner.com'
s.authors = [ 'Tasos Laskos' ]
s.licenses = ['Apache-2.0', 'Proprietary']
s.licenses = ['Arachni Public Source License v1.0']
s.files += Dir.glob( 'config/**/**' )
s.files += Dir.glob( 'gfx/**/**' )
......@@ -78,8 +78,9 @@ Gem::Specification.new do |s|
# Printing complex objects.
s.add_dependency 'awesome_print', '~> 1.2.0'
# JSON reporter.
s.add_dependency 'json', '~> 1.8.1'
# Optimized JSON.
s.add_dependency 'oj', '~> 2.12.9'
s.add_dependency 'oj_mimic_json'
# For the Arachni console (arachni_console).
s.add_dependency 'rb-readline', '0.5.1'
......@@ -111,7 +112,7 @@ Documentation - http://arachni-scanner.com/wiki
Support - http://support.arachni-scanner.com
GitHub page - http://github.com/Arachni/arachni
Code Documentation - http://rubydoc.info/github/Arachni/arachni
License - Apache License v2.0/Proprietary
License - Arachni Public Source License v1.0
(https://github.com/Arachni/arachni/blob/master/LICENSE.md)
Author - Tasos "Zapotek" Laskos (http://twitter.com/Zap0tek)
Twitter - http://twitter.com/ArachniScanner
......@@ -123,9 +124,8 @@ or report a bug (via GitHub Issues) if you come across any problem.
MSG
s.description = <<DESCRIPTION
Arachni is an Open Source, feature-full, modular, high-performance Ruby framework
aimed towards helping penetration testers and administrators evaluate the security
of web applications.
Arachni is a feature-full, modular, high-performance Ruby framework aimed towards
helping penetration testers and administrators evaluate the security of web applications.
It is smart, it trains itself by monitoring and learning from the web application's
behavior during the scan process and is able to perform meta-analysis using a number of
......
......@@ -14,7 +14,7 @@
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
#
# @version 0.3.1
# @version 0.3.2
#
# @see http://cwe.mitre.org/data/definitions/94.html
# @see http://php.net/manual/en/function.eval.php
......@@ -32,7 +32,7 @@ class Arachni::Checks::CodeInjectionTiming < Arachni::Check::Base
php: 'sleep(__TIME__/1000);',
perl: 'sleep(__TIME__/1000);',
python: 'import time;time.sleep(__TIME__/1000);',
jsp: 'Thread.sleep(__TIME__);',
java: 'Thread.sleep(__TIME__);',
asp: 'Thread.Sleep(__TIME__);',
}.inject({}) do |h, (platform, payload)|
h[platform] = [ ' %s', ';%s', "\";%s#", "';%s#" ].map { |s| s % payload }
......@@ -53,7 +53,7 @@ a time delay.
},
elements: ELEMENTS_WITH_INPUTS,
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>',
version: '0.3.1',
version: '0.3.2',
platforms: payloads.keys,
issue: {
......
......@@ -140,7 +140,7 @@ class Arachni::Checks::CSRF < Arachni::Check::Base
audited( "#{url}::#{name}" )
log( vector: form )
log( vector: form, proof: form.source )
print_ok "Found unprotected form with name '#{name}' at '#{page.url}'"
end
......@@ -155,7 +155,7 @@ checks them for lack of anti-CSRF tokens.
},
elements: [ Element::Form ],
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com> ',
version: '0.3.4',
version: '0.3.5',
issue: {
name: %q{Cross-Site Request Forgery},
......
......@@ -9,7 +9,7 @@
# File inclusion check.
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
# @version 0.1.4
# @version 0.1.6
#
# @see http://cwe.mitre.org/data/definitions/98.html
# @see https://www.owasp.org/index.php/PHP_File_Inclusion
......@@ -21,22 +21,21 @@ class Arachni::Checks::FileInclusion < Arachni::Check::Base
regexp: {
unix: [
/DOCUMENT_ROOT.*HTTP_USER_AGENT/,
/(root|mail):.+:\d+:\d+:.+:[0-9a-zA-Z\/]+/im
/:.+:\d+:\d+:.+:[0-9a-zA-Z\/]+/im
],
windows: [
/\[boot loader\].*\[operating systems\]/im,
/\[fonts\].*\[extensions\]/im
],
tomcat: [
java: [
/<web\-app/im
],
# Generic PHP errors.
php: [
/An error occurred in script/,
/Failed opening '.*?' for inclusion/,
/Failed opening required/,
/failed to open stream:.*/,
/failed to open stream:/,
/<b>Warning<\/b>:\s+file/,
/<b>Warning<\/b>:\s+read_file/,
/<b>Warning<\/b>:\s+highlight_file/,
......@@ -81,7 +80,7 @@ class Arachni::Checks::FileInclusion < Arachni::Check::Base
'/windows/win.ini',
'/winnt/win.ini'
].map { |p| [p, "c:#{p}", "#{p}#{'.'* 700}", p.gsub( '/', '\\' ) ] }.flatten,
tomcat: [ '/WEB-INF/web.xml', '\WEB-INF\web.xml' ]
java: [ '/WEB-INF/web.xml', '\WEB-INF\web.xml' ]
}.inject({}) do |h, (platform, payloads)|
h.merge platform => payloads.map { |p| [p, "file://#{p}" ] }.flatten
end
......@@ -101,7 +100,7 @@ content or errors in the HTTP response body.
},
elements: ELEMENTS_WITH_INPUTS,
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com> ',
version: '0.1.4',
version: '0.1.6',
platforms: options[:regexp].keys,
issue: {
......
......@@ -9,7 +9,7 @@
# Simple OS command injection check.
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
# @version 0.2.4
# @version 0.2.5
#
# @see http://cwe.mitre.org/data/definitions/78.html
# @see https://www.owasp.org/index.php/OS_Command_Injection
......@@ -19,7 +19,7 @@ class Arachni::Checks::OsCmdInjection < Arachni::Check::Base
@options ||= {
regexp: {
unix: [
/(root|mail):.+:\d+:\d+:.+:[0-9a-zA-Z\/]+/im
/:.+:\d+:\d+:.+:[0-9a-zA-Z\/]+/im
],
windows: [
/\[boot loader\].*\[operating systems\]/im,
......@@ -73,7 +73,7 @@ Tries to find Operating System command injections.
},
elements: ELEMENTS_WITH_INPUTS,
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com> ',
version: '0.2.4',
version: '0.2.5',
platforms: payloads.keys,
issue: {
......
......@@ -9,7 +9,7 @@
# Path Traversal check.
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
# @version 0.4.4
# @version 0.4.6
#
# @see http://cwe.mitre.org/data/definitions/22.html
# @see https://www.owasp.org/index.php/Path_Traversal
......@@ -25,13 +25,13 @@ class Arachni::Checks::PathTraversal < Arachni::Check::Base
regexp: {
unix: [
/DOCUMENT_ROOT.*HTTP_USER_AGENT/,
/(root|mail):.+:\d+:\d+:.+:[0-9a-zA-Z\/]+/im
/:.+:\d+:\d+:.+:[0-9a-zA-Z\/]+/im
],
windows: [
/\[boot loader\].*\[operating systems\]/im,
/\[fonts\].*\[extensions\]/im
],
tomcat: [
java: [
/<web\-app/im
]
},
......@@ -58,9 +58,9 @@ class Arachni::Checks::PathTraversal < Arachni::Check::Base
end,
skip_like: proc do |m|
# Tomcat payloads begin with a traversal which won't be preserved
# Java payloads begin with a traversal which won't be preserved
# via LinkTemplate injections so don't bother.
m.is_a?( LinkTemplate ) && m.audit_options[:platform] == :tomcat
m.is_a?( LinkTemplate ) && m.audit_options[:platform] == :java
end
}
end
......@@ -90,7 +90,7 @@ class Arachni::Checks::PathTraversal < Arachni::Check::Base
h
end
@payloads[:tomcat] = [ '/../../', '../../', ].map do |trv|
@payloads[:java] = [ '/../../', '../../', ].map do |trv|
[ "#{trv}WEB-INF/web.xml", "file://#{trv}WEB-INF/web.xml" ]
end.flatten
......@@ -111,7 +111,7 @@ of relevant content in the HTML responses.
},
elements: ELEMENTS_WITH_INPUTS,
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com> ',
version: '0.4.4',
version: '0.4.6',
platforms: payloads.keys,
issue: {
......
......@@ -9,7 +9,7 @@
# HTTP Response Splitting check.
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
# @version 0.2.1
# @version 0.2.3
#
# @see http://cwe.mitre.org/data/definitions/20.html
# @see https://www.owasp.org/index.php/HTTP_Response_Splitting
......@@ -30,7 +30,12 @@ class Arachni::Checks::ResponseSplitting < Arachni::Check::Base
# and pass a block that will check for a positive result
audit( header, submit: { follow_location: false } ) do |response, element|
next if response.headers[header_name].to_s.downcase != 'no'
log vector: element, response: response
log(
vector: element,
response: response,
proof: response.headers_string[/#{header_name}.*$/i]
)
end
end
......@@ -42,7 +47,7 @@ Injects arbitrary and checks if any of them end up in the response header.
},
elements: ELEMENTS_WITH_INPUTS,
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com> ',
version: '0.2.1',
version: '0.2.3',
issue: {
name: %q{Response Splitting},
......@@ -68,7 +73,7 @@ other attacks.
},
tags: %w(response splitting injection header),
cwe: 20,
severity: Severity::MEDIUM,
severity: Severity::HIGH,
remedy_guidance: %q{
It is recommended that untrusted data is never used to form the contents of the
response header.
......
......@@ -20,7 +20,7 @@
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
#
# @version 0.1.1
# @version 0.1.2
class Arachni::Checks::SessionFixation < Arachni::Check::Base
def token
......@@ -48,7 +48,11 @@ class Arachni::Checks::SessionFixation < Arachni::Check::Base
select { |c| c.name == name }.first
next if !cookie || !cookie.value.include?( token )
log vector: element, response: response
log(
vector: element,
response: response,
proof: cookie.source
)
end
end
end
......@@ -62,7 +66,7 @@ Checks whether or not the session cookie can be set to an arbitrary value.
},
elements: [ Element::Form, Element::Link, Element::LinkTemplate ],
author: 'Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>',
version: '0.1.1',
version: '0.1.2',
issue: {
name: %q{Session fixation},
......
......@@ -11,7 +11,7 @@
# source code.
#
# @author Tasos "Zapotek" Laskos <tasos.laskos@arachni-scanner.com>
# @version 0.2.2
# @version 0.2.3
#
# @see http://cwe.mitre.org/data/definitions/540.html
class Arachni::Checks::SourceCodeDisclosure < Arachni::Check::Base
......@@ -20,15 +20,15 @@ class Arachni::Checks::SourceCodeDisclosure < Arachni::Check::Base
@options ||= {