Commit 39435bfd authored by Sophie Brun's avatar Sophie Brun

New upstream version 1.0+git20180309

parent 63e9b6f0
Please provide the following details.
### Host System
- OS :
- Python version (`python --version`) :
### Error Description
Please provide the details of the error. Try to provide the **output** and also **steps to reproduce** (if possible).
If you cloned Datasploit, also provide the output of `git log -n 1 --pretty=format:"%B"`.
......@@ -82,4 +82,5 @@ target/
.ipynb_checkpoints
profile_pic/
reports/json/
[![ToolsWatch Best Tools](https://www.toolswatch.org/badges/toptools/2016.svg)](https://www.toolswatch.org/2013/12/2013-top-security-tools-as-voted-by-toolswatch-org-readers/)
[![ToolsWatch Best Tools](https://www.toolswatch.org/badges/toptools/2016.svg)](http://www.toolswatch.org/2017/02/2016-top-security-tools-as-voted-by-toolswatch-org-readers/)
[![Black Hat Arsenal](https://www.toolswatch.org/badges/arsenal/2016.svg)](https://www.blackhat.com/us-16/arsenal.html#datasploit) US
[![Arsenal-2017-EU](https://rawgit.com/toolswatch/badges/master/arsenal/2017.svg)](http://www.toolswatch.org/2017/09/black-hat-arsenal-europe-2017-lineup/) - EUROPE
[![Black Hat Arsenal](https://www.toolswatch.org/badges/arsenal/2016.svg)](https://www.blackhat.com/us-16/arsenal.html#datasploit) EU
[![Arsenal-2017-US](https://rawgit.com/toolswatch/badges/master/arsenal/2017.svg)](http://www.toolswatch.org/2017/06/the-black-hat-arsenal-usa-2017-phenomenal-line-up-announced/) - USA
[![Arsenal-2017-ASIA](https://rawgit.com/toolswatch/badges/master/arsenal/2017.svg)](http://www.toolswatch.org/2017/02/the-black-hat-arsenal-asia-2017-great-line-up/) - ASIA
[![Arsenal-2016-EU](https://www.toolswatch.org/badges/arsenal/2016.svg)](http://www.toolswatch.org/2016/09/the-black-hat-arsenal-europe-2016-line-up/) - EUROPE
[![Arsenal-2016-US](https://www.toolswatch.org/badges/arsenal/2016.svg)](http://www.toolswatch.org/2016/06/the-black-hat-arsenal-usa-2016-remarkable-line-up/) - USA
[![DEFCON 25 ReconVillage](https://img.shields.io/badge/DEFCON%2025-Recon%20Village-red.svg)](http://reconvillage.org/) [![DEFCON 24 Demolabs](https://img.shields.io/badge/DEFCON%2024-Demo%20Labs-red.svg)](https://www.defcon.org/html/defcon-24/dc-24-demolabs.html)
[![Join Datasploit Slack](https://img.shields.io/badge/slack-open-e01563.svg)](http://datasploit.slack.com "Join our Slack community")
[![Follow Datasploit on Twitter](https://img.shields.io/twitter/follow/datasploit.svg?style=social&label=Follow%20%40datasploit)](https://twitter.com/intent/user?screen_name=datasploit "Follow Datasploit on Twitter")
# Overview of the tool:
* Performs OSINT on a domain / email / username / phone and find out information from different sources.
* Correlates and collaborate the results, show them in a consolidated manner.
* Correlate and collaborate the results, show them in a consolidated manner.
* Tries to find out credentials, api-keys, tokens, subdomains, domain history, legacy portals, etc. related to the target.
* Use specific script / launch automated OSINT for consolidated data.
* Performs Active Scans on collected data.
......@@ -39,9 +48,17 @@ Options:
# Required Setup:
* Python 2.7 (because bunch of dependencies do not support Python 3.0)
* Bunch of python libraries (use requirements.txt)
* In **Kali Linux**, please install the requirements using the command `pip install --upgrade --force-reinstall -r requirements.txt`
## Detailed Tool Documentation:
> [http://datasploit.readthedocs.io/en/latest/](http://datasploit.readthedocs.io/en/latest/)
> [https://datasploit.github.io/datasploit/](https://datasploit.github.io/datasploit/)
### Lead Developers
* Shubham Mittal - https://github.com/upgoingstar
* Sudhanshu Chauhan - https://github.com/sudhanshuc
* Kunal Aggarwal - https://github.com/KunalAggarwal
### Social Media
* [Official Website](https://datasploit.github.io/datasploit/)
* [Twitter](https://twitter.com/datasploit)
* [Facebook](https://facebook.com/datasploit)
File mode changed from 100644 to 100755
File mode changed from 100644 to 100755
#Store all your config's here.
#added to gitignore so will not be syned
#Backup in ~/Desktop/config.py_backup
shodan_api=""
bing_api=""
github_access_token=""
......@@ -7,28 +10,35 @@ builtwith_api=""
censysio_id=""
censysio_secret=""
facebook_access_token = ""
google_cse_key= ""
google_cse_cx = ""
flickr_api=""
google_api=""
google_cse_key=""
google_cse_cx = ""
hashes_api=""
instagram_api=""
instagram_secret=""
ipinfodb_api=""
jigsaw_api=""
jigsaw_password=""
jigsaw_username=""
linkedin_api=""
linkedin_secret=""
pwnedlist_api=""
pwnedlist_iv=""
pwnedlist_secret=""
reddit_id = ""
reddit_secret = ""
spyonweb_access_token = ""
twitter_consumer_key=""
twitter_consumer_secret=""
twitter_access_token = ""
twiter_access_token_secret = ""
twitter_access_token_secret = ""
zoomeyeuser = ""
zoomeyepass = ""
clearbit_apikey = ""
emailhunter=""
jsonwhois=""
instagram_token = ""
instagram_client_id = ""
instagram_client_secret = ""
fullcontact_api = ""
mailboxlayer_api = ""
virustotal_public_api = ""
virustotal_public_api =""
upgoingstar
nutanpanda
sudhanshu_c
kunalaggarwal92
\ No newline at end of file
aggkunal
#!/usr/bin/env python
import dep_check
dep_check.check_dependency()
import re
import sys
import optparse
import shutil
import os
import textwrap
import argparse
import emailOsint
import domainOsint
import ipOsint
import usernameOsint
from tld import get_tld
from netaddr import IPAddress,AddrFormatError
parser = optparse.OptionParser()
parser.add_option('-a', '--active', action="store", dest="domain", help="Launches Active Scans (work in progress)",
default="spam")
options, args = parser.parse_args()
def printart():
print "\t "
print "\t ____/ /____ _ / /_ ____ _ _____ ____ / /____ (_)/ /_"
print "\t / __ // __ `// __// __ `// ___// __ \ / // __ \ / // __/"
print "\t / /_/ // /_/ // /_ / /_/ /(__ )/ /_/ // // /_/ // // /_ "
print "\t \__,_/ \__,_/ \__/ \__,_//____// .___//_/ \____//_/ \__/ "
print "\t /_/ "
print "\t "
print "\t Open Source Assistant for #OSINT "
print "\t Website: www.datasploit.info "
print "\t "
def main(user_input):
printart()
print "User Input: %s" % user_input
if re.match('[^@]+@[^@]+\.[^@]+', user_input):
print "Looks like an EMAIL, running emailOsint...\n"
emailOsint.run(user_input)
elif re.match('^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', user_input):
print "Looks like an IP, running ipOsint...\n"
ipOsint.run(user_input)
elif re.match('^[a-zA-Z\d-]{,63}(\.[a-zA-Z\d-]{,63}).$', user_input):
print "Looks like a DOMAIN, running domainOsint...\n"
domainOsint.run(user_input)
else:
print "Looks like a Username, running usernameOsint...\n"
usernameOsint.run(user_input)
def main(argv):
output=None
desc="""
____/ /____ _ / /_ ____ _ _____ ____ / /____ (_)/ /_
/ __ // __ `// __// __ `// ___// __ \ / // __ \ / // __/
/ /_/ // /_/ // /_ / /_/ /(__ )/ /_/ // // /_/ // // /_
\__,_/ \__,_/ \__/ \__,_//____// .___//_/ \____//_/ \__/
/_/
if __name__ == "__main__":
Open Source Assistant for #OSINT
www.datasploit.info
"""
epilog=""" Connect at Social Media: @datasploit
"""
# Set all parser arguments here.
parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter,description=textwrap.dedent(desc),epilog=epilog)
parser.add_argument("-i","--input",help="Provide Input",dest='single_target')
parser.add_argument("-f","--file",help="Provide Input",dest='file_target')
parser.add_argument("-a","--active",help="Run Active Scan attacks",dest='active',action="store_false")
parser.add_argument("-q","--quiet",help="Run scans in automated manner accepting default answers",dest='quiet',action="store_false")
parser.add_argument("-o","--output",help="Provide Destination Directory",dest='output')
# check and ensure the config file is present otherwise create one. required for all further operations
ds_dir=os.path.dirname(os.path.realpath(__file__))
config_file_path = os.path.join(ds_dir,"config.py")
config_sample_path= os.path.join(ds_dir,"config_sample.py")
print os.path.exists(config_file_path)
if not os.path.exists(config_file_path):
print "[+] Looks like a new setup, setting up the config file."
shutil.copyfile(config_sample_path,config_file_path)
print "[+] A config file is added please follow guide at https://datasploit.github.io/datasploit/apiGeneration/ to fill API Keys for better results"
# We can think about quiting at this point.
# parse arguments in case they are provided.
x=parser.parse_args()
active=x.active
quiet=x.quiet
single_input=x.single_target
file_input=x.file_target
output=x.output
# if no target is provided print help and quit.
if not (single_input or file_input):
print "\nSingle target or file input required to run\n"
parser.print_help()
sys.exit()
# Banner print
print textwrap.dedent(desc)
if single_input:
try:
auto_select_target(single_input, output)
except KeyboardInterrupt:
print "\nCtrl+C called Quiting"
if file_input:
try:
if os.path.isfile(file_input):
print "File Input: %s" % file_input
with open(file_input, 'r') as f:
for target in f:
auto_select_target(target.rstrip(), output)
print "\nDone processing %s" % file_input
else:
print "%s is not a readable file" % file_input
print "Exiting..."
except KeyboardInterrupt:
print "\nCtrl+C called Quiting"
def auto_select_target(target, output=None):
"""Auto selection logic"""
print "Target: %s" % target
try:
user_input = sys.argv[1]
inp=IPAddress(target);
if inp.is_private() or inp.is_loopback():
print "Internal IP Detected : Skipping"
sys.exit()
else:
print "Looks like an IP, running ipOsint...\n"
ipOsint.run(target, output)
except SystemExit:
print "exiting"
except AddrFormatError:
if re.match('[^@]+@[^@]+\.[^@]+', target):
print "Looks like an EMAIL, running emailOsint...\n"
emailOsint.run(target, output)
elif get_tld(target, fix_protocol=True,fail_silently=True) is not None:
print "Looks like a DOMAIN, running domainOsint...\n"
domainOsint.run(target, output)
else:
print "Nothing Matched assuming username, running usernameOsint...\n"
usernameOsint.run(target, output)
except:
print "\n[-] Invalid Input. Exiting now..\n"
sys.exit(0)
main(user_input)
print "Unknown Error Occured"
if __name__ == "__main__":
main(sys.argv[1:])
import pip
import sys
def check_dependency():
list_deps = []
missing_deps = []
with open('requirements.txt') as f:
list_deps = f.read().splitlines()
pip_list = sorted([(i.key) for i in pip.get_installed_distributions()])
for req_dep in list_deps:
if req_dep not in pip_list:
missing_deps.append(req_dep)
if missing_deps:
print "You are missing a module for Datasploit. Please install them using: "
print "pip install -r requirements.txt"
sys.exit()
......@@ -11,19 +11,9 @@ python domain_subdomains.py example.com
```
While, domainOsint will call all other domain_* scripts and list down data as well as dump the same in mongoDb, domain_subdomains and other such scripts will just list down data specific to their function.
domainOsint.py generates a JSON and an HTML report in reports folder as following hirarchy (example files are based on abcd.com domain)
../datasploit/reports
---------------------|------abcd.com
---------------------|------|------abcd.com_YYYY-MM-DD-HH-MM-SS.html
---------------------|------|------abcd.com_YYYY-MM-DD-HH-MM-SS.json
---------------------|------|------abcd.com_YYYY-MM-DD-HH-MM-SS.subdomains.txt
---------------------|------|------abcd.com_YYYY-MM-DD-HH-MM-SS.emails.txt
domainOsint.py generates a JSON report in reports folder as following hirarchy (example files are based on abcd.com domain)
```
python domainOsint.py -d example.com -o json
```
Writing custom modules/scripts for dataSploit
=============================================
dataSploit has been made with a modular approach in mind. We wanted to make things simple for even a novice user to understand. Thus, writing a new module is as simple as adding a new script to a module folder and that's it. dataSploit takes care of the rest. There are a few simple guidelines to follow when writing a new module/script. Please read the documentation below to get a better understanding of how dataSploit is structured.
Following is the tree structure of a basic git clone of the dataSploit repository:
```bash
datasploit/
├── active_default_file_check.py
├── active_scan.py
├── base.html
├── check_urls.txt
├── config.py
├── contributors.txt
├── datasploit_config.py
├── datasploit.py
├── docs
│ ├── apiGeneration.md
│ ├── contributors.md
│ ├── home.md
│ ├── index.md
│ ├── setupGuide.md
│ └── Usage.md
├── domain
│ ├── base.py
│ ├── domain_censys.py
│ ├── domain_checkpunkspider.py
│ ├── domain_dnsrecords.py
│ ├── domain_emailhunter.py
│ ├── domain_forumsearch.py
│ ├── domain_github.py
│ ├── domain_GooglePDF.py
│ ├── domain_history.py
│ ├── domain_pagelinks.py
│ ├── domain_pastes.py
│ ├── domain_shodan.py
│ ├── domain_subdomains.py
│ ├── domain_wappalyzer.py
│ ├── domain_whois.py
│ ├── domain_wikileaks.py
│ ├── domain_zoomeye.py
│ ├── __init__.py
│ └── template.py
├── domainOsint.py
├── emailOsint.py
├── emails
│ ├── base.py
│ ├── email_basic_checks.py
│ ├── email_clearbit.py
│ ├── email_fullcontact.py
│ ├── email_haveibeenpwned.py
│ ├── email_pastes.py
│ ├── email_scribd.py
│ ├── email_slideshare.py
│ ├── email_whoismind.py
│ ├── __init__.py
│ └── template.py
├── __init__.py
├── ip
│ ├── base.py
│ ├── __init__.py
│ ├── ip_shodan.py
│ ├── ip_virustotal.py
│ ├── ip_whois.py
│ └── template.py
├── ipOsint.py
├── LICENSE
├── mkdocs.yml
├── osint_runner.py
├── README.md
├── reports
│ └── README
├── requirements.txt
├── roadmap.txt
├── username
│ ├── base.py
│ ├── __init__.py
│ ├── template.py
│ ├── username_gitscrape.py
│ ├── username_gituserdetails.py
│ ├── username_keybase.py
│ ├── username_profilepic.py
│ ├── username_twitterdetails.py
│ └── username_usernamesearch.py
└── usernameOsint.py
```
Out of the box, 4 main modules have been provided, as can be seen from the tree structure above, namely:
1. domain
2. emails
3. ip
4. username
Each of these folder houses scripts of it's own kind, i.e., scripts working on domain name as input are located in the domain folder and so on. You may have also noticed, each script in a module has a naming convention, i.e., a common prefix followed by an underscore and followed by the script name (e.g., domain_shodan.py). Please note this is required to be followed when adding a new script to the module. This way, dataSploit can dynamically pick up the script at runtime when triggered from the module's Osint script.
### Writing a new script for an existing module
To write a new script for a module, there is a `template.py` located in each module directory to help you get started quickly. Following is the contents of the template.py file in the domain module:
```python
#!/usr/bin/env python
import base
import config as cfg
import sys
# Control whether the module is enabled or not
ENABLED = True
def banner():
# Write a cool banner here
pass
def main(domain):
# Use the domain variable to do some stuff and return the data
print domain
return []
def output(data, domain=""):
# Use the data variable to print out to console as you like
for i in data:
print i
if __name__ == "__main__":
try:
domain = sys.argv[1]
banner()
result = main(domain)
output(result, domain)
except Exception as e:
print e
print "Please provide a domain name as argument"
```
In short there are 3 functions that need to be implemented for a script:
1. `def banner()`
This function is an optional implementation, used only to print out the banner at the start of the script when executed as standalone tool.
2. `def main(input)`
This function is a mandatory implementation. The parameter input to this function is what is passed as command line argument to the scripts. All data processing needs to be done in this function and it needs to return the data from this function. Please note, it is advised not to output anything in this function. The return value from here gets passed onto the next function for display.
3. `def output(data, input="")`
This function is another mandatory implementation. It is used to handle how the data returned by the main function above is to be rendered on the console. The parameter data is the returned data from the mail function. The parameter input is an optional parameter and holds the value of the command line argument passed to the script.
There is also another important variable in the script towards the top named `ENABLED`. By default, even in the template ENABLED is set to True. This variable can be used like a switch to control whether the script gets picked up for execution when running using either the parent datasploit.py script or either one of the Osint.py scripts.
Once you are done with modifying the template.py file and finalizing your code, you need to simply rename the file using the predefined format, i.e., the module prefix then an underscore and then the script name. For example, writing a new script for the domain module, the file needs to be renamed as `domain_scriptname.py`. Once this is done, the script will get automatically picked by datasploit.py or domainOsint.py.
### Adding a new module to dataSploit
Adding a new module is also pretty straight forward. For example, let's say we want to add a module named mobile to dataSploit. Given below are the steps to add a new module.
1. Create a directory in the datasploit folder named mobile and move to the newly created directory.
```bash
mkdir mobile
cd mobile
```
2. In this new directory, create a file named `base.py`
```bash
touch base.py
vi base.py
```
And add the following contents to the file
```python
import sys
import os
dir_path = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
sys.path.insert(0, dir_path)
```
3. Also in the same directory, create another file called `__init__.py`. This might be a good time to decide a prefix that will be used to name scripts in this module, for this document purpose we'll be picking the prefix name mobile.
```bash
touch __init__.py
vi __init__.py
```
And add the following contents to the file:
```python
from os.path import dirname, basename, isfile, abspath
import glob, importlib, sys
modules = glob.glob(dirname(__file__) + "/mobile_*.py")
__all__ = [basename(f)[:-3] for f in modules if isfile(f)]
sys.path.append(dirname(abspath(__file__)))
for m in __all__:
__import__(m, locals(), globals())
del m, f, dirname, basename, isfile, abspath, glob, importlib, sys, modules
```
*Please note*: Line number 4:
```python
modules = glob.glob(dirname(__file__) + "/mobile_*.py")
```
This is where the prefix of the script name comes into play. Please change this accordingly to whatever prefix you decide.
4. Assuming you're in the mobile folder created above, go one level up to the datasploit folder and create a file called `mobileOsint.py`. This script will be used to execute all scripts in the module mobile as a consolidated package.
```bash
cd ..
touch mobileOsint.py
vi mobileOsint.py
```
Add the following contents to the file.
```python
#!/usr/bin/env python
import sys
import osint_runner
def run(email):
osint_runner.run("mobile", "mobile", mobile)
if __name__ == "__main__":
mobile = sys.argv[1]
run(mobile)
```
Take note of the osint_runner.run() function inside the run function. The first parameter is the prefix name of the scripts. The second parameter is the name of the module directory.
Scripts can be added to this folder as mentioned in the guide above to create new script for an existing module.
5. The last step is adding the newly created module to the datasploit.py script. For that, simply edit the datasploit.py file and first add an import to the top:
```python
import mobileOsint
```
Then in the main function, add the handler to identify the user input and basis of that add the following line of code to call the mobileOsint module:
```python
mobileOsint.run(user_input)
```
That's all. This configures the new mobile module to either run as a whole using the datasploit.py or mobileOsint.py file or as standalone scripts using the mobile_scriptname.py files inside the mobile folder.
The possibilities of extending dataSploit are endless. New modules and scripts are easily integrable as mentioned above. We look forward to seeing contribution from the community to help increase the capabilites of dataSploit.
\ No newline at end of file
We need following API keys to run this tool efficiently:
shodan_api, censysio_id, censysio_secret, zoomeyeuser, zoomeyepass, clearbit_apikey, emailhunter, fullcontact, google_cse_key, google_cse_cx.
shodan_api, censysio_id, censysio_secret, zoomeyeuser, zoomeyepass, clearbit_apikey, emailhunter, fullcontact, google_cse_key, google_cse_cx, github_travis_key.
## Shodan_api
* [Register](https://account.shodan.io/register) an account in shodan.
......@@ -49,6 +49,17 @@ shodan_api, censysio_id, censysio_secret, zoomeyeuser, zoomeyepass, clearbit_api
* Go to https://cse.google.com/cse/all again and click on the search engine you just created.
* Click on the 'Search engine id' button and copy your search engine id. This is the value for *google_cse_cx* field in config.py file.
## SpyOnWeb Access Token
* [Register](https://api.spyonweb.com/users/sign_up) an account in SpyOnWeb.
* SpyOnWeb will send a confirmation email with a link you will need to use to confirm the account.
* [Login](https://api.spyonweb.com/users/sign_in) and your Access Token will be on the main page.
* Copy the Access Token and use as the value for the *spyonweb_access_token* field in the config.py file.
## Reddit API
* [Register](https://www.reddit.com/login) for an account on Reddit.
* Go [Here](https://www.reddit.com/prefs/apps/) and click the *create app* button at the bottom.
* Name the app whatever you want, select *script* type, for a redirect uri (if needed) you can use anything (eg. `http://www.example.com/unused/redirect/uri`).
* Copy the *personal use script ID* to `reddit_id` and *secret* to `reddit_secret`.
## Zoomeye Username and Password
* [Register](https://www.zoomeye.org/accounts/register) an user with zoomeye and use the credentials for this tool. (Don't worry if you are redirected to sso.telnet404.com. *This is how it works.)*
......@@ -63,3 +74,9 @@ shodan_api, censysio_id, censysio_secret, zoomeyeuser, zoomeyepass, clearbit_api
* Now your account is activated and use those credentials in the tool.
* Email ID which you have used to sign up is your username and is the value for *zoomeyeuser* field in config.py
* Your account password is the value for *zoomeyepass* field in the config.py
## Travis CI Access Token
* [Register](https://github.com/join?source=header-home) an account with Github.
* Login into Github (https://github.com/login) and go to the page 'New personal access token' (https://github.com/settings/tokens/new).
* Generate a Github access token based on the instructions present at https://travispy.readthedocs.io/en/stable/getting_started/.
* Copy the Access Token and use as the value for the *github_access_token* field in the config.py file.
......@@ -16,9 +16,6 @@ Following API configs are mandatory for proper results in domainOsint.py:
Other modules:
* github_access_token
* instagram_token
* instagram_client_id
* instagram_client_secret
* jsonwhois
......
## Overview
* Performs automated OSINT on a domain / email / username / phone and find out relevant information from different sources.
* Useful for Pen-testers, Cyber Investigators, Product companies, defensive security professionals, etc.
* Correlates and collaborate the results, show them in a consolidated manner.
* Tries to find out credentials, api-keys, tokens, subdomains, domain history, legacy portals, etc. related to the target.
* Available as single consolidating tool as well as standalone scripts.
* Performs Active Scans on collected data.
* Generates HTML, JSON reports along with text files.
* Performs automated OSINT on a domain / email / username / IP and find out relevant information from different sources.
* Easy to contribute OSINT Framework.
* Code for Banner, Main and Output function. Datasploit automagically do rest of the things for you.
* Useful for Pen-testers, Bug Bounty Hunters, Cyber Investigators, Product companies, Security Engineers, etc.
* Collaborate the results, show them in a consolidated manner.
* Tries to find out credentials, api-keys, tokens, subdomains, domain history, legacy portals, usernames, dumped accounts, etc. related to the target.
* Can be used as library, automated scripts or standalone scripts.
* Can generate lists which can be feeded to active scan tools.
* Generates HTML, along with text files.
## Why DataSploit???
Irrespective of whether you are attacking a target or defending one, you need to have a clear picture of the threat landscape before you get in. This is where DataSploit comes into the picture. Utilizing various Open Source Intelligence (OSINT) tools and techniques that we have found to be effective, DataSploit brings them all into one place, correlates the raw data captured and gives the user, all the relevant information about the domain / email / phone number / person, etc. It allows you to collect relevant information about a target which can expand your attack/defence surface very quickly. Sometimes it might even pluck the low hanging fruits for you without even touching the target and give you quick wins. Of course, a user can pick a single small job (which do not correlates obviously), or can pick up the parent search which will launch a bunch of queries, call other required scripts recursively, correlate the data and give you all juicy information in one go.
Irrespective of whether you are attacking a target or defending one, you need to have a clear picture of the threat landscape before you get in. This is where DataSploit comes into the picture. Utilizing various Open Source Intelligence (OSINT) tools and techniques that we have found to be effective, DataSploit brings them all into one place, correlates the raw data captured and gives the user, all the relevant information about the domain / email / IP / person, etc. It allows you to collect relevant information about a target which can expand your attack/defence surface very quickly. Sometimes it might even pluck the low hanging fruits for you without even touching the target and give you quick wins. Of course, a user can pick a single small job (which do not correlates obviously), or can pick up the parent search which will launch a bunch of queries, call other required scripts recursively, correlate the data and give you all juicy information in one go.
## Tool Background
Created using our beloved Python, DataSploit simply requires the bare minimum data (such as domain name, email ID, person name, etc.) before it goes out on a mining spree. Once the data is collected, firstly the noise is removed, after which data is correlated and after multiple iterations it is stored locally in a database which could be easily visualised on the UI provided. The sources that have been integrated are all hand picked and are known to be providing reliable information. We have used them previously during different offensive as well as defensive engagements and found them helpful.
Apart from being a tool, DataSploit is an easy to use OSINT framework where people can contribute other modules and help the community with cutting edge researches.
People can either write modules for DataSploit or can simpley import datasploit as library and write their own tools.
## Setup
Worried about setup? Well, there are two major requirements here:
Worried about setup? We got you. You should be worried about two things:
* Setting up the db, django, libraries, etc. We will soon have a script which will automate this for you, so can just go ahead and shoot the OSINT job.
* Install the required python dependencies. Either use requirements.txt or simpley pip install datasploit.
* Feeding specific API keys for few specific sources. We are going to have a knowledge base where step by step instructions to generate these API keys will be documented. Sweet deal?
* [Click here to check step by step setup guide](/setupGuide/)
## Roadmap
Apart from this, in order to make it more useful in daily life of a pen-tester, we are working to make the tool as an extension of the other tools that pen-testers commonly use such as Burp Suite, Maltego etc. so that you can feel at home during the usage.
1. While focusing on new modules, we didnt paid much attention to Exception handling. This is something we are keen to work on. Any contibutors? We already love you.
2. Apart from this, in order to make it more useful in daily life of a pen-tester, we are working to active scripts which can get data from datasploit and use the OSINT data agregated from multiple sources.
We are always in process of improving Datasploit as an OSINT framework in order to serve its sole objective, help people in their jobs and make their life easier. We realized there were some issues in installation and excpetion handling and we have resolved many of them. Having said that, we still have a lot more work to do. There is surely a large number of things which we think can be improved/added in datasploit.
Following are few of the things we are planning to work on, in near future. If you think you have some idea/suggestions, please feel free to reach us on our Slack Channel [https://datasploit.slack.com](https://datasploit.slack.com). (If you are not registered, you can use http://selfinvite.datasploit.info).
And if you think you can pick any task out of the following features/sources/enhancements, Well, what could be better? Please do send in your Pull Requests. Cheers.
### Features/Enhancement(s):
1. Check API keys before executing any module. This should be a framework level check.
2. JSON/Txt outputs for every module.
3. Visualization for the data that comes in. This could be done either in JPG images locally stored. Or setup a front end with Django/Flask/etc.
4. Option to run DataSploit against a file containing a list of emails/domains/subdomains/usernames/etc. This will be an optional switch that will take file as --filename (-f) option.
5. Dump data in Sqlite3 (keys as one column and whole json document as value), and explore this option to be used as database. Our users earlier faced a lot of problems while setting up MongoDB.
### New Source(s):
1. Locate files for a domain on search engines like Google, Yahoo etc. This could be passed to metadata_extractor module.
2. Fetch Twitter Sleeping time of a user. More Twitter OSINT is possible.
3. Add https://urlquery.net
4. Extract files, metadata and enumerate information from metadata
5. More sources for Username Enumeration : Include WhatsMyName by @WebBreacher
6. Module to find All Websites hosted on a domain's server.
7. Add AbuseIPDb
8. Reverse Image Search
9. Search on Darknet.
10. Company OSINT. Use Open-Corporates for this. User can pass a keyword for which a list of companies (with few details like location, Year of Est., etc.) matching the keyword will be listed. User needs to select the company he/she is targeting for complete search.
\ No newline at end of file
This page holds the setup guide you will need before kicking off the datasploit in your system. Please note that all the documentation is as per *nix machines, and the tool has not been thoroughly tested on Windows platform. If you would like to volunteer for the same, give us a shout at helpme@datasploit.info. Following are the quick steps to get you going:
dataSploit as a Framework / Tool
================================
If you want to work with web gui, follow the steps till 7. Otherwise, follow till 5th and you should be good to go.
dataSploit is now available as a framework and can be used in 2 ways, either as a library such that it can be incorporated into other projects or as a standalone tool. Given below are the methods to install and use dataSploit in the mentioned 2 ways.
### Step 1 - Download DataSploit to your system.
## 1. Set it up as a Library
You can either use the git command line tools using the following command:
```
git clone https://github.com/datasploit/datasploit.git
```
, or you can simply download the zip file *([link](https://github.com/datasploit/datasploit/archive/master.zip))* and extract the same using unzip.
```
unzip master.zip
dataSploit is now available on the [pypi repository](https://pypi.python.org/pypi/datasploit) for easy installation using pip. If you're a developer looking for a way to leverage the capabilities of dataSploit in your own projects, this method is probably the way to go for you. To use dataSploit as a library to incorporate in your own projects, please follow the below instructions to set it up.
```bash
# pip install datasploit
```