Commit a7e262b1 authored by Devon Kearns's avatar Devon Kearns

Imported Upstream version 0.4.5

parents
The MIT License
Copyright (c) 2010 Craig Heffner
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
DESCRIPTION
Binwalk is a tool for searching a given binary image for embedded file types. Specifically,
it was designed for identifying files embedded inside of firmware images. Binwalk file signatures
are compatible with the magic signatures used by the Unix file utility.
Binwalk includes a custom magic signature file, 'magic.binwalk'. This file contains improved
signatures for files that are commonly found in firmware images such as compressed/archived files,
Linux kernels, bootloaders, filesystems, etc.
Since version 0.3.3 an additional option, -C, is included. Specifying this option displays the
value of each file offset in various data types (long, short, date, etc), as defined in 'magic.bincast'.
This is useful for identifying header fields such as date and length values.
Since version 0.3.8 an additional option, -A, is included. This option scans the specified file(s) for
executable code by searching for opcodes associated with the function prologues/epiloges of various
architectures. These opcode signatures are defined in 'magic.binarch'.
USAGE
The only required options to Binwalk are the file(s) that you want to search:
$ binwalk firmware1.bin firmware2.bin firmware3.bin
By default binwalk will include short signatures for gzip, lzma and jffs2 file fomats, and exclude
invalid results. These default filters can be disabled with the -d option, which will speed up the
scan time but may cause binwalk to miss gzip, lzma or jffs2 files:
$ binwalk -d firmware.bin
If searching for specific files, the scan time can be significantly improved by specifying the -y
option. The -y option only searches for signatures that match the specified string(s):
$ binwalk -y jffs2 firmware.bin
$ binwalk -y jffs2 -y cramfs firmware.bin
By default binwalk will use the signatures from the binwalk.magic file, but you may specify any other
libmagic-compatible signature file with the -m option. Note that for full maigc file compatibility,
you must specify the -s option to disable 'smart' matching:
$ binwalk -m /usr/share/misc/magic -s firmware.bin
By default binwalk will check for valid file signatures anywhere in the target file. This means that
scanning a 4MB file is the equivalent of running the Unix file utility 4 million times. To
decrease scan time, you may specify the byte alignment via the -b option. If, for example,
you specify a byte alignment of 16, then binwalk will assume that everything in the file is
16-byte aligned and will only look for signatures every 16 bytes:
$ binwalk -b 16 firmware.bin
You may also specify at what offset into the firmware image to start searching, and how many
bytes should be searched. The following command searches 1000 bytes of data starting at an offset
of 100:
$ binwalk -o 100 -l 1000 firmware.bin
All integer arguments, such as -o, and -l, can be entered as decimal (ex: 16) or hexadecimal
(ex: 0x10, \x10, 10H, 10h) values.
By default, all magic signatures that are only two bytes long are ignored as they have a high
rate of false positive matches. To include these magic signatures, specify the -a option:
$ binwalk -a firmware.bin
By default, binwalk will apply several default filters in order to improve scan reliability.
These filters can be explicitly disabled with the -d option:
$ binwalk -d firmware.bin
You can also include individual signatures from the default exclude list with the -i option:
$ binwalk -i gzip firmware.bin
Include and exclude filters may also be specified in order to limit the search results. Multiple
include / exclude filters may be specified, and are case insensitive. If an include filter is specified,
only descriptions that match that filter will be displayed. If an exclude filter is specified, all
results will be displayed except those that match the exclude filter. If both exclude and include
filters are specified, exclude filters trump include filters.
Only search for gzip results:
$ binwalk -y gzip firmware.bin
Search for everything except results that contain the string 'system':
$ binwalk -x system firmware.bin
Search only for results that are file systems, but that are not JFFS2 file systems:
$ binwalk -y filesystem -x jffs2 firmware.bin
To update to the latest magic file definitions, use the -u option:
# binwalk -u
Some scans can take some time to complete and may not display many results during this time.
You can press the enter key at any time to force binwalk to display its current scan progress:
$ binwalk -v firmware.bin
Scan Time: Dec 09, 2011 @ 18:00:42
Magic File: /usr/local/etc/binwalk/magic.binwalk
Signatures: 76
Target File: firmware.bin
MD5 Checksum: 1c802dbacdcfc0b96b900f8680d9d196
DECIMAL HEX DESCRIPTION
------------------------------------------------------------------------------------------
<Enter>
Progress: 1595 / 12074736 (0.01%)
<Enter>
Progress: 8015 / 12074736 (0.07%)
<Enter>
Progress: 12424 / 12074736 (0.10%)
INSTALLATION
To build and install binwalk, run:
$ ./configure
$ make
# make install
DEPENDENCIES
Binwalk is currently supported on the Linux and Mac OSX platforms.
To build from source, you must have the libmagic, zlib and libcurl libraries.
Debian users can install zlib and libcurl via apt-get:
$ sudo apt-get install libmagic-dev zlib1g-dev libcurl4-openssl-dev
Note that some distributions/platforms may not have libmagic readily available,
or may use a version of libmagic that is incompatible with binwalk. In this
case, you may download the source code for the Unix file utility at:
ftp://ftp.astron.com/pub/file/
Building and installing the file utility will also install libmagic.
FILES
docs/README Project README file
docs/COPYING Project license file
src/binwalk.c Main binwalk source code file
src/binwalk.h Main binwalk source header file
src/common.c Common functions used by binwalk
src/common.h Common function declarations and definitions
src/dd.c Code for dumping sections of the target file to disk
src/dd.h DD code functions header file.
src/filter.c Result filtering functions
src/filter.h Filter functions header file
src/magic.binarch Custom magic signature file for opcode scans
src/magic.bincast Custom magic signature file for casting data types
src/magic.binwalk Custom magic signature file for binwalk
src/md5.c MD5 algorithm code by Peter Deutsch
src/md5.h MD5 algorithm header by Peter Deutsch
src/mparse.c Minimal magic file parsing library
src/mparse.h Parsing library header file
src/update.c Magic file update routines
src/update.h Updates header file
CC=@CC@
CFLAGS=@CFLAGS@
CPPFLAGS=@CPPFLAGS@
LDFLAGS=@LDFLAGS@
MAGIC=magic
MAGICFILES=$(wildcard $(MAGIC)/$(MAGIC).*)
prefix=@prefix@
exec_prefix=@exec_prefix@
MAGICPATH=@sysconfdir@/binwalk/$(MAGIC)
ETC=$(DESTDIR)@sysconfdir@/binwalk
BIN=$(DESTDIR)@bindir@
.PHONY: all magi install uninstall clean cleanall distclean
all: binwalk
binwalk: dd.o common.o md5.o mparse.o filter.o update.o
$(CC) $(CFLAGS) $(CPPFLAGS) -DMAGIC='"$(MAGICPATH).binwalk"' -DMAGIC_CAST='"$(MAGICPATH).bincast"' -DMAGIC_ARCH='"$(MAGICPATH).binarch"' binwalk.c -o binwalk *.o $(LDFLAGS)
dd.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c dd.c
common.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c common.c
md5.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c md5.c
mparse.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c mparse.c
filter.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c filter.c
update.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c update.c
magi:
cat $(MAGICFILES) > $(MAGIC).binwalk
install:
mkdir -p $(ETC)
mkdir -p $(BIN)
cp $(MAGIC).bin* $(ETC)
cp binwalk $(BIN)/binwalk
uninstall:
rm -rf $(ETC)
rm -f $(BIN)/binwalk
clean:
rm -f binwalk *.o
cleanall: clean
rm -rf config.* *.cache
rm -f Makefile
distclean: cleanall
This diff is collapsed.
/*
* Copyright (c) 2010 Craig Heffner
*
* This software is provided under the MIT license. For the full text of this license, please see
* the COPYING file included with this code, or visit http://www.opensource.org/licenses/MIT.
*/
#ifndef BINWALK_H
#define BINWALK_H
#include <magic.h>
#include "md5.h"
#include "mparse.h"
#include "filter.h"
#include "common.h"
#include "dd.h"
#include "config.h"
/* These should be defined in the Makefile. If not, default to /etc/binwalk/magic.bin*. */
#ifndef MAGIC
#define MAGIC "/etc/binwalk/magic.binwalk"
#endif
#ifndef MAGIC_CAST
#define MAGIC_CAST "/etc/binwalk/magic.bincast"
#endif
#ifndef MAGIC_ARCH
#define MAGIC_ARCH "/etc/binwalk/magic.binarch"
#endif
#define DATA "data"
#define DATA_SIZE 4
#define TEXT "text"
#define DEFAULT_BYTE_ALIGN 1
#define PROGRESS_INTERVAL 1000
#define MULTIPLE_MATCH_DELIM "\\012- "
#define MULTIPLE_MATCH_NEWLINE "\r\n\t\t\t\t"
#define MULTIPLE_MATCH_SIZE 6
#define GZIP_FILTER "gzip"
#define LZMA_FILTER "lzma"
#define JFFS_FILTER "jffs2"
#define INVALID_FILTER "invalid"
#define USAGE_OPTIONS "\
\t-o, --offset=<int> File offset to start searching at\n\
\t-l, --length=<int> Number of bytes to search\n\
\t-b, --align=<int> Set byte alignment\n\
\t-f, --file=<file> Log results to file\n\
\t-m, --magic=<file> Magic file to use [%s]\n\
\t-g, --grep=<string> Only display results that contain the text <string>\n\
\t-r, --raw-bytes=<string> Search for a sequence of raw bytes inside the target file (implies -a, -d, -I)\n\
\t-y, --search=<filter> Only search for matches that have <filter> in their description (implies -t, -d, -k)\n\
\t-x, --exclude=<filter> Exclude matches that have <filter> in their description\n\
\t-i, --include=<filter> Include matches that are normally excluded and that have <filter> in their description *\n\
\t-D, --dd=<type:ext:count> Extract <count> number of entries whose descriptions match <type> and save them with the file extension <ext> ****\n\
\t-a, --all Search for all matches, including those that are normally excluded *\n\
\t-d, --defaults Speed up scan by disabling default filters **\n\
\t-I, --show-invalid Show results marked as invalid ***\n\
\t-t, --fast Speed up scan by only loading signatures specified by -i or -y\n\
\t-u, --update Update magic signature files\n\
\t-v, --verbose Enable verbose mode\n\
\t-s, --smart Disable smart matching (implies -a)\n\
\t-k, --keep-going Don't stop at the first match (implies -I)\n\
\t-c, --validate Validate magic file\n\
\t-q, --quiet Supress output to stdout\n\
\t-A, --opcodes Scan for executable code (implies -a)\n\
\t-C, --cast Cast file contents as various data types (implies -k)\n\
\n\n\
* Signatures of two bytes or less are excluded by default. Use -i or -a to include them in the search.\n\
\n\
** Default filters include '%s', '%s' and '%s' results.\n\
Disabling the default filters (-d) will speed up scan time, but may miss these file types.\n\
\n\
*** By default, all results that contain the text '%s' will not be shown. Use -I to display them.\n\
\n\
**** If count is not specified or is 0, all matching entries will be extracted. The type '%s' will match everything.\n\
"
struct binconf
{
int smart;
int verbose;
int flags;
int offset;
int align;
int length;
char *magic;
magic_t cookie;
};
struct progress_t
{
size_t length;
size_t offset;
} progress;
void usage(char *progname);
void *display_progress(void *arg);
void parse_dd_option(char *optarg);
int process_file(char *bin_file, struct binconf *config, struct magic_signature **signatures, int num_sigs, struct magic_filter **filters, int filter_count);
#endif
/*
* Copyright (c) 2010 Craig Heffner
*
* This software is provided under the MIT license. For the full text of this license, please see
* the COPYING file included with this code, or visit http://www.opensource.org/licenses/MIT.
*/
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <time.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <fcntl.h>
#include <ctype.h>
#include "common.h"
#ifdef __linux
#include <linux/fs.h>
#endif
/* Convert decimal and hexadecimal strings to integers */
int str2int(char *str)
{
int val = 0;
int strsize = 0;
int base = 10;
char *strptr = NULL;
char buffer[INT_MAX_STR_SIZE] = { 0 };
if(str != NULL)
{
strsize = strlen(str);
if(strsize < INT_MAX_STR_SIZE)
{
memcpy((void *) &buffer, str, strsize);
strptr = (char *) &buffer;
/* If the string starts with '0x' or '\x', or if it ends in 'h' or 'H', treat it as hex */
if(strsize >= 2)
{
if(strptr[1] == 'x')
{
strptr += 2;
base = 16;
}
else if(strptr[strsize-1] == 'h' || strptr[strsize-1] == 'H')
{
strptr[strsize-1] = '\0';
base = 16;
}
}
val = strtol(strptr,NULL,base);
}
}
return val;
}
/* Reads in and returns the contents and size of a given file */
const void *file_read(char *file, size_t *fsize)
{
int fd = 0;
size_t file_size = 0;
struct stat _fstat = { 0 };
const void *buffer = NULL;
fd = open(file, O_RDONLY);
if(!fd)
{
perror(file);
goto end;
}
if(stat(file, &_fstat) == -1)
{
perror(file);
goto end;
}
if(_fstat.st_size > 0)
{
file_size = _fstat.st_size;
}
#ifdef __linux
else
{
long long long_file_size = 0;
/* Special files may report a zero size in stat(); must get their file size via an ioctl call */
if(ioctl(fd, BLKGETSIZE64, &long_file_size) == -1)
{
perror("ioctl");
goto end;
}
else
{
file_size = (size_t) long_file_size;
}
}
#endif
if(file_size > 0)
{
buffer = mmap(NULL, file_size, PROT_READ, (MAP_SHARED | MAP_NORESERVE), fd, 0);
if(buffer == MAP_FAILED)
{
perror("mmap");
buffer = NULL;
}
else
{
*fsize = file_size;
}
}
end:
if(fd) close(fd);
return buffer;
}
/* Create a temporary magic file to search for a specific string sequence */
char *create_magic_file(char *search_string)
{
FILE *fp = NULL;
char *file_path = NULL, *file_contents = NULL;
int file_contents_size = 0;
file_contents_size = (strlen(search_string) * 2) + strlen(STRING_ENTRY_FORMAT);
file_contents = malloc(file_contents_size + 1);
if(file_contents)
{
memset(file_contents, 0, (file_contents_size + 1));
sprintf(file_contents, STRING_ENTRY_FORMAT, search_string, search_string);
fp = fopen(TMP_MAGIC_FILE, "wb");
if(fp)
{
fwrite(file_contents, 1, strlen(file_contents), fp);
fclose(fp);
file_path = strdup(TMP_MAGIC_FILE);
}
else
{
perror(TMP_MAGIC_FILE);
}
free(file_contents);
}
else
{
perror("malloc");
}
return file_path;
}
/* Remove any temporary magic files created during execution */
void cleanup_magic_file(void)
{
unlink(TMP_MAGIC_FILE);
}
/* Print messages to both the log file and stdout, as appropriate */
void print(const char* format, ...)
{
va_list file_args;
va_list stdout_args;
va_start(file_args, format);
va_start(stdout_args, format);
if(globals.fsout != NULL)
{
vfprintf(globals.fsout, format, file_args);
fflush(globals.fsout);
}
if(globals.quiet == 0)
{
vfprintf(stdout, format, stdout_args);
fflush(stdout);
}
va_end(file_args);
va_end(stdout_args);
return;
}
/* Returns the current timestamp as a string */
char *timestamp()
{
time_t t = { 0 };
struct tm *tmp = NULL;
char *ts = NULL;
t = time(NULL);
tmp = localtime(&t);
if(!tmp)
{
perror("Localtime failure");
goto end;
}
ts = malloc(MAX_TIMESTAMP_SIZE);
if(!ts)
{
perror("Malloc failure");
goto end;
}
memset(ts,0,MAX_TIMESTAMP_SIZE);
if(strftime(ts,MAX_TIMESTAMP_SIZE-1,TIMESTAMP_FORMAT,tmp) == 0)
{
if(ts) free(ts);
ts = NULL;
}
end:
return ts;
}
/* Performs a case-insensitive string search */
int string_contains(char *haystack, char *needle)
{
char *my_haystack = NULL, *my_needle = NULL;
int retval = 0;
/* Duplicate the strings, as we will be converting them to all uppercase */
my_haystack = strdup(haystack);
my_needle = strdup(needle);
if(!my_haystack || !my_needle)
{
perror("strdup");
}
else
{
/* Convert string to all upper case */
uppercase(my_haystack);
uppercase(my_needle);
/* Search for needle in haystack */
if(strstr(my_haystack, my_needle) != NULL)
{
retval = 1;
}
}
if(my_haystack) free(my_haystack);
if(my_needle) free(my_needle);
return retval;
}
/* Convert a given string to all upper case */
void uppercase(char *string)
{
int i = 0;
for(i=0; i<strlen(string); i++)
{
string[i] = toupper(string[i]);
}
return;
}
/*
* Copyright (c) 2010 Craig Heffner
*
* This software is provided under the MIT license. For the full text of this license, please see
* the COPYING file included with this code, or visit http://www.opensource.org/licenses/MIT.
*/
#ifndef COMMON_H
#define COMMON_H
#define INT_MAX_STR_SIZE 1024
#define MAX_TIMESTAMP_SIZE 1024
#define TIMESTAMP_FORMAT "%b %d, %Y @ %T"
#define STRING_ENTRY_FORMAT "0\tstring\t%s\t\"%s\""
#define TMP_MAGIC_FILE "/tmp/binwalk.magic.raw.search"
/* These globals are set once, read many */
struct bin_globals
{
FILE *fsout;
int quiet;
} globals;
int str2int(char *str);
const void *file_read(char *file, size_t *fsize);
void print(const char* format, ...);
char *timestamp();
char *create_magic_file(char *search_string);
void cleanup_magic_file(void);
void uppercase(char *string);
int string_contains(char *haystack, char *needle);
#endif
This diff is collapsed.
dnl Many thanks to Anthony Basile for the initial configure script and Makefile
AC_PREREQ([2.65])
AC_INIT([binwalk],[0.4.5],[http://code.google.com/p/binwalk/issues/list])
AC_CONFIG_SRCDIR([md5.c])
AC_PROG_CC
AC_LANG(C)
CFLAGS="-Wall $CFLAGS"
LIBS="$LIBS -lz -lpthread -lmagic"
LDFLAGS="$LDFLAGS $LIBS"
AC_ARG_ENABLE([static], [ --enable-static perform static build ], [STATIC=y], [STATIC=n])
AC_ARG_ENABLE([updates], [ --disable-updates disable update features that rely on libcurl ], [CURL=n], [CURL=y])
AC_TYPE_SIZE_T
AC_FUNC_MALLOC
AC_HEADER_STDC
AC_CHECK_HEADERS([arpa/inet.h fcntl.h stdlib.h string.h unistd.h pthread.h],[],[echo "error: missing necessary header file" && exit 1])
AC_CHECK_FUNCS([memset strdup strstr strtol pthread_create])
AC_CHECK_LIB([magic],[magic_buffer],[],[echo "error: missing libmagic library, get it from ftp://ftp.astron.com/pub/file/" && exit 1],[])
AC_CHECK_HEADERS([magic.h],[],[echo "error: missing libmagic header files, get them from ftp://ftp.astron.com/pub/file/" && exit 1])
if test $CURL = y
then
AC_CHECK_LIB([curl],[curl_easy_init],[],[echo "error: missing libcurl library" && exit 1],[])
AC_CHECK_HEADERS([curl/curl.h],[],[echo "error: missing curl header files" && exit 1])
LDFLAGS="-lcurl $LDFLAGS"
else
CFLAGS="-DNOCURL $CFLAGS"
fi
dnl curl static libs only required if we're doing a static build
if test $STATIC = y
then
CLIBS=$(curl-config --static-libs)
LDFLAGS="-static $CLIBS $LDFLAGS"
fi
cp confdefs.h config.h
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "common.h"
#include "dd.h"
int dd_rule_count = 0;
struct dd_rule **dd_rules = NULL;
/* Add a DD rule to extract matching entries from the target file */
void add_dd_rule(char *match, char *extension, int max)
{
void *tmp = NULL;
tmp = dd_rules;
dd_rules = realloc(dd_rules, ((dd_rule_count+1) * sizeof(struct dd_rule *)));
if(!dd_rules)
{
perror("realloc");
if(tmp) free(tmp);
}
else
{
dd_rules[dd_rule_count] = malloc(sizeof(struct dd_rule));
if(dd_rules[dd_rule_count])
{
memset(dd_rules[dd_rule_count], 0, sizeof(struct dd_rule));
dd_rules[dd_rule_count]->match = strdup(match);
dd_rules[dd_rule_count]->ext = strdup(extension);
dd_rules[dd_rule_count]->max = max;
dd_rules[dd_rule_count]->count = 0;
dd_rule_count++;
}
}
}
/* Frees all DD rule entries */
void free_dd_rules(void)
{
int i = 0;
if(dd_rules)
{
for(i=0; i<dd_rule_count; i++)
{
if(dd_rules[i])