Commit f5a91705 authored by Stephen Stafford's avatar Stephen Stafford

Initial revision

Julian Assange <>
original concept / code
Christian Weisgerber <>
elvi: fast, thesaurus, webster
Christian Surchi <>
elvi: debcontents, debbugs, debpackages
Jason Harris <>
elvi: freebsd, openbsd, filesearching, ftpfind
Copyright (c) 2000-2001 Julian Assange <>
Copyright (c) 2001 Australian Institute for Collaborative Research
Copyright (c) 2000 Melbourne Institute for Advanced Study
The copyright holders listed above assert no rights on this release
of the software ``surfraw'' and thereby explicity place this release into
the into the public domain. Do what you will.
See NEWS for a list of high level changes and the cvs
repository for a list of low level changes
______ _ _ ______ _______ ______ _______ _ _ _
/ _____)(_) (_)(_____ \ (_______)(_____ \ (_______)(_)(_)(_)
( (____ _ _ _____) ) _____ _____) ) _______ _ _ _
\____ \ | | | || __ / | ___) | __ / | ___ || || || |
_____) )| |___| || | \ \ | | | | \ \ | | | || || || |
(______/ \_____/ |_| |_||_| |_| |_||_| |_| \_____/
Surfraw - Shell Users' Revolutionary Front Rage Against the Web
Oh Baybe
I need some
Deep Linking
Let us go
Surfin' in the raw!
Or... How to Eat The CHEESE
1) First, find a front end (`an elvi, types of elvis -- makes evil
nice') in the elvi/ directory that does something similar to what you
want. In this example we shall use the `google' elvis.
2) Copy the `google' elvis to the name of your new elvis.
In this example we shall create an elvis for the
Peoples' Front Against Forum Stealing Dogs mailing-list search service
3) At the top of the file, you will see something like:
# elvis: google -- Search the web using Google (
Keep the "elvis:" take and change the rest of the line. This
description is used at Surfraw installation time to generate a list
of Surfraw clients
Immediately after we see:
. surfraw || exit 1
Which pulls in the Surfraw functions. Keep it.
4) Next we (optionally) define w3_config_hook(). It is called
by w3_config and is used to define default configuration
variables. It is called after the global configuration file
is interpreted, but before ~/.surfraw.
You will notice the use of:
def -- define configuration variable unless already defined
(i.e by the environment)
defyn -- define boolean configuration variable unless
already defined
Which are pre-defined surfraw functions, but you can
of course use your own arbitrary broken shell code instead.
5) Any non-trivial program will then define w3_usage_hook().
$w3_argv0 is set to contain $0, otherwise usage is self
explanatory. You must call w3_global_usage at the end
of this function and follow the surfraw usage style
least you be marked a quiche eater.
6) Next define w3_parse_option_hook(). This is called at w3_config()
time, and takes two arguments. The first is the current
option being examined, the second is only present if the
option was of the form `-foo=bar'. It's the `bar'.
w3_parse_option_hook() is called before Surfraw global options
are parsed and so you can selectively over-ride them.
Return value: 0 if the option was recognised, 1 otherwise.
7) After this, w3_config() and w3_parse_args "$@" are called.
w3_config() scans the configuration files and calls
w3_config_hook(). w3_parse_args "$@" processes options,
called w3_parse_option_hook() as needed and builds
$w3_args -- the list non-option arguments.
This ends the regular initialisation code.
8) What happens next depends on the type of evil we are dealing
with. In the Peoples' Front Against Forum Stealing Dogs case, we have
previously performed a search on the search
service, and noted (with the '=' key in lynx, links) the url
returned. We then analyse this url to work out which CGI variable is
being used to convey search the search string, number of results et
Q: What about those proslytizers of iniquity that worship idols like
A: Most sites will take GETs in the place of POSTs, even if they
preach method=POST in forms. For those that don't? Walking with
the devil should not be a an exercise in relaxed discourse. The
problem is getting the browser to automatically submit the data
using POST as a method. One partial solution is to have surfraw
generate a pre-filled method=POST form as a local file and have
the browser read it. Exploration of other ideas is encouraged.
w3_url_of_arg() is then used to escape the command line search
string argument into something that can be seen in a URL without
i.e escaped_args=`w3_url_of_arg $w3_args`
In our last exhilarating moments we then build the final URL and
pass it onto the currently selected browser with w3_browse_url,
proud in the knowledge that the Shell Users' Revolutionary Front
Against the WWW has brought the holy word of the command line to
yet another heathen land.
-Julian Assange <>
______ _ _ ______ _______ ______ _______ _ _ _
/ _____)(_) (_)(_____ \ (_______)(_____ \ (_______)(_)(_)(_)
( (____ _ _ _____) ) _____ _____) ) _______ _ _ _
\____ \ | | | || __ / | ___) | __ / | ___ || || || |
_____) )| |___| || | \ \ | | | | \ \ | | | || || || |
(______/ \_____/ |_| |_||_| |_| |_||_| |_| \_____/
Surfraw - Shell Users' Revolutionary Front Rage Against the Web
Oh Baybe
I need some
Deep Linking
Let us go
Surfin' in the raw!
See README for Surfraw specific usage information
Basic Installation
These are generic installation instructions.
The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation. It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions. Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, a file
`config.cache' that saves the results of its tests to speed up
reconfiguring, and a file `config.log' containing compiler output
(useful mainly for debugging `configure').
If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release. If at some point `config.cache'
contains results you don't want to keep, you may remove or edit it.
The file `' is used to create `configure' by a program
called `autoconf'. You only need `' if you want to change
it or regenerate `configure' using a newer version of `autoconf'.
The simplest way to compile this package is:
1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system. If you're
using `csh' on an old version of System V, you might need to type
`sh ./configure' instead to prevent `csh' from trying to execute
`configure' itself.
Running `configure' takes awhile. While running, it prints some
messages telling which features it is checking for.
2. Type `make' to compile the package.
3. Optionally, type `make check' to run any self-tests that come with
the package.
4. Type `make install' to install the programs and any data files and
5. You can remove the program binaries and object files from the
source code directory by typing `make clean'. To also remove the
files that `configure' created (so you can compile the package for
a different kind of computer), type `make distclean'. There is
also a `make maintainer-clean' target, but that is intended mainly
for the package's developers. If you use it, you may have to get
all sorts of other programs in order to regenerate files that came
with the distribution.
Compilers and Options
Some systems require unusual options for compilation or linking that
the `configure' script does not know about. You can give `configure'
initial values for variables by setting them in the environment. Using
a Bourne-compatible shell, you can do that on the command line like
CC=c89 CFLAGS=-O2 LIBS=-lposix ./configure
Or on systems that have the `env' program, you can do it like this:
env CPPFLAGS=-I/usr/local/include LDFLAGS=-s ./configure
Compiling For Multiple Architectures
You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory. To do this, you must use a version of `make' that
supports the `VPATH' variable, such as GNU `make'. `cd' to the
directory where you want the object files and executables to go and run
the `configure' script. `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'.
If you have to use a `make' that does not supports the `VPATH'
variable, you have to compile the package for one architecture at a time
in the source code directory. After you have installed the package for
one architecture, use `make distclean' before reconfiguring for another
Installation Names
By default, `make install' will install the package's files in
`/usr/local/bin', `/usr/local/man', etc. You can specify an
installation prefix other than `/usr/local' by giving `configure' the
option `--prefix=PATH'.
You can specify separate installation prefixes for
architecture-specific files and architecture-independent files. If you
give `configure' the option `--exec-prefix=PATH', the package will use
PATH as the prefix for installing programs and libraries.
Documentation and other data files will still use the regular prefix.
In addition, if you use an unusual directory layout you can give
options like `--bindir=PATH' to specify different values for particular
kinds of files. Run `configure --help' for a list of the directories
you can set and what kinds of files go in them.
If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
Optional Features
Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System). The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.
For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.
Specifying the System Type
There may be some features `configure' can not figure out
automatically, but needs to determine by the type of host the package
will run on. Usually `configure' can figure that out, but if it prints
a message saying it can not guess the host type, give it the
`--host=TYPE' option. TYPE can either be a short name for the system
type, such as `sun4', or a canonical name with three fields:
See the file `config.sub' for the possible values of each field. If
`config.sub' isn't included in this package, then this package doesn't
need to know the host type.
If you are building compiler tools for cross-compiling, you can also
use the `--target=TYPE' option to select the type of system they will
produce code for and the `--build=TYPE' option to select the type of
system on which you are compiling the package.
Sharing Defaults
If you want to set default values for `configure' scripts to share,
you can create a site shell script called `' that gives
default values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/' if it exists, then
`PREFIX/etc/' if it exists. Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.
Operation Controls
`configure' recognizes the following options to control how it
Use and save the results of the tests in FILE instead of
`./config.cache'. Set FILE to `/dev/null' to disable caching, for
debugging `configure'.
Print a summary of the options to `configure', and exit.
Do not print messages saying which checks are being made. To
suppress all normal output, redirect it to `/dev/null' (any error
messages will still be shown).
Look for the package's source code in directory DIR. Usually
`configure' can determine that directory automatically.
Print the version of Autoconf used to generate the `configure'
script, and exit.
`configure' also accepts some other, not widely useful, options.
# $Id$
SUBDIRS = elvi
bin_SCRIPTS = surfraw
sysconf_DATA = surfraw.conf
devel_extra = banner prebuild
aclocal.m4 HACKING \
htdocs: index.html
scp index.html
README: index.html
st=`stty -g`; stty cols 80; lynx -dump $< >$@ || rm -f $@; stty $$st
scp $(PACKAGE)-$(VERSION).tar.gz
# ncftpput -F /incoming $(PACKAGE)-$(VERSION).tar.gz
This diff is collapsed.
______ _ _ ______ _______ ______ _______ _ _ _
/ _____)(_) (_)(_____ \ (_______)(_____ \ (_______)(_)(_)(_)
( (____ _ _ _____) ) _____ _____) ) _______ _ _ _
\____ \ | | | || __ / | ___) | __ / | ___ || || || |
_____) )| |___| || | \ \ | | | | \ \ | | | || || || |
(______/ \_____/ |_| |_||_| |_| |_||_| |_| \_____/
Surfraw - Shell Users' Revolutionary Front Rage Against the Web
Oh Baybe
I need some
Deep Linking
Let us go
Surfin' in the raw!
Wed Dec 26 02:48:06 EST 2001 - Surfraw 1.0.7
Fix major sed lossage in 1.0.6 which removed all references to
http:// (not just in the documentation strings as was intended)
Sun Dec 23 21:02:57 EST 2001 - Surfraw 1.0.6
New elvi: freebsd, openbsd, filesearching, ftpfind
Updated elvi: translate (babelfish), ftpsearch, fast, netbsd
Rename linuxdoc elvi to slinuxdoc, to prevent name clash with sgmltools
Use test -r instead of test -e which inexplicably chocked Solaris8
Documentation changes
Tue Nov 6 11:14:21 EST 2001 - Surfraw 1.0.5
Fix wayward indentation introduced in 1.0.4
which caused -browser to fail
Note that SURFRAW_browser was made obsolete in 1.0.4.
SURFRAW_text_browser and SURFRAW_graphical_browser
replace its functionality
even Redhat users get it.
Thu Oct 25 05:32:25 EST 2001 - Surfraw 1.0.4
New elvi: debbugs, debcontents, debpackages, swhois, woffle
Dead elvi: ircsearch, newscientist
-g and -t switch between graphical & text browsers
-new opens a new window on -remote comprehending browsers
Support "--" option to end all options
Fix url space escaping problem
Surfraw placed into the public domain
Fri Jan 12 14:51:10 EST 2001 - Surfraw 1.0.3
New elvi: amazon
Add `scaleplus' elvi to elvi/
Minor documentation bug fixes
Wed Sep 27 06:37:48 EST 2000 - Surfraw 1.0.2
Viva l`explorateur de texte!
Translate elvi didn't correctly parse -from or -to
HACKING file was missing from EXTRA_DIST
Fri Aug 25 23:42:31 EST 2000 - Surfraw 1.0.1
New elvi: britannica
Updated elvi: freshmeat
Fri Aug 25 02:26:31 EST 2000 - Surfraw 0.5.8
New elvi: stockquote (nasdaq,, yahoo), jake
Don't open new screen(1)s by default
Zorch usage of local variables (not POSIX)
Sat Aug 12 20:00:45 EST 2000 - Surfraw 0.5.7
New elvi: fast netbsd translate W wetandwild.
Auto detect browser during configure phase.
Built in support for screen(1)
Built in support for netscape-remote
`surfraw' on its own now displays a list of elvi
Sun Aug 6 11:43:50 EST 2000 - Surfraw 0.5.6
New elvi: newscientist pubmed freshmeat
Sat Aug 5 16:07:55 EST 2000 - Surfraw 0.5.5
New elvi: appwatch cnn excite ftpsearch linuxdoc scaleplus sundocs
sunsolve thesaurus weather webster
Wed Aug 2 08:56:04 EST 2000
Name change, surfaw -> surfraw
Web-site is now
Sun Jul 30 17:51:50 EST 2000 - Surfaw 0.5.4
A Lazy sunday. A tired, but happy mood. The afternoon drowses by and
we have the initial version.
Surfraw - Shell Users' Revolutionary Front Rage Against the Web
Home Page: [1]
______ _ _ ______ _______ ______ _______ _ _ _
/ _____)(_) (_)(_____ \ (_______)(_____ \ (_______)(_)(_)(_)
( (____ _ _ _____) ) _____ _____) ) _______ _ _ _
\____ \ | | | || __ / | ___) | __ / | ___ || || || |
_____) )| |___| || | \ \ | | | | \ \ | | | || || || |
(______/ \_____/ |_| |_||_| |_| |_||_| |_| \_____/
Surfraw - Shell Users' Revolutionary Front Rage Against the Web
Oh Baybe
I need some
Deep Linking
Let us go
Surfin' in the raw!
Surfraw provides a fast unix command line interface to a variety of
popular WWW search engines and other artifacts of power. It reclaims
google, altavista, babelfish, dejanews, freshmeat, research index,
slashdot and many others from the false-prophet, pox-infested heathen
lands of html-forms, placing these wonders where they belong, deep in
unix heartland, as god loving extensions to the shell.
Surfraw abstracts the browser away from input. Doing so lets it get on
with what it's good at. Browsing. Interpretation of linguistic forms
is handed back to the shell, which is what it, and human beings are
good at. Combined with netscape-remote or incremental text browsers,
such as links (, w3m
(, and screen(1) a Surfraw liberateur is capable
of navigating speeds that leave GUI tainted idolaters agape with fear
and wonder.
For example:
$ ask why is jeeves gay?
$ google -results=100 RMS, GNU, which is sinner, which is sin?
$ austlii -method=phrase dog like
$ rhyme -method=perfect Julian
Global options are common to all Surfraw elvi (clients). You can
get a list of the currently installed elvi by just typing `surfraw'.
For example:
$ surfraw
W -- Activate Surfraw defined web-browser
altavista -- Search the web using Altavista (
amazon -- Search the bookstore
appwatch -- Search entries on Appwatch (
ask -- Question the web using Ask Jeeves (
austlii -- Search Australian Law docs (
britannica -- Look up word in Encyclopedia Britannica (
cite -- Search computer science papers (
cnn -- Search on CNN (
currency -- Abuse the Universal Currency Converter (
debbugs -- Search the debian BTS (
debcontents -- Search contents of debian packages(
debpackages -- Search debian packages (
deja -- Search usenet using Deja News (
excite -- Search on Excite (
fast -- Search the web using FAST Search (
filesearching -- Search files on filesearching (
freebsd -- Search FreeBSD related information (
freshmeat -- Search Freshmeat (
ftpfind -- Search files on ftpfind (
ftpsearch -- Search files on ftpsearch (
google -- Search the web using Google (
jake -- Search journal/databases in jake (
linuxdoc -- Search entries in LDP (
netbsd -- Search NetBSD related information (
openbsd -- Search OpenBSD related information (
pubmed -- Search medical/molbio databases (
raging -- Search the web using Raging (
rhyme -- Search for rhymes et al using Lycos Rhyme (
scaleplus -- Search Australian Law using ScalePlus (
slashdot -- Search stories on Slashdot (
sourceforge -- Search SourceForge (
stockquote -- Get a single stock quote (multiple providers)
sundocs -- Search docs on (
sunsolve -- Search public docs on Sunsolve (
swhois -- Search Whois (
thesaurus -- Look up word in Merriam-Webster's Thesaurus (
translate -- Translate human languages (various providers)
webster -- Look up word in Merriam-Webster's Dictionary (
wetandwild -- Real time weather information (many sources)
woffle -- Search the web using Woffle (localhost:8080)
xxx -- Superior XXX, hot and ready to press (
yahoo -- Search Yahoo categories (
All elvi have useful low calorie help, for example:
Usage: rhyme [options] [search words]...
Surfraw search for rhymes and other word correlations using
Lycos Rhyme (
$ rhyme Julian
$ rhyme -method=rel surfer
Hurclean St. Julien.
surfboarder, bather, natator, swimmer
Local options:
-method= Type of word correlation to search for
prefect | Perfect rhyme
syn | Synonyms
hom | Homophones
cons | Consonsant rhymes only
rel | Semantically related words
sub | Phrases
spell | Similar spellings
pic | Pictures
shake | Match with Shakespeare archives
def Find wordnet definition
Default: perfect
Environment: SURFRAW_rhyme_method
Global options:
-browser=EXECUTABLE Set browser
Default: mozilla
-elvi List Surfraw mechanims for conquering evil
-escape-url-args=yes|no Apply url escaping to arguments
Default: yes
Environment: SURFRAW_escape_url_args
-g | -graphical Get some windowed sin
-p0rn=yes|no Yes, yes, harder, deeper, faster, oh baybe
Default: yes
Environment: SURFRAW_p0rn
-help What you're reading now, dude
-quiet=yes|no I can't enjoy it with you talking all the time
Default: no
Environment: SURFRAW_quiet
-new[=yes|no] Start in a new window
Default: no
Environment: SURFRAW_new_window
-t | -text Back to the yellow brick road
-q | -quote Quote arguments with " characters
Default: no
Environment: SURFRAW_quote_args
-version Display Surfraw version (1.0.4)
Copyright (c) 2000-2001 Julian Assange
Copyright (c) 2001 Australian Institute for Collaborative Research
Copyright (c) 2000 Melbourne Institute for Advanced Study
The system wide configuration file can be found in:
SURFRAW_global_conf @sysconfdir@/surfraw.conf
Where @sysconfdir@ is something like /etc, or /usr/local/etc
Each user can also specify their own overrides in:
SURFRAW_conf $HOME/.surfraw.conf
Read HACKING. Surfrawize the soul of your favourite internet wonder.
Join the Shell Users' Revolutionary Front Against the WWW by
submitting code. Reclaim heathen lands. Bear witness to the
truth. Its love will set you free.
- Julian Assange
[3]Julian Assange
Last modified: Sun Dec 23 21:02:17 EST 2001
Outstanding requests: (feel free to contribute!) -- internet movie database -- search engine -- update deja search to take advantage of new google features
dnl aclocal.m4 generated automatically by aclocal 1.4
dnl Copyright (C) 1994, 1995-8, 1999 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
dnl This program is distributed in the hope that it will be useful,
dnl but WITHOUT ANY WARRANTY, to the extent permitted by law; without
dnl even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# Do all the work for Automake. This macro actually does too much --
# some checks are only needed if your package does certain things.
# But this isn't really a big deal.
# serial 1
dnl Usage:
dnl AM_INIT_AUTOMAKE(package,version, [no-define])
dnl test to see if srcdir already configured
if test "`cd $srcdir && pwd`" != "`pwd`" && test -f $srcdir/config.status; then
AC_MSG_ERROR([source directory already configured; run "make distclean" there first])
AC_DEFINE_UNQUOTED(VERSION, "$VERSION", [Version number of package]))
dnl FIXME This is truly gross.
missing_dir=`cd $ac_aux_dir && pwd`
AM_MISSING_PROG(ACLOCAL, aclocal, $missing_dir)
AM_MISSING_PROG(AUTOCONF, autoconf, $missing_dir)
AM_MISSING_PROG(AUTOMAKE, automake, $missing_dir)
AM_MISSING_PROG(AUTOHEADER, autoheader, $missing_dir)
AM_MISSING_PROG(MAKEINFO, makeinfo, $missing_dir)
# Check to make sure that the build environment is sane.
[AC_MSG_CHECKING([whether build environment is sane])
# Just in case
sleep 1
echo timestamp > conftestfile
# Do `set' in a subshell so we don't clobber the current shell's
# arguments. Must try -L first in case configure is actually a
# symlink; some systems play weird games with the mod time of symlinks
# (eg FreeBSD returns the mod time of the symlink's containing
# directory).
if (
set X `ls -Lt $srcdir/configure conftestfile 2> /dev/null`
if test "[$]*" = "X"; then
# -L didn't work.
set X `ls -t $srcdir/configure conftestfile`
if test "[$]*" != "X $srcdir/configure conftestfile" \
&& test "[$]*" != "X conftestfile $srcdir/configure"; then
# If neither matched, then we have a broken ls. This can happen
# if, for instance, CONFIG_SHELL is bash and it inherits a
# broken ls alias from the environment. This has actually
# happened. Such a system could not be considered "sane".
AC_MSG_ERROR([ls -t appears to fail. Make sure there is not a broken
alias in your environment])
test "[$]2" = conftestfile
# Ok.
AC_MSG_ERROR([newly created file is older than distributed files!
Check your system clock])
rm -f conftest*
dnl The program must properly implement --version.
[AC_MSG_CHECKING(for working $2)
# Run test in a subshell; some versions of sh will print an error if
# an executable is not found, even if stderr is redirected.
# Redirect stdin to placate older versions of autoconf. Sigh.
if ($2 --version) < /dev/null > /dev/null 2>&1; then