Commit 301918f6 authored by Google Code Exporter's avatar Google Code Exporter

Migrating wiki contents from Google Code

# Introduction #
Waf already supports quite a number of tools (gcc, g++, java, python, intltool etc. etc.) that compile software. If you happen to use a tool that waf does not support yet this page describes how to add that support. Once you have written the code please file an issue in our [Issue tracker]( and attach the patch, or send it to the [development mailing list](
# Waf tool definition #
A Waf tool is a file located in the `wafadmin/Tools` folder. It is used to extend the Waf behavior in the following areas:
1. detect project settings (when running `waf configure`)
1. provide additional configuration helpers (detecting libraries)
1. provide code for building files (define how to compile c++ code)
The Waf tools contain python code in no particular order, except for the function `detect(conf)`. It is executed during the configuration, when the code `conf.check_tool('tool_name')` is called
# Writing Waf tools #
## 1. Detecting the project settings ##
The body of the function `detect(conf)` is used to detect the configuration:
* the `conf` object is an instance of `Configure.Configuration`
* the attribute `conf.env` holds the configuration variables: `print conf.env['CXX']`
* `conf.env` can be modified to store the project configuration, for example `conf.env['CXX'] = 'g++'`
* the conf object has various methods (find\_program, check\_library) to help configuring the project
* `detect(conf)` does not return any special value
## 2. Extending the Configuration class ##
The `conf` object can be extended by **attaching** new methods to it (`conf.hook`). The methods usually throw `ConfigurationError` if something fails.
The purpose is to extend the configuration routines and to limit the amount of `import` to write.
Here is a quick example (the module `` provides several useful examples):
def check_dummy(self):
return True
def detect(conf):
"attach the checks to the conf object"
# now use the hook attached to the conf instance
result = conf.check_dummy()
## 3. Defining new compilation rules ##
Waf separates the high-level interfaces (declaring programs, shared libraries) from the low-level tasks (compile this file and add the results to the files to link). The user script almost never access the low-level apis, but this is needed for defining new kinds of file processing.
The elements needed for defining new compilation rules are the following (they are not all needed):
* Defining new Tasks
* Extending the class task\_gen
* Adding file extensions
### a. Defining new Tasks ###
Waf decomposes the file transformations into units of change. These units are called `tasks`, they can be linked to other tasks, and they can be run in parallel or one by one (depending on the constraints).
The `Task` objects can be created in three ways:
* Using `Tass.simple_task_type`
* Using `Task.task_type_from_func`
* using the `Task` constructor
The most popular way is to use the `simple_task_type`:
Task.simple_task_type('task_name', '${COMPILER} ${SRC} > ${TGT}', color='BLUE', prio=40)
The variables such as `${COMPILER`} represent environment variables, while the `SRC` and `TGT` are special variables representing the input and output files
Another optional way is to define your own 'builder' function, then call `task_type_from_func`:
Task.task_type_from_func('task_name', builder_func, color='CYAN', prio=120)
### b. Extending the class task\_gen ###
The classes instantiated when calling `bld.create_obj` are instances of ``. That class provides settings for calling methods (adding, removing, changing methods is possible), and for declaring the order in which to call the code.
The core of the `task_gen` class is the method `apply` which takes the methods to execute, performs a topological sort on them, and calls them one by one (code execution in parallel would even be possible).
Here is how to declare a new method on task\_gen:
* use the following imports `from Object import after, before, taskgen, feature`
* write a function with a unique parameter: `def function(self):`
* add the decorator `@taskgen` to attach a function to the task\_gen class
* use the decorators `@after('apply_core')` or `@before('apply_core')` to set the order
* use the decorator `@feature('name')` to declare the feature it belongs to. Features like cc, cxx, are used to add groups of methods upon execution
The new methods may perform the following operations:
* modify the state of the `task_gen` instance (change attributes)
* create `task` instances
* modify the environment instance
* add or remove methods, even if the method sorting has been performed
As an example:
from Object import taskgen, after, before, feature
def special_install(self):
if not getattr(self, "special", None): return
print "disabling the regular installation"
self.inst_var = 0
### c. Adding file extensions ###
The class `task_gen` usually executes the method `apply_core` which is used to process the source files (the "source" attribute).
The source files are transformed one by one into `Node` objects and are added to a temporary list. The list is then processed using a mapping `file_extension -> method`
Here is how to declare a new extension that process files of extension ".coin" into ".cpp" files:
from Object import extension
def coin_file(self, node):
out_source = node.change_ext('.cpp')
tsk = self.create_task('task_name')
# the out file is to be processed as a cpp file
## 4. Implementation details ##
Instead of adding types, subtypes, and extensions through inheritance on task\_gen (, the following scheme is now used:
1. until a task generator is posted (asked to create the tasks), nothing happens
1. when posted, the table of features are used to add groups of methods
1. the methods inserted are reordered with the ordering constraints (topological sort)
1. the methods are then called one by one for creating the tasks
Here are a few examples, they only represent the ideas, not the actual code:
* if a task\_generator has an attribute "cpp", the methods "apply\_defines" and "apply\_core" are added to the list of methods to execute
* if it has an attribute "shlib", the method "create\_shlib" is added
* if msvc is to be used (env['MSVC']==True), the "create\_shlib" method is replaced by "create\_shlib\_msvc"
Using such meta-data adds enough abstraction without impacting performance (calling fewer methods) or flexibility (the methods can be added manually, or inheritance is still possible).
\ No newline at end of file
#summary Various code snippets for common use cases.
# Introduction #
It often happens that people have similar goals and needs when writing build scripts for Waf. And so it happens they write similar code. This page is intended to list some common examples of code which might be interesting for others and maybe re-used.
# Snippets #
### Retrieve latest change Subversion revision in current directory ###
Author: eht16 (Enrico Tröger)
Description: The following function reads the output of **svn info** and retrieves the revision number from the **Last Changed Rev** field in its output. The revision number is returned as a string object or '-1' if an error occurred.
import subprocess
def conf_get_svn_rev():
p = subprocess.Popen(['svn', 'info', '--non-interactive'], stdout=subprocess.PIPE, \
stderr=subprocess.STDOUT, close_fds=False, env={'LANG' : 'C'})
stdout = p.communicate()[0]
if p.returncode == 0:
lines = stdout.splitlines(True)
for line in lines:
if line.startswith('Last Changed Rev'):
key, value = line.split(': ', 1)
return value.strip()
return '-1'
return '-1'
### Install post action ###
(author: dkovalkov)
def shutdown():
if Options.commands['install']:
# Your actions
### Disable install ###
(author: dkovalkov)
def build(bld):
obj = bld(features='cxx cprogram')
obj.install_path = None
For 1.6, you can also do it this way:
def build(bld):
### Disable color output ###
(author: dkovalkov)
import Logs
Logs.colors_lst['USE'] = False
(.. or set NOCOLOR in the shell environment)
### Get the relative path from the caller wscript to the main wscript ###
def get_top_level(bld):
Get the relative path from the caller wscript to main wscript.
import traceback, os
stack = traceback.extract_stack(limit=2)
caller = os.path.dirname(stack[0][0])
root = bld.srcnode.abspath()
root_to_caller = caller[len(root):].strip(os.path.sep)
caller_to_root = ''
for entry in root_to_caller.split(os.path.sep):
caller_to_root += '..' + os.path.sep
caller_to_root = caller_to_root.rstrip(os.path.sep)
return caller_to_root
### Run LINT on all source files ###
see playground/lint/
\ No newline at end of file
# Editors #
It is recommended to include a header in wscript files
#! /usr/bin/env python
# encoding: utf-8
## Vim ##
Sometimes the developers will omit it, making it difficult to read in text editors. Here is a line to add to the ~/vimrc file:
au BufNewFile,BufRead wscript* set filetype=python
The build outputs can be parsed by vim to scroll to the errors. Use the following to build with waf:
:set makeprg=waf
## Emacs ##
In emacs put this into yout .emacs file:
(setq auto-mode-alist (cons '("wscript" . python-mode) auto-mode-alist))
\ No newline at end of file
# Introduction #
This page tries to list the OS environment variables that affect WAF behaviour. For instance, in a unix system, with a bourne shell, the variable CXX can be used to select a particular compiler, like this:
CXX=/usr/bin/g++-3.3 waf configure
# Details #
## Variables that always have effect ##
> When set, color is disabled in WAF output; see also TERM.
> If the value of TERM is 'dumb', color is disabled in WAF output; see also NOCOLOR.
> Only used in the win32 platform; controls the command interpreter that is used to run commands
> Selects WAF modules installation directory.
> Directory to use for project configuration caching generated by WAF
> Name of the lock file (named '.lock-wscript' by default)
## Variables that affect waf configure ##
These variables only have effect when running 'waf configure', and are completely ignored in 'waf build' or 'waf'.
> Selects compilation options for when the C compiler is used, e.g. "-Wall".
> Selects compilation options for when the C++ compiler is used, e.g. "-Wall".
> Selects C preprocessor options, e.g. "-DFOO=bar"
> Extra linker options, e.g. "-L/usr/local -lsome-library"
> The C compiler that will be used instead of the platform default. Example CC=/usr/bin/gcc-2.95
> The C++ compiler that will be used instead of the platform default. Example CXX=/usr/bin/g++-2.95
> The default installation prefix to be used, if no --prefix option is given.
## Variables that affect waf build ##
These variables only have effect when running 'waf build'.
> The amount of parallel jobs when building targets, if no --jobs option is given.
## Variables that affect waf install ##
These variables only have effect when running 'waf install'.
> Used in "waf install", sets the base installation directory, i.e. a path that is used as prefix to all installation paths. This is typically in packaging to ensure that files are installed to a temporary folder during packaging even though they would normally go to a different directory under normal installation. Example:
$ DESTDIR=/tmp waf install
Compilation finished successfully
* installing build/default/foo.pyc as /tmp/usr/local/lib/python2.5/site-packages/foo.pyc
* installing as /tmp/usr/local/lib/python2.5/site-packages/
* installing build/default/foo.pyo as /tmp/usr/local/lib/python2.5/site-packages/foo.pyo
* installing build/default/ as /tmp/usr/local/lib/python2.5/site-packages/
* installing build/default/test as /tmp/usr/local/bin/test
Installation finished successfully
\ No newline at end of file
# Waf Users #
# Who is who #
The project members are visible on the [front page](
# wscripts maintainers and Waf developers #
## paths ##
Q: why does foo.cpp compile to `foo.cpp.<n>.o` where `<n>` is some number?
A: else the same files might be compiled in different contexts and overwritten
## variants ##
(see also: [Variants](
### on build ###
get environment by variant name:
# The waf philosophy #
## Why does not waf do more error checking on user scripts? ##
The waf approach is to provide an extensible build framework in which
only a few restrictions are enforced. Checks for common errors are executed
when calling "waf -v -v"
If more error checking is needed right now, the waf core functions
may be wrapped to perform stricter error checking.
## Why is packaging of waf in distributions discouraged? ##
With most build systems developers need to spend a lot of time and
effort ensuring that their projects will build correctly with many versions
of the build tool they use. This is needed because the developers have
very little control over the age of the distribution that their
package is being built on, and requiring that all people who want to
build their project update to a specific version of the build system
can be a significant burden.
You might think that this problem can be solved by careful design of
the build systems APIs, but history has shown that even with careful
design it is difficult to create a build system that avoids these
problems. For example, the venerable 'autotools' package that is so
widely used by free software packages has never managed to stabilise
its API enough to solve this problem. Many projects use a
script to cope with autotools variants and many others are shipping massive
'configure' scripts (sometimes several MB in size) with their projects
to avoid relying on the version of autotools installed on a users
system being able to work with their project.
The waf script is designed to be small enough to include with your
project, which completely avoids these issues and allows you to take
advantage of the latest additions to waf without ever being concerned
that your users may hit a problem that only happens with an earlier
version of waf.
If waf is packaged with distributions then end users may inadvertently
end up using the distribution version rather than the version that you
have carefully tested with your project.
You do still need a copy of python installed of course, but we have
put a lot of effort into waf to ensure it works well with a wide range
of python versions.
# Common problems and solutions #
## The same files are always built ##
1. Understanding why
> > The following command may provide a few ideas `./waf -v --zones=task`
1. Generated files already exist in the source directory
> > If generated files exist in the source directory, the file signature will be incorrect. Two possibilities exist:
* remove the generated file from the source directory and call `waf clean build`
* update the file signature, for example `bld(rule='touch ${TGT}', target='foo.txt', update_outputs=True)`
1. Several object files use the same name as output
> > This occurs if the same source files are used to produce different times the same target files. If the same task generators are declared by accident, remove the duplicates. Or, if the intent is to create several similar targets, use variants.
> > In the case of c/c++ applications the object file extension can be changed, for example:
idx = 55,
source = 'test.c',
target = 'test1')
idx = 66,
source = 'test.c',
target = 'test2')
1. The command-line changes
> > If the order of the source file change, the command-line will be different, causing the corresponding task to be executed (sometimes randomly).
## The files are not recompiled when the headers change ##
The include paths should contain the paths where the headers are located, relative to the current wscript file: `obj.includes = '. .. src'`
Waf does not look in folders such as `/usr/include` for performance reasons. To add a dependency on an external folder easily, you may:
**compute a checksum of the needed headers, and add the checksum as a command-line parameter such as `-Dexternal=84509734`** compute a checksum of the needed headers and set conf.env.CCDEPS, conf.env.CXXDEPS and perhaps conf.env.LINKDEPS
**use from waflib import preproc; c\_preproc.go\_absolute = True**
## Turning off file content hashing ##
Waf hashes the file contents to obtain the version of the files. This is required for the WAFCACHE to operate properly. In rare cases it is necessary to turn off file content hashing (projects with more than 10000 files), this can be performed by replacing the hash function like this:
from waflib import Utils
import stat
def h_file(filename):
st = os.stat(filename)
if stat.S_ISDIR(st): raise IOError, 'not a file'
m = md5()
return m.digest()
Utils.h_file = h_file
## How do I build static C binaries? ##
For gcc, set conf.env.SHLIB\_MARKER = '-Wl,-Bstatic' to link all libraries in static mode, and add '-static' to the linkflags to make a fully static binary.
# General questions #
## translations, intltool, gettext, msgfmt ##
See demos/intltool in the waf distribution
## distcheck ##
Waf provides a 'distcheck' (similar to the one found in autotools) since waf 1.2
## checkinstall ##
Turn off the libc tricks:
checkinstall --fstrans=no --nodoc ./waf install --destdir=test
## How do I ask waf to continue in spite of errors, attempt to perform all tasks and finally show all errors? ##
Use the -k option when invoking waf build
## are there any licensing issues with including waf in your project? ##
Waf is distributed under the new BSD license (see
The waf developers consider including the waf script in your project to meet the conditions for a source distribution of waf. This means that as long as your project
is compatible with the BSD license (which nearly everythinig is!) then you are fine
including the waf script in your project source tree.
## where does the Waf name come from? ##
The name 'Waf' was the shortest and easily typed name we could find at the time. There is no particular meaning.
\ No newline at end of file
# General recommendations #
* Make as little code as possible (unlike autotools, it is not necessary to have a script in each folder)
* Look carefully at the demos provided with the Waf source code
* Use the `uselib` features
# C/C++ specific #
* Remove object files (files with extension `.o`) from the source directory
1. they may cause build errors or unnecessary rebuilds
1. there is a risk to add them to the source control
* Avoid static libraries:
1. copy pasting binary objects wastes space
1. flag order is tricky to get right
1. use bld.path.ant\_glob to match files across several folders
* Move the `#ifdef` conditions to the configuration settings
1. try to replace complicated structures such as `#if PROGRAM_VERSION(2, 4, 2)` by `#if VERSION_1`, where the defines are set in a config.h file
1. errors should occur at configuration time, not at build time
\ No newline at end of file
# C/C++ Dependencies in waf #
The module contains the base class for all c/c++ programs/libraries/objects:
* add the include paths
* add the defines
* create the tasks (c/c++ or other that produce c source)
* add the library flags
* add extra object files
Source files are not analyzed until all c/c++ tasks are created. To infer the dependencies on the c/c++ source files, scanner classes are used. The roles of the scanner classes are:
* to find the files on which the sources depend
* create a signature of a c/c++ task in a consistent manner to know if it needs to be run
The scanners can currently operate on two modes:
* scan raw `#include` lines in the source
* preprocess the source files when they change (default behaviour)
# System includes #
To scan the files in reasonable time, waf does not look for includes on the system like /usr/include, this means that for a block like this, the "foo.h" include will be ignored if the macro is not defined within the project:
#if MY_EVIL_MACRO(5, 6, 234)
#include "foo.h"
To make the project more modular and maintainable, it is recommended to put all platform-specific test that lead to include changes into a configuration header (config.h), and to use simple configuration defines:
#ifdef IS_WIN32
#include "foo.h"
# Strict quotes #
With the following:
import preproc
only includes using double quotes will be used:
#include "foo.h" /* will be considered */
#include <bar.h> /* will be ignored */
The strict behaviour is disabled (0) by default.
# Configuration header : config.h #
A script similar to this one:
srcdir = '.'
blddir = 'build'
def set_options(opt):
def configure(conf):
conf.define('HELLO', None)
conf.define('NAME', 'ita')
conf.define('ANOTHER', 1)
def build(bld):
obj = bld.create_obj('cc', 'program')
obj.source = 'main.c' = 'test'
obj.includes = '.'
will produce a configuration header named `config.h` in the build dir `build` with the following content upon `waf configure`:
/* configuration created by waf */
#ifndef _CONFIG_H_WAF
#define _CONFIG_H_WAF
#define HELLO
#define ANOTHER 1
#define NAME "ita"
#endif /* _CONFIG_H_WAF */
The defines are stored on `conf.define`, to write several configuration headers, it may be necessary to reset the values:
conf.defines = {}
\ No newline at end of file
**WARNING: Waf is moving to github.**
Waf is a Python-based framework for configuring, compiling and installing applications. Here are perhaps the most important features of Waf:
* **Automatic build order**: the build order is computed from input and output files, among others
* **Automatic dependencies**: tasks to execute are detected by hashing files and commands
* **Performance**: tasks are executed in parallel automatically, the startup time is meant to be fast (separation between configuration and build)
* **Flexibility**: new commands and tasks can be added very easily through subclassing, bottlenecks for specific builds can be eliminated through dynamic method replacement
* **Extensibility**: though many programming languages and compilers are already supported by default, many others are available as extensions
* **IDE support**: Eclipse, Visual Studio and Xcode project generators (waflib/extras/)
* **Documentation**: the application is based on a robust model documented in [The Waf book]( and in the [API docs](
* **Python compatibility**: cPython 2.5 to 3.4, Jython 2.5, IronPython, and Pypy
Waf is used in particular by [innovative companies]( such as [Avalanche studios]( In the open-source world, Waf is used by a few projects such as [Samba]( Learn more about Waf by reading [The Waf book](
For researchers and build system writers, Waf is also a framework for creating [custom build systems]( and [package distribution systems](
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment