Commit 90395131 authored by Johan Bloemberg's avatar Johan Bloemberg

Restructure dev workflow around Tox some more.

parent 18894aa7
......@@ -5,6 +5,9 @@ export DEBUG=1
# during development, just have ipv6 on.
export NETWORK_SUPPORTS_IPV6=1
# record path of Python executable outside of Tox environment to prevent conflicts
export PYTHON_BIN=$(which python3.6)
# use virtualenv created by Tox for development
PATH_add $PWD/.tox/default/bin/
......
......@@ -38,12 +38,12 @@ Linux or MacOS capable of running Python3 and git.
Download and install below system requirements to get started:
- [git](https://git-scm.com/downloads) (download and install)
- [python3](https://www.python.org/downloads/) (download and install)
- [python3.6](https://www.python.org/downloads/) (download and install)
- [Tox](http://tox.readthedocs.io/) (`pip3 install --user tox`)
- [direnv](https://direnv.net/) (optional, download and install, then follow [setup instructions](https://direnv.net/), see Direnv section below)
- [direnv](https://direnv.net/) (recommended, download and install, then follow [setup instructions](https://direnv.net/), see Direnv section below)
- [Docker](https://docs.docker.com/engine/installation/) (optional, recommended, follow instructions to install.)
# Obtaining the software
# Quickstart
In a directory of your choosing:
......@@ -65,9 +65,35 @@ After completing succesfully the application is available to run:
failmap -h
# Quickstart
To perform non-Docker development, make sure all 'Requirements' are installed. Run the following command to setup a development instance:
tox -e setup
After this run to the following command to start a development server:
failmap runserver
Now visit the [map website](http://127.0.0.1:8000/) and/or the
[admin website](http://127.0.0.1:8000/admin/) at http://127.0.0.1:8000 (credentials: admin:faalkaart).
For the quickstart we assume the usage of Docker as it offers the most complete environment for development. This project aims to be environment agnostic and development without Docker is possible. See Development section below.
The setup script performs the following steps:
# creates the database
failmap migrate
# create a user to view the admin interface
failmap load_dataset development
# loads a series of sample data into the database
failmap load_dataset testdata
# calculate the scores that should be displayed on the map
failmap rebuild_ratings
# Using Docker [WIP]
For the Docker Quickstart we assume the usage of Docker as it offers the most complete environment for development. This project aims to be environment agnostic and development without Docker is possible, but Docker allows for a more managable development environment and less 'need-to-know-what-you-are-doing'.
Below commands result in a failmap installation that is suitable for testing and development. It is
capable of handling thousands of urls and still be modestly responsive.
......@@ -123,40 +149,6 @@ For more information consult docker composer [documentation](https://docs.docker
docker-compose -h
# Development
To perform non-Docker development, make sure all Requirements are installed. Run the following script to setup a development instance:
tools/dev_setup.sh
After this run:
# finally start the development server
failmap runserver
Now visit the [map website](http://127.0.0.1:8000/) and/or the
[admin website](http://127.0.0.1:8000/admin/) at http://127.0.0.1:8000 (credentials: admin:faalkaart).
The setup script performs the following steps:
# download even more requirements needed to run this software
pip3 install -e .
# and download the development requirements
pip3 install -r requirements.dev.txt
# creates the database
failmap migrate
# create a user to view the admin interface
failmap load_dataset development
# loads a series of sample data into the database
failmap load_dataset testdata
# calculate the scores that should be displayed on the map
failmap rebuild_ratings
# Scanning services (beta)
Some scanners require redis to be installed. We're currently in transition from running scanners
......@@ -235,17 +227,21 @@ To run code quality checks and unit tests run:
tox
For a comprehensive test run:
tox -e check,test,datasets
To make life easier you can use `autopep8`/`isort` before running `tox` to automatically fix most style issues:
tox -e autofix
To run only a specific test use:
tox -- -k test_name
tox -e test -- -k test_name
To only run a specific test suite use for example:
.tox/default/bin/failmap test tests/test_smarturl.py
.tox/default/bin/failmap test tests/scanners/test_dummy.py
A coverage report is generated after running tests, on OSX it can be viewed using:
......@@ -258,13 +254,13 @@ Pytest allows to drop into Python debugger when a tests fails. To enable run:
## Direnv / Virtualenv
This project has [direnv](https://direnv.net/) configuration to automatically manage the Python
virtual environment. Install direnv and run `direnv allow` to enable.
virtual environment. Install direnv and run `direnv allow` to enable it initially. After this the environment will by automatically loaded/unloaded every time you enter/leave the project directory.
Alternatively you can manually create a virtualenv using:
virtualenv venv
Be sure to active the environment before starting development every time:
Be sure to active the environment before starting development every time and see `.envrc` for other settings that are normally enabled by direnv:
. venv/bin/activate
export DEBUG=1
......
......@@ -2,7 +2,6 @@
import json
import pytest
from django.core.management import call_command
TEST_ORGANIZATION = 'faalonië'
......@@ -14,4 +13,5 @@ def test_dummy(responses, db, faalonië):
result = json.loads(call_command('scan_dummy', '-v3', '-o', TEST_ORGANIZATION))
assert result[0]['status'] == 'success'
# dummy returns random success/failure results, only check if there is a result, not the result itself
assert result[0]
#!/bin/bash
set -ve
# run all commands needed to setup dev. environment (like mentioned in README.md)
if -z "$VIRTUAL_ENV"; then
echo "Refusing to run with a Virtualenv, please see README.md->Direnv/Virtualenv."
exit 1
fi
pip3 install -e .
failmap migrate
# load development environment users/settings
failmap load-dataset development
# load a test data set
failmap load-dataset testdata -v0
failmap rebuild-ratings -v0
[tox]
# list of test environment to run when none is explicitly specified by the '-e' argument.
envlist = check,test,datasets
envlist = check,test
# default configuration for all tox test environments
[testenv]
......@@ -8,7 +8,8 @@ envlist = check,test,datasets
# and this reduces time required to build the same environment for every command
envdir = {toxworkdir}/default
basepython = python3.6
# force use of specific Python version outside of Tox environment to be used by Tox itself
basepython = {env:PYTHON_BIN:/usr/local/bin/python3.6}
usedevelop = True
extras =
development
......@@ -63,3 +64,16 @@ commands =
[testenv:integration]
commands = pytest -k 'integration' {posargs}
# setup Failmap for development (create test database, load testdata, etc)
[testenv:setup]
commands =
# initialize or update database
failmap migrate
# load development environment users/settings
failmap load_dataset development
# load a test data set
failmap load_dataset testdata -v0
failmap rebuild_ratings -v0
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment