Commit 790c733f authored by Johan Bloemberg's avatar Johan Bloemberg

rename failmap_admin to failmap

parent ec9ba344
......@@ -10,15 +10,15 @@
htmlcov
error.log
db_backup_*.sqlite3
failmap_admin/map/static/CACHE/*
failmap_admin/map/static/images/screenshots/*
failmap_admin/scanners/resources/wordlists/knownsubdomains.txt
failmap/map/static/CACHE/*
failmap/map/static/images/screenshots/*
failmap/scanners/resources/wordlists/knownsubdomains.txt
dist/
build/
code.json
codeclimate.json
static/
!failmap_admin/*/static/
!failmap/*/static/
.direnv
version
vendor/dnsrecon/
......
......@@ -12,7 +12,7 @@ COPY requirements.deploy.txt /source/
RUN /pyenv/bin/pip install -r /source/requirements.deploy.txt
# copy all relevant files for python installation
COPY ./failmap_admin/ /source/failmap_admin/
COPY ./failmap/ /source/failmap/
# add wildcard to version file as it may not exists (eg: local development)
COPY setup.py setup.cfg MANIFEST.in requirements.dev.txt version* /source/
......@@ -37,7 +37,7 @@ COPY --from=build /pyenv /pyenv
COPY --from=build /source /source
# expose relevant executable(s)
RUN ln -s /pyenv/bin/failmap-admin /usr/local/bin/
RUN ln -s /pyenv/bin/failmap /usr/local/bin/
RUN ln -s /pyenv/bin/uwsgi /usr/local/bin/
RUN ln -s /pyenv/bin/celery /usr/local/bin/
......@@ -46,22 +46,22 @@ WORKDIR /
# configuration for django-uwsgi to work correct in Docker environment
ENV UWSGI_GID root
ENV UWSGI_UID root
ENV UWSGI_MODULE failmap_admin.wsgi
ENV UWSGI_MODULE failmap.wsgi
# serve static files (to caching proxy) from collected/generated static files
ENV UWSGI_STATIC_MAP /static=/srv/failmap-admin/static
ENV UWSGI_STATIC_MAP /static=/srv/failmap/static
# set proxy and browser caching for static files to 1 month
ENV UWSGI_STATIC_EXPIRES /* 2678400
# collect all static files form all django applications into static files directory
RUN /pyenv/bin/failmap-admin collectstatic
RUN /pyenv/bin/failmap collectstatic
# Compress JS/CSS before serving, using django-compressor, run after collectstatic
# COMPRESS=1 is a hack to disable django_uwsgi app as it currently conflicts with compressor
# https://github.com/django-compressor/django-compressor/issues/881
RUN env COMPRESS=1 /pyenv/bin/failmap-admin compress
RUN env COMPRESS=1 /pyenv/bin/failmap compress
EXPOSE 8000
ENTRYPOINT [ "/usr/local/bin/failmap-admin" ]
ENTRYPOINT [ "/usr/local/bin/failmap" ]
CMD [ "help" ]
include version
include requirements.txt
recursive-include failmap_admin/*/templates *
recursive-include failmap_admin/*/static *
recursive-include failmap_admin/*/fixtures *.yaml *.json
recursive-include failmap_admin/*/locale *.mo *.po
recursive-include failmap/*/templates *
recursive-include failmap/*/static *
recursive-include failmap/*/fixtures *.yaml *.json
recursive-include failmap/*/locale *.mo *.po
......@@ -81,15 +81,15 @@ Notice: MySQL/Redis connection errors might be shown during startup. This is nor
You can now visit the [map website](http://127.0.0.1:8000/) and/or the
[admin website](http://127.0.0.1:8000/admin/) at http://127.0.0.1:8000 (credentials: admin:faalkaart).
The environment is aware of code changes in the `failmap_admin` folder. Services are automatically restarted to reflect the latest changes.
The environment is aware of code changes in the `failmap` folder. Services are automatically restarted to reflect the latest changes.
To stop the entire stack run: `docker-compose down`
There is a command-line application available to perform administrative tasks. To run it do:
docker-compose exec admin failmap-admin
docker-compose exec admin failmap
Further in this documentation the `failmap-admin` command is mentioned, when using the Docker environment always prepend `docker-compose exec admin` before the command.
Further in this documentation the `failmap` command is mentioned, when using the Docker environment always prepend `docker-compose exec admin` before the command.
To see all running components:
......@@ -124,7 +124,7 @@ To perform non-Docker development, make sure all Requirements are installed. Run
After this run:
# finally start the development server
failmap-admin runserver
failmap runserver
Now visit the [map website](http://127.0.0.1:8000/) and/or the
[admin website](http://127.0.0.1:8000/admin/) at http://127.0.0.1:8000 (credentials: admin:faalkaart).
......@@ -138,16 +138,16 @@ The setup script performs the following steps:
pip3 install -r requirements.dev.txt
# creates the database
failmap-admin migrate
failmap migrate
# create a user to view the admin interface
failmap-admin load_dataset development
failmap load_dataset development
# loads a series of sample data into the database
failmap-admin load_dataset testdata
failmap load_dataset testdata
# calculate the scores that should be displayed on the map
failmap-admin rebuild_ratings
failmap rebuild_ratings
# Scanning services (beta)
......@@ -162,19 +162,19 @@ Each of the below commands requires their own command line window:
redis-server
# start a worker
failmap-admin celery worker -ldebug
failmap celery worker -ldebug
These services help fill the database with accurate up to date information. Run each one of them in
a separate command line window and keep them running.
# handles all new urls with an initial (fast) scan
failmap-admin onboard_service
failmap onboard_service
# slowly gets results from qualys
failmap-admin scan_tls_qualys_service
failmap scan_tls_qualys_service
# makes many gigabytes of screenshots
failmap-admin screenshot_service
failmap screenshot_service
# Using the software
......@@ -237,7 +237,7 @@ To run only a specific test use:
To only run a specific test suite use for example:
.tox/default/bin/failmap-admin test tests/test_smarturl.py
.tox/default/bin/failmap test tests/test_smarturl.py
A coverage report is generated after running tests, on OSX it can be viewed using:
......
......@@ -52,10 +52,10 @@ services:
DEBUG: 1
# mount current source into container to allow changes to propagate without container rebuild
volumes:
- ./failmap_admin:/source/failmap_admin
- ./failmap:/source/failmap
- ./tools:/source/tools
# use watchdog to provide auto restart functionality on changed code
entrypoint: [ '/source/tools/autoreload.sh', 'failmap-admin']
entrypoint: [ '/source/tools/autoreload.sh', 'failmap']
command: [ "celery", "worker", "-l", "info", "-c", "6" ]
# web interface
......@@ -78,12 +78,12 @@ services:
UWSGI_STATIC_MAP:
# mount current source into container to allow changes to propagate without container rebuild
volumes:
- ./failmap_admin:/source/failmap_admin
- ./failmap:/source/failmap
- ./tools:/source/tools
ports:
- 8000:8000
# use watchdog to provide auto restart functionality on changed code
entrypoint: [ '/source/tools/autoreload.sh', 'failmap-admin']
entrypoint: [ '/source/tools/autoreload.sh', 'failmap']
command: [ "runuwsgi" ]
# migrate
......@@ -95,7 +95,7 @@ services:
links:
- database:mysql
volumes:
- ./failmap_admin:/source/failmap_admin
- ./failmap:/source/failmap
- ./tools:/source/tools
environment:
DJANGO_DATABASE: production
......@@ -113,7 +113,7 @@ services:
links:
- database:mysql
volumes:
- ./failmap_admin:/source/failmap_admin
- ./failmap:/source/failmap
- ./tools:/source/tools
environment:
DJANGO_DATABASE: production
......
......@@ -4,4 +4,4 @@ Application specific fixtures (user accounts, settings) for different environmen
Contains a admin user for use during development. Username `admin`, password: `faalkaart`.
failmap-admin loaddata development
failmap loaddata development
......@@ -6,20 +6,20 @@
fields: {every: 3, period: days}
- model: django_celery_beat.periodictask
pk: 1
fields: {name: Rebuild ratings, task: failmap_admin.map.tasks.rebuild_ratings, interval: 1,
fields: {name: Rebuild ratings, task: failmap.map.tasks.rebuild_ratings, interval: 1,
crontab: null, args: '[]', kwargs: '{}', queue: null, exchange: null, routing_key: null,
expires: null, enabled: true, last_run_at: null,
total_run_count: 8, date_changed: ! '2017-10-31 15:11:21+00:00',
description: ''}
- model: django_celery_beat.periodictask
pk: 2
fields: {name: scan-security-headers, task: failmap_admin.scanners.scanner_security_headers.scan,
fields: {name: scan-security-headers, task: failmap.scanners.scanner_security_headers.scan,
interval: 1, crontab: null, solar: null, args: '[[]]', kwargs: '{}', queue: null,
exchange: null, routing_key: null, expires: null, enabled: true, last_run_at: null, total_run_count: 38, date_changed: ! '2017-10-31 15:11:21+00:00',
description: ''}
- model: django_celery_beat.periodictask
pk: 3
fields: {name: scan-tls-qualys, task: failmap_admin.scanners.scanner_tls_qualys.scan,
fields: {name: scan-tls-qualys, task: failmap.scanners.scanner_tls_qualys.scan,
interval: 2, crontab: null, solar: null, args: '[]', kwargs: '{}', queue: null,
exchange: null, routing_key: null, expires: null, enabled: true, last_run_at: null, total_run_count: 38, date_changed: ! '2017-10-31 15:11:21+00:00',
description: ''}
......@@ -10,8 +10,8 @@ from celery.result import ResultSet
from django.conf import settings
from django.core.management.base import BaseCommand
from failmap_admin.app.common import ResultEncoder
from failmap_admin.celery import app
from failmap.app.common import ResultEncoder
from failmap.celery import app
log = logging.getLogger(__name__)
......
......@@ -7,7 +7,7 @@
/*
https://masterofallscience.com/
npm install -g datauri-cli
ls *.jpg | xargs -I% basename % .jpg |xargs -n1 -I% datauri --css failmap_admin/app/static/css/500.css --class=image_% %.jpg
ls *.jpg | xargs -I% basename % .jpg |xargs -n1 -I% datauri --css failmap/app/static/css/500.css --class=image_% %.jpg
*/
.image_1082039 {
......@@ -32,7 +32,7 @@ def test_job(db, mock, celery):
assert job.result_id
assert job.status == 'completed'
assert job.result == 'result'
assert job.task == 'failmap_admin.app.tests.dummy()'
assert job.task == 'failmap.app.tests.dummy()'
assert job.created_by == user
assert str(job) == job.result_id
......@@ -45,4 +45,4 @@ def test_job_no_result(db, celery):
assert job.result_id
assert job.status == 'completed'
assert job.result == '-- task generated no result object --'
assert job.task == 'failmap_admin.app.tests.dummy(False)'
assert job.task == 'failmap.app.tests.dummy(False)'
......@@ -9,11 +9,11 @@ import time
from celery import Celery, Task
from django.conf import settings
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "failmap_admin.settings")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "failmap.settings")
app = Celery(__name__)
app.config_from_object('django.conf:settings', namespace='CELERY')
# autodiscover all celery tasks in tasks.py files inside failmap_admin modules
# autodiscover all celery tasks in tasks.py files inside failmap modules
appname = __name__.split('.', 1)[0]
app.autodiscover_tasks([app for app in settings.INSTALLED_APPS if app.startswith(appname)])
......
......@@ -4,7 +4,7 @@ import sys
def main():
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "failmap_admin.settings")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "failmap.settings")
from django.core.management import execute_from_command_line
......
......@@ -2,7 +2,7 @@ import logging
from django.core.management.base import BaseCommand
from failmap_admin.map.rating import default_ratings
from failmap.map.rating import default_ratings
logger = logging.getLogger(__package__)
......
......@@ -2,7 +2,7 @@ import logging
from django.core.management.base import BaseCommand
from failmap_admin.map.models import OrganizationRating, UrlRating
from failmap.map.models import OrganizationRating, UrlRating
logger = logging.getLogger(__package__)
......
from django.core.management.base import BaseCommand
from failmap_admin.organizations.models import Organization
from failmap.organizations.models import Organization
class Command(BaseCommand):
......
from django.core.management.base import BaseCommand
from failmap_admin.map.stats import update_stats
from failmap.map.stats import update_stats
class Command(BaseCommand):
......
from django.core.management.base import BaseCommand
from failmap_admin.map.rating import DetermineRatings
from failmap.map.rating import DetermineRatings
class Command(BaseCommand):
......
import logging
from failmap_admin.app.management.commands._private import TaskCommand
from failmap.app.management.commands._private import TaskCommand
from ...rating import rebuild_ratings_async
......
import logging
from failmap_admin.app.management.commands._private import TaskCommand
from failmap.app.management.commands._private import TaskCommand
from ...rating import rerate_organizations
......
from django.db import models
from jsonfield import JSONField
from failmap_admin.organizations.models import Organization, Url
from failmap.organizations.models import Organization, Url
# Create your models here.
......
......@@ -8,8 +8,8 @@ from deepdiff import DeepDiff
from django.core.exceptions import ObjectDoesNotExist
from django.db.models import Q
from failmap_admin.organizations.models import Organization, Url
from failmap_admin.scanners.models import Endpoint, EndpointGenericScan, TlsQualysScan
from failmap.organizations.models import Organization, Url
from failmap.scanners.models import Endpoint, EndpointGenericScan, TlsQualysScan
from ..celery import app
from .models import OrganizationRating, UrlRating
......@@ -17,7 +17,7 @@ from .points_and_calculations import points_and_calculation
logger = logging.getLogger(__package__)
"""
failmap-python -m cProfile -o rebuild_complete_3 -s time `which failmap-admin` rebuild_ratings
failmap-python -m cProfile -o rebuild_complete_3 -s time `which failmap` rebuild_ratings
Expensive are:
Deepdiff,
......