Commit 59a264c3 authored by Serial Lab's avatar Serial Lab

Initial commit

parents
Pipeline #11477794 failed with stage
in 2 minutes and 6 seconds
[run]
omit =
*/__init__.py/*
*/migrations/*
serialbox/common/admin.py
serialbox/tests/settings.py
serialbox/apps.py
source =
serialbox
\ No newline at end of file
*.log
*.pot
*.pyc
*.db
*.sublime-project
.env
.project
.pytdevproject
local_settings.py
/static
/htmlcov/
/.settings/
*.idea
/.metadata/
.idea/*
/docs/site/
\ No newline at end of file
services:
- postgres
variables:
POSTGRES_DB: $SERIALBOX_DB
POSTGRES_USER: $SERIALBOX_USER
POSTGRES_PASSWORD: $SERIALBOX_PASSWORD
stages:
- test-python
- build-docs
- deploy
python3_5_unit_test:
image: seriallab/python3.5dev
stage: test-python
script:
- pip install -r requirements.txt
- pip install coverage
- python manage.py migrate
- python manage.py collectstatic --noinput
- coverage run manage.py test
- coverage report -m
- coverage html
artifacts:
paths:
- htmlcov/
python2_7_unit_test:
image: seriallab/python2.7dev
stage: test-python
script:
- pip install -r requirements.txt
- pip install coverage
- python manage.py migrate
- python manage.py collectstatic --noinput
- coverage run manage.py test
- coverage html
artifacts:
paths:
- htmlcov/
deploy:
stage: deploy
image: docker:latest
services:
- docker:dind
script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- docker build -t registry.gitlab.com/serial-lab/serialbox:latest .
- docker push registry.gitlab.com/serial-lab/serialbox:latest
only:
- tags
environment: production
pages:
image: seriallab/python3.5dev
stage: build-docs
script:
- pip install mkdocs
- mkdocs build -d ./public
artifacts:
paths:
- public
only:
- tags
Rob Magee
\ No newline at end of file
FROM python:3-onbuild
RUN apt-get update && apt-get install -y \
git \
nginx \
supervisor \
vim \
&& rm -rf /var/lib/apt/lists/*
RUN pip install uwsgi
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
COPY nginx-app.conf /etc/nginx/sites-available/default
COPY supervisor-app.conf /etc/supervisor/conf.d/
#the host name of the database container
#in the docker-compose file is the container name
#and is referenced via this variable in the
#django settings file for SerialBox
ENV SERIALBOX_DB_HOST 'postgres'
EXPOSE 80
CMD ["supervisord", "-n"]
RUN python manage.py collectstatic --noinput
# Installation
Installing and running *SerialBox* requires a working knowledge of
[Django](http://www.thedjangoproject.com). Prior to installing SerailBox,
you should understand how to start a Django project/app and configure it for
database access.
## Create a new Django Project
```
mkdir ~/Downloads/sbdemo
python django-admin.py startproject sbdemo ~/Downloads/sbdemo
cd ~/Downloads/dbdemo
```
## Modify Your settings.py Module
Add django rest framework and serialbox to your installed apps by putting
`serialbox.pools` in your settigns file's `INSTALLED_APPS` as in the example
below:
```
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'serialbox.pools',
]
```
## Add Serialbox URLS to your urls.py Module
```
from django.conf.urls import url, include
from serialbox.api import urls
urlpatterns = [
url(r'^serialbox/', include(urls)),
]
```
## Run Migrations
```
python manage.py makemigrations
python manage.py migrate
```
## Execute the Unit Tests
### Using Your Settings File
```
python manage.py test serialbox
```
### Using the Test Settings File
The SerialBox unit tests can be run independently using a sqlite database
by executing the unit tests as in the example below:
```
python manage.py test serialbox --settings=serialbox.pools.tests.settings
```
Your SerialBox installation should be complete at this point. You can
run the Django development server to test or hook into your favorite
web server. For more on running and configuring Django see the
[The Django Project web site.](http://www.thedjangoproject.com)
This diff is collapsed.
recursive-include templates static
\ No newline at end of file
[![coverage report](https://gitlab.com/serial-lab/serialbox/badges/master/coverage.svg?job=python3_5_unit_test)](https://gitlab.com/serial-lab/serialbox/commits/master) [![build status](https://gitlab.com/serial-lab/serialbox/badges/master/build.svg)](https://gitlab.com/serial-lab/serialbox/commits/master)
```
____ _ _ ____
/ ___| ___ _ __(_) __ _| | __ ) _____ __
\___ \ / _ \ '__| |/ _` | | _ \ / _ \ \/ /
___) | __/ | | | (_| | | |_) | (_) > <
|____/ \___|_| |_|\__,_|_|____/ \___/_/\_\
```
SerialBox solves the non-trivial problem of generating and distributing serial number information from system to system for the use in manufacturing and supply chain environments. SerialBox was built with Marijuana and Pharmaceutical production and distribution systems in mind but can be used for any application that requires a unique serial number distribution API.
# Open, Simple, Tested and Well Documented
SerialBox is easy to install, provides a simple RESTful API for fast and clean implementation, comes with a comprehensive suite of unit tests and is fully documented.
# Get Serial
SerialBox is distributed via the code on this site under the GPLv3 license and also via the gitlab docker registry.
## Docker Compose Project
Check out the docker compose project here:
[SerialLab serial-box-docker-compose Project](https://gitlab.com/serial-lab/serial-box-docker-compose)
## Documentation
Installation and configuration instructions can be found here:
[SerialBox Dox](https://serial-lab.gitlab.io/serialbox/installation/index.html)
version: '2'
services:
postgres:
image: postgres
environment:
POSTGRES_DB: ${SERIALBOX_DB}
POSTGRES_USER: ${SERIALBOX_USER}
POSTGRES_PASSWORD: ${SERIALBOX_PASSWORD}
web:
image: registry.gitlab.com/serial-lab/serialbox
ports:
- "80:80"
depends_on:
- postgres
# Allocation API
## Allocate
The *Allocate* API exposes the The Pool Request View- which is the primary
view by which other systems will access the API for Pool and Region
number allocation requests.
### Forming Requests
#### Format
The url should be formatted as follows. See the options table below for more
info. ::
http[s]://[servername]:[port]/allocate/[pool machine-name]/[size of the request]/?format=[format]
#### Options
* `servername` the host name or IP address of SerialBox host
* `port` The TCP port SerialBox is being hosted on. If you are using standard HTTP or HTTPS this is not required.
* `pool machine-name` The machine-name from which the numbers are being requested.
* `size of the request` A positive integer value expressing the size of the number block being requested. This can be throttled by using the *SizeLimitRule* pre-processing rule.
* `format` By default the `xml`, `json` and `csv` formats are enabled. However, this is primarily a fucntion of your [Django Rest Framework](http://www.django-rest-framework.org) configuration.
#### Example
So, for example, to request 1000 numbers from a SerialBox pool with machine
name *my-foo-bar* in *JSON*
formatting, on the host *www.foobar.com*, one would format the request url
as below: ::
```
http[s]://www.foobar.com/allocate/my-foo-bar/1000/?format=json
```
> **Important Note**: Paging is not active for this view since the number list is an
attribute of the response object.
## Handling Responses
SerialBox will return the following response fields in it's default configuration.
** numbers **
A list/array of numbers that represent the reply to the request for numbers.
**Sequential Replies**
If the request is from a *Pool* defined with *Sequential Regions* then the
reply will be sequntial in nature and will only include two numbers: the first
and last number. The range is implied via these two numbers and requesting
systems are able to use any number that falls betweent the two. The first
number will always be lesser than the last.
**Non-Sequential Replies**
The list will include all of the numbers explicitly defined and will be sized
according to the *size* parameter supplied in the request.
**fulfilled**
A boolean value representing whether or not SerialBox could fulfill the
request. This value would be false, for example, if a request were made
for 1000 numbers from a *Pool*/*Region* that only had 500 left. SerialBox
will return the 500 numbers with a fullfilled flag set to `false`.
**type**
This will be either `sequential`, `random` or `list`. Sequential will
always return **two** numbers. *Random* and *list* will always return
a list of numbers sized according to the inbound *size* parameter.
**encoding**
This will be either `decimal`, `hex` or `base-36`.
* **decimal**
A base-10 number with values from 0-9. In other words, regular old numbers.
* **hex**
A base-16 hexadecimal number with values from 0-9a-f.
* **base-36**
A base-36 number with values from 0-9a-z.
**region**
The *machine-name* of the *Region* that furninshed the request within the
*Pool* specified in the *pool machine-name* parameter of the request.
**size_granted**
The size granted. This will typically be equal to the inbound *size*
paramter; however, when the *fulfilled* flag is set to false, this will
represent the amount of numbers SerialBox was able to return to the client.
### JSON Response Example
{
"numbers": "[395, 404]",
"fulfilled": true,
"type": "",
"encoding": "decimal",
"region": "Test Region Two",
"size_granted": 10,
}
### XML Response Example
The *root* node is, aptly enough, named **root**. This is a function of the
[Django Rest Framework](http://django-rest-framework.org) and can be changed
if need be via a custom
[XML renderer](http://www.django-rest-framework.org/api-guide/renderers/).
<root>
<numbers>[985, 994]</numbers>
<fulfilled>True</fulfilled>
<type>sequential</type>
<encoding>decimal</encoding>
<region>sqr2</region>
<size_granted>10</size_granted>
</root>
### CSV Response Example
First line is, effectively, the header.
encoding,fulfilled,numbers,region,size_granted,type
decimal,True,"[995, 1004]",sqr2,10,sequential
# Getting Started: Basic Concepts
## Pools
A `pool` is the highest level concept to understand in SerialBox. A pool
defines a border around which groups of *numbers* can be organized. For example,
if you think of a pool of voters where each voter is an independent entity
but, at the same time, falls within the group due to a party affiliation you
are on the right track.
A pool contains various `region`s of numbers that are independent entities, much like the voters
in the example above, but are (through affiliation in the database) linked
to a pool. This affiliation allows the API to group requests of numbers from
specific pools in such a way that those requests can be serviced by many
independent `region` entities.
![Image of Pool](pool.png)
## Regions
As the name might suggest, a `region` is a stateful component that has a specific *begginning*
and *ending*. I use those terms because unlike, for example, sequential
number ranges that have a start value and an end value, you may have a random
region that has different types of stateful values relative to it's size, current number, etc.
Regions are what, ultimately, serve up discrete responses from the SerialBox API.
Each `region`, as it becomes depleted, enters and inactive state and the next
`region` that is available in a given pool can start servicing requests for numbers.
Depleted `region`s are marked as inactive and the next `region` available is determined
by each `region`'s `order` property.
## Generators
Generators are tightly coupled to regions. Each defined type of region (sequential,
random, word, etc.) must have a generator that knows how to respond to requests
for numbers from the pool it is coupled with. Much like each restaurant must
have waiters to service the customers, each *type* of region must have a
generator to service API requests. Out-of-the-box, SerialBox only comes
with a single Generator to handle Sequential number requests from the
`SequentialRegion` class defined within the framework; however, when SerialBox
is extended via `FlavorPackApp`s, new `Generator` classes must be defined and
paired with any new types of `region`s.
## Number Allocation
*Number Allocation* is a term used throughout the documentation to describe
the process by which *numbers* are received by request through the SerialBox
API. The request for numbers, any logging that takes place and the response
received from the allocation API all fall within the
*number allocation* process.
## FlavorPacks
*FlavorPacks* allow developers to extend and compliment the SerialBox
framework by enabling custom `Region`s, and `Generator`s...as you will see
in this documentation's [Custom FlavorPacks](flavorpacks/) documentation
example.
from recommonmark.parser import CommonMarkParser
source_parsers = {
'.md': CommonMarkParser,
}
source_suffix = ['.md']
\ No newline at end of file
# Custom Rules
Here you will find a tutorial on implementing custom pre and post processing
rules for any `Generator` class.
## Step 1. Implement a PreProcessingRule Class
Rules are very simple to both create and configure for use in SerialBox.
Start by creating a class that inherits from `serialbox.rules.common.PreProcessingRule`
or `serialbox.rules.common.PostProcessingRule` (if you want do do post-processing of
requests).
Override/implement the `execute` method on the base class which takes the
following arguments:
* `request` - The Django Rest Framework (HTTP) Request object.
* `pool` - The `serialbox.models.Pool` instance from which
numbers are being requested.
* `region` - The `serialbox.models.SequentialRegion` from which numbers
are being requested.
* `size` - an integer representing the size of the request.
In addition, if you require a custom error message or HTTP status be returned
to your client then create a custom error class that inherits from the
`serialbox.rules.errors.RuleError` and specify a custom return message along
with, if needed, an HTTP status that applies to the error. The default
HTTP status for the `RuleError` is HTTP 400 BAD REQUEST.
The example below is an implementation of a PreProcessingRule class
that checks request sizes and raises a custom exception when the a number allocation
request specifies an odd number of serial numbers.
### Example Custom Rule and Error Classes
from rest_framework import status
from serialbox.rules.common import PreprocessingRule
from serialbox.rules.errors import RuleError
class OddNumberError(RuleError):
def __init__(self, detail=None):
self.default_detail = ('No odd number requests are allowed! '
'To disable this remove the OddNumberError '
'rule from the GENERATOR_PREPROCESSING_RULES settings.')
# NOTE:
# the status is 400 by default, just including here as an example!!!
self.status_code = status.HTTP_400_BAD_REQUEST
RuleError.__init__(self, detail=detail)
class NoOddRequests(PreprocessingRule):
'''
Example rule that doesn't allow requests for odd numbers.
'''
def execute(self, request, pool, region, size):
# check the size
if size % 2 != 0:
raise OddNumberError()
Override/implement the `execute` method on the base class which takes the
following arguments:
* `request` - The Django Rest Framework (HTTP) Request object.
* `pool` - The `serialbox.models.Pool` instance from which
numbers are being requested.
* `region` - The `serialbox.models.SequentialRegion` from which numbers
are being requested.
* `size` - an integer representing the size of the request.
3. Add the fully qualified class name of your python class to the *SequentialGenerator*
list of the SerialBox GENERATOR_PREPROCESSING_RULES dictionary.
>NOTE: Make sure your class is on the python path and, in addition, make sure
you don't forget to add the existing (default) rules by concatenating the
existing rule settings with any custom ones. See the settings example below
and the [SerialBox settings](settings.md) section for more.
## Step 2. Modify your Settings File
In your Django settings file, you will want to import either `GENERATOR_PREPROCESSING_RULES`
or `GENERATOR_POSTPROCESSING_RULES` depending on what type of rule you are
implementing. This is done so that you can import the default SerialBox
rule implementation and add your rule to it. However, this is only necessary
if you wish to implement all of the default rules (which is usually a good idea)...
but, by all means, feel free to implement only the rules you wish to.
### Example Settings Entry
```python
from serialbox.serialbox_settings import GENERATOR_PREPROCESSING_RULES
GENERATOR_PREPROCESSING_RULES['serialbox.generators.sequential.SequentialGenerator'] = \
GENERATOR_PREPROCESSING_RULES['serialbox.generators.sequential.SequentialGenerator'] + \
['sbdemo.odd_number_rule.NoOddRequests']
```
Restart your server and execute an odd-numbered request against one of your
pools and you should get the error defined in the custom error class.
\ No newline at end of file
#Setting Up the .env File
SerialBox uses the python-dotenv package to maintain a file that includes
a list of specific environment variables that are used by both it's Django
settings module and it's Dockerfile. python-dotenv evaluates a file named
`.env` within your SerialBox root directory and uses those values to connect
to a Djago database backend (postgresql by default).
1. Create a .env File in your SerialBox root directory
2. Add the following environment values:
Environment Variable | Description
-------------------- | --------
SERIALBOX_USER | The SerialBox database user.
SERIALBOX_PASSWORD | The SerialBox database user's password.
SERIALBOX_DB | The name of the SerialBox database.
SERIALBOX_DB_HOST | The host of the database (not required- default is localhost)
SERIALBOX_DB_PORT | The port of the database (not required- default is 5432)
For example:
SERIALBOX_USER=myuser
SERIALBOX_PASSWORD=verysecurepassword
SERIALBOX_DB=serialbox
SERIALBOX_DB_HOST=db.host.xyz
SERIALBOX_DB_PORT=5432
\ No newline at end of file
# Extending SerialBox
SerialBox functionality can be enhanced and extended via pre and post processing
rules and FlavorPacks.
## Rules
SerialBox `rules` are executed before and after each allocation request and
are specified in a given application's Django Settings module. The order
in which rules are executed before and after each allocation API request
is determined in the dictionary and are specified by `Generator`. See the
[GENERATOR_PREPROCESSING_RULES](settings/#generator_preprocessing_rules)
documentation.
If there are no rules specified for pre or post processing using the full
class name of the `Generator` then any `default` rule specifications defined
in the either rules dictionary will be used. See the [default pre-processing rules](settings#default)
for an example of how (and what) default rules are declared.
## FlavorPacks
FlavorPacks are Django Apps that can extend the `pool` API and allow
SerialBox to serve up types of *number* regions not currently defined.
In the [Custom FlavorPacks](flavorpacks/) section you will find
a very thorough presentation of this idea.
\ No newline at end of file
This diff is collapsed.
# SerialBox Documentation
SerialBox solves the non-trivial problem of allocating serial numbers among
many distributed (and often disparate) systems where the non-duplication of serial data is of the utmost
importance. SerialBox solves this problem by becoming a centralized, easy to
implement hub where number range pools can be defined and accessed via a clean
and simple RESTful API.
## Notes
Serial box was written using the
[Django](http://www.djangoproject.com) and
[Django Rest Framework](http://django-rest-framework.org) projects.
In order to
understand some of the finer aspects of configuring and deploying SerialBox you should have
some cursory understanding of [Django](http://www.djangoproject.com) and
the [Django Rest Framework](http://django-rest-framework.org). However, for most
simple number range operations, SerialBox should function fine out of the box!
## License
SerialBox is distributed and licensed under the
[GNU General Public License v3](http://www.gnu.org/licenses/gpl-3.0.html)
Copyright (c) 2015 SerialLab LLC, All rights reserved.
# Installation
Installing and running *SerialBox* requires a working knowledge of
[Django](http://www.thedjangoproject.com) and the
[Django Rest Framework](http://django-rest-framework.org). Prior to installing SerialBox,
you should probably understand how to start a Django project/app and configure it for
database access. Nevertheless, instructions for getting everything up and
running are below...
## Create a new Django Project
```
mkdir ~/sbdemo
python django-admin.py startproject sbdemo ~/sbdemo
cd ~/dbdemo
```
## Download and Install SerialBox
If you downloaded the SerialBox code from the repository and execute the following
in the root directory.
```
python setup.py install
```
...otherwise, execute:
```
pip install serialbox
```
SerialBox will install Django and the Django Rest Framework as dependencies- along
with the `djangorestframework-xml` and `djangorestframework-csv` packages which are
necessary for XML and CSV support in the SerialBox API.
## Modify Your settings.py Module
Add django rest framework and serialbox to your installed apps by adding
`serialbox` and then `rest_framework` in your settings file's `INSTALLED_APPS` as in the example
below.
### Optionally Use the .env File to set Environment Variables
If you wish to use environment variables to drive your database connections,
see the [Environment Variables](environment_variables.md) section.
SerialBox uses the [Django Rest Framwork](http://django-rest-framework.org),
if you wish to utilize the CSV, XML and Form functionality,
you can import the REST_FRAMEWORK settings as below or enable the available
features of the framework according to your needs. JSON will be enabled
by default if you choose not to enable the additional renderers and parsers
defined in the REST_FRAMEWORK settings.
### Add serialbox and rest_framework to INSTALLED_APPS
```
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'serialbox',
'rest_framework',
]
from serialbox.serialbox_settings import REST_FRAMEWORK
```
## Configure Database Access
See the [Django Docs](https://docs.djangoproject.com/en/dev/ref/databases/) for
instructions on how to set up database access in your *settings.py* file. Make
sure to reference the proper documentation for the version of Django you are running.
*The link above is to the current development version.*
>**Note:** SerialBox will run fine on the default **SQLite** database if you want to configure
a more robust database at a later point in time.
## Production Configuration Considerations