docs: Update installation instructions

#160

BUILDSTREAM_README.rst

-Temp Demo Instructions
-======================
-
-A quick guide to getting remote execution working with BuildStream. Please change URL and certifcates / keys to your own.
-
-Downloaded and build::
-
-  https://gitlab.com/BuildStream/buildbox
-
-Copy build to bin/.
-
-Checkout branch::
-
-  https://gitlab.com/BuildStream/buildstream/tree/jmac/source_pushing_experiments
-
-Update to your URL::
-
-  https://gitlab.com/BuildStream/buildstream/blob/jmac/source_pushing_experiments/buildstream/sandbox/_sandboxremote.py#L73
-
-Start artifact server::
-
-  bst-artifact-server --port 11001 --server-key server.key --server-cert server.crt --client-certs client.crt --enable-push /home/user/
-
-Start bgd server::
-
-  bgd server start
-
-Run::
-
-  bgd bot buildbox
-
-Update project.conf in build area with::
-
-  artifacts:
-      url: https://localhost:11001
-      server-cert: server.crt
-
-      # Optional client key pair for authentication
-      client-key: client.key
-      client-cert: client.crt
-
-      push: true
-
-Run build with::
-
-  bst build --track something.bst

docs/source/installation.rst

@@ -21,20 +21,24 @@ How to install BuildGrid directly onto your machine.
 Prerequisites
 ~~~~~~~~~~~~~
 
-BuildGrid only supports ``python3 >= 3.5`` but has no system requirements. Main
-Python dependencies, automatically handled during installation, include:
+BuildGrid only supports ``python3 >= 3.5.3`` but has no system requirements.
+Main Python dependencies, automatically handled during installation, include:
 
 - `boto3`_: the Amazon Web Services (AWS) SDK for Python.
 - `click`_: a Python composable command line library.
 - `grpcio`_: Google's `gRPC`_ Python interface.
+- `janus`_: a mixed sync-async Python queue.
 - `protobuf`_: Google's `protocol-buffers`_ Python interface.
+- `PyYAML`_: a YAML parser and emitter for Python.
 
 .. _boto3: https://pypi.org/project/boto3
 .. _click: https://pypi.org/project/click
 .. _grpcio: https://pypi.org/project/grpcio
 .. _gRPC: https://grpc.io
+.. _janus: https://pypi.org/project/janus
 .. _protobuf: https://pypi.org/project/protobuf
 .. _protocol-buffers: https://developers.google.com/protocol-buffers
+.. _PyYAML: https://pypi.org/project/PyYAML
 
 
 .. _install-host-source-install:
@@ -42,20 +46,30 @@ Python dependencies, automatically handled during installation, include:
 Install from sources
 ~~~~~~~~~~~~~~~~~~~~
 
-BuildGrid has ``setuptools`` support. In order to install it to your home
-directory, typically under ``~/.local``, simply run:
+BuildGrid has ``setuptools`` support. We recommend installing it in a dedicated
+`virtual environment`_. In order to do so in an environment named ``env``
+placed in the source tree, run:
 
 .. code-block:: sh
 
-   git clone https://gitlab.com/BuildGrid/buildgrid.git && cd buildgrid
-   pip3 install --user --editable .
+   git clone https://gitlab.com/BuildGrid/buildgrid.git
+   cd buildgrid
+   python3 -m venv env
+   env/bin/python -m pip install --upgrade setuptools pip wheel
+   env/bin/python -m pip install --editable .
 
-Additionally, and if your distribution does not already include it, you may
-have to adjust your ``PATH``, in ``~/.bashrc``, with:
+.. hint::
+
+   Once created, the virtual environment can be *activated* by sourcing the
+   ``env/bin/activate`` script. In an activated terminal session, simply run
+   ``deactivate`` to later *deactivate* it.
+
+Once completed, you can check that installation succeed by locally starting the
+BuildGrid server with default configuration. Simply run:
 
 .. code-block:: sh
 
-   export PATH="${PATH}:${HOME}/.local/bin"
+   env/bin/bgd server start data/config/default.conf -vvv
 
 .. note::
 
@@ -63,66 +77,112 @@ have to adjust your ``PATH``, in ``~/.bashrc``, with:
    ``tests``. They declare required dependency for, respectively, authentication
    and authorization management, generating documentation and running
    unit-tests. They can be use as helpers for setting up a development
-   environment. To use them simply run:
+   environment. To use them run:
 
    .. code-block:: sh
 
-      pip3 install --user --editable ".[auth,docs,tests]"
+      env/bin/python -m pip install --editable ".[auth,docs,tests]"
+
+.. _virtual environment: https://docs.python.org/3/library/venv.html
 
 
 .. install-docker:
 
-Installation through Docker
----------------------------
+Install through Docker
+----------------------
 
-How to build a Docker image that runs BuildGrid.
+BuildGrid comes with Docker support for local development use-cases.
 
-.. _install-docker-prerequisites:
+.. caution::
 
-Prerequisites
-~~~~~~~~~~~~~
+   The Docker manifests are intended to be use for **local development only**.
+   Do **not** use them in production.
 
-A working Docker installation. Please consult `Docker's Getting Started Guide`_ if you don't already have it installed.
+Please consult the `Get Started with Docker`_ guide if you are looking for
+instructions on how to setup Docker on your machine.
 
-.. _`Docker's Getting Started Guide`: https://www.docker.com/get-started
+.. _`Get Started with Docker`: https://www.docker.com/get-started
 
 
 .. _install-docker-build:
 
-Docker Container from Sources
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Docker build
+~~~~~~~~~~~~
 
-To clone the source code and build a Docker image, simply run:
+BuildGrid ships a ``Dockerfile`` manifest for building images from source using
+``docker build``. In order to produce a ``buildgrid:local`` base image, run:
 
 .. code-block:: sh
 
-   git clone https://gitlab.com/BuildGrid/buildgrid.git && cd buildgrid
-   docker build -t buildgrid_server .
+   git clone https://gitlab.com/BuildGrid/buildgrid.git
+   cd buildgrid
+   docker build --tag buildgrid:local .
 
 .. note::
 
-   The image built will contain the contents of the source code directory, including
-   configuration files.
-   
+   The image built will contain the Python sources, including example
+   configuration files. The main endpoint is the ``bgd`` CLI tools and the
+   default command shall run the BuildGrid server loading default configuration.
+
+Once completed, you can check that build succeed by locally starting in a
+container the BuildGrid server with default configuration. Simply run:
+
+.. code-block:: sh
+
+   docker run --interactive --publish 50051:50051 buildgrid:local
+
 .. hint::
 
-    Whenever the source code is updated or new configuration files are made, you need to re-build 
-    the image.
+   You can run any of the BuildGrid CLI tool using that image, simply pass extra
+   arguments to ``docker run`` the same way you would pass them to ``bgd``.
+
+    Bear in mind that whenever the source code or the configuration files are
+    updated, you **must** re-build the image.
+
+
+.. _install-docker-compose:
 
-After building the Docker image, to run BuildGrid using the default configuration file 
-(found in `data/config/default.conf`), simply run:
+Docker Compose
+~~~~~~~~~~~~~~
+
+BuildGrid ships a ``docker-compose.yml`` manifest for building and running a
+grid locally using ``docker-compose``. In order to produce a
+``buildgrid:local`` base image, run:
 
 .. code-block:: sh
 
-   docker run -i -p 50051:50051 buildgrid_server
+   git clone https://gitlab.com/BuildGrid/buildgrid.git
+   cd buildgrid
+   docker-compose build
+
+Once completed, you can start a minimal grid by running:
+
+.. code-block:: sh
+
+   docker-compose up
 
 .. note::
 
-    To run BuildGrid using a different configuration file, include the relative path to the
-    configuration file at the end of the command above. For example, to run the default 
-    standalone CAS server (without an execution service), simply run:
+   The grid is composed of three containers:
+
+   - An execution and action-cache service available at
+     ``http://localhost:50051``.
+   - An CAS service available at ``http://localhost:50052``.
+   - A single ``local`` instance with one host-tools based worker bot attached.
 
-       .. code-block:: sh
+.. hint::
+
+   You can spin up more bots by using ``docker-compose`` scaling capabilities:
+
+   .. code-block:: sh
+
+      docker-compose up --scale bots=12
+
+.. hint::
 
-            docker run -i -p 50052:50052 buildgrid_server buildgrid/_app/settings/cas.yml
+   The contained services configuration files are bind mounted into the
+   container, no need to rebuild the base image on configuration update.
+   Configuration files are read from:
 
+   - ``data/config/controller.conf`` for the execution service.
+   - ``data/config/storage.conf`` for the CAS and action-cache service.