Commit dd96bd16 authored by Jonathan Maw's avatar Jonathan Maw
Browse files

doc: Add documentation for source mirroring

parent 7f5550f4
Loading
Loading
Loading
Loading
+27 −0
Original line number Diff line number Diff line
@@ -65,6 +65,33 @@ these methods are mandatory to implement.

  **Optional**: If left unimplemented, this will default to calling
  :func:`Source.stage() <buildstream.source.Source.stage>`

* :func:`Source.get_source_fetchers() <buildstream.source.Source.get_source_fetchers>`

  Get the objects that are used for fetching.

  **Optional**: This only needs to be implemented for sources that need to
  download from multiple URLs while fetching (e.g. a git repo and its
  submodules). For details on how to define a SourceFetcher, see
  :ref:`SourceFetcher <core_source_fetcher>`.


.. _core_source_fetcher:

SourceFetcher - Object for fetching individual URLs
===================================================


Abstract Methods
----------------
SourceFetchers expose the following abstract methods. Unless explicitly
mentioned, these methods are mandatory to implement.

* :func:`SourceFetcher.fetch() <buildstream.source.SourceFetcher.fetch>`

  Fetches the URL associated with this SourceFetcher, optionally taking an
  alias override.

"""

import os
+33 −0
Original line number Diff line number Diff line
@@ -198,6 +198,39 @@ You can also specify a list of caches here; earlier entries in the list
will have higher priority than later ones.


.. _project_essentials_mirrors:

Mirrors
~~~~~~~
A list of mirrors can be defined that couple a location to a mapping of aliases to a
list of URIs, e.g.

.. code:: yaml

  mirrors:
  - name: middle-earth
    aliases:
      foo:
      - http://www.middle-earth.com/foo/1
      - http://www.middle-earth.com/foo/2
      bar:
      - http://www.middle-earth.com/bar/1
      - http://www.middle-earth.com/bar/2
  - name: oz
    aliases:
      foo:
      - http://www.oz.com/foo
      bar:
      - http://www.oz.com/bar

The order that the mirrors (and the URIs therein) are consulted is in the order
they are defined when fetching, and in reverse-order when tracking.

A default mirror to consult first can be defined via
:ref:`user config <config_default_mirror>`, or the command-line argument
:ref:`--default-mirror <invoking_bst>`.


.. _project_plugins:

External plugins
+22 −0
Original line number Diff line number Diff line
@@ -89,6 +89,28 @@ modifying some low level component.
   the ``--strict`` and ``--no-strict`` command line options.


.. _config_default_mirror:

Default Mirror
~~~~~~~~~~~~~~

When using :ref:`mirrors <project_essentials_mirrors>`, a default mirror can
be defined to be fetched first.
The default mirror is defined by its name, e.g.

.. code:: yaml

  projects:
    project-name:
      default-mirror: oz


.. note::

   It is possible to override this at invocation time using the
   ``--default-mirror`` command-line option.


Default configuration
---------------------
The default BuildStream configuration is specified here for reference: