This fix a problem with the garbage collector not being able to
clean the MetaElements that are loaded.

On small projects this is not a problem, but in bigger projects, this
can save a few hundred of MBs at runtime

The reason behind this is, whenever we have a "stack" element, which
has no stack.yaml configuration, since it doesn't need it, we would
get an exception thrown when initiating the first one, as loading the
yaml file would fail.

This would capture the frame in which this command was executed, which
references meta_elements. Therefore, as long as another exception is not
thrown, the garbage collector would not be able to clean all the
MetaElements.

Cleanup MetaElement local state

See merge request !1147

This affects the cache key version (updated to 7) and introduces
a dependency on `ujson` which is BSD licenced as of the version
locked in `requirements.txt`

Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>

Update cache keys to use JSON

See merge request !1151

We can easily rebiuld MetaSource from any source so there is no
reason to keep them around.

This will slightly improve memory usage.

Don't keep MetaSource around in Source

See merge request !1150

Resolving symlinks during staging causes various issues:

* Split rules may not work properly as the resolved paths will differ
  depending on whether another artifact with a directory symlink has
  been staged in the same root directory or not, e.g., as part of
  compose.

* The order of symlinks in file lists is difficult to get right to
  guarantee consistent and predictable behavior as paths in a file list
  might rely on symlinks in the same file list. See #647 and #817.

* Staging order differences can lead to surprising results. See #390.

* Difficult to properly support absolute symlinks. Absolute symlinks are
  currently converted to relative symlinks, however, this doesn't always
  work. See #606 and #830.

This will require changes in projects that rely on the current behavior.
However, the changes are expected to be small and are often a sign of
buggy element files. E.g., elements that don't fully obey `bindir` or
`sbindir` variables.

This matches the change in utils._process_list().

This also removes the _Resolver class as it is now unused. We may want
to support controlled symlink resolution in the future, in which case
the _Resolver class can be resurrected from this commit.

Copy symlinks as they are, absolute or relative. We no longer resolve
symlinks when copying files, which makes this safe.

Do not resolve or mangle symlinks during staging

See merge request !1140

Chandan Singh authored Feb 13, 2019 and Chandan Singh committed Feb 15, 2019

This script leverages the recently added format strings
(`%{build-deps}`, `%{runtime-deps}`) to `bst show` to print a graph in
DOT format. This requires users to have the `graphviz` python package
installed.

Additionally, users can also render the graph using the `--format`
option if they have the `graphviz` command line tool installed.

contrib/bst-graph: Add script to print graph in DOT format

Closes #705

See merge request !1148

Makes artifactdir and builddir obsolete.

Fixes #870

Will check and move old artifact directory if it exists, and create
symlink linking old directory to new.

Part of #870

This sits in Context allowing artifact cache to check the cas quota
while not being used for CASServer.

A lot of code that checks cache quota has been touched.

Part of #870

ArtifactCache now derives from BaseCache, and so will the SourceCache.
Methods relevant to both have been moved here, such as generating specs
from the config.

WIP

Part of #440

This introduced the source cache which is used in sources to avoid full
staging and will allow fetching of sources from a configured remote CAS.

Part of #440