Some integration shell tests fail when run on their own

Summary

Depending on how they are run a few test cases will fail.

Steps to reproduce

Run tox -- --integration "tests/integration/shell.py::test_env_assign[Pony]" and the test fails. Other tests such as tests/integration/shell.py::test_isolated_no_mount also fail. If the tests are run together like tox -- --integration "tests/integration/shell.py" they pass, though not with -n 4 specified.

What is the current bug behavior?

Gives the following error pointing to something in _yaml.pyx

cli = <buildstream.testing.runcli.CliIntegration object at 0x7f6c9cd6bd30>, datafiles = local('/home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_env_assign_Pony_0'), animal = 'Pony'

    @pytest.mark.parametrize("animal", [("Horse"), ("Pony")])
    @pytest.mark.datafiles(DATA_DIR)
    @pytest.mark.skipif(not HAVE_SANDBOX, reason='Only available with a functioning sandbox')
    def test_env_assign(cli, datafiles, animal):
        project = str(datafiles)
        expected = animal + '\n'
    
        result = execute_shell(cli, project, ['/bin/sh', '-c', 'echo ${ANIMAL}'], config={
            'shell': {
                'environment': {
>                   'ANIMAL': animal
                }
            }
        })

tests/integration/shell.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/integration/shell.py:37: in execute_shell
    project=project, project_config=config, args=['build', element])
src/buildstream/testing/runcli.py:564: in run_project_config
    base_config = _yaml.load_data(config)
src/buildstream/_yaml.pyx:407: in buildstream._yaml.load_data
    cpdef Node load_data(str data, int file_index=_SYNTHETIC_FILE_INDEX, str file_name=None, bint copy_tree=False):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   f_info = <FileInfo> _FILE_LIST[file_index]
E   IndexError: list index out of range

src/buildstream/_yaml.pyx:440: IndexError

What is the expected correct behavior?

Test should pass regardless of how it's run. //: # (What you should see instead)

Other relevant information

Affects master, even after a git clean.


Edited by Raoul Hidalgo Charman