NID: testenv-runcmd errors
Using the testenv-runcmd
target to run a NSO CLI command will always succeed, even if the command failed.
The commit below failed but the test didn't stop. In this particular case, the test fails later when verifying the action was performed by observing the results. But users would rightfully expect that the step would fail if the NSO commands fails.
make[2]: Entering directory '/home/mzagozen/foss/nso/post-upgrade-actions'
docker exec -t testenv-post-upgrade-actions-5.3-mzagozen-nso bash -lc 'echo -e "configure\n set post-upgrade-actions foo-factor xpath /pua-tester:pua-tester action-name increment enabled\n commit" | ncs_cli -u admin -g ncsadmin'
set post-upgrade-actions
syntax error: element does not exist
[error][2020-02-05 07:40:40]
make[2]: Leaving directory '/home/mzagozen/foss/nso/post-upgrade-actions'
Now, for the solution, it isn't clear cut. ncs_cli
does exit with different exit codes, but those correspond to its own errors in communication with NSO daemon, not to commands the user executes. If ncs_cli
is able to successfully execute the commands for the user, its exit code will always be 0. Building a complex wrapper around ncs_cli
to parse out all possible error messages seems like a hack too - we have perfectly good error reporting built into NETCONF already :). I propose every invocation of testenv-runcmd
also includes success criteria, something like | grep '$SUCCESS'
. The success criteria could be passed with another variable, like $CMD
. I would have done this already, but I don't like the fact that grep
swallows the output. Is there an elegant way of running grep in pass-through mode?