Generate Compatibility Chart Fails to Properly Analyze Multiple Result Documents
I just tested the case where on 2021-08-03 all the results were successful and then on 2021-08-05 I forced all the results to be a failure for Idempotency. However, they are all still being marked as idempotent with the green check.
Attached are the two results I placed in molecule/.results
before running ansibler --generate-compatibility-chart --molecule-results-dir molecule/.results
with the latest version installed.
The "Tested On" was properly updated but the Idempotent column used the value from 2021-08-03. Please ensure the fix is made for both Status and Idempotent.
For Idempotency, in the PLAY RECAP, if the changed count, unreachable count, or failed count is anything but 0 then the case must be marked as failure for Idempotency.
For the regular Status, in the PLAY RECAP, if the unreachable count or failed count is anything but 0 then it should be marked as a failure. Combine this logic with whatever you whipped up when you originally created the Status test.