Skip to content
  • Junio C Hamano's avatar
    Sane use of test_expect_failure · 41ac414e
    Junio C Hamano authored
    
    
    Originally, test_expect_failure was designed to be the opposite
    of test_expect_success, but this was a bad decision.  Most tests
    run a series of commands that leads to the single command that
    needs to be tested, like this:
    
        test_expect_{success,failure} 'test title' '
    	setup1 &&
            setup2 &&
            setup3 &&
            what is to be tested
        '
    
    And expecting a failure exit from the whole sequence misses the
    point of writing tests.  Your setup$N that are supposed to
    succeed may have failed without even reaching what you are
    trying to test.  The only valid use of test_expect_failure is to
    check a trivial single command that is expected to fail, which
    is a minority in tests of Porcelain-ish commands.
    
    This large-ish patch rewrites all uses of test_expect_failure to
    use test_expect_success and rewrites the condition of what is
    tested, like this:
    
        test_expect_success 'test title' '
    	setup1 &&
            setup2 &&
            setup3 &&
            ! this command should fail
        '
    
    test_expect_failure is redefined to serve as a reminder that
    that test *should* succeed but due to a known breakage in git it
    currently does not pass.  So if git-foo command should create a
    file 'bar' but you discovered a bug that it doesn't, you can
    write a test like this:
    
        test_expect_failure 'git-foo should create bar' '
            rm -f bar &&
            git foo &&
            test -f bar
        '
    
    This construct acts similar to test_expect_success, but instead
    of reporting "ok/FAIL" like test_expect_success does, the
    outcome is reported as "FIXED/still broken".
    
    Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    41ac414e