Regression tests don't yet report how many have been completed so far
My thought process is largely here:
!3282 (comment 1717215133)
Other than reporting how many tests have been completed, I noted that because using fmt.Println
rather than fmt.Print
there was an opportunity to improve blank line consistency.
Develop-branch commit d8fa5f98:
RUN=core PARALLELISM=5 make test-regression
Log
fatal: not a git repository (or any of the parent directories): .git
2:27PM INF cmd/main.go:82 > running tests count=5 parallelism=5
2:27PM INF cmd/run.go:29 > Loading regression test path=suites/core/initialize.yaml
2:27PM INF cmd/run.go:100 > Running regression test blocks=1 path=suites/core/initialize.yaml
2:27PM INF cmd/run.go:106 > >>> [1] state line=0
2:27PM INF cmd/run.go:209 > Executing 4 operations
2:27PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:27PM INF cmd/run.go:211 > >>> [3] check line=6
2:27PM INF cmd/run.go:211 > >>> [4] check line=11
2:27PM INF cmd/run.go:211 > >>> [5] check line=16
2:27PM INF cmd/run.go:227 > All operations succeeded
2:27PM INF cmd/export.go:221 > State export matches expected
2:27PM INF cmd/run.go:29 > Loading regression test path=suites/core/memo.yaml
2:27PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/memo.yaml
2:27PM INF cmd/run.go:106 > >>> [1] state line=0
2:27PM INF cmd/run.go:106 > >>> [2] state line=3
2:27PM INF cmd/run.go:209 > Executing 7 operations
2:27PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:27PM INF cmd/run.go:211 > >>> [4] tx-observed-in line=8
2:27PM INF cmd/run.go:211 > >>> [5] create-blocks line=31
2:27PM INF cmd/run.go:211 > >>> [6] check line=34
2:27PM INF cmd/run.go:211 > >>> [7] tx-observed-out line=41
2:27PM INF cmd/run.go:211 > >>> [8] create-blocks line=61
2:27PM INF cmd/run.go:211 > >>> [9] check line=64
2:27PM INF cmd/run.go:227 > All operations succeeded
2:27PM INF cmd/export.go:221 > State export matches expected
2:27PM INF cmd/run.go:29 > Loading regression test path=suites/core/send.yaml
2:27PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/send.yaml
2:27PM INF cmd/run.go:106 > >>> [1] state line=0
2:27PM INF cmd/run.go:209 > Executing 9 operations
2:27PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:27PM INF cmd/run.go:211 > >>> [3] tx-send line=6
2:27PM INF cmd/run.go:211 > >>> [4] tx-send line=16
2:27PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:27PM INF cmd/run.go:211 > >>> [6] check line=26
2:27PM INF cmd/run.go:211 > >>> [7] tx-send line=31
2:27PM INF cmd/run.go:211 > >>> [8] tx-send line=42
2:27PM INF cmd/run.go:211 > >>> [9] create-blocks line=50
2:27PM INF cmd/run.go:211 > >>> [10] check line=53
2:27PM INF cmd/run.go:227 > All operations succeeded
2:27PM INF cmd/export.go:221 > State export matches expected
2:27PM INF cmd/run.go:29 > Loading regression test path=suites/core/vault-frozen.yaml
2:27PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/vault-frozen.yaml
2:27PM INF cmd/run.go:106 > >>> [1] state line=0
2:27PM INF cmd/run.go:106 > >>> [2] state line=3
2:27PM INF cmd/run.go:209 > Executing 10 operations
2:27PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:27PM INF cmd/run.go:211 > >>> [4] tx-tss-keysign line=8
2:27PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:27PM INF cmd/run.go:211 > >>> [6] check line=26
2:27PM INF cmd/run.go:211 > >>> [7] tx-deposit line=32
2:27PM INF cmd/run.go:211 > >>> [8] create-blocks line=39
2:27PM INF cmd/run.go:211 > >>> [9] check line=42
2:27PM INF cmd/run.go:211 > >>> [10] tx-observed-out line=47
2:27PM INF cmd/run.go:211 > >>> [11] create-blocks line=67
2:27PM INF cmd/run.go:211 > >>> [12] check line=70
2:27PM INF cmd/run.go:227 > All operations succeeded
2:27PM INF cmd/export.go:221 > State export matches expected
2:27PM INF cmd/run.go:29 > Loading regression test path=suites/core/outbound-fee.yaml
2:27PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/outbound-fee.yaml
2:27PM INF cmd/run.go:106 > >>> [1] state line=0
2:27PM INF cmd/run.go:106 > >>> [2] state line=3
2:27PM INF cmd/run.go:209 > Executing 13 operations
2:27PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:27PM INF cmd/run.go:211 > >>> [4] check line=8
2:27PM INF cmd/run.go:211 > >>> [5] tx-deposit line=14
2:27PM INF cmd/run.go:211 > >>> [6] create-blocks line=21
2:27PM INF cmd/run.go:211 > >>> [7] check line=24
2:27PM INF cmd/run.go:211 > >>> [8] tx-observed-out line=29
2:27PM INF cmd/run.go:211 > >>> [9] create-blocks line=49
2:27PM INF cmd/run.go:211 > >>> [10] check line=52
2:27PM INF cmd/run.go:211 > >>> [11] check line=57
2:27PM INF cmd/run.go:211 > >>> [12] tx-observed-in line=63
2:27PM INF cmd/run.go:211 > >>> [13] create-blocks line=83
2:27PM INF cmd/run.go:211 > >>> [14] check line=86
2:27PM INF cmd/run.go:211 > >>> [15] check line=92
2:27PM INF cmd/run.go:227 > All operations succeeded
2:27PM INF cmd/export.go:221 > State export matches expected
Succeeded: 5
- suites/core/initialize.yaml
- suites/core/memo.yaml
- suites/core/send.yaml
- suites/core/vault-frozen.yaml
- suites/core/outbound-fee.yaml
Failed: 0
(no blank line under the 'running tests' line at the start,
two blank lines before the 'Succeeded:' line near the bottom)
RUN=core make test-regression
Log
fatal: not a git repository (or any of the parent directories): .git
2:29PM INF cmd/main.go:82 > running tests count=5 parallelism=1
2:29PM INF cmd/run.go:29 > Loading regression test path=suites/core/outbound-fee.yaml
2:29PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/outbound-fee.yaml
2:29PM INF cmd/run.go:106 > >>> [1] state line=0
2:29PM INF cmd/run.go:106 > >>> [2] state line=3
2:29PM INF cmd/run.go:209 > Executing 13 operations
2:29PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:29PM INF cmd/run.go:211 > >>> [4] check line=8
2:29PM INF cmd/run.go:211 > >>> [5] tx-deposit line=14
2:29PM INF cmd/run.go:211 > >>> [6] create-blocks line=21
2:29PM INF cmd/run.go:211 > >>> [7] check line=24
2:29PM INF cmd/run.go:211 > >>> [8] tx-observed-out line=29
2:29PM INF cmd/run.go:211 > >>> [9] create-blocks line=49
2:29PM INF cmd/run.go:211 > >>> [10] check line=52
2:29PM INF cmd/run.go:211 > >>> [11] check line=57
2:29PM INF cmd/run.go:211 > >>> [12] tx-observed-in line=63
2:29PM INF cmd/run.go:211 > >>> [13] create-blocks line=83
2:29PM INF cmd/run.go:211 > >>> [14] check line=86
2:29PM INF cmd/run.go:211 > >>> [15] check line=92
2:29PM INF cmd/run.go:227 > All operations succeeded
2:29PM INF cmd/export.go:221 > State export matches expected
2:29PM INF cmd/run.go:29 > Loading regression test path=suites/core/vault-frozen.yaml
2:29PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/vault-frozen.yaml
2:29PM INF cmd/run.go:106 > >>> [1] state line=0
2:29PM INF cmd/run.go:106 > >>> [2] state line=3
2:29PM INF cmd/run.go:209 > Executing 10 operations
2:29PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:29PM INF cmd/run.go:211 > >>> [4] tx-tss-keysign line=8
2:29PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:29PM INF cmd/run.go:211 > >>> [6] check line=26
2:29PM INF cmd/run.go:211 > >>> [7] tx-deposit line=32
2:29PM INF cmd/run.go:211 > >>> [8] create-blocks line=39
2:29PM INF cmd/run.go:211 > >>> [9] check line=42
2:29PM INF cmd/run.go:211 > >>> [10] tx-observed-out line=47
2:29PM INF cmd/run.go:211 > >>> [11] create-blocks line=67
2:29PM INF cmd/run.go:211 > >>> [12] check line=70
2:29PM INF cmd/run.go:227 > All operations succeeded
2:29PM INF cmd/export.go:221 > State export matches expected
2:29PM INF cmd/run.go:29 > Loading regression test path=suites/core/memo.yaml
2:29PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/memo.yaml
2:29PM INF cmd/run.go:106 > >>> [1] state line=0
2:29PM INF cmd/run.go:106 > >>> [2] state line=3
2:29PM INF cmd/run.go:209 > Executing 7 operations
2:29PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:29PM INF cmd/run.go:211 > >>> [4] tx-observed-in line=8
2:29PM INF cmd/run.go:211 > >>> [5] create-blocks line=31
2:29PM INF cmd/run.go:211 > >>> [6] check line=34
2:29PM INF cmd/run.go:211 > >>> [7] tx-observed-out line=41
2:29PM INF cmd/run.go:211 > >>> [8] create-blocks line=61
2:29PM INF cmd/run.go:211 > >>> [9] check line=64
2:29PM INF cmd/run.go:227 > All operations succeeded
2:29PM INF cmd/export.go:221 > State export matches expected
2:29PM INF cmd/run.go:29 > Loading regression test path=suites/core/send.yaml
2:29PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/send.yaml
2:29PM INF cmd/run.go:106 > >>> [1] state line=0
2:29PM INF cmd/run.go:209 > Executing 9 operations
2:29PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:29PM INF cmd/run.go:211 > >>> [3] tx-send line=6
2:29PM INF cmd/run.go:211 > >>> [4] tx-send line=16
2:29PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:29PM INF cmd/run.go:211 > >>> [6] check line=26
2:29PM INF cmd/run.go:211 > >>> [7] tx-send line=31
2:29PM INF cmd/run.go:211 > >>> [8] tx-send line=42
2:29PM INF cmd/run.go:211 > >>> [9] create-blocks line=50
2:29PM INF cmd/run.go:211 > >>> [10] check line=53
2:29PM INF cmd/run.go:227 > All operations succeeded
2:29PM INF cmd/export.go:221 > State export matches expected
2:29PM INF cmd/run.go:29 > Loading regression test path=suites/core/initialize.yaml
2:29PM INF cmd/run.go:100 > Running regression test blocks=1 path=suites/core/initialize.yaml
2:29PM INF cmd/run.go:106 > >>> [1] state line=0
2:29PM INF cmd/run.go:209 > Executing 4 operations
2:29PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:29PM INF cmd/run.go:211 > >>> [3] check line=6
2:29PM INF cmd/run.go:211 > >>> [4] check line=11
2:29PM INF cmd/run.go:211 > >>> [5] check line=16
2:29PM INF cmd/run.go:227 > All operations succeeded
2:29PM INF cmd/export.go:221 > State export matches expected
Succeeded: 5
- suites/core/outbound-fee.yaml
- suites/core/vault-frozen.yaml
- suites/core/memo.yaml
- suites/core/send.yaml
- suites/core/initialize.yaml
Failed: 0
Regression Coverage: 17.0%
(different--a blank line under the 'running tests' line at the start, and only one blank line above the 'Succeeded:' line near the bottom)
Proposed commit 3acbad65:
RUN=core PARALLELISM=5 make test-regression
Log
fatal: not a git repository (or any of the parent directories): .git
2:20PM INF cmd/main.go:84 > running tests count=5 parallelism=5
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/initialize.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=1 path=suites/core/initialize.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:209 > Executing 4 operations
2:20PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:20PM INF cmd/run.go:211 > >>> [3] check line=6
2:20PM INF cmd/run.go:211 > >>> [4] check line=11
2:20PM INF cmd/run.go:211 > >>> [5] check line=16
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 1/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/memo.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/memo.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:106 > >>> [2] state line=3
2:20PM INF cmd/run.go:209 > Executing 7 operations
2:20PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:20PM INF cmd/run.go:211 > >>> [4] tx-observed-in line=8
2:20PM INF cmd/run.go:211 > >>> [5] create-blocks line=31
2:20PM INF cmd/run.go:211 > >>> [6] check line=34
2:20PM INF cmd/run.go:211 > >>> [7] tx-observed-out line=41
2:20PM INF cmd/run.go:211 > >>> [8] create-blocks line=61
2:20PM INF cmd/run.go:211 > >>> [9] check line=64
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 2/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/send.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/send.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:209 > Executing 9 operations
2:20PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:20PM INF cmd/run.go:211 > >>> [3] tx-send line=6
2:20PM INF cmd/run.go:211 > >>> [4] tx-send line=16
2:20PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:20PM INF cmd/run.go:211 > >>> [6] check line=26
2:20PM INF cmd/run.go:211 > >>> [7] tx-send line=31
2:20PM INF cmd/run.go:211 > >>> [8] tx-send line=42
2:20PM INF cmd/run.go:211 > >>> [9] create-blocks line=50
2:20PM INF cmd/run.go:211 > >>> [10] check line=53
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 3/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/vault-frozen.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/vault-frozen.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:106 > >>> [2] state line=3
2:20PM INF cmd/run.go:209 > Executing 10 operations
2:20PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:20PM INF cmd/run.go:211 > >>> [4] tx-tss-keysign line=8
2:20PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:20PM INF cmd/run.go:211 > >>> [6] check line=26
2:20PM INF cmd/run.go:211 > >>> [7] tx-deposit line=32
2:20PM INF cmd/run.go:211 > >>> [8] create-blocks line=39
2:20PM INF cmd/run.go:211 > >>> [9] check line=42
2:20PM INF cmd/run.go:211 > >>> [10] tx-observed-out line=47
2:20PM INF cmd/run.go:211 > >>> [11] create-blocks line=67
2:20PM INF cmd/run.go:211 > >>> [12] check line=70
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 4/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/outbound-fee.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/outbound-fee.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:106 > >>> [2] state line=3
2:20PM INF cmd/run.go:209 > Executing 13 operations
2:20PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:20PM INF cmd/run.go:211 > >>> [4] check line=8
2:20PM INF cmd/run.go:211 > >>> [5] tx-deposit line=14
2:20PM INF cmd/run.go:211 > >>> [6] create-blocks line=21
2:20PM INF cmd/run.go:211 > >>> [7] check line=24
2:20PM INF cmd/run.go:211 > >>> [8] tx-observed-out line=29
2:20PM INF cmd/run.go:211 > >>> [9] create-blocks line=49
2:20PM INF cmd/run.go:211 > >>> [10] check line=52
2:20PM INF cmd/run.go:211 > >>> [11] check line=57
2:20PM INF cmd/run.go:211 > >>> [12] tx-observed-in line=63
2:20PM INF cmd/run.go:211 > >>> [13] create-blocks line=83
2:20PM INF cmd/run.go:211 > >>> [14] check line=86
2:20PM INF cmd/run.go:211 > >>> [15] check line=92
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 5/5 regression tests completed.
Succeeded: 5
- suites/core/initialize.yaml
- suites/core/memo.yaml
- suites/core/send.yaml
- suites/core/vault-frozen.yaml
- suites/core/outbound-fee.yaml
Failed: 0
Regression Coverage: 17.0%
('regression tests completed' lines in order,
blank line under the 'running tests' line at the start,
only one blank line above the 'Succeeded:' line near the bottom)
RUN=core PARALLELISM=1 make test-regression
Log
fatal: not a git repository (or any of the parent directories): .git
2:19PM INF cmd/main.go:84 > running tests count=5 parallelism=1
2:19PM INF cmd/run.go:29 > Loading regression test path=suites/core/outbound-fee.yaml
2:19PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/outbound-fee.yaml
2:19PM INF cmd/run.go:106 > >>> [1] state line=0
2:19PM INF cmd/run.go:106 > >>> [2] state line=3
2:19PM INF cmd/run.go:209 > Executing 13 operations
2:19PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:19PM INF cmd/run.go:211 > >>> [4] check line=8
2:19PM INF cmd/run.go:211 > >>> [5] tx-deposit line=14
2:19PM INF cmd/run.go:211 > >>> [6] create-blocks line=21
2:19PM INF cmd/run.go:211 > >>> [7] check line=24
2:19PM INF cmd/run.go:211 > >>> [8] tx-observed-out line=29
2:19PM INF cmd/run.go:211 > >>> [9] create-blocks line=49
2:19PM INF cmd/run.go:211 > >>> [10] check line=52
2:19PM INF cmd/run.go:211 > >>> [11] check line=57
2:19PM INF cmd/run.go:211 > >>> [12] tx-observed-in line=63
2:19PM INF cmd/run.go:211 > >>> [13] create-blocks line=83
2:19PM INF cmd/run.go:211 > >>> [14] check line=86
2:19PM INF cmd/run.go:211 > >>> [15] check line=92
2:19PM INF cmd/run.go:227 > All operations succeeded
2:19PM INF cmd/export.go:221 > State export matches expected
2:19PM INF cmd/main.go:130 > 1/5 regression tests completed.
2:19PM INF cmd/run.go:29 > Loading regression test path=suites/core/vault-frozen.yaml
2:19PM INF cmd/run.go:100 > Running regression test blocks=4 path=suites/core/vault-frozen.yaml
2:19PM INF cmd/run.go:106 > >>> [1] state line=0
2:19PM INF cmd/run.go:106 > >>> [2] state line=3
2:19PM INF cmd/run.go:209 > Executing 10 operations
2:19PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:20PM INF cmd/run.go:211 > >>> [4] tx-tss-keysign line=8
2:20PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:20PM INF cmd/run.go:211 > >>> [6] check line=26
2:20PM INF cmd/run.go:211 > >>> [7] tx-deposit line=32
2:20PM INF cmd/run.go:211 > >>> [8] create-blocks line=39
2:20PM INF cmd/run.go:211 > >>> [9] check line=42
2:20PM INF cmd/run.go:211 > >>> [10] tx-observed-out line=47
2:20PM INF cmd/run.go:211 > >>> [11] create-blocks line=67
2:20PM INF cmd/run.go:211 > >>> [12] check line=70
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 2/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/memo.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/memo.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:106 > >>> [2] state line=3
2:20PM INF cmd/run.go:209 > Executing 7 operations
2:20PM INF cmd/run.go:211 > >>> [3] create-blocks line=5
2:20PM INF cmd/run.go:211 > >>> [4] tx-observed-in line=8
2:20PM INF cmd/run.go:211 > >>> [5] create-blocks line=31
2:20PM INF cmd/run.go:211 > >>> [6] check line=34
2:20PM INF cmd/run.go:211 > >>> [7] tx-observed-out line=41
2:20PM INF cmd/run.go:211 > >>> [8] create-blocks line=61
2:20PM INF cmd/run.go:211 > >>> [9] check line=64
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 3/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/send.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=3 path=suites/core/send.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:209 > Executing 9 operations
2:20PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:20PM INF cmd/run.go:211 > >>> [3] tx-send line=6
2:20PM INF cmd/run.go:211 > >>> [4] tx-send line=16
2:20PM INF cmd/run.go:211 > >>> [5] create-blocks line=23
2:20PM INF cmd/run.go:211 > >>> [6] check line=26
2:20PM INF cmd/run.go:211 > >>> [7] tx-send line=31
2:20PM INF cmd/run.go:211 > >>> [8] tx-send line=42
2:20PM INF cmd/run.go:211 > >>> [9] create-blocks line=50
2:20PM INF cmd/run.go:211 > >>> [10] check line=53
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 4/5 regression tests completed.
2:20PM INF cmd/run.go:29 > Loading regression test path=suites/core/initialize.yaml
2:20PM INF cmd/run.go:100 > Running regression test blocks=1 path=suites/core/initialize.yaml
2:20PM INF cmd/run.go:106 > >>> [1] state line=0
2:20PM INF cmd/run.go:209 > Executing 4 operations
2:20PM INF cmd/run.go:211 > >>> [2] create-blocks line=3
2:20PM INF cmd/run.go:211 > >>> [3] check line=6
2:20PM INF cmd/run.go:211 > >>> [4] check line=11
2:20PM INF cmd/run.go:211 > >>> [5] check line=16
2:20PM INF cmd/run.go:227 > All operations succeeded
2:20PM INF cmd/export.go:221 > State export matches expected
2:20PM INF cmd/main.go:130 > 5/5 regression tests completed.
Succeeded: 5
- suites/core/outbound-fee.yaml
- suites/core/vault-frozen.yaml
- suites/core/memo.yaml
- suites/core/send.yaml
- suites/core/initialize.yaml
Failed: 0
Regression Coverage: 17.0%
(same)
That said,
https://gitlab.com/thorchain/thornode/-/jobs/5878475834
bears further thought, for instance with its count=60 parallelism=8
displaying
fatal: not a git repository (or any of the parent directories): .git
2:38PM INF cmd/main.go:84 > running tests count=60 parallelism=8
2:39PM INF cmd/main.go:130 > 1/60 regression tests completed.
2:38PM INF cmd/run.go:29 > Loading regression test path=suites/lending/lending.yaml
2:39PM INF cmd/run.go:211 > >>> [78] check line=606
2:39PM INF cmd/run.go:227 > All operations succeeded
2:39PM INF cmd/export.go:221 > State export matches expected
2:39PM INF cmd/main.go:130 > 4/60 regression tests completed.
2:39PM INF cmd/main.go:130 > 5/60 regression tests completed.
2:38PM INF cmd/run.go:29 > Loading regression test path=suites/savers/streaming.yaml
with no blank lines
(perhaps unavoidable in a pipeline job?)
and out-of-order logging display timing(/timestamps)
(which I would definitely prefer to avoid if possible).
(Since
fmt.Print(buf.String())
and
log.Info().Msg(fmt.Sprintf("%d/%d regression tests completed.", completed, total))
are the source of the different lines,
and they take place within the same mu.Lock()
/mu.Unlock()
region,
how can it be possible for them to be viewed out-of-order?)
2024-01-08 update:
Looking into a remembered out-of-order aspect of broken invariant displays,
6abd779b
was prepared, noting incidentally the leftover files
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/mnt/exports/suites_core_memo.json
( https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/mnt/exports/core/memo.json )
and
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/mnt/exports/suites_swaps_streaming-swaps.json
( https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/mnt/exports/swaps/streaming-swaps.json )
.
make test regression
fatal: not a git repository (or any of the parent directories): .git
7:59AM INF cmd/main.go:82 > running tests count=1 parallelism=1
7:59AM INF cmd/run.go:29 > Loading regression test path=suites/minimal-broken-invariant.yaml
7:59AM INF cmd/run.go:100 > Running regression test blocks=1 path=suites/minimal-broken-invariant.yaml
7:59AM INF cmd/run.go:106 > >>> [1] state line=0
7:59AM INF cmd/run.go:106 > >>> [2] state line=3
7:59AM INF cmd/run.go:106 > >>> [3] state line=5
7:59AM INF cmd/run.go:209 > Executing 1 operations
7:59AM INF cmd/run.go:211 > >>> [4] create-blocks line=20
7:59AM INF cmd/run.go:227 > All operations succeeded
BTC.BTC pool has 100000000 more than its vaults
Succeeded: 0
Failed: 1
- suites/minimal-broken-invariant.yaml
Normal, 'more than its vaults' message after the operations' lines.
PARALLELISM=2 make test-regression
fatal: not a git repository (or any of the parent directories): .git
7:57AM INF cmd/main.go:82 > running tests count=1 parallelism=2
BTC.BTC pool has 100000000 more than its vaults
7:57AM INF cmd/run.go:29 > Loading regression test path=suites/minimal-broken-invariant.yaml
7:57AM INF cmd/run.go:100 > Running regression test blocks=1 path=suites/minimal-broken-invariant.yaml
7:57AM INF cmd/run.go:106 > >>> [1] state line=0
7:57AM INF cmd/run.go:106 > >>> [2] state line=3
7:57AM INF cmd/run.go:106 > >>> [3] state line=5
7:57AM INF cmd/run.go:209 > Executing 1 operations
7:57AM INF cmd/run.go:211 > >>> [4] create-blocks line=20
7:57AM INF cmd/run.go:227 > All operations succeeded
Succeeded: 0
Failed: 1
- suites/minimal-broken-invariant.yaml
No blank line at the top and two blank lines at the bottom (solution already noted above),
but the 'more than its vaults' is printed before the operations' lines.
This is an out-of-order matter apparently unrelated to the pipeline job out-of-order log display.
Corresponding (parallelism=8
) pipeline job also demonstrating this:
https://gitlab.com/thorchain/thornode/-/jobs/5880758359
Relevant code:
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/main.go#L109-114
// create a buffer to capture the logs
var out io.Writer = os.Stderr
buf := new(bytes.Buffer)
if parallelism > 1 {
out = buf
}
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/main.go#L133
err = run(out, file, routine)
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/run.go#L26
localLog := consoleLogger(out)
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/main.go#L145
err = export(out, file, routine)
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/export.go#L26
localLog := consoleLogger(out)
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/main.go#L123-125
if parallelism > 1 {
fmt.Println(buf.String())
}
I wonder then why the downstream log is apparently being added to the top of the buffer,
and whether passing a common consoleLogger(out)
(for each file) could avoid the out-of-order effect.
Edit:
--No, 39febe4f does not change the observed behaviour.
Second edit:
Ahh.
https://gitlab.com/thorchain/thornode/-/blob/v1.125.3/test/regression/cmd/export.go#L171-185
Printf
within checkExportInvariants
rather than logging, thus explaining why it unintuitively gets printed before the operations;
my impulse is then to try changing those Printf
s to log lines
(specifically https://gitlab.com/thorchain/thornode/-/jobs/5883629192 )
, though I remain puzzled by the pipeline job out-of-order logs noted above.
Third edit:
The common logger approach appears to resolve the out-of-order pipeline job logs matter
(saving the reporting line to the buffer rather than trying to log it after the buffer printing):
https://gitlab.com/thorchain/thornode/-/jobs/5883225267
as I'm uncertain about the behaviour of the drainLogs
calls, at present I'm leaning towards a single-use localLog instead, which appears to resolve it similarly:
https://gitlab.com/thorchain/thornode/-/jobs/5883257105
Overall suggested commits at this time:
https://gitlab.com/thorchain/thornode/-/commits/fa2ae734
( https://gitlab.com/thorchain/thornode/-/jobs/5883742024 )
More specifically: !3352 (merged)
'Report how many regression tests have been completed so far'.