Speed: Create WebCheckStrategy specific for use with WebRunnerMachine
Problem
WebCheckStrategy was designed for use with the proxy code. This design is not optimal for use with the runner the code and places limitations on threading work.
Proposal
Create a new WebRunnerCheckStrategy class refactored to work better with WebRunnerMachine and is ready for threading work.
Goals:
- Simplify interface
- Eye towards concurrent usage
- Don't use global state
- Remove global Lists
Strategy was originally written to contain a hidden for loop that would work using the proxy code. _mutationsPending contained an the unrolled for loop with each MutationContext instances holding an index value.
In the new version, Strategy will produce work items that can be used to parallelize the work. Each work item will be one of the following:
- Active check
- Parameter + Mutation check
These will by scheduled by the WebCheckRunner.
Types of Work Items
- Parameter + mutation check
- Active check
TODO
-
Verify Active checks are working as work items -
EvaluateScope section of Record is not reentrant. Needs a lock or redesign. -
_mutationEnumerator should be passed in instead of class variable -
_mutationsPending -
_shouldContinue -
What about _logsink? Threading will cause logs to be seen across mutations giving false positives. -
_testCaseName -
SavingJob -- Move this to a timer event instead of checking time on each iteration -
System.Timers.Timer
-
-
[ ] HttpConnectionSettings - This can change with the route!- Moved to threading issue.
-
How do we know when to print Finished testing operation message? -
Verify stats are working -
Verify tests are passing/update tests - Outstanding as of 5/16 -
Peach.Web.Test.Integration.Runner.IntegrationTests.TestMissingClientSideCerts
-
Peach.Web.Test.Integration.Runner.IntegrationTests.TestErrorTimeout
-
test_large_har_report_with_assets
-
[ ]false positive, test is flakytest_with_openapi_tls_url_over_https
-
tests_int_config_fuzzing-quick
- Partially identified but not resolved
-
-
Investigate why check counts are wrong in "Finished Testing" section of console output -
Console output re-Testing/Finished testing message. -
Too many Finished testing operation
messages -
Output improvements: -
Change operations with no work items to say Operation skipped
. -
Mention it is likely a duplicate. -
Put duplicates last.
-
-
Why are operations with no work items not showing up first? -
Compare console output with master to confirm
-
-
Stats console output -- Performed N requests are wrong. They always get bigger. -
[ ] RestoreWe decided the code it was wrapping was not complex enough to need this special handling.InnermostException()
handling -
Verify that e2e tests pass in feature branch before merging
Edited by David Nelson