Broken link test enhancements

Originally from #77 (closed). This issue captures the complete set of broken link enhancements. Implementation for each is captured in #105 (closed), #112 (closed), and #115 as noted below.

Add the following configuration options:

  • ignoreDuplicates - Do not repeat checks for links already checked in this run (default: true) (see #112 (closed))
  • ignoreQuerystring - Ignore querystring checking links (default: false) (see #132)
  • checkLocalFiles - Check local file system for file if file:// and pass if found (default: true) (see #115)
  • checkLinksFromBrowser - Fetch links from browser page rather than check with axios (default: false) (see #105 (closed))

With this, can update processing logic (bold - new capabilities):

  • If relative link on same page
    • Check page for element
  • Normalize URL. Remove hash, and querystring if ignoreQuerystring. (see #112 (closed), #132)
  • If URL is in ignored links array, return 100
  • If ignoreDuplicates and URL has been processed during this test run, return previous result (see #112 (closed))
  • If file and checkLocalFiles (see #115, #112 (closed))
    • Convert URL to file path, removing hash, and if file exists return 200, otherwise 404
    • Save checked URL without hash
  • If checkLinksFromBrowser (see #105 (closed), #112 (closed))
    • Send fetch to page and return response
    • Save checked URL without hash
  • Make head request with axios.
    • If pass, or fail with 429 (too many requests), return response.
    • If failed with other code, retry with get request and return response.
Edited by Aaron Goldenthal