Retrieve api spec documents from protected URIs by using overrides config

Problem

API security is able to consume specification documents (HAR/Postman/OpenApi) from URLs. Some URLs are protected thus they are not available to anyone with just the URL. The API security does not support any mechanism to allow consuming documents from protected URLs.

A customer is running into this issue currently. See linked issue.

Solution

Reuse Overrides configuration when requesting specification document (HAR/Postman/OpenApi) from a URL.

Remark

• OpenApi files can be split into multiple files using $ref. The support for $ref in OpenApi 3.x is based on JSON Schema, while in OpenApi 2.0 is limited to specific sections of the document. It may be required to research how to load external resources which are protected.
• Postman Collection specification may use a complementary file Postman Variables file. It may be required to load complimentary files from a protected location.

Proposal

1. MR to update FileLoader

   1. Add a FileLoader.LoadUrl overload that accepts a OverrideDocument instance
      1. If OverrideDocument instance is provided then modify instance of HttpRequestMessage to override:
         1. headers with OverrideDocument.Headers
         2. cookies with OverrideDocument.Cookies
         3. query string with OverrideDocument.Query
      2. Add Unit test(s). If HTTP access is required consider using in-process TestHttpListener (e.g. CreateTargetServer).
      3. Test each override option
         1. For OverrideDocument.Query test if overwrites an existing query-string parameter

2. MR to update Overrides

   1. Modify the Overrides service to
      1. allow retrieving OverrideDocument instance.
         1. If the service is running, it could retrieve last updated instance.
         2. If the service is not running, it should not start the service, but load the document.
            1. document could be loaded from the file. The file may not exist but be generated by using override_cmd.
            2. document could be loaded from env var.
      2. Add unit(s) test for new method/property which retrieves OverrideDocument instance

3. MR to update Readers and Validators

   1. Update Reader to use OverrideDocument when calling FileLoader.Load for HAR, OpenAPI ($ref), Postman (postman variables), GraphQL.
      1. Follow rules from note to decide whether or not to pass OverrideDocument.
      2. Add unit tests for each format and related files
   2. Update Validator to use OverrideDocument when calling FileLoader.Load for HAR, OpenAPI ($ref), Postman (postman variables), GraphQL. Follow rules from note
      1. Follow rules from note to decide whether or not to pass OverrideDocument.
      2. Add unit tests for each format and related files
   3. Integration tests
      1. Update flask/graphql target to reject API spec document and related files. e.g. look for a specific token in Header, Cookie, or Query. If the token is provided then the content is delivered, otherwise, 4xx should be returned.
      2. Add integration test for each API spec format (each test using related files, e.g. postman variables)