Improve handling of unknown content types in API fuzzing and DAST API
Problem
OpenAPI supports specifying different media types for the same Operation. The OpenAPI reader component can generate sample data for specific media types. These facts can lead to:
-
No media type can be used for a given Url. For instance, an OpenAPI document could have many operations and OpenAPI reader cannot generate sample data for any of them. The job output does not provide feedback about this scenario or how to work around it.
-
Multiple media types can be used for the same operation. For instance, the same operation can be tested using
application/json
orapplication/xml
. This leads to generating two different requests for the same operation. This behavior might not always be desirable.
Links
Originally, there was an issue about a non-supported type, and it was producing an exception. A quick fix was delivered to prevent crashes for non-supported content types. It now displays Body unavailable
.
Related issue #333026 (closed)
The issue seemed to happen with application/json
content. although, it was possible to reproduce with multipart/form-data
content.
This issue is to follow up and propose a more complete solution.
Proposal
-
In the case of no media type can be used for the given operations. -
Scanner -
OpenAPI reader to throw a ReaderException when no request has been generated and at least one of them has a content body specified. (keep in mind that GET
operations do not haveContent-Type
thus we could have many operations withoutContent-Type
). The exception should suggest user-actionable items, e.g.: suggest using supported media types in at least one operation, point out review documentation to see supported media types.
-
-
Worker-Entry -
Should be able to detect this case when invoking OpenAPIController.Validate
. Ensure the proper message is delivered to Job output on this scenario. (integration test?)
-
-
Tests -
Add C# integration test only with GET operations. Assert that no exception should be thrown. -
Add C# integration test where all operations have non-supported Content-Type. Assert that expected ReaderException is thrown. -
Add C# integration test where all operations have non-supported Content-Type, but one. Assert no exception is thrown. -
Add C# integration test to validate this specific ReaderException -
Add/Update tests with multiple operations and at least one has a supported media type. -
Add/Update tests with multiple operations and at none has a supported media type.
-
-
Documentation -
Document exception in the troubleshooting section -
Explain this behavior in OpenAPI section.
-
-
- In the case of multiple media types can be used for the same operation
- Introduce
_OPENAPI_ALL_MEDIA_TYPES
. Optional variable, Disabled by default. When set to any value, then the flag is enabled. When enabled, it uses all supported media types, otherwise, API security uses only one of any supported media type.-
Worker-Entry -
Add support to a new variable _OPENAPI_ALL_MEDIA_TYPES
. if this variable has any value (not null or not empty), it will produce requests for all the supported media types.
-
-
Scanner -
Update OpenAPIReader to check _OPENAPI_ALL_MEDIA_TYPES
variable. When_OPENAPI_ALL_MEDIA_TYPES
is set to any value and If the operation provides multiple supported media types, then all supported media types should be used, otherwise, only one supported media type should be used to create the requests.
-
-
Tests -
Existing tests were expecting requests for each supported media type. Update them to use _OPENAPI_ALL_MEDIA_TYPES
and keep their existing assertions. -
Add tests where multiple supported media types are provided for the same operation, _OPENAPI_ALL_MEDIA_TYPES
is not set. Then assert that only one of the media types gets generated requests. Assert that output provides information about skipped media types. - [-]
Add/Update integration tests (could be C# integration tests) where media type is enabledUsing existing tests. - [-]
Add/Update integration tests (could be C# integration tests) where media type is disabledUsing existing tests.
-
-
Documentation -
Add an entry for new variable _OPENAPI_ALL_MEDIA_TYPES
-
Explain the new behavior in the OpenAPI section
-
-
Publish container
-
- Introduce
_OPENAPI_MEDIA_TYPES
. Optional variable, disabled by default. To enable it should be set to a list of media types. This list of the media types is used to filter request generation-
Worker-Entry -
Add support to a new variable _OPENAPI_MEDIA_TYPES
. Pass it toOpenApiController.Validate
-
-
Scanner -
Update OpenAPIReader to check _OPENAPI_MEDIA_TYPES
variable., when set. The list of supported media types should intersect the list provided by_OPENAPI_MEDIA_TYPES
.
-
-
Tests -
Add test to Scanner (C# integration or unit test) -
Add test to Worker-Entry (integration-test)
-
-
-
Ensure _OPENAPI_ALL_MEDIA_TYPES
and_OPENAPI_MEDIA_TYPES
are mutually exclusive. Only one of them should be set during a scan session.-
Update Scanner -
Throw a ReaderException if both variables are set. -
Add test (C# integration or unit test)
-
-
Update Worker-Entry -
Add an error when both environmental variables are set -
Add integration test (pytests)
-
-
- Introduce