Support JSON references ($ref) in OpenAPI schemas

API Security supports the full Open API spec, including JSON References. However, API Security testing does not correctly parse a provided OpenAPI 3 schema which uses a JSON reference directly in the requestBody property:

{
  "paths": {
    "/pets": {
      "post": {
        "requestBody": {
          "$ref": "#/components/requestBodies/pets"
        }

This is due to a known issue in the third-party library we use to parse the schema.

High-Level Fix Options

Listing out our options for now, I will do more research on these:

  1. Pre-process the document expanding $refs. The reference syntax is part of JSON Schema, not OpenAPI specific. The pre-processing would result in a document that doesn't have any $ref statements, avoiding the limitations of the NSwag library. This could be done in Python or C#. In C# we would likely need to roll our own resolver, which it looks like Newtonsoft JSON would let us do. For Python there is a jsonref library that might do this for us.
  2. Contact the library author and request the issue be fixed. The issue has been open for 5 years, so I don't have high hopes for this.
  3. Implement the fix ourselves and submit it to the library. The library does seem to be maintained and MRs are merged semi-regularly; however there are also 100+ open MRs...
  4. Fix the library in an internal fork. Probably the easiest short-term option, but it creates a long-term maintenance issue (and also carries additional effort around consuming the library from our fork instead of via Nuget).
  5. Find another library.
  6. Write our own parser.

Analysis

Is this a bug?

The Open API docs make it clear (see Places Where $ref Can Be Used) that "$ref is only allowed in places where the OpenAPI 3.0 Specification explicitly states that the value may be a reference". The spec defines the requestBody field as optionally being a Reference Object, so the $ref property should be respected.

Where is the bug?

NSwag uses the IJsonReference interface, usually via the JsonSchema base class, to mark types which represent parts of the spec that can be defined via reference; see for example OpenApiParameter. The OpenApiRequestBody type does not implement IJsonReference, and so the $ref property is ignored.

Can we fix the main problem without an update to the library?

The built-in behavior of NSwag (via NJsonSchema) is to statically define where $ref can be used, capture the $ref during deserialization, and then post-process it to update the object from the reference. We cannot fix the post-processing because the $ref property was lost during deserialization due to not being statically defined; and we cannot change the static definition of the OpenApiRequestBody type to capture the $ref property without updating the library.

Can we resolve the reference ourselves during deserialization?

NJsonSchema uses Newtonsoft.Json to deserialize the OpenApiDocument. Newtonsoft.Json has limited support for custom reference resolvers; however that support is designed around the library's implementation of references, which identifies reference objects via an $id property and then references them by that ID in the $ref property. This is very different from how $id and $ref are used in Open API and JSON Schema (which did not exist when Newtonsoft.Json was created), and the mismatch is irreconcilable.

[TODO: What about a type converter?]

Can we preprocess the $ref property while making use of existing functionality?

NJsonSchema has reasonably good separation between identifying what references to resolve and actually resolving them. The former is where the bug is; if we identify ourselves what needs to be resolved, we can make use the built-in resolution functionality.

Open Questions

  • Do we want to respect the Open API restriction that $ref is not respected everywhere but only where it is allowed?
    • If we have pre-validated the spec, can we assume that any $ref property we find is a valid reference?
Edited by David Nelson