Use Json Schema instead of our Schema

We should use JSON Schema https://json-schema.org/ instead of our own home rolled schema.

The is an MIT implementation https://github.com/gregsdennis/json-everything but this will require us to switch from JSON.Net to Text.JSON.

Changelog Summary

We now use JSON Schema for Schemas

Schemas are a way to ensure that your data has a particular structure, for example you can control which properties are present and what types they have.

Steps which use JSON Schema:

  • CreateSchema creates a schema from an array of entities.

The step will produce the most restrictive schema possible.

- CreateSchema [('Foo': 1), ('Foo': 2)]
| Log
  • Transform tries to adjust entities so that they fit a schema.

This example transforms the 'Foo' value from a string to an integer.

- <schema> = FromJSON '{"type": "object", "properties": {"foo": {"type": "integer"}}}'
- <entities> = [('Foo': '1'), ('Foo': '2'), ('Foo': '3')]
- <results> = Transform <entities> <schema>
- <results> | ForEach | Log

You can provide additional arguments to control

  • Validate ensures that every entity in an array exactly matches the schema

This example filters the entities to only those where 'Foo' is a multiple of 2.

You could change the Error Behavior to do act differently for elements which do not match.

- <schema> = FromJSON '{"type": "object", "properties": {"foo": {"type": "integer", "MultipleOf": 2}}}'
- <entities> = [('Foo': 1), ('Foo': 2), ('Foo': 3), ('Foo': 4)]
- <results> = Validate <entities> <schema> ErrorBehavior: 'Skip'
- <results> | ForEach | Log

Resources for JSON Schemas:

Edited by Mark Wainwright