Commit 54ed57b9 authored by Bob Van Landuyt's avatar Bob Van Landuyt 💬
Browse files

Merge branch 'feat-json-schema-ext-ref' into 'master'

feat: Adding validation for feature categories using an external reference

Closes gitlab-com/gl-infra/observability/team#4219

See merge request !200

Merged-by: Bob Van Landuyt's avatarBob Van Landuyt <bob@gitlab.com>
Approved-by: Elliot Forbes's avatarElliot Forbes <eforbes@gitlab.com>
Approved-by: Bob Van Landuyt's avatarBob Van Landuyt <bob@gitlab.com>
Approved-by: default avatarPeter Leitzen <pleitzen@gitlab.com>
Reviewed-by: default avatarPeter Leitzen <pleitzen@gitlab.com>
Reviewed-by: default avatarGitLab Duo <gitlab-duo@gitlab.com>
Co-authored-by: Hercules Merscher's avatarHercules Merscher <hmerscher@gitlab.com>
parents 4dd97248 46d766c6
Loading
Loading
Loading
Loading
Loading
+3 −3
Original line number Diff line number Diff line
{
  "$schema": "http://json-schema.org/draft-06/schema#",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby/-/raw/master/config/user_experience_slis/schema.json",
  "title": "User Experience SLI Definition",
  "description": "Schema for GitLab User Experience SLI files",
@@ -12,8 +12,8 @@
    },
    "feature_category": {
      "type": "string",
      "minLength": 1,
      "description": "GitLab feature category this experience belongs to"
      "description": "GitLab feature category this experience belongs to",
      "$ref": "https://gitlab.com/gitlab-org/gitlab/-/raw/master/config/feature_categories/schema.json"
    },
    "urgency": {
      "type": "string",
+2 −1
Original line number Diff line number Diff line
@@ -25,7 +25,8 @@ Gem::Specification.new do |spec|
  spec.add_runtime_dependency "grpc", ">= 1.75" # Be sure to update the "grpc-tools" dev_dependency too
  spec.add_runtime_dependency "google-protobuf", ">= 3.25", "< 5.0"
  spec.add_runtime_dependency "jaeger-client", "~> 1.1.0"
  spec.add_runtime_dependency 'json-schema', '~> 5.1'
  spec.add_runtime_dependency 'json_schemer', '~> 2.4.0'
  spec.add_runtime_dependency "openssl", "~> 3.3.2"
  spec.add_runtime_dependency "opentracing", "~> 0.4"
  spec.add_runtime_dependency "pg_query", ">= 6.1.0", "< 7.0"
  spec.add_runtime_dependency "prometheus-client-mmap", "~> 1.2.9"
+74 −0
Original line number Diff line number Diff line
# JSON Schema Reference Resolver

## Overview

The `RefResolver` class provides HTTP/HTTPS-based resolution for external JSON schema references. It enables validation of JSON data against schemas that reference remote schema definitions, with built-in caching and timeout protection.

## Why Use RefResolver?

When working with JSON schemas, you may encounter `$ref` properties that point to external schema definitions hosted remotely:

```json
{
  "$ref": "https://example.com/schemas/common.json#/definitions/address"
}
```

The `RefResolver` solves several key challenges:

- **Remote Schema Resolution**: Automatically fetches and parses external schema definitions over HTTP/HTTPS
- **Performance Optimization**: Caches fetched schemas to avoid redundant network requests
- **Reliability**: Implements configurable timeouts to prevent hanging on slow or unresponsive endpoints
- **Error Handling**: Provides clear error messages for network failures, timeouts, and invalid JSON responses

## How to Use

### Basic Usage

The `RefResolver` is designed to work with the [JSONSchemer](https://github.com/davishmcclurg/json_schemer) gem:

```ruby
require 'labkit/json_schema/ref_resolver'

# Create a resolver instance
resolver = Labkit::JsonSchema::RefResolver.new

# Use with JSONSchemer
schema = JSONSchemer.schema(
  your_schema_hash,
  ref_resolver: resolver
)

# Validate data
schema.valid?(your_data)
```

### Custom Timeout Configuration

By default, the resolver uses a 2-second timeout for both connection and read operations. You can customize this:

```ruby
# Use a 5-second timeout
resolver = Labkit::JsonSchema::RefResolver.new(timeout_s: 5)
```

### Cache Management

The resolver maintains a class-level cache to share fetched schemas across instances:

```ruby
# Access the cache
Labkit::JsonSchema::RefResolver.cache

# Clear the cache if needed
Labkit::JsonSchema::RefResolver.cache.clear
```

## Use Case Example

The `RefResolver` is used in the User Experience SLI Registry [lib/labkit/user_experience_sli/registry.rb](lib/labkit/user_experience_sli/registry.rb) to validate user experience definitions against a JSON schema.

## Implementation Details

- Only HTTP and HTTPS schemes are supported. Other schemes (FTP, file://, etc.) will raise an error.
- The class-level cache is shared across all instances. In multi-threaded environments, consider using appropriate synchronization mechanisms if cache consistency is critical.
+63 −0
Original line number Diff line number Diff line
# frozen_string_literal: true

require 'net/http'
require 'uri'
require 'json'

module Labkit
  module JsonSchema
    # This class resolves JSON schema references (e.g., "$ref": "http://example.com/schema.json")
    # by fetching remote schemas over HTTP/HTTPS and caching them.
    # It is used by JSONSchemer to resolve external schema definitions.
    # We need it to validate JSON data against schemas that might be hosted remotely.
    class RefResolver
      mattr_accessor :cache, default: {}

      def initialize(timeout_s: 2)
        @timeout_s = timeout_s
      end

      def call(uri)
        uri_str = uri.to_s

        return cache[uri_str] if cache.key?(uri_str)

        cache[uri_str] = fetch_remote_schema(uri_str)
      end

      private

      def fetch_remote_schema(uri_str)
        uri = URI(uri_str)

        raise(JSONSchemer::UnknownRef, "Unsupported URI scheme: #{uri_str}") unless %w[http https].include?(uri.scheme)

        response = Net::HTTP.start(
          uri.host,
          uri.port,
          use_ssl: uri.scheme == 'https',
          open_timeout: @timeout_s,
          read_timeout: @timeout_s
        ) do |http|
          request = Net::HTTP::Get.new(uri.request_uri)
          http.request(request)
        end

        unless response.is_a?(Net::HTTPSuccess)
          raise(
            JSONSchemer::UnknownRef,
            "Failed to fetch #{uri_str}: #{response.code} #{response.message}"
          )
        end

        JSON.parse(response.body)
      rescue Net::OpenTimeout, Net::ReadTimeout => e
        raise(JSONSchemer::UnknownRef, "Timeout fetching #{uri_str}: #{e.message}")
      rescue JSON::ParserError => e
        raise(JSONSchemer::UnknownRef, "Invalid JSON at #{uri_str}: #{e.message}")
      rescue SocketError, Errno::ECONNREFUSED => e
        raise(JSONSchemer::UnknownRef, "Connection failed for #{uri_str}: #{e.message}")
      end
    end
  end
end
+2 −1
Original line number Diff line number Diff line
@@ -16,11 +16,12 @@ module Labkit
  module UserExperienceSli
    # Configuration class for UserExperienceSli
    class Configuration
      attr_accessor :logger, :registry_path
      attr_accessor :logger, :registry_path, :ref_resolver_timeout

      def initialize
        @logger = Labkit::Logging::JsonLogger.new($stdout)
        @registry_path = File.join("config", "user_experience_slis")
        @ref_resolver_timeout = 2
      end
    end

Loading