Skip to content

RDF migration

Luana Martelli requested to merge RDF-Migration into develop

Description

Contracts and signatures have transitioned from being stored in JSON format within MongoDB to being represented in RDF format and stored within Jena Fuseki databases. This migration impacts the ACS/PDP and CMS components.

Event Manager and Message Queue are gone and will be replaced by the Audit Trail component (MR !270 (merged)).

Changes

Global architecture

  • The component can be executed using the provided docker-compose files.
  • Currently, there is no support for Kubernetes available.
  • A dedicated network named "custodian" has been established, and containers must be situated within it to facilitate communication among themselves. The setup and configuration details are outlined in the platform/ Readme.
  • IP addresses are hardcoded in both the docker-compose files and the configuration files (of PDP and CMS).
  • User authentication verification is not enforced by the PDP and ACS components. This will be added in a further version (with keycloak check)

Platform

  • SSL certificates (for the communication with the HaProxy) have been pushed to make deployment and tests easier. They are for test purpose only.
  • PDP and CMS have different users
  • TODO : For prod, the certificates will be generated for K8s and stored in secrets. Dummy secrets/certificates have been pushed for tests only.
  • TODO : For tests and POC deployment, requests are made in HTTP -> change it to HTTPS as soon as possible
  • TODO : Should we encrypt data before storing them? By default, Jena does not have built-in encryption methods

CMS

  • Merged signatures and contracts management to a single folder
  • Inputs are sanitized based on regex
  • Examples with curl are provided in the Readme
  • The CMS signs the requests on the user's behalf
  • TODO : signature verification is implemented, but not used yet.

The container can be run and all functions are expected to operate correctly

PDP

  • GetAuthorization method has been completely changed to use RDF technology
  • Inputs are sanitized
  • Legacy code removed
  • Examples with curl are provided in the Readme
  • TODO : The structure of the variable userid, projectid, and datasetname requires discussion and validation
  • TODO : Missing functions to grant data access depending on the use case (require discussion and validation)
  • TODO : Swagger/OpenAPI is not implemented yet
  • TODO : Comment k8s deployment (but left it for the future)

The container can be run and should be functional

Reviewers

@martinfontanet @sabinem

Edited by Luana Martelli

Merge request reports