Digital Lab

Digital Lab Wiki

Ecosystem

Overview of the Digital Lab ecosystem, consisting of three services: virtual labs, Themis, and RepoMan.Red indicates external services,indicates internal services developed by the Digital Lab. Each internal service is a standalone application that can optionally integrate with other internal/external services.

---
Title: "Ecosystem"
---

graph BT
    LTI["LTI (Brightspace)"]

    
    REPO_MAN -- "Create Repositories, Teams, etc." --> SCM_PLATFORM

    SCM_PLATFORM -- "Deploy Environment" --> VIRTUAL_LABS

    REPO_MAN -.-> LTI
    THEMIS -.-> LTI
    VIRTUAL_LABS -.-> LTI

    subgraph Digital Lab
        VIRTUAL_LABS["Virtual Labs"]
        THEMIS["Themis"]
        REPO_MAN["RepoMan"]

        THEMIS -- "Deploy Judgment" --> VIRTUAL_LABS
        VIRTUAL_LABS -- "Submit Assignment" --> THEMIS

        REPO_MAN ~~~ THEMIS ~~~ VIRTUAL_LABS
    end

    SCM_PLATFORM["SCM Platform (Gitlab)"]

Global

graph LR
    LTI --- AUTH["Authentication & Authorization"]
    IDP --- IDP_EXTERNAL["idp: SurfConext"]
    
    subgraph Digital Lab
        %% input / UI's
        ADMIN_UI["Admin UI"]
        WEB_UI["Web UI"]
        CLI["Command Line Interface"]
        LTI_CONNECTOR["LTI Connector"]
        API["API Gateway"]

        ADMIN_UI --> API
        WEB_UI --> API
        CLI --> API
        LTI_CONNECTOR --> API
        API --> AUTH

        %% AUTH
        AUTH["Authentication & Authorization"]
        IDP["idp: KeyCloack"]
        AUTH --- IDP["idp: KeyCloack"]


        %% DL services
        USERS["Users"]
        COURSES["Courses"]
        GRADES["Grades"]
        PROVISION["Provision"]
        CONFIG["Configuration"]

        API ~~~ CONFIG
        
        API --> USERS
        API --> COURSES
        API --> REPOS

        CONFIG --> PROVISION
        CONFIG --> COURSES
        CONFIG --> GRADES
        CONFIG --> REPOS

        COURSES -.- REPOS
        COURSES -.- ASSIGNMENTS
        USERS -.- REPOS
        USERS -.- ASSIGNMENTS

        subgraph Themis
            ASSIGNMENTS["Assignments"]
            RUNTIME_ORCH["Runtime Orchestrator"]
            COMP_CONT["Companion Container"]

            ASSIGNMENTS --> RUNTIME_ORCH
            
            RUNTIME_ORCH --> COMP_CONT
            RUNTIME_ORCH --> JUDGMENTS["Judgements"]
        end

        subgraph RepoMan
            REPOS["Repositories"] --> SCM["SCM Platform Worker"]
        end

        SCM_PLATFORM["SCM Platform"]
        SCM --> SCM_PLATFORM
    end

Container Image Layers

flowchart TB
    subgraph Container Image
        BASE["Base Layer (OS)"]
        COURSE["Course Layer (Dependencies)"]
        ASSIGNMENT["Assignment Layer (Insertables)"]
        
        BASE -.- Persisted
        COURSE -.- Persisted
        ASSIGNMENT -.- Optional
    end

Themis Global Overview

A global view of the different components and services that will be part of Themis v2.

flowchart TB
subgraph Digital Lab
    API["API Gateway"]
    COURSES["Courses"]
    GRADES["Grades"]
    PROVISION["Provision"]
    CONFIG["Configuration"]
    
    %% external to internal
    API --> CONFIG
    API --> COURSES
    API -- "POST assignments" --> ASSIGNMENTS

    %% internal 
    COURSES ~~~ CONFIG 
    CONFIG -- "UPDATE images" --> PROVISION
    CONFIG -- "PUSH config" --> COURSES
    COURSES --> DB[(MariaDB)]

    subgraph Themis
        ASSIGNMENTS["Assignments"]
        RUNTIME_ORCH["Runtime Orchestrator"]
        COMP_CONT["Companion Container"]
        ENV["Environments"]
        JMT["Judgements"]

        THEMIS_DB[(MariaDB)]
        THEMIS_MINIO[(Minio)]

        ASSIGNMENTS --> THEMIS_DB
        ASSIGNMENTS --> THEMIS_MINIO
        
        ASSIGNMENTS ~~~ RUNTIME_ORCH

        RUNTIME_ORCH -- "GET insertables and overridables" --> ASSIGNMENTS
        ASSIGNMENTS -- "New submission event" --> RUNTIME_ORCH
        
        subgraph Single Submission
            RUNTIME_ORCH --> ENV
            ENV --> COMP_CONT
        end

        RUNTIME_ORCH -- "Test outputs" --> JMT
        JMT --> JMT_DB[(MariaDB)]
        JMT --> JMT_MINIO[(Minio)]
    end

    JMT -- "Update Grade" -->   GRADES
end

RepoMan Global Overview

Not that the GitWorker is per SCM platform.

flowchart TB
WEB_UI["Web UI"]

subgraph Digital Lab
    API["API Gateway"]
    LTI["LTI Service"]
    COURSES["Courses & Assignments Service"]
    GRADES["Grades"]

    subgraph RepoMan
        MQ["Message Queue (Kafka)"]
        REPO["Repository Service"]
        DB[("MariaDB")]
        GITWORKER["Git Worker"]

        REPO -.- MQ
        MQ -.- GITWORKER
        REPO -.- DB
    end

    REPO -.- LTI
    GRADES -.- LTI
end

WEB_UI --> API
API --> GRADES
API --> REPO

COURSES -.- GRADES
REPO -.- GRADES

SCM_GL["GitLab"]
SCM_GH["GitHub"]
SCM_BB["Bitbucket"]

GITWORKER --> SCM_GL
GITWORKER --> SCM_GH
GITWORKER --> SCM_BB

Quick Links

Git Policy