Scheduler

The scheduler is responsible for triggering the various ingestion activities on a configurable schedule. For this initial implementation is is recommended that one Cloud Run instance be selected as the scheduler leader through Terraform configuration. In the future, moving to a leader election algorithm would provide flexibility and recovery ability for the system.

Initially use a similar schedule to how often the components are run by the deployment project scheduled jobs.

The scheduler operates by loading the schedule configuration from the state table, including the last time notifications were made. The scheduler then waits and triggers actions by posting PubSub messages.

This component is part of the worker. See the worker design for additional information.