We're running into some limitations with the current GitLab CI scheduling system in the [meltano](https://gitlab.com/meltano) project. We are using GitLab CI to coordinate data extract jobs, and so have some different needs than the standard CI/CD use case. * We have jobs that need to run at different frequencies, for a variety of reasons. Right now for us, that's typically every 2 hours, or every 24. The new method of tying a job to a scheduled pipeline based on variables can help here, but it's not clean. We'd have to go in and mark every job as either a 2 hour frequency or 24, so we don't get parallel runs of the same job when they overlap. This will also impact our review app pipelines, unless we create an additional parallel set of review jobs. This will add significant complexity to our CI YML. * We also have the reality that a job may fail and need to be re-run. For example if a 24 hour frequency job failed, perhaps due to the upstream API being down for maintenance or a temporary network issue, it should be retried before the next 24 hour run. We can sort of solve this by setting `retries`, but if its a network or upstream issue it's unlikely to be fixed in the short duration of the runs. * It would be nice to have real tri-state support within GitLab CI: failed, passed with warnings, or success. Right now there is no warning state, instead it is a job that failed but was allowed to. This would allow us to convey additional information: ELT is in process but hit an API limit before it could fully complete, a schema change was made that resulted in a partial ELT, etc. This is being tracked in https://gitlab.com/gitlab-org/gitlab-ce/issues/46298. For the above reasons, we are not using the GitLab CI flow control right now. Right now we have explicit time checks in the jobs, but will move away from that soon and store job metadata in the database. The job metadata will provide us the ability to: * Control the frequency of jobs without duplicating jobs in the CI YML. * Retry a job if it fails at the next interval. For example if we have a 2hour scheduled pipeline, we could retry a nightly job that failed 2 hours ago, with it more likely to succeed. * Report more complete status information, like the full tri-state. Some other background issues: * https://gitlab.com/meltano/meltano/issues/63 * https://gitlab.com/meltano/meltano/issues/69 * https://gitlab.com/meltano/meltano/issues/126 * https://gitlab.com/gitlab-org/gitlab-ce/issues/46298 #### Proposal It would be great to be able to define a frequency for each job, where it would not be re-ran again on that branch provided it succeeded. This would allow us to continue to have a 2 hour scheduled pipeline, but it would retry a nightly job every 2 hours until it succeeded. This type of behavior really isn't possible today. It would also avoid having the job duplication that would be required to use the current variable based system.