feat(autoflow): run AutoFlow on the rig

Runs AutoFlow next to the monolith on the rig, as the first milestone of gitlab-org/theseus&17: a gitlab-dev-stack release with one CloudNativePG database for autocore's central schema and workflow database 1 plus Valkey, a small autoflow-secrets chart that plays Vault's role for the shared secret between Rails and AutoFlow and AutoFlow's own secrets, and the AutoFlow chart installed from the Relay repository's master checkout with the image that repository's CI publishes, pinned by commit sha; Renovate moves that sha to the head of Relay's master through a git-refs manager, and each bump runs the integration job. PostgreSQL and Valkey reach AutoFlow through the infrastructure contract. Rails is pointed at the API Service over gRPC; the GitLab chart's own kas subchart stays off. Nothing serves Kubernetes agents. The chart's seed Job, off by default, is turned on here and registers workflow database 1 with shards 1-8; the Helm deployer waits for the rollout, so that post-install hook runs after AutoFlow has applied the central schema.

mise run autoflow -- run -s WORKFLOW.star runs the autoflow CLI against the rig's AutoFlow with no Rails involved, for trying definitions by hand; get and cancel work the same way. The task runs scripts/autoflow.sh, which builds the CLI from the Relay checkout, reads the secret from relay-secrets and port-forwards the Service. mise run autoflow:ctl -- central status --live runs autocorectl in the AutoFlow pod with the configuration the pod runs with. docs/autoflow.md documents all of it. scripts/ci/smoke-autoflow.sh checks the rollout, Rails to AutoFlow over the API Service, a hello workflow through that wrapper, shards 1-8 on workflow database 1 and both schemas in the one database; the integration autoflow CI job runs it. Edit mode comes from the Relay checkout's caproni-project.yaml and is opted into in caproni.local.yaml.

Trying it

From a checkout of this branch, with nothing else required beyond the rig's usual mise install:

cp caproni.local.yaml.example caproni.local.yaml   # uncomment fragments/caproni.autoflow.yaml under extends
caproni up

cat > hello.star <<'STAR'
def main(w, name="world"):
    print("hello, " + name)
    return "hello, " + name
STAR
mise run autoflow -- run -s hello.star --kwarg 'name="caproni"'

The wrapper logs its phases: building the autoflow CLI from repos/autoflow (minutes the first time, since it installs the Go toolchain and downloads modules; seconds afterwards), reading the API secret, port-forwarding the autoflow Service, then the CLI's own output ending in state=WORKFLOW_STATE_COMPLETED value="string_value:\"hello, caproni\"". The print line lands in AutoFlow's log:

caproni kubectl -n autoflow logs deploy/autoflow -c app | grep is_script_print
mise run autoflow:ctl -- central status --live        # workflow database 1, shards 1..8, all owned
scripts/ci/smoke-autoflow.sh                          # everything the integration job checks

Refs #51 (closed)

Edited by Timo Furrer

Merge request reports

Loading
Loading