Skip to content

fix: workaround for adding large scripts to CLI_DO_INIT_TASKS

Artur Gaspar requested to merge artur/lms-job-script-size into main

Description

Adds a workaround to break up scripts in CLI_DO_INIT_TASKS that are too large to be passed to one invocation of sh.

Testing instructions

Steps to test the changes:

  1. Add a large amount of data to the site_configuration of each GROVE_ADDITIONAL_DOMAINS, so that none individually exceed 128KiB, but do in total.
  2. See that deploy fails with exec /usr/bin/sh: argument list too long.
    • https://gitlab.com/opencraft/client/asu-moe/grove-development/-/jobs/6038477451
    • $ ./kubectl logs --namespace=opencraft --follow lms-job-20240128192139-67b29
docker run -i -t --rm --platform linux/amd64 --workdir /workspace --mount type=bind,source=/home/artur/Opencraft/client/asu-moe/grove-development,destination=/workspace --mount type=bind,source=/home/artur/Opencraft/client/asu-moe/grove-development/kubeconfig-private.yml,destination=/home/grove/.kube/config -e TF_VAR_cluster_name -e TF_VAR_cluster_provider -e TF_VAR_aws_region -e TF_VAR_ami_id -e TF_VAR_max_worker_node_count -e TF_VAR_worker_node_size -e TF_VAR_rds_instance_class -e TF_VAR_rds_min_storage -e TF_VAR_rds_max_storage -e TF_VAR_rds_ca_cert_identifier -e TF_VAR_mongodbatlas_project_id -e DEPLOYMENT_BRANCH -e TUTOR_DOCKER_REGISTRY -e TUTOR_PLUGINS_ROOT -e TF_VAR_mongodb_version -e TF_VAR_cluster_domain -e TF_VAR_enable_monitoring_ingress -e TF_VAR_opensearch_persistence_size -e GITLAB_PROJECT_NUMERIC_ID -e GITLAB_USERNAME -e CI_REGISTRY_IMAGE -e GITLAB_PASSWORD -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e MONGODB_ATLAS_PUBLIC_KEY -e MONGODB_ATLAS_PRIVATE_KEY -e DIGITALOCEAN_TOKEN -e SPACES_ACCESS_KEY_ID -e SPACES_SECRET_ACCESS_KEY -e NEW_RELIC_API_KEY -e NEW_RELIC_MONITORING_EMAILS -e NEW_RELIC_REGION -e TF_VAR_gitlab_group_deploy_token_username -e TF_VAR_gitlab_group_deploy_token_password -e TF_VAR_gitlab_cluster_agent_token -e TF_VAR_do_token -e TF_STATE_NAME -e TF_VAR_gitlab_project_id -e TF_USERNAME -e TF_PASSWORD -e GITLAB_TOKEN -e TF_ROOT -e TF_ADDRESS -e CI_PROJECT_DIR registry.gitlab.com/opencraft/dev/grove/tools-container:viadanna-patch-redis-postdeploy kubectl logs --namespace=opencraft --follow lms-job-20240128192139-67b29
exec /usr/bin/sh: argument list too long
  3. Use this branch of tutor-contrib-grove, deploy again.
  4. See that deploy succeeds.
  5. See that the correct init commands are run.

Checklist

If any of the items below is not applicable, do not remove them, but put a check in it.

  • Git commit history is clean
  • Git commits are following conventional-commit practices
  • Unit tests are added/updated
  • Documentation is added/updated

Additional context

Internal-ref: https://tasks.opencraft.com/browse/BB-8515

Edited by Artur Gaspar

Merge request reports