Commits on Source 37
-
Daniel P. Berrangé authoredAfter installing meson+python, symbol errors are present: ld-elf.so.1: /lib/libc.so.7: version FBSD_1.7 required by /usr/local/lib/libpython3.9.so.1.0 not found This is usually a sign that the port was built on a newer version of the FreeBSD image. Since 13.1 is now available in Cirrus CI we should update it to regardless. Signed-off-by:Daniel P. Berrangé <berrange@redhat.com>
-
Pavel Hrdina authoredRecent change in systemd packaging in Fedora Rawhide discovered that in projects installing systemd unit files we are not depending on this package explicitly which broke libvirt-dbus package build. Packaging guidelines are clear about this that we should explicitly depend on this package. [1] [1] <https://docs.fedoraproject.org/en-US/packaging-guidelines/Systemd/#packaging > Signed-off-by:
Pavel Hrdina <phrdina@redhat.com>
-
Keep the inventory.target_facts dictionary immutable; instead, create a copy in the group_vars array, and only pass to ansible the targets that are actually used. Signed-off-by:Paolo Bonzini <pbonzini@redhat.com>
-
The ansible/group_vars/ directory is not directly related to what is passed to Ansible in group variables; those also include packages. Furthermore, it is also used to generate Dockerfiles which does not involve Ansible. Move it to another name which matches the name of the variables in the code. Signed-off-by: -
The ansible/vars/ directory is not passed directly to Ansible, it is passed in the group variables after processing in Python. Furthermore, it is also used to generate Dockerfiles which does not involve Ansible. Move it to another name which matches the name of the variables. Signed-off-by: -
The target name is simply the name of the file, it need not be included in the .yml file. Signed-off-by: -
The unit tests validate that target names (and hence filenames) are consistent with the OS name and version in the YAML file itself. Check it also at runtime to be safe and to provide better error messages in the unit tests. Signed-off-by: -
Reverse the direction in which facts[target] is created. Instead of starting with a copy of the shared facts and overriding them with target-specific facts, add the missing shared facts to facts[target]. With the old algorithm, subdictionaries could not be merged; the change enables that change, for example to allow different paths to come from different YAML files. Signed-off-by: -
Daniel P. Berrangé authored
This ensures we always have a stack trace available for any log messages associated with caught exceptions. Signed-off-by: -
Daniel P. Berrangé authored
When certain problems happen we catch and wrapper exceptions. Unfortunately this means that we loose the stack trace of the original exception. Logging the original exception lets the developer re-run with --debug to see the root cause trace. Signed-off-by: -
Daniel P. Berrangé authored
The 'except' clause is a legacy mechanism replaced by the more general purpose 'rules' clause. Signed-off-by: -
Daniel P. Berrangé authored
The package lists are supposed to be vertically aligned with the end of the package install command eg dnf install \ foo \ bar the alignment calls did not account for usage of 'nosync' so ended up with nosync dnf install \ foo \ bar This commit fixes it to nosync dnf install \ foo \ bar Signed-off-by: -
Daniel P. Berrangé authored
This sorting change will make later diffs easier to follow. Signed-off-by: -
Daniel P. Berrangé authored
The _format_section_native and _format_section_foreign methods construct a set of docker commands that execute shell commands to prepare a build host. We wish to use the shell commands outside the context of docker, so need to refactor the methods to separate the docker commands. Signed-off-by: -
Daniel P. Berrangé authored
The _format_section_native and _format_section_foreign methods construct a set of docker commands that export environment variables to jobs in the container. We wish to use the environment variables outside the context of docker, so need to refactor the methods to separate the docker commands. Signed-off-by: -
Daniel P. Berrangé authored
Re-arrange the methods so all the dockerfile specific methods come last in the class. This is to facilitate splitting the class in the subsequent commit. Signed-off-by: -
Daniel P. Berrangé authored
With the shell script logic separated from the dockerfile commands, it is now possible to introduce BuildEnvFormatter as a parent class to DockerfileFormatter, holding only the logic related to the shell command formatting. Signed-off-by: -
Daniel P. Berrangé authored
When building containers we remove excess package in order to reduce the container storage footprint. If installing a local env or throwaway CI env this cleanup can be skipped to speed things up a little. Signed-off-by: -
Daniel P. Berrangé authored
When building containers we use nosync to avoid flushing I/O to disk, since if anything goes wrong the container image can be thrown away. If installing a local env on a machine, however, this is undesirable so usage of nosync must be skipped. Signed-off-by: -
Daniel P. Berrangé authored
The new formatter class generates a shell script that contains a function 'install_buildenv'. When called this shell function will install all the packages needed to build the project. The shell script will also export any relevant environment variables. In terms of the build dependancies, the result should be functionally equivalent to what's present in the dockerfiles. In theory the dockerfiles could actually just source these shell scripts and call the functions, but then the dockerfiles would no longer be self-contained. Thus we just maintain the commonality in the formatter classes, and accept the duplication in the generated outputs. Signed-off-by: -
Daniel P. Berrangé authored
The new 'buildenvscript' command generates a shell script that contains the commands needed to setup a build environment. The commands are shared with those used in the dockerfiles. Signed-off-by: -
Daniel P. Berrangé authored
In preparation for some major changes to CI pipelines / jobs, introduce workflow rules that control what scenarios we allow pipelines to launch: - Push to branch - Manual launch from web UI - Manual launch from REST API - Scheduled job This will avoid repeating similar expressions in job rules. Signed-off-by: -
Daniel P. Berrangé authored
Some of the job rules and scripts refer to the upstream namespace of the project. With the namespace hardcoded in this way, it can be difficult to test CI changes prior to them being merged upstream. This commit introduces a 'RUN_UPSTREAM_NAMESPACE' variable that can be set when triggering a pipeline. That way CI maintainers can create a fork of their fork, and treat the 1st level fork as "upstream" for the purposes of testing changes in their 2nd level fork. Signed-off-by: -
Daniel P. Berrangé authored
The jobs all have the same pattern, only differing in the name of the script to run and the file extension of the output. Signed-off-by: -
Daniel P. Berrangé authored
Currently all Linux targets build and use containers for the CI jobs. A following commit is going to eliminate creation of cointainers in repo forks, in favour of running a shell script for populating the buildenv on the fly. This change causes the manifest command to generate suitable shell scripts in ci/buildenv/*.sh Signed-off-by: -
Daniel P. Berrangé authored
With the changes in the free tier, users are limited to 5 GB of storage quota. This is quickly consumed when CI pipelines publish 10-20 containers in the fork. The only option is to stop publishing containers in fork repositories. With this change, containers are only published in the upstream project after code is committed to the default branch. There now needs to be two sets of build jobs, one set will run using the pre-built container from upstream. The other set will use the shell buildenv script to dynamically populate a fresh build environment from the 3rd party distro base container image. Build jobs running upstream post commit on the default branch can use the pre-built containers, but those running other (stable) branches must always populate a fresh environment. In forks, pipelines will always populate a fresh environment, as it can't be determined through CI env variables alone, whether there are CI changes vs upstream. The 'changes:' clause is not usable for pushes in forks, as it reports on the delta vs the previous fork branch state, not the delta vs upstream. As well as saving storage quota in forks there are some other benefits. Pipelines testing commits which do not modify CI config will now be faster, since we're always skipping the container jobs. This new scheme will also unlock the ability to trigger pipelines in merge request context. Previously this was impossible, as the container jobs would publish containers for code that is yet to be reviewed or merged. As mentioned earlier, it is also now possible to run CI for upstream stable branches, since it won't mistakenly use containers associated with latest mainline. The effect of this change is a variety of different jobs will run depending on scenario - Push upstream default branch with CI changes * $TARGET-container * $TARGET-prebuilt-env - Push upstream default branch without CI changes * $TARGET-prebuilt-env - Push upstream non-default branch * $TARGET-local-env - Push fork * $TARGET-local-env NB, that the decision to use a prebuilt vs local env build jobs is per target, when triggered based on CI changes. IOW if a fork dockerfile for fedora-rawhide-mingw64 changes, only that target will use local env, remaining targets will continue with the prebuilt env. Signed-off-by: -
Daniel P. Berrangé authored
Currently pipelines are triggered from branch pushes, even though they are set as a pre-condition for merge request acceptance. This is wasteful of contributors' limited CI quota as often there is a need to push separately from submitting code for merge. To stop wasting CI quota this switches to only trigger pipelines when a merge request is opened. This also enables projects to make use of merge trains, whereupon the CI is re-run after the merge request is accepted, to prove it still passes even if HEAD has moved on. Contributors can force a pipeline on push by setting 'RUN_PIPELINE=1' as an environment variable. Signed-off-by: -
Daniel P. Berrangé authored
When working on CI jobs it is often unclear what variables are defined in what scenario. The 'debug' job can be triggered by setting the CI variable 'RUN_DEBUG=1', and will print all defined CI variables. Signed-off-by: -
Signed-off-by:Richard W.M. Jones <rjones@redhat.com>
-
Erik Skultety authored
- Alpine Edge has a rust-clippy package - Alpine 3.15 doesn't have a package and requires usage of Rustup - FreeBSD packages clippy as part of the rust package Signed-off-by:Erik Skultety <eskultet@redhat.com>
-
Erik Skultety authored
Tumbleweed now includes python minor version in package names and includes a 3.9 version of flake8. Signed-off-by: -
Next patch will introduce the libnbd project, let's add their build and unit test dependencies. Signed-off-by:
-
Erik Skultety authored
Dependencies were added in the previous patch, this patch formally adds the YAML configuration file and the corresponding generated Dockerfiles. Signed-off-by: -
Erik Skultety authored
The out data were apparently hand-cooked and now if anyone regenerates the test data, these changes will automatically be pulled in as well. Fix it by regenerating the data properly. Fixes: 3c1adcac Signed-off-by:
-
Erik Skultety authored
Unlike with RPM, apt is very sensitive on fetching fresh repository metadata before installing a package. Any Debian/Ubuntu template base images have been stripped from these metadata to save space before publishing. If a VM was created from such a base image, we need to make sure we update the repo cache before installing any packages. Signed-off-by: -
Brad Smith authored
Signed-off-by:Brad Smith <brad@comstyle.com>
-
Daniel P. Berrangé authored
Previously the manifest had relied on the $CI_PROJECT_NAME variable to build a path to the upstream container registry. This was incorrect though, because the repo fork may have had its name changed relative to upstream. This the current manifest code uses the lcitool project name to identify the upstream cohntainer registry. This means that project names need to match the git repo names, but we never updated the libvirt Go projects after we switched development to new repos to enable Go modules usage. Signed-off-by:
