Loading
Commits on Source 23
-
This script will be used by the autobuild-run script to generate the configuration to test. It is put in the utils directory because it can also be called directly to allow users to test things. For now, it is a direct copy of the relevant functions from the autobuild-run script. The only changes are: - unneeded import statements are removed; - code/decode wrappers are limited to decode_byte_list; - __main__ handling is added. For now, the only supported arguments are the ones needed for autobuild-run. Follow-up patches will refactor things and also change the way the script is called. In this version, it can be called from the autobuild-run script as: subprocess.call([os.path.join(srcdir, "utils/genrandconfig"), "-i", str(kwargs['instance']), "--toolchains-url", kwargs['toolchains_url']], stdout=log, stderr=log) Signed-off-by:Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> --- I elected to go for argparse rather than docopt because it is much more standard and I find it easier to work with. --- v7: new patch
-
Popen is more complicated and more difficult to understand. check_output raises an exception if the exit code is non-zero, but that's probably what we want if ldd can't be executed. Signed-off-by: -
Warnings fixed: E731 do not assign a lambda expression, use a def -> urlopen_closing is defined with a def. urlopen is not used elsewhere so inlined. E302 expected 2 blank lines E501 line too long -> long lines due to a long string are NOT split E701 multiple statements on one line (colon) E722 do not use bare except' -> use "except Exception", so KeyInterrupt and SystemExit are still passed. We never intended to catch those. E741 ambiguous variable name 'l' -> variable name is replaced with the much more descriptive toolchains_csv E271 multiple spaces after keyword E231 missing whitespace after ',' Signed-off-by: -
kwargs is a left-over from the use of docopt, it's better to use argparse's Namespace object directly. In addition, most functions use just one or two fields of args, so these can just as well be passed directly as arguments to the function. Particularly for outputdir it doesn't make sense to reconstruct it all the time. Signed-off-by: -
The SystemInfo class is instantiated globally and passed down to all functions, but it is really only used in fixup_config. So instead, instantiate it there. Signed-off-by: -
The output of genrandconfig is currently very terse, which is annoying for debugging the script or generally seeing what is going on. Also the timing information added by log_write isn't very useful when the script is used stand-alone. In the new setup, (verbose) output goes to stdout and error output goes to stderr. Also the "INFO: generate the configuration" message is eliminated - it should go in the autobuild-run script. We also add an explicit message when a toolchain can't be used after the first defconfig, otherwise autobuild-run will just silently restart. Note that, since the output of make is no longer redirected to /dev/null, we get one more message on stderr that will be recorded in the autobuilder's log file: KCONFIG_SEED=0xXXXXXXXX. This approach allows us to optimise the error handling to use exceptions, where appropriate, which can be caught at the top level and converted to an error message to stderr. This, in turn, allows us to use subprocess.check_call, which eliminates a lot of conditions. Signed-off-by: -
This prepares for passing outputdir as an argument. Signed-off-by: -
This prepares for passing buildrootdir as an argument. Signed-off-by: -
The --instance argument is just an artifact of genrandconfig's history as part of autobuild-run. It is much more logical to pass the output directory and the buildroot directory as arguments, with sane defaults. This also allows us to remove the hack of creating a symlink in the instance directory if it doesn't exist yet. Note that the default outputdir 'output' doesn't work yet, because in that case Buildroot will put the config file in the buildroot directory instead of the output directory. This will be fixed in a follow-up patch. After this change, the script should be called from autobuild-run as: subprocess.call([os.path.join(srcdir, "utils/genrandconfig"), "-o", outputdir, "-b", srcdir, "--toolchains-url", kwargs['toolchains_url']], stdout=devnull, stderr=log) Signed-off-by: -
The path to the .config file is calculated in several places - replace it with a single calculation, and pass configfile as an argument to is_toolchain_usable and fixup_config. These functions also don't need outputdir any more. This makes it easier to fix the case when configfile is not in outputdir. Signed-off-by: -
When outputdir is 'output' (the default), genrandconfig didn't work correctly because it expects the configfile in outputdir, while Buildroot puts it in the buildroot directory. Fix this by explicitly checking if outputdir == buildrootdir/output. Because abspath is used for both paths, string comparison works reliably. Signed-off-by: -
This minimal configuration is also very useful outside test-pkg. In addition, it will simplify the config merge in a later patch. Signed-off-by: -
We really want test-pkg to do the test with a paranoid unsafe path. Signed-off-by: -
The user shouldn't need to pass this manually when creating a test-pkg config file. It's an absolutely harmless option to enable always. Signed-off-by: -
This has a number of side-effects which must be handled. The lines in minimal.config may be overridden by the random lines added by amending the configuration, so is_toolchain_usable() shouldn't take those into account, or indeed the random lines added. Therefore, make a copy of the config before appending minimal.config and the random lines. While we're at it, rename the variable to the more appropriate toolchainconfig. minimal.config sets BR2_INIT_NONE=y, but we really also want to test with BR2_INIT_BUSYBOX=y. Therefore, add a random line to use the busybox init system. We set its probability rather high. The probabilities of systemd and eudev are increased since they're now in the else branch of BR2_INIT_BUSYBOX, which halves the probability that we even get there. We now also generate configurations without busybox. Previously, busybox was almost always selected due to BR2_INIT_BUSYBOX=y. Only if systemd is selected there was a chance to build without busybox. We now set BR2_SYSTEM_BIN_SH_NONE=y, the other /bin/sh options are never tested. However, this is not really something that is relevant to test in the autobuilders. Signed-off-by: -
We currently have a list of toolchain configurations that are used by the autobuilders at [1]. However, this makes it a little more difficult for people to use these configurations, and also to have a different list of configurations for different branches. For example if a new architecture is introduced, the 2017.02.x branch doesn't have support for this architecture yet so it shouldn't try to run those configs. Therefore, include the autobuild config fragments directly in Buildroot, so they can be branched together with the rest. We create a new directory under support/ to store them. Generated with wget -nd --no-parent --recursive http://autobuild.buildroot.net/toolchains/configs/ The index.html file is removed. The toolchain-configs.csv file is adapted so the URLs become relative paths pointing to the config fragments. [1] http://autobuild.buildroot.net/toolchains/configs/toolchain-configs.csv Signed-off-by:
-
Now we have the toolchain config fragments in the buildroot directory itself, it is no longer necessary to fetch it from the toolchain URL. Signed-off-by: -
For now, testing a package requires network access. However, there are situations where everything is already cached locally (especially the toolchains tarballs) and network is not available (e.g. in the train, travelling back from FOSDEM...) Alternatively, one may also want to test against a subset of the default toolchains (e.g. the ones known to have a specific issue), or against additional toolchains (e.g. the ones used within the company). Add an option to use an alternate CSV file containing the config fragments of toolchains to try. Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by:
-
Now we have the toolchain config fragments in the buildroot directory itself, it is no longer necessary to fetch it from the toolchain URL. The --toolchains-url option is renamed to --toolchains-csv. The paths in the toolchains_csv file should be either absolute, or relative to buildrootdir. After this change, the script should be called from autobuild-run as: subprocess.call([os.path.join(srcdir, "utils/genrandconfig"), "-o", outputdir, "-b", srcdir, "--toolchains-csv", kwargs['toolchains_csv']], stdout=devnull, stderr=log) Signed-off-by: -
This column is not used by either genrandconfig or test-pkg, so remove it. Signed-off-by: -
The rules for the *config targets are all very similar, so factor them together using $@. Signed-off-by: -
It is used by Kconfig's merge_config.sh. No alldefpackageconfig is added, since it's rather pointless: it would only enable busybox. Signed-off-by: -
It is supposedly more robust than just concatenating. Signed-off-by: