Cargo source doesn't support using an url alias if there are no refs listed. `bst track` fails

The cargo plugin can be specified in a bst file with or without a ref section, and with or without a url section. Using an alias in the url section seems to cause bst track to fail, but only if there is no ref section.

Specifically, the bug occurs when there is no ref section for the cargo source in the actual .bst file. Refs in the project.refs file will not resolve the issue.

This issue was discovered on the Gnome-Build-Meta project, in relation to the librsvg element as well as two other elements with cargo sources.

Gnome-build-meta uses a project.refs file rather than listing refs in the individual bst files, which is probably why the problem was noticed here.

Steps to replicate

  1. Clone the gnome-build-meta project.
  2. Update the aliases section of project.conf with a new alias: crates: https://static.crates.io/.
  3. Update the cargo source in elements/sdk/librsvg.bst, by adding a url field: url: crates:crates
  4. run bst track sdk/librsvg.bst

Expected results

bst track should complete without any errors (which may involve creating or updating project.refs).

Actual results

bst track fails. See first comment for the full python Traceback. The last few lines are:

      File "/home/douglaswinship/.local/lib/python3.9/site-packages/buildstream/source.py", line 547, in translate_url
        self.mark_download_url(url, primary=primary)
      File "/home/douglaswinship/.local/lib/python3.9/site-packages/buildstream/source.py", line 614, in mark_download_url
        assert (url in self.__marked_urls or not _extract_alias(url)), \
    AssertionError: URL was not seen at configure time: crates:crates

Investigation

If there is no url at all, or if the full URL (https://static.crates.io/crates) is used, then bst track does not fail. If there is a ref section, then it doesn't seem to fail even when an alias is used. The failure only seems to occur if there is:

  • no ref section for the cargo source in the file
  • a url given
  • an alias used in the url

According to the BuildStream documentation the function translate_url "must be called for every URL in the configuration during Plugin.configure() if Source.mark_download_url() is not called."

I think this is the source of the problem. translate_url is called several times in the cargo source, but in separate disconnected places, none of which is in configure(). Depending on the inputs, it seems to be possible for the plugin to be invoked without translate_url ever being called.

Possible Workaround

It seems to be possible to add a 'dummy' refs section. This allows bst track to complete without failing, and I don't think it prevents the element from building (needs more testing).

- kind: cargo
  url: crates:crates
  ref:
  - name: null      # Dummy ref, needed to avoid
    version: null   # https://gitlab.com/BuildStream/bst-external/-/issues/53
Edited by Douglas Winship