plugins/sources/git.py: Avoid downloading unused submodules

Currently we have configuration in place to disable use of
submodules, with the `checkout-submodules` git plugin option
and the individual `checkout` options for each submodule, but
these unused submodules are still downloaded at fetch time.

This patch fixes the plugin to just completely ignore the
submodules which are configured to be unused.

This was previously fixed in the master branch by !996, and
this patch backports the relevant fix to the 1.2 branch,
fixing issue #804.

buildstream/plugins/sources/git.py

@@ -421,13 +421,7 @@ class GitSource(Source):
         with self.timed_activity("Staging {}".format(self.mirror.url), silent_nested=True):
             self.mirror.stage(directory)
             for mirror in self.submodules:
-                if mirror.path in self.submodule_checkout_overrides:
-                    checkout = self.submodule_checkout_overrides[mirror.path]
-                else:
-                    checkout = self.checkout_submodules
-
-                if checkout:
-                    mirror.stage(directory)
+                mirror.stage(directory)
 
     def get_source_fetchers(self):
         yield self.mirror
@@ -465,6 +459,10 @@ class GitSource(Source):
         #
         for path, url in self.mirror.submodule_list():
 
+            # Completely ignore submodules which are disabled for checkout
+            if self.ignore_submodule(path):
+                continue
+
             # Allow configuration to override the upstream
             # location of the submodules.
             override_url = self.submodule_overrides.get(path)
@@ -478,6 +476,16 @@ class GitSource(Source):
 
         self.submodules = submodules
 
+    # Checks whether the plugin configuration has explicitly
+    # configured this submodule to be ignored
+    def ignore_submodule(self, path):
+        try:
+            checkout = self.submodule_checkout_overrides[path]
+        except KeyError:
+            checkout = self.checkout_submodules
+
+        return not checkout
+
 
 # Plugin entry point
 def setup():

[Seppo Yli-Olli]

Potentially also

checkout = self.submodule_checkout_overrides.get(path, self.checkout_submodules)

[Tristan Van Berkom]

@nanonyme Good point, it could have been written that way, but it's written with try/except in master, maybe worth changing it there...

[Seppo Yli-Olli]

Sure. Sorry for commenting on wrong branch.