Skip to content

Add svn:externals support

Taylor Smock requested to merge smocktaylor/reposurgeon:feature/externals into master

This will fix #436 (closed) .

Text copied from #436 (closed) comment 2.

I've got some WIP code (I've gotten to the point where I need to write to the new repo).

Basically, SVN externals have one of the following forms:

  • {RELATIVE DIRECTORY} [-rREVISION] {REPO URL}
  • [-rREVISION] {REPO URL} {RELATIVE DIRECTORY}
  • {REPO URL}[@REVISION] {RELATIVE DIRECTORY}

Notably, the svn:externals property can be set on any arbitrary directory (unlike git submodules; there is one source of truth for those).

What I've done in my WIP code is:

  • Parse the externals to a struct
  • Store the externals in a map of strings to lists, where the key is the directory where svn:externals was set and the value being a list of externals for that directory
    • This map is currently regenerated on every commit with svn:externals.
  • Print out the externals (which can be done on a per-commit basis, if I didn't bork something)
  • Search on an external URL/revision/path combo (although I think I need to fix the path -- I was just comparing the relative path) -- this was to make it easier to set the equivalent git URL, revision, and (maybe) a different path if the repo URL was checking out a subdirectory.

Converting from an external to a .gitmodule is something I've done manually (which was a PITA). As an example, svn:externals on the path applications/editors/josm is

http://josm.openstreetmap.de/svn/trunk/tools 00_core_tools
http://josm.openstreetmap.de/svn/trunk/test/lib 00_core_test_lib
https://github.com/JOSM/JOSM-areaselector/trunk areaselector

Converting it to git submodules would look like this:

[submodule "applications/editors/josm/core"]
  path = applications/editors/josm/core
  url = ../josm.git
  branch = master
[submodule "applications/editors/josm/plugins/areaselector"]
  path = applications/editors/josm/plugins/areaselector
  url = ../areaselector.git
  branch = master

Notably, since subversion lets us check out a subdirectory instead of the full repository, we need to additionally add symbolic links to the "real" location.

Example (works with reposurgeon 4.18):

blob :4000001 <<EOF
../core/tools
EOF
<30121> add M 120000 :4000001 applications/editors/josm/plugins/00_core_tools

blob :4000002 <<EOF
../core/test/lib
EOF
<30552> add M 120000 :4000002 applications/editors/josm/plugins/00_core_test_lib

So, what I need to do with my WIP code:

  1. Enable editing of repo information
  2. Generate the .gitmodules file every time the externals change; this should be fairly trivial, I'll just want to make it idempotent (probably sorting by full path).
  3. Set the directory checkout information to a fake blob when no revision information is given; this is where I got stuck on my initial version of the code (I was doing the all of the processing in svnread, and git didn't like the generated hashes)
  4. Create symbolic links, where necessary
    1. If a repository is already checked out somewhere and at the same revision, the symbolic link should go there
    2. Otherwise, the repository should be checked-out to a well-known location (e.g. .gitmodules-sources in the root directory)

Merge request reports