Migration scripts for Blender.org (svn -> git)

Archived project! Repository is read-only

Name Last Update
authors.txt Loading commit data...
authors_extract_mail.py Loading commit data...
blender.lift Loading commit data...
blender_lift_generate_delete_commands.py Loading commit data...
convert.sh Loading commit data...
convert_all.sh Loading commit data...
convert_post_gc.sh Loading commit data...
convert_post_upload.sh Loading commit data...
convert_pre_svn_dump.sh Loading commit data...
convert_pre_svn_sync.sh Loading commit data...
git_log_all_files.py Loading commit data...
readme.html Loading commit data...
readme.rst Loading commit data...
svn_find_last_path.py Loading commit data...
todo.txt Loading commit data...

Blender Git Migration


The basic steps to converting svn to git with reposurgeon are:

# Dump the repository with svnadmin dump # Load the repository in reposurgeon and build the git repository.

Everything else ontop of this is mainly to get a better quality conversion and remove usless info.

You will have to manually make a dump, but after that, scripts are included in this repo to perform other steps.

Example conversion command.

REPOSURGEON="pypy reposurgeon" time ./convert

Reposurgeon Quickstart

For anyone who is reading this page and never used reposurgeon, heres a quick way to convert your repo.

From the command line dump

svnadmin dump /path/to/my/repo > mysvn.dump

Create a lift file: myproject.lift

read mysvn.dump
prefer git
rebuild mynewrepo

Run the conversion

reposurgeon "script blender.lift"

Now You’ll now have ./mynewrepo/ as a new git repo.

Useful Commands

Converting overnight

./convert_all.sh 2>&1 > convert.log ; shutdown now

Find large files

Once converted, finding large files.

git rev-list --objects --all | sort -k 2 > allfileshas.txt

git verify-pack -v objects/pack/pack-*.idx | egrep "^\w+ blob\W+[0-9]+ [0-9]+ [0-9]+$" | sort -k 3 -n -r > bigobjects.txt

for SHA in `cut -f 1 -d\  < bigobjects.txt`; do
    echo $(grep $SHA bigobjects.txt) $(grep $SHA allfileshas.txt) | awk '{print $1,$3,$7}' >> bigtosmall.txt

See: http://naleid.com/blog/2012/01/17/finding-and-purging-big-files-from-git-history/

Find missing authors

Listing all authors (with their email addresses)

git log --format='%aN = %aN <%aE>' | awk '{arr[$0]++} END{for (i in arr){print arr[i], i;}}' | sort -rn | cut -d\  -f2-

# and to find missing from authors.txt
git log --format='%aN = %aN <%aE>' | awk '{arr[$0]++} END{for (i in arr){print arr[i], i;}}' | sort -rn | cut -d\  -f2- | grep -v "@"

List all files ever

List all files ever committed to a git repo. ... as well as interesting files.

git log --all --pretty=format: --name-only --diff-filter=A | sort --unique - > allfiles.txt

# and to find interesting files
cat allfiles.txt | grep -v $"\.h$\|\.py$\|\.mk$\|\.cpp$\|\.cc$\|\.c$\|\.f\|\.hpp$\|\.txt$\|\.bat$\|\.vcproj$\|\.osl$\|\.ico$\|Makefile$\|SConscript$\|\.sln$\|\.rst$\|\.spi1d\|\.dsp\|\.dsw\|\.cmake$"

List branches sorted by date

When reviewing brancghes I found it handy to list by date rather then alphabetically.

git for-each-ref --sort=-committerdate --format='%(refname:short)' refs/heads/

List all commits containing a root path

This command checks all commits for the directory 'blender' (since it was a root directory that should never have been).

for REV in $(git rev-list --all) ; do ; git cat-file -p $REV^{tree} | egrep $" tree .*\s+blender$" > /dev/null && echo BAD $REV ; done

List all redundant commits

Find all commits that are not merges and don't change any files (possibly svn props change), most likely these can be removed.

git log --all --max-parents=1 --format="format:%H" --shortstat | pcregrep -v -M "^[a-z0-9]+\n " | egrep -v '^\s*$'

List all commits with 3+ parents

These commits are often faulty merges

git log --min-parents=3 --all

Uploading the repo

Need to include more info... basic commands.

git push --mirror