Add git push test to the Performance Toolkit

Added test for git push based on https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols#_uploading_data and https://gitlab.com/gitlab-org/gitaly/blob/master/internal/service/smarthttp/receive_pack_test.go

Scenario:

1. Generate packfile binary data for git push
2. Send git push:

• GET /qa-perf-testing/gitlabhq.git/info/refs?service=git-receive-pack
• POST /qa-perf-testing/gitlabhq.git/git-receive-pack

1. Send another git push to undo the push above, so the project data stays consistent.

Caveat: Added test is not completely "real", because it uses existing commits data for push requests, we don't create a new data with this test. We will need to create a test framework which spins up virtual users that use real git commands in the shell and ensure that we have a stable RPS for these requests.

Issues that should be addressed:

• What data should be sent?
• Need to change ACCESS_TOKEN permissions for: pre, pre-puma, staging - add write_repository permission.

Git push test details

When user calls git push to upload data, the send-pack and receive-pack processes are involved. The send-pack process runs on the client and connects to a receive-pack process on the remote side.

1. The first request is a GET to the repository:

GET /qa-perf-testing/gitlabhq.git/info/refs?service=git-receive-pack HTTP/1.1
Host: localhost
Accept: */*
Accept-Encoding: deflate, gzip
Pragma: no-cache
Authorization: Basic {basicAuthToken}

Response example:

001f# service=git-receive-pack
000000adeedcd0db2cbc11f683f152ef61a9b9a266563eff refs/heads/1-1-auto-deploy-0000001report-status delete-refs side-band-64k quiet atomic ofs-delta push-options agent=git/2.22.0
0050b7524d0889c8e1d4afb698d9c6982ae6116049b5 refs/heads/1-1-auto-deploy-0000002
(...)

2. The second request to push objects to receive-pack(in the current implementation data = 0000):

POST /qa-perf-testing/gitlabhq.git/git-receive-pack HTTP/1.1
Host: localhost
Accept: application/x-git-receive-pack-result
Authorization: Basic {basicAuthToken}
Content-Type: application/x-git-receive-pack-request

{ Packfile binary data }

Packfile that is being sent to receive-pack has a similar structure:

len(pkt-line)+ oldhead(0 means a new one) + newhead(existing_commit_sha) + packProtocolCapabilities + pktFlushStr + PACK + git binary data

The beginning of the request body:

009befc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 b028afaede664580e04cddf0dd98faeff28ffbef refs/heads/git-pushtest report-status side-band-64k agent=git/2.22.00000

• 009b - pkt-line length(hex value)
• efc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 - The current head commit SHA.
• b028afaede664580e04cddf0dd98faeff28ffbef - The new head commit SHA that will be set for the specified branch.
• refs/heads/git-pushtest - Branch name
• report-status side-band-64k agent=git/2.22.0 - Pack protocol capabilities

The information above should be combined with binary data. A packfile MUST be sent if either create or update command is used, even if the server already has all the necessary objects. Since we're using existing commits, we can use any pack binary data, it will be ignored by Git anyway. We utilize hardcoded binary data for this purpose. We're generating the beginning of the request manually and combine it with hardcoded valid binary data with prepare_git_push_data_for_project.

To undo the commit we need to switch current head commit and new commit SHAs, for example:

1. First request - to make b028afae a new head of the branch git-pushtest - like git reset --soft HEAD~1

009befc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 b028afaede664580e04cddf0dd98faeff28ffbef refs/heads/git-pushtest report-status side-band-64k agent=git/2.22.00000{hardcoded_binary}

2. Second request - to restore efc9f72c as head commit of the branch git-pushtest - like git reset 'HEAD@{1}':

009bb028afaede664580e04cddf0dd98faeff28ffbef efc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 refs/heads/git-pushtest report-status side-band-64k agent=git/2.22.00000{hardcoded_binary}

To create /delete a new branch we're doing the similar thing:

1. First request - to create a new branch git-pushtest from efc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 - like git checkout -b git-pushtest efc9f72c9ffaa76f966b8c162a9d184d7aa1ff18:

009b0000000000000000000000000000000000000000 efc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 refs/heads/git-pushtest report-status side-band-64k agent=git/2.22.00000{hardcoded_binary}

• 0000000000000000000000000000000000000000 because there was no old head commit and we want to create a new branch.

2. Second request - to delete a new branch like git push origin --delete git-pushtest:

009befc9f72c9ffaa76f966b8c162a9d184d7aa1ff18 0000000000000000000000000000000000000000 refs/heads/git-pushtest report-status side-band-64k agent=git/2.22.00000{hardcoded_binary}

Example POST request responses:

time="2020-02-13T16:37:51+03:00" level=info msg="0042\x01000eunpack ok\n002bok refs/heads/12-1-auto-deploy-0011201\n00000085\x02\nTo create a merge request for 12-1-auto-deploy-0011201, visit:\n  https://staging.gitlab.com/qa-perf-testing/gitlabhq/-/merge_re004b\x02quests/new?merge_request%5Bsource_branch%5D=12-1-auto-deploy-0011201\n\n0000"
time="2020-02-13T16:37:51+03:00" level=info msg="0042\x01000eunpack ok\n002bok refs/heads/12-1-auto-deploy-0011201\n00000085\x02\nTo create a merge request for 12-1-auto-deploy-0011201, visit:\n  https://staging.gitlab.com/qa-perf-testing/gitlabhq/-/merge_re004b\x02quests/new?merge_request%5Bsource_branch%5D=12-1-auto-deploy-0011201\n\n0000"
time="2020-02-13T16:37:52+03:00" level=info msg=Running i=1 t=3.659269516s
time="2020-02-13T16:37:53+03:00" level=info msg=Running i=1 t=4.661263565s
time="2020-02-13T16:37:54+03:00" level=info msg="0042\x01000eunpack ok\n002bok refs/heads/12-1-auto-deploy-0011201\n00000085\x02\nTo create a merge request for 12-1-auto-deploy-0011201, visit:\n  https://staging.gitlab.com/qa-perf-testing/gitlabhq/-/merge_re004b\x02quests/new?merge_request%5Bsource_branch%5D=12-1-auto-deploy-0011201\n\n0000"

Since we're getting responses like this, it should mean that refs were updated and hooks were fired, GitLab responses with a link to create MR:

0042\x01000eunpack ok
002bok refs/heads/12-1-auto-deploy-0011201\n00000085\x02
To create a merge request for 12-1-auto-deploy-0011201, visit:
https://staging.gitlab.com/qa-perf-testing/gitlabhq/-/merge_re004b\x02quests/new?merge_request%5Bsource_branch%5D=12-1-auto-deploy-0011201

0000

Closes #97 (closed)