Skip to content

Support to detect git clone/pull audit event

What does this MR do and why?

Related issue: #384960

Support to detect git clone/pull audit event since now

GitHub support to detect both audit events since 2020.10, technically it is doable. Gitlab needs it too.

We can pass the want/have info from Gitaly to Rails to generate the streaming audit for clone/pull/push event.

By the workflow in #384960 (comment 1386091357):

sequenceDiagram
    User->>+Workhorse: git clone or pull (HTTPS)
    Workhorse->>Rails: request /git-upload-pack
    Rails->>Workhorse: authentication info (need an audit?)
    Workhorse->>Gitaly: PostUploadPackWithSidechannel
    Gitaly->>Workhorse: PostUploadPackWithSidechannelResponse(stats={Wants=9, Haves=0, ...})
    Workhorse-->>Rails: Report audit event if needed
    Workhorse->>-User: Return
sequenceDiagram
    User->>+GitLabShell: git clone or pull (SSH)
    GitLabShell->>Rails: request /allowed
    Rails->>GitLabShell: authentication info (need an audit?)
    GitLabShell->>Gitaly: UploadPackWithSidechannel
    Gitaly->>GitLabShell: UploadPackWithSidechannelResponse(stats={Wants=9, Haves=0, ...})
    GitLabShell-->>Rails: Report audit event if needed
    GitLabShell->>-User: Return

and the discussion from #384960 (comment 1431604700),

This MR creates a new separate internal API file lib/api/internal/shellhorse.rb to deal with all the API needs to AUTH both the Gitlab-Workhorse-Api-Request and Gitlab-Shell-Api-Request header. The new API path is api/v4/internal/shellhorse/git_audit_event

Screenshots or screen recordings

Just add a new field in the streaming audit event JSON: details --> custom_message --> verb

The new verb field's value should be these: clone/pull/push

{
  "id":"4795387c-4ac6-48a4-9ceb-40d9ff4d160b",
  "author_id":1,
  "entity_id":25,
  "entity_type":"Project",
  "details":{
    "author_name":"Administrator",
    "author_class":"User",
    "target_id":25,
    "target_type":"Project",
    "target_details":"pipeline-demo",
    "custom_message":{
      "protocol":"ssh",
      "action":"git-upload-pack",
      "verb":"clone"
    },
    "ip_address":"127.0.0.1",
    "entity_path":"tragedxy/pipeline-demo"
  },
  "ip_address":"127.0.0.1",
  "author_name":"Administrator",
  "entity_path":"tragedxy/pipeline-demo",
  "target_details":"pipeline-demo",
  "created_at":"2023-06-08T10:47:07.535Z",
  "target_type":"Project",
  "target_id":25,
  "event_type":"repository_git_operation"
}

How to set up and validate locally

  1. Register a free http API test account in https://pipedream.com, create a RequestBin workflow
  2. Add the RequestBin workflow as the streaming destination in Gitlab, just follow the Gitlab doc reference steps
  3. Enable the feature flag log_git_streaming_audit_events
  4. Use the Curl command to request the new internal API like below:
curl --location 'http://gdk.test:3000/api/v4/internal/git_audit_event' \
--header 'Content-Type: application/json' \
--header 'Cookie: BetterErrors-2.10.0-CSRF-Token=4849bd3f-c113-439e-b625-a25468586a29' \
--data '{
  "protocol": "ssh",
  "action": "git-upload-pack",
  "username": "root",
  "gl_repository": "project-27",
  "packfile_stats": {
    "wants": 0,
    "haves": 0,
    "payload_size": 0,
    "packets": 0,
    "caps": ["", ""],
    "shallows": 0,
    "deepen": "",
    "filter": ""
  }
}'
  1. Keeping monitor the pipedream.com RequestBin workflow page, the streaming audit event with the new verb field should be sent to it if all correct above, like below screenshot:

image

Related to #384960

/cc @chaomao

Edited by xfyuan

Merge request reports