Commit 184385ac authored by Kirill Smelkov's avatar Kirill Smelkov
Browse files

Add support to connect gitlab-shell to Unicorn via UNIX socket 

It is well known that UNIX sockets are faster than TCP over loopback.

E.g. on my machine according to lmbench[1] they have ~ 2 times
lower latency and ~ 2-3 times more throughput compared to TCP over
loopback:

    *Local* Communication latencies in microseconds - smaller is better
    ---------------------------------------------------------------------
    Host                 OS 2p/0K  Pipe AF     UDP  RPC/   TCP  RPC/ TCP
                            ctxsw       UNIX         UDP         TCP conn
    --------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
    teco      Linux 4.2.0-1  13.8  29.2 26.8  45.0  47.9  48.5  55.5  45.

    *Local* Communication bandwidths in MB/s - bigger is better
    -----------------------------------------------------------------------------
    Host                OS  Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                                 UNIX      reread reread (libc) (hand) read write
    --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
    teco      Linux 4.2.0-1 1084 4353 1493 2329.1 3720.7 1613.8 1109.2 3402 1404.

The same ratio usually holds for servers.

Also UNIX sockets, since they reside on filesystem, besides being faster with
less latency, have one another nice property: access permissions to them are
managed the same way access to files is.

Because of lower latencies and higher throughput - for performance reasons, and
for easier security, it makes sense to interconnect services on one machine via
UNIX sockets and talk via TCP only to outside world.

All internal services inside GitLab can talk to each other via UNIX socket
already and only gitlab-shell was missing support to talk to Unicorn via UNIX
socket.

Let's teach gitlab-shell to talk via UNIX sockets.

[1] http://www.bitmover.com/lmbench/

~~~~

In this patch we

- add URI::HTTPUNIX to handle http+unix:// URI scheme
- add Net::HTTPUNIX to handle "connect via unix socket and then talk http"
- adjust GitlabNet#http_client_for() accordingly
- adjust documentation in config.yml.example

The http+unix:// scheme is not reinvented anew: the idea about its structure is
quite logical an was already established at least in requests-unixsocket python
package:

    http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/
    https://github.com/msabramo/requests-unixsocket
parent 79fdf65c

config.yml.example

+3 −1
Original line number Diff line number Diff line
@@ -10,7 +10,9 @@ user: git 
# Default: http://localhost:8080/

# You only have to change the default if you have configured Unicorn

# to listen on a custom port, or if you have configured Unicorn to

# only listen on a Unix domain socket.

# only listen on a Unix domain socket. For Unix domain sockets use

# "http+unix://<urlquoted-path-to-socket>/", e.g.

# "http+unix://%2Fpath%2Fto%2Fsocket/"

gitlab_url: "http://localhost:8080/"



# See installation.md#using-https for additional HTTPS configuration details.

lib/gitlab_net.rb

+5 −0
Original line number Diff line number Diff line
@@ -5,6 +5,7 @@ require 'json' 
require_relative 'gitlab_config'

require_relative 'gitlab_logger'

require_relative 'gitlab_access'

require_relative 'httpunix'



class GitlabNet

  class ApiUnreachableError < StandardError; end
@@ -63,7 +64,11 @@ class GitlabNet 
  end



  def http_client_for(uri)

    if uri.is_a?(URI::HTTPUNIX)

      http = Net::HTTPUNIX.new(uri.hostname)

    else

      http = Net::HTTP.new(uri.host, uri.port)

    end



    if uri.is_a?(URI::HTTPS)

      http.use_ssl = true

lib/httpunix.rb

0 → 100644
+54 −0
Original line number Diff line number Diff line 
# support for http+unix://... connection scheme

#

# The URI scheme has the same structure as the similar one for python requests. See:

#   http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/

#   https://github.com/msabramo/requests-unixsocket



require 'uri'

require 'net/http'



module URI

  class HTTPUNIX < HTTP

    def hostname

      # decode %XX from path to file

      v = self.host

      URI.decode(v)

    end



    # port is not allowed in URI

    DEFAULT_PORT = nil

    def set_port(v)

      return v unless v

      raise InvalidURIError, "http+unix:// cannot contain port"

    end

  end

  @@schemes['HTTP+UNIX'] = HTTPUNIX

end



# Based on:

# - http://stackoverflow.com/questions/15637226/ruby-1-9-3-simple-get-request-to-unicorn-through-socket

# - Net::HTTP::connect

module Net

  class HTTPUNIX < HTTP

    def initialize(socketpath, port=nil)

      super(socketpath, port)

      @port    = nil    # HTTP will set it to default - override back -> set DEFAULT_PORT

    end



    # override to prevent ":<port>" being appended to HTTP_HOST

    def addr_port

      address

    end



    def connect

      D "opening connection to #{address} ..."

      s = UNIXSocket.new(address)

      D "opened"

      @socket = BufferedIO.new(s)

      @socket.read_timeout = @read_timeout

      @socket.continue_timeout = @continue_timeout

      @socket.debug_output = @debug_output

      on_connect

    end

  end

end

spec/httpunix_spec.rb

0 → 100644
+55 −0
Original line number Diff line number Diff line 
require_relative 'spec_helper'

require_relative '../lib/httpunix'

require 'webrick'



describe URI::HTTPUNIX do

  describe :parse do

    uri = URI::parse('http+unix://%2Fpath%2Fto%2Fsocket/img.jpg')

    subject { uri }



    it { should be_an_instance_of(URI::HTTPUNIX) }

    its(:scheme) { should eq('http+unix') }

    its(:hostname) { should eq('/path/to/socket') }

    its(:path) { should eq('/img.jpg') }

  end

end





# like WEBrick::HTTPServer, but listens on UNIX socket

class HTTPUNIXServer < WEBrick::HTTPServer

  def listen(address, port)

    socket = Socket.unix_server_socket(address)

    socket.autoclose = false

    server = UNIXServer.for_fd(socket.fileno)

    socket.close

    @listeners << server

  end

end



def tmp_socket_path

  File.join(ROOT_PATH, 'tmp', 'socket')

end



describe Net::HTTPUNIX do

  # "hello world" over unix socket server in background thread

  FileUtils.mkdir_p(File.dirname(tmp_socket_path))

  server = HTTPUNIXServer.new(:BindAddress => tmp_socket_path)

  server.mount_proc '/' do |req, resp|

    resp.body = "Hello World  (at #{req.path})"

  end

  Thread.start { server.start }



  it "talks via HTTP ok" do

    VCR.turned_off do

      begin

        WebMock.allow_net_connect!

        http = Net::HTTPUNIX.new(tmp_socket_path)

        expect(http.get('/').body).to eq('Hello World  (at /)')

        expect(http.get('/path').body).to eq('Hello World  (at /path)')



      ensure

        WebMock.disable_net_connect!

      end

    end

  end

end