• Kirill Smelkov's avatar
    Add support to connect gitlab-shell to Unicorn via UNIX socket · 184385ac
    Kirill Smelkov authored
    It is well known that UNIX sockets are faster than TCP over loopback.
    
    E.g. on my machine according to lmbench[1] they have ~ 2 times
    lower latency and ~ 2-3 times more throughput compared to TCP over
    loopback:
    
        *Local* Communication latencies in microseconds - smaller is better
        ---------------------------------------------------------------------
        Host                 OS 2p/0K  Pipe AF     UDP  RPC/   TCP  RPC/ TCP
                                ctxsw       UNIX         UDP         TCP conn
        --------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
        teco      Linux 4.2.0-1  13.8  29.2 26.8  45.0  47.9  48.5  55.5  45.
    
        *Local* Communication bandwidths in MB/s - bigger is better
        -----------------------------------------------------------------------------
        Host                OS  Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                                     UNIX      reread reread (libc) (hand) read write
        --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
        teco      Linux 4.2.0-1 1084 4353 1493 2329.1 3720.7 1613.8 1109.2 3402 1404.
    
    The same ratio usually holds for servers.
    
    Also UNIX sockets, since they reside on filesystem, besides being faster with
    less latency, have one another nice property: access permissions to them are
    managed the same way access to files is.
    
    Because of lower latencies and higher throughput - for performance reasons, and
    for easier security, it makes sense to interconnect services on one machine via
    UNIX sockets and talk via TCP only to outside world.
    
    All internal services inside GitLab can talk to each other via UNIX socket
    already and only gitlab-shell was missing support to talk to Unicorn via UNIX
    socket.
    
    Let's teach gitlab-shell to talk via UNIX sockets.
    
    [1] http://www.bitmover.com/lmbench/
    
    ~~~~
    
    In this patch we
    
    - add URI::HTTPUNIX to handle http+unix:// URI scheme
    - add Net::HTTPUNIX to handle "connect via unix socket and then talk http"
    - adjust GitlabNet#http_client_for() accordingly
    - adjust documentation in config.yml.example
    
    The http+unix:// scheme is not reinvented anew: the idea about its structure is
    quite logical an was already established at least in requests-unixsocket python
    package:
    
        http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/
        https://github.com/msabramo/requests-unixsocket
    184385ac
config.yml.example 2.42 KB