• Stan Hu's avatar
    Avoid Gitaly RPCs in rate-limited raw blob requests · 5200d89d
    Stan Hu authored
    As observed in
    https://gitlab.com/gitlab-com/gl-infra/scalability/issues/77, a
    rate-limited blob request would still cause disk I/O since
    `assign_ref_vars` called FindCommit to look up the given commit.
    
    To avoid this, we need to do a number of things:
    
    1. Move rate-limiting to occur before the `assign_ref_vars` filter.
    
    2. Call `extracts_path` directly to pull out the filename of the
    request.
    
    3. Use the path as the key for rate-limiting. This means that filenames
    with different commits will now be rate-limited in the same bucket.
    That should be okay since the intent of the rate-limiting is to block
    abusive behavior, and requesting the same filename with different commit
    IDs at a high rate could create performance issues.
    
    4. Redirecting to another blob path would still cause Gitaly RPCs
    because the controller would call `assign_ref_vars`. Now, just render a
    plain text messsage with a 429 response code.
    
    Closes https://gitlab.com/gitlab-com/gl-infra/scalability/issues/77
    5200d89d
sh-skip-findcommit-lookup-rate-limit.yml 110 Bytes