1. 23 Feb, 2018 8 commits
    • Yorick Peterse's avatar
      Optimise searching for users using short queries · cda22187
      Yorick Peterse authored
      This optimises searching for users when using queries consisting out of
      one or two characters such as "ab". We optimise such cases by searching
      for `LOWER(name)` and `LOWER(username)` instead of using `ILIKE`. Using
      `LOWER` produces a _much_ better performing query.
      
      For example, when searching for all users matching the term "a" we'd
      produce the following plan:
      
           Limit  (cost=637.69..637.74 rows=20 width=805) (actual time=41.983..41.995 rows=20 loops=1)
             Buffers: shared hit=8330
             ->  Sort  (cost=637.69..638.61 rows=368 width=805) (actual time=41.982..41.990 rows=20 loops=1)
                   Sort Key: (CASE WHEN ((name)::text = 'a'::text) THEN 0 WHEN ((username)::text = 'a'::text) THEN 1 WHEN ((email)::text = 'a'::text) THEN 2 ELSE 3 END), name
                   Sort Method: top-N heapsort  Memory: 35kB
                   Buffers: shared hit=8330
                   ->  Bitmap Heap Scan on users  (cost=75.47..627.89 rows=368 width=805) (actual time=9.452..41.305 rows=277 loops=1)
                         Recheck Cond: (((name)::text ~~* 'a'::text) OR ((username)::text ~~* 'a'::text) OR ((email)::text = 'a'::text))
                         Rows Removed by Index Recheck: 7601
                         Heap Blocks: exact=7636
                         Buffers: shared hit=8327
                         ->  BitmapOr  (cost=75.47..75.47 rows=368 width=0) (actual time=8.290..8.290 rows=0 loops=1)
                               Buffers: shared hit=691
                               ->  Bitmap Index Scan on index_users_on_name_trigram  (cost=0.00..38.85 rows=180 width=0) (actual time=4.369..4.369 rows=4071 loops=1)
                                     Index Cond: ((name)::text ~~* 'a'::text)
                                     Buffers: shared hit=360
                               ->  Bitmap Index Scan on index_users_on_username_trigram  (cost=0.00..34.41 rows=188 width=0) (actual time=3.896..3.896 rows=4140 loops=1)
                                     Index Cond: ((username)::text ~~* 'a'::text)
                                     Buffers: shared hit=328
                               ->  Bitmap Index Scan on users_email_key  (cost=0.00..1.94 rows=1 width=0) (actual time=0.022..0.022 rows=0 loops=1)
                                     Index Cond: ((email)::text = 'a'::text)
                                     Buffers: shared hit=3
           Planning time: 3.912 ms
           Execution time: 42.171 ms
      
      With the changes in this commit we now produce the following plan
      instead:
      
           Limit  (cost=13257.48..13257.53 rows=20 width=805) (actual time=1.567..1.579 rows=20 loops=1)
             Buffers: shared hit=287
             ->  Sort  (cost=13257.48..13280.93 rows=9379 width=805) (actual time=1.567..1.572 rows=20 loops=1)
                   Sort Key: (CASE WHEN ((name)::text = 'a'::text) THEN 0 WHEN ((username)::text = 'a'::text) THEN 1 WHEN ((email)::text = 'a'::text) THEN 2 ELSE 3 END), name
                   Sort Method: top-N heapsort  Memory: 35kB
                   Buffers: shared hit=287
                   ->  Bitmap Heap Scan on users  (cost=135.66..13007.91 rows=9379 width=805) (actual time=0.194..1.107 rows=277 loops=1)
                         Recheck Cond: ((lower((name)::text) = 'a'::text) OR (lower((username)::text) = 'a'::text) OR ((email)::text = 'a'::text))
                         Heap Blocks: exact=277
                         Buffers: shared hit=287
                         ->  BitmapOr  (cost=135.66..135.66 rows=9379 width=0) (actual time=0.152..0.152 rows=0 loops=1)
                               Buffers: shared hit=10
                               ->  Bitmap Index Scan on yorick_test_users  (cost=0.00..124.75 rows=9377 width=0) (actual time=0.101..0.101 rows=277 loops=1)
                                     Index Cond: (lower((name)::text) = 'a'::text)
                                     Buffers: shared hit=4
                               ->  Bitmap Index Scan on index_on_users_lower_username  (cost=0.00..1.94 rows=1 width=0) (actual time=0.035..0.035 rows=1 loops=1)
                                     Index Cond: (lower((username)::text) = 'a'::text)
                                     Buffers: shared hit=3
                               ->  Bitmap Index Scan on users_email_key  (cost=0.00..1.94 rows=1 width=0) (actual time=0.014..0.014 rows=0 loops=1)
                                     Index Cond: ((email)::text = 'a'::text)
                                     Buffers: shared hit=3
           Planning time: 0.303 ms
           Execution time: 1.687 ms
      
      Here we can see the new query is 25 times faster compared to the old
      query.
      cda22187
    • Yorick Peterse's avatar
      Removed pagination from AutocompleteUsersFinder · 37e681ac
      Yorick Peterse authored
      The frontend code doesn't use this so there's no practical point in
      supporting this. We also hardcode the limit to 20 so users can no longer
      request their own limit, which could overload the database (depending on
      any upper bounds perhaps enforced by Kaminari).
      37e681ac
    • Yorick Peterse's avatar
      Don't pluck IDs in AutocompleteUsersFinder · ce1f0ad2
      Yorick Peterse authored
      We can instead just use a UNION. This removes the need for plucking
      hundreds if not thousands of IDs into memory when a project has many
      members.
      ce1f0ad2
    • James Lopez's avatar
      Merge branch 'ce-to-ee-2018-02-23' into 'master' · abd17639
      James Lopez authored
      CE upstream - 2018-02-23 00:25 UTC
      
      See merge request gitlab-org/gitlab-ee!4685
      abd17639
    • Sean McGivern's avatar
      Merge branch 'bvl-optimize-mr-approvals' into 'master' · 316b5507
      Sean McGivern authored
      Optimize number of queries for approvals
      
      Closes #4903
      
      See merge request gitlab-org/gitlab-ee!4492
      316b5507
    • Douwe Maan's avatar
      Merge branch 'bvl-external-auth' into 'master' · fec98d0f
      Douwe Maan authored
      External authorization service
      
      Closes #4216
      
      See merge request gitlab-org/gitlab-ee!4675
      fec98d0f
    • Grzegorz Bizon's avatar
      Merge branch 'ee-42431-add-auto-devops-and-clusters-button-to-projects' into 'master' · 3cc868a1
      Grzegorz Bizon authored
      Add a button on the project page to set up a Kubernetes cluster and enable Auto DevOps -- EE merge edition
      
      See merge request gitlab-org/gitlab-ee!4543
      3cc868a1
    • GitLab Bot's avatar
  2. 22 Feb, 2018 32 commits