1. 10 Feb, 2021 2 commits
    • Patrick Steinhardt's avatar
      Explicitly put timestamp into Gitaly requests · d1df1c5c
      Patrick Steinhardt authored
      When creating git commits, the resulting commit hash is computed over
      all contents of the commit. This not only includes hashes of referenced
      objects like e.g. its tree, but also soft information like the author's
      and committer's timestamps. As a result, commits generated at different
      points in time have different object IDs.
      
      This hasn't been a problem for Gitaly until now, a standalone Gitaly
      instance still doesn't care. But this is slowly changing with the
      not-so-recent introduction of Praefect and reference transactions. If
      any RPC is called under a transaction, then the result will only be
      committed if all Gitaly nodes taking part in the transaction arrive at
      the same result. If any node tries to update a git reference to
      something else than the others, then it will abort the transaction and
      not modify the on-disk state.
      
      This is where the problem is: when for example two Gitalies are part of
      a transaction where the RPC shall compute a commit and update a branch
      to point to that commit, then both Gitaly instances need to compute the
      exact same commit, including the date. If there is time drift between
      both nodes or if they didn't compute the the commit at the exact same
      point in time, then chances are high they arrive at different commits
      and thus fail the transaction.
      
      Gitaly has thus introduced a new "timestamp" field for RPCs which create
      commits. If it is set, its date will be used for the author and/or
      committer signature. Given that this same request is then distrisbuted
      to all Gitalies, they stop relying on their own local time and only use
      what was provided.
      
      The only remaining piece is that there needs to be a central location
      where this time is reliably set, and the GitLab app is the natural
      location to do so. So this commit starts injecting timestamps into the
      requests to make transactions work reliably.
      d1df1c5c
    • Patrick Steinhardt's avatar
      Update Gitaly Gem to v13.9.0-rc1 · 4fa3ff69
      Patrick Steinhardt authored
      In order to get hold of the new "timestamp" field added to a subset of
      Gitaly RPCs, this commit updates the Gitaly Gem to v13.9.0-rc1.
      4fa3ff69
  2. 09 Feb, 2021 34 commits
  3. 08 Feb, 2021 4 commits
    • Amy Qualls's avatar
      Merge branch 'docs-aqualls-crosslink-todos' into 'master' · 7de5a0b0
      Amy Qualls authored
      Cross-link and fix to-do items API page
      
      See merge request gitlab-org/gitlab!53655
      7de5a0b0
    • Dylan Griffith's avatar
      Set 1s server side timeout on Elasticsearch counts · a893b326
      Dylan Griffith authored
      These count requests are loaded one per tab every time the search page
      loads. This means a single search for one type of document will trigger
      up to 7 other searches just to get the counts for the other tabs.
      
      These tab counts are often incredibly expensive requests too especially
      relative to the cheaper searches. For example an issue search may take
      1s while a blobs count will take 30s. Due to a limited thread pool on
      the Elasticsearch side we regularly see these count queries being the
      cause of queuing which is slowing down otherwise fast searches on
      GitLab.com.
      
      As such we want to set a timeout on these. This timeout is just a
      server side Elasticsearch timeout for now which is a soft limit because
      Elasticsearch is asynchronous and it may actually take Elasticsearch
      longer to realise it's timed out and cancel the query. As such we may
      see searches take a few seconds before they timeout even though the
      timeout is 1s. This is not perfect but benchmarking in the related issue
      shows this still can drastically improve throughput and this is one of
      the easiest steps to take now.
      
      One thing to also note about this approach is that users will still see
      a count in the event of a timeout. The count may be a partial count and
      actually lower than the true count. If they switch to the tab they will
      see a true count. I think this is probably still better than displaying
      nothing since the main value the tab counts have is showing whether or
      not there are searches on that tab at all.
      
      Later we may wish to introduce client side timeouts on our ES client but
      it's trickier to accomplish since we use a single client configuration
      which has a global timeout for all Elasticsearch queries. Additionally
      client side timeouts will result in errors that we may wish to handle
      specially to show some indicator on the tab.
      
      Read more at https://gitlab.com/gitlab-org/gitlab/-/issues/301146
      a893b326
    • Mayra Cabrera's avatar
      Merge branch 'fix_gitlab_experiment_keyword' into 'master' · 6f28cd1d
      Mayra Cabrera authored
      Fix ruby keyword warning in gitlab_experiment.rb
      
      See merge request gitlab-org/gitlab!53444
      6f28cd1d
    • Stan Hu's avatar
      Merge branch 'aa-record-conversion-for-3-experiment' into 'master' · d9730360
      Stan Hu authored
      Improve Tracking for 4 experiments in onboarding flow
      
      See merge request gitlab-org/gitlab!53583
      d9730360