An error occurred fetching the project authors.
  1. 22 Jul, 2021 1 commit
  2. 30 Jun, 2021 1 commit
    • George Koltsov's avatar
      Preserve user authorship during bulk imports · 31245e13
      George Koltsov authored
       - Cache source_user_id => destination_user_id during
         MembersPipeline in redis in order to use it later on
         in other pipelines that reference users
       - Replace MembersMapper with UsersMapper in NdjsonPipeline
         as it has the same public API which implements user
         authorship in ndjson piplines that reference users
         (e.g. epics)
      
      Changelog: added
      31245e13
  3. 10 Jun, 2021 1 commit
    • Kassio Borges's avatar
      GithubImporter: Optimize Pull Request Review Importer · f60fd4e3
      Kassio Borges authored
      = Problem
      
      The Github API does not provide a way to fetch all the pull requests
      reviews of a project (repo), like it provides for comments, instead we
      have to fetch the reviews by Pull Request.
      
      For this reason, the
      Gitlab::GithubImport::Importer::PullRequestsReviewsImporter¹ have to
      iterate over the imported pull requests and for each one do request the
      reviews, which might be more than one page.
      
      If the importer hits a rate limit, the process restarts, and the
      imported pull requests are skipped², but the importer goes over all the
      review pages again.
      
      In other words, for some projects with large number of pull requests and
      large number of reviews per pull request, we might end up with
      duplicated reviews and unnecessary API requests, which would lead to
      longer importing times.
      
      = Proposed solution
      
      - To avoid duplicated comments, besides caching the Pull Requests ids,
        also cache the review ids and skip the already processed ones.
      
      - To avoid unnecessary API requests, use the PageCounter to only request
        pages that weren't yet imported.
      
      Related to: https://gitlab.com/gitlab-org/gitlab/-/issues/331315
      Changelog: changed
      MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62036
      f60fd4e3
  4. 15 Jun, 2020 1 commit
  5. 26 Mar, 2020 1 commit
  6. 12 Mar, 2020 1 commit