• Timothy Andrew's avatar
    Improve performance of the cycle analytics page. · ba25e2f1
    Timothy Andrew authored
    1. These changes bring down page load time for 100 issues from more than
       a minute to about 1.5 seconds.
    
    2. This entire commit is composed of these types of performance
       enhancements:
    
         - Cache relevant data in `IssueMetrics` wherever possible.
         - Cache relevant data in `MergeRequestMetrics` wherever possible.
         - Preload metrics
    
    3. Given these improvements, we now only need to make 4 SQL calls:
    
        - Load all issues
        - Load all merge requests
        - Load all metrics for the issues
        - Load all metrics for the merge requests
    
    4. A list of all the data points that are now being pre-calculated:
    
        a. The first time an issue is mentioned in a commit
    
          - In `GitPushService`, find all issues mentioned by the given commit
            using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
            flag for each of them.
    
          - There seems to be a (pre-existing) bug here - files (and
            therefore commits) created using the Web CI don't have
            cross-references created, and issues are not closed even when
            the commit title is "Fixes #xx".
    
        b. The first time a merge request is deployed to production
    
          When a `Deployment` is created, find all merge requests that
          were merged in before the deployment, and set the
          `first_deployed_to_production_at` flag for each of them.
    
        c. The start / end time for a merge request pipeline
    
          Hook into the `Pipeline` state machine. When the `status` moves to
          `running`, find the merge requests whose tip commit matches the
          pipeline, and record the `latest_build_started_at` time for each
          of them. When the `status` moves to `success`, record the
          `latest_build_finished_at` time.
    
        d. The merge requests that close an issue
    
          - This was a big cause of the performance problems we were having
            with Cycle Analytics. We need to use `ReferenceExtractor` to make
            this calculation, which is slow when we have to run it on a large
            number of merge requests.
    
          - When a merge request is created, updated, or refreshed, find the
            issues it closes, and create an instance of
            `MergeRequestsClosingIssues`, which acts as a join model between
            merge requests and issues.
    
          - If a `MergeRequestsClosingIssues` instance links a merge request
            and an issue, that issue closes that merge request.
    
    5. The `Queries` module was changed into a class, so we can cache the
       results of `issues` and `merge_requests_closing_issues` across
       various cycle analytics stages.
    
    6. The code added in this commit is untested. Tests will be added in the
       next commit.
    ba25e2f1
schema.rb 52.4 KB