Reduce SQL queries when no pipeline hooks are active

If no pipeline hooks are active for a project, we still go through the
trouble of generating the hook data, which may make a number of SQL,
Gitaly, and Redis calls unnecessarily.

Most projects don't have any hooks, so we can reduce the amount of load
by only building the hook data if there are active hooks or
services. This requires several cheap SQL queries. Note that the optimal
way would be to lazily load the hook data, but that requires more
complex changes.

This is a similar optimization that was implemented for PostReceive
(https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31741).

Relates to
https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/12055

app/models/ci/pipeline.rb

@@ -831,9 +831,8 @@ module Ci
     end
 
     def execute_hooks
-      data = pipeline_data
-      project.execute_hooks(data, :pipeline_hooks)
-      project.execute_services(data, :pipeline_hooks)
+      project.execute_hooks(pipeline_data, :pipeline_hooks) if project.has_active_hooks?(:pipeline_hooks)
+      project.execute_services(pipeline_data, :pipeline_hooks) if project.has_active_services?(:pipeline_hooks)
     end
 
     # All the merge requests for which the current pipeline runs/ran against
@@ -1159,7 +1158,9 @@ module Ci
     end
 
     def pipeline_data
-      Gitlab::DataBuilder::Pipeline.build(self)
+      strong_memoize(:pipeline_data) do
+        Gitlab::DataBuilder::Pipeline.build(self)
+      end
     end
 
     def merge_request_diff_sha

changelogs/unreleased/sh-avoid-build-hooks-data.yml

+---
+title: Reduce SQL queries when no pipeline hooks are active
+merge_request: 49186
+author:
+type: performance

spec/models/ci/pipeline_spec.rb

@@ -2578,6 +2578,14 @@ RSpec.describe Ci::Pipeline, :mailer, factory_default: :keep do
           it 'receives a pending event once' do
             expect(WebMock).to have_requested_pipeline_hook('pending').once
           end
+
+          it 'builds hook data once' do
+            create(:pipelines_email_service, project: project)
+
+            expect(Gitlab::DataBuilder::Pipeline).to receive(:build).once.and_call_original
+
+            pipeline.execute_hooks
+          end
         end
 
         context 'when build is run' do
@@ -2639,6 +2647,12 @@ RSpec.describe Ci::Pipeline, :mailer, factory_default: :keep do
       it 'did not execute pipeline_hook after touched' do
         expect(WebMock).not_to have_requested(:post, hook.url)
       end
+
+      it 'does not build hook data' do
+        expect(Gitlab::DataBuilder::Pipeline).not_to receive(:build)
+
+        pipeline.execute_hooks
+      end
     end
 
     def create_build(name, stage_idx)